AWS S3 Writer

This component allows you to efficiently write your database data into Amazon S3 in Parquet format. For more background details, please refer to the README. Before proceeding, make sure you have completed the getting started instructions.

IAM Permissions Required

To successfully sync your database data with S3, ensure that your AWS IAM policy grants at least minimal permissions for listing, creating, reading, and writing to the S3 bucket. Below is a sample IAM policy. Note that this example may provide more permissions than the absolute minimum required.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3ConsoleAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetAccountPublicAccessBlock",
        "s3:GetBucketAcl",
        "s3:GetBucketLocation",
        "s3:GetBucketPolicyStatus",
        "s3:GetBucketPublicAccessBlock",
        "s3:ListAccessPoints",
        "s3:ListAllMyBuckets"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ListObjectsInBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": [
        "arn:aws:s3:::olake-s3-test"
      ]
    },
    {
      "Sid": "AllObjectActions",
      "Effect": "Allow",
      "Action": "s3:*Object",
      "Resource": [
        "arn:aws:s3:::olake-s3-test/*"
      ]
    }
  ]
}