Last updated:|... min read
Key | Description | Data Type | Probable Values |
---|---|---|---|
type | Specifies the output file format for writing data. Currently, only the "PARQUET" format is supported. | string | "PARQUET" |
s3_bucket | The name of the Amazon S3/GCS bucket (without s3:// or gs:// ) where your output files will be stored. Ensure that the bucket exists and that you have proper access. | string | A valid S3 bucket name (e.g. "olake-s3-test" ) |
s3_region | The AWS/GCS region where the specified S3 bucket is hosted. | string | AWS/GCS region codes such as "us-west-2" , "ap-south-1" , etc. |
s3_access_key | The AWS/GCS HMAC access key used for authenticating S3 requests. | string | A valid AWS/GCS HMAC access key |
s3_secret_key | The AWS/GCS HMAC secret key used for S3 authentication. This key should be kept secure. | string | A valid AWS/GCS HMAC access key |
s3_endpoint | (Optional) The custom endpoint for S3-compatible services. Required for GCS using HMAC keys. | string | "https://storage.googleapis.com" |
s3_path | (Optional) The specific path (or prefix) within the S3 bucket where data files will be written. This is typically a folder path that starts with a / (e.g. "/data" ). | string | A valid path string |