Skip to main content
KeyDescriptionData TypeProbable Values
typeSpecifies the output file format for writing data. Currently, only the "PARQUET" format is supported.string"PARQUET"
s3_bucketThe name of the Amazon S3/GCS bucket (without s3:// or gs://) where your output files will be stored. Ensure that the bucket exists and that you have proper access.stringA valid S3 bucket name (e.g. "olake-s3-test")
s3_regionThe AWS/GCS region where the specified S3 bucket is hosted.stringAWS/GCS region codes such as "us-west-2", "ap-south-1", etc.
s3_access_keyThe AWS/GCS HMAC access key used for authenticating S3 requests.stringA valid AWS/GCS HMAC access key
s3_secret_keyThe AWS/GCS HMAC secret key used for S3 authentication. This key should be kept secure.stringA valid AWS/GCS HMAC secret key
s3_endpoint(Optional) The custom endpoint for S3-compatible services. Required for GCS using HMAC keys.string"https://storage.googleapis.com"
s3_path(Optional) The specific path (or prefix) within the S3 bucket where data files will be written. This is typically a folder path that starts with a / (e.g. "/data").stringA valid path string
max_file_size_mb(Optional) Rolls a partition into a new parquet file once the current file reaches this size in MB. Fractional values are allowed. Defaults to 512 when unset.numberA positive number (integers e.g. 512, 128 or fractional e.g. 0.1 for 100KB)


💡 Join the OLake Community!

Got questions, ideas, or just want to connect with other data engineers?
👉 Join our Slack Community to get real-time support, share feedback, and shape the future of OLake together. 🚀

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!