Skip to main content
ParameterSample ValueDescription
iceberg_s3_path
required
s3://<BUCKET_NAME>/S3 bucket path where Iceberg table data and metadata files will be stored.
aws_region
required
ap-south-1AWS Region where your S3 bucket and Glue Data Catalog exist. Required so OLake Go calls the correct regional S3 and Glue endpoints.
aws_access_keyXXXAWS access key ID for S3 authentication. Optional if using IAM roles or instance profiles.

If using IAM: add the required IAM environment variables in your shell or runtime environment.
aws_secret_keyXXXAWS secret access key for S3 authentication. Optional if using IAM roles or instance profiles.

If using IAM: add the required IAM environment variables in your shell or runtime environment.
s3_endpointhttps://glue.ap-south-1.amazonaws.comS3 API endpoint for reading/writing Iceberg data files. Optional for AWS S3 but required for S3-compatible storage like MinIO.
glue_additional_configtrue/falseEnable custom Glue catalog configuration when Glue catalog settings differ from S3 (different credentials, region, or endpoint). Leave off when Glue and S3 use the same AWS setup.
arrow_writesfalseWrites data and delete files using Apache Arrow based writer and registers them in Iceberg.

When using separate Glue catalog settings, set glue_additional_config to true in destination.json and add the fields below. These configure Glue catalog connection settings separately from the S3 fields above.

ParameterSample ValueDescription
glue_catalog_id123456789012AWS account ID used as the Glue Data Catalog identifier.
glue_access_keyXXXAccess key for authenticating Glue catalog requests. Required when Glue credentials differ from S3 credentials.
glue_secret_keyXXXSecret key for authenticating Glue catalog requests. Required when Glue credentials differ from S3 credentials.
glue_endpointhttps://glue.ap-south-1.amazonaws.comCustom endpoint URL for AWS Glue or a Glue-compatible catalog service.
glue_regionap-south-1Region for the Glue catalog, if different from the S3 region. Falls back to AWS Region if not set.


💡 Join the OLake Community!

Got questions, ideas, or just want to connect with other data engineers?
👉 Join our Slack Community to get real-time support, share feedback, and shape the future of OLake together. 🚀

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!