Last updated:|... min read
| Parameter | Sample Value | Description |
|---|---|---|
iceberg_s3_path required | s3://<BUCKET_NAME>/ | S3 bucket path where Iceberg table data and metadata files will be stored. |
aws_region required | ap-south-1 | AWS Region where your S3 bucket and Glue Data Catalog exist. Required so OLake Go calls the correct regional S3 and Glue endpoints. |
| aws_access_key | XXX | AWS access key ID for S3 authentication. Optional if using IAM roles or instance profiles. If using IAM: add the required IAM environment variables in your shell or runtime environment. |
| aws_secret_key | XXX | AWS secret access key for S3 authentication. Optional if using IAM roles or instance profiles. If using IAM: add the required IAM environment variables in your shell or runtime environment. |
| s3_endpoint | https://glue.ap-south-1.amazonaws.com | S3 API endpoint for reading/writing Iceberg data files. Optional for AWS S3 but required for S3-compatible storage like MinIO. |
| glue_additional_config | true/false | Enable custom Glue catalog configuration when Glue catalog settings differ from S3 (different credentials, region, or endpoint). Leave off when Glue and S3 use the same AWS setup. |
| arrow_writes | false | Writes data and delete files using Apache Arrow based writer and registers them in Iceberg. |
When using separate Glue catalog settings, set glue_additional_config to true in destination.json and add the fields below. These configure Glue catalog connection settings separately from the S3 fields above.
| Parameter | Sample Value | Description |
|---|---|---|
| glue_catalog_id | 123456789012 | AWS account ID used as the Glue Data Catalog identifier. |
| glue_access_key | XXX | Access key for authenticating Glue catalog requests. Required when Glue credentials differ from S3 credentials. |
| glue_secret_key | XXX | Secret key for authenticating Glue catalog requests. Required when Glue credentials differ from S3 credentials. |
| glue_endpoint | https://glue.ap-south-1.amazonaws.com | Custom endpoint URL for AWS Glue or a Glue-compatible catalog service. |
| glue_region | ap-south-1 | Region for the Glue catalog, if different from the S3 region. Falls back to AWS Region if not set. |