Skip to main content
ParameterSample ValueDescription
iceberg_s3_path
required
s3://warehouse/ or gs://hive-dataproc-generated-bucket/hive-warehouseS3 bucket path where Iceberg table data and metadata files will be stored.
aws_region
required
us-east-1AWS Region where your S3 bucket exist. Required so OLake Go calls the correct regional S3 endpoints.
hive_uri
required
thrift://<hostname>:9083 or thrift://METASTORE_IP:9083Thrift endpoint URL for your Hive Metastore service. OLake Go uses this to read and write Iceberg catalog metadata.
hive_clients
required
5Specifies the number of Hive clients allocated for managing interactions with the Hive Metastore.
aws_access_keyXXXAWS access key ID for authentication. Optional if using IAM roles or instance profiles.

If using IAM:
Docker Compose: add the required IAM environment variables under x-envs in your compose file. See Service Environment Variables for setup details.
Kubernetes: set up pod IAM in Cloud IAM Integration.
aws_secret_keyXXXAWS secret access key for authentication. Optional if using IAM roles or instance profiles.

If using IAM:
Docker Compose: add the required IAM environment variables under x-envs in your compose file. See Service Environment Variables for setup details.
Kubernetes: set up pod IAM in Cloud IAM Integration.
s3_endpointhttp://S3_ENDPOINTS3 API endpoint for writing Iceberg data files. Optional for AWS S3 but required for S3-compatible storage like MinIO.
catalog_nameolake_icebergName of the Iceberg catalog OLake Go registers tables under. Defaults to olake_iceberg if left empty.
s3_use_sslfalse/trueControls whether OLake Go uses HTTPS when connecting to the S3 API for writing Iceberg files. Set to false when S3 Endpoint uses http:// (for example, local MinIO). Set to true when the endpoint uses https:// (for example, AWS S3 or TLS-enabled MinIO).
s3_path_stylefalse/trueControls how OLake Go formats S3 request URLs. Path-style puts the bucket name in the URL path instead of the hostname. This is required for MinIO and other S3-compatible storage that do not support AWS-style virtual-hosted URLs.
hive_sasl_enabledfalse/trueEnables SASL authentication when OLake Go connects to the Hive Metastore over Thrift. Required only if your metastore enforces SASL-based client authentication.
arrow_writesfalse/trueWrites data and delete files using Apache Arrow based writer and registers them in Iceberg.


💡 Join the OLake Community!

Got questions, ideas, or just want to connect with other data engineers?
👉 Join our Slack Community to get real-time support, share feedback, and shape the future of OLake together. 🚀

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!