Skip to main content
ParameterSample ValueDescription
iceberg_s3_paths3://warehouse/ or gs://hive-dataproc-generated-bucket/hive-warehouseDetermines the S3 path or storage location for Iceberg data. The value s3://warehouse/ represents the designated S3 bucket or directory. If using GCP, use dataproc hive metastore bucket. Use s3a:// if using Minio
aws_regionus-east-1Specifies the AWS region associated with the S3 bucket where the data is stored.
aws_access_keyXXXAWS access key with sufficient permissions for S3. Optional if using IAM role attached to running instance/pod.
aws_secret_keyXXXAWS secret key with sufficient permissions for S3. Optional if using IAM role attached to running instance/pod.
s3_endpointhttp://S3_ENDPOINTSpecifies the endpoint URL for the S3 service. This may be used when connecting to an S3-compatible storage service like MinIO running on localhost.
hive_urithrift://<hostname>:9083 or thrift://METASTORE_IP:9083Defines the URI of the Hive Metastore service that the writer will connect to for catalog interactions. METASTORE_IP will be provided by GCP's Hive dataproc metastore or thrift://localhost:9083 or thrift://host.docker.internal:9083 if you are using local setup using docker compose
s3_use_sslfalseIndicates whether SSL is enabled for S3 connections. "false" means that SSL is disabled for these communications.
s3_path_styletrueDetermines if path-style access is used for S3. "true" means that the writer will use path-style addressing instead of the default virtual-hosted style.
hive_clients5Specifies the number of Hive clients allocated for managing interactions with the Hive Metastore.
hive_sasl_enabledfalseIndicates whether SASL authentication is enabled for the Hive connection. "false" means that SASL is disabled.
iceberg_dbiceberg_dbSpecifies the name of the Iceberg database to be used by the destination configuration.


💡 Join the OLake Community!

Got questions, ideas, or just want to connect with other data engineers?
👉 Join our Slack Community to get real-time support, share feedback, and shape the future of OLake together. 🚀

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!