Skip to main content
ParameterSample ValueDescription
catalog_typehiveIndicates the catalog type used by the writer. "hive" means that the writer uses the Hive Metastore for catalog operations.
iceberg_s3_paths3://warehouse/ or gs://hive-dataproc-generated-bucket/hive-warehouseDetermines the S3 path or storage location for Iceberg data. The value "s3://warehouse/" represents the designated S3 bucket or directory. If using GCP, use dataproc hive metastore bucket. Use s3a:// if using Minio.
aws_regionus-east-1Specifies the AWS region associated with the S3 bucket where the data is stored.
aws_access_keyadminProvides the AWS access key used for authentication when connecting to S3.
aws_secret_keypasswordProvides the AWS secret key used for authentication when connecting to S3.
s3_endpointhttp://localhost:9000Specifies the endpoint URL for the S3 service. This may be used when connecting to an S3-compatible storage service like MinIO running on localhost.
hive_urithrift://localhost:9083 or thrift://METASTORE_IP:9083Defines the URI of the Hive Metastore service that the writer will connect to for catalog interactions. METASTORE_IP will be provided by GCP's Hive dataproc metastore. If you are using separate docker for hive then it can be thrift://host.docker.internal:9083
s3_use_sslfalseIndicates whether SSL is enabled for S3 connections. "false" means that SSL is disabled for these communications.
s3_path_styletrueDetermines if path-style access is used for S3. "true" means that the writer will use path-style addressing instead of the default virtual-hosted style.
hive_clients5Specifies the number of Hive clients allocated for managing interactions with the Hive Metastore.
hive_sasl_enabledfalseIndicates whether SASL authentication is enabled for the Hive connection. "false" means that SASL is disabled.
iceberg_dbolake_icebergSpecifies the name of the Iceberg database to be used by the destination configuration.


πŸ’‘ Join the OLake Community!

Got questions, ideas, or just want to connect with other data engineers?
πŸ‘‰ Join our Slack Community to get real-time support, share feedback, and shape the future of OLake together. πŸš€

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!