Skip to main content
ParameterSample ValueDescription
catalog_typehiveIndicates the catalog type used by the writer. "hive" means that the writer uses the Hive Metastore for catalog operations.
iceberg_s3_paths3://warehouse/ or gs://hive-dataproc-generated-bucket/hive-warehouseDetermines the S3 path or storage location for Iceberg data. The value "s3://warehouse/" represents the designated S3 bucket or directory. If using GCP, use dataproc hive metastore bucket. Use s3a:// if using Minio.
aws_regionus-east-1Specifies the AWS region associated with the S3 bucket where the data is stored.
aws_access_keyadminProvides the AWS access key used for authentication when connecting to S3.
aws_secret_keypasswordProvides the AWS secret key used for authentication when connecting to S3.
s3_endpointhttp://localhost:9000Specifies the endpoint URL for the S3 service. This may be used when connecting to an S3-compatible storage service like MinIO running on localhost.
hive_urithrift://localhost:9083 or thrift://METASTORE_IP:9083Defines the URI of the Hive Metastore service that the writer will connect to for catalog interactions. METASTORE_IP will be provided by GCP's Hive dataproc metastore. If you are using separate docker for hive then it can be thrift://host.docker.internal:9083
catalog_nameolake_icebergEnter a name for your catalog.
s3_use_sslfalseIndicates whether SSL is enabled for S3 connections. "false" means that SSL is disabled for these communications.
s3_path_styletrueDetermines if path-style access is used for S3. "true" means that the writer will use path-style addressing instead of the default virtual-hosted style.
hive_clients5Specifies the number of Hive clients allocated for managing interactions with the Hive Metastore.
hive_sasl_enabledfalseIndicates whether SASL authentication is enabled for the Hive connection. "false" means that SASL is disabled.


💡 Join the OLake Community!

Got questions, ideas, or just want to connect with other data engineers?
👉 Join our Slack Community to get real-time support, share feedback, and shape the future of OLake together. 🚀

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!