Last updated:|... min read
Parameter | Sample Value | Description |
---|---|---|
catalog_type | hive | Indicates the catalog type used by the writer. "hive" means that the writer uses the Hive Metastore for catalog operations. |
iceberg_s3_path | s3a://warehouse/ or gs://hive-dataproc-generated-bucket/hive-warehouse | Determines the S3 path or storage location for Iceberg data. The value "s3a://warehouse/" represents the designated S3 bucket or directory. If using GCP, use dataproc hive metastore bucket. |
aws_region | us-east-1 | Specifies the AWS region associated with the S3 bucket where the data is stored. |
aws_access_key | admin | Provides the AWS access key used for authentication when connecting to S3. |
aws_secret_key | password | Provides the AWS secret key used for authentication when connecting to S3. |
s3_endpoint | http://localhost:9000 | Specifies the endpoint URL for the S3 service. This may be used when connecting to an S3-compatible storage service like MinIO running on localhost. |
hive_uri | thrift://localhost:9083 or thrift://METASTORE_IP:9083 | Defines the URI of the Hive Metastore service that the writer will connect to for catalog interactions. METASTORE_IP will be provided by GCP's Hive dataproc metastore. |
s3_use_ssl | false | Indicates whether SSL is enabled for S3 connections. "false" means that SSL is disabled for these communications. |
s3_path_style | true | Determines if path-style access is used for S3. "true" means that the writer will use path-style addressing instead of the default virtual-hosted style. |
hive_clients | 5 | Specifies the number of Hive clients allocated for managing interactions with the Hive Metastore. |
hive_sasl_enabled | false | Indicates whether SASL authentication is enabled for the Hive connection. "false" means that SASL is disabled. |
iceberg_db | olake_iceberg | Specifies the name of the Iceberg database to be used by the destination configuration. |