Last updated:|... min read
Parameter | Sample Value | Description |
---|---|---|
Iceberg S3 Path | s3a://warehouse/ or gs://hive-dataproc-generated-bucket/hive-warehouse | Determines the S3 path or storage location for Iceberg data. The value "s3a://warehouse/" represents the designated S3 bucket or directory. If using GCP, use dataproc hive metastore bucket. |
AWS Region | us-east-1 | Specifies the AWS region associated with the S3 bucket where the data is stored. |
AWS Access Key | admin | Provides the AWS access key used for authentication when connecting to S3. |
AWS Secret Key | password | Provides the AWS secret key used for authentication when connecting to S3. |
S3 Endpoint | http://localhost:9000 | Specifies the endpoint URL for the S3 service. This may be used when connecting to an S3-compatible storage service like MinIO running on localhost. |
Hive URI | thrift://localhost:9083 or thrift://METASTORE_IP:9083 | Defines the URI of the Hive Metastore service that the writer will connect to for catalog interactions. METASTORE_IP will be provided by GCP's Hive dataproc metastore. |
Use SSL for S3 | false | Indicates whether SSL is enabled for S3 connections. "false" means that SSL is disabled for these communications. |
Use Path Style for S3 | true | Determines if path-style access is used for S3. "true" means that the writer will use path-style addressing instead of the default virtual-hosted style. |
Hive Clients | 5 | Specifies the number of Hive clients allocated for managing interactions with the Hive Metastore. |
Enable SASL for Hive | false | Indicates whether SASL authentication is enabled for the Hive connection. "false" means that SASL is disabled. |
Iceberg Database | olake_iceberg | Specifies the name of the Iceberg database to be used by the destination configuration. |