Configuration Fields

Parameter	Sample Value	Description
catalog_type required	rest	Defines the catalog type used by the writer.
rest_catalog_url required	http://<REST_ENDPOINT>:8181	Specifies the endpoint URL for the REST catalog service that the writer will connect to.
iceberg_s3_path required	s3://<BUCKET_NAME>	S3 bucket path where Iceberg table data and metadata files will be stored.
aws_region required	ap-south-1	AWS Region where your S3 bucket exists. Required so OLake Go calls the correct regional S3 endpoint for writing Iceberg data files.
catalog_name	olake_iceberg	Name of the Iceberg catalog OLake Go registers tables under. Defaults to olake_iceberg if left empty.
s3_endpoint	https://glue.ap-south-1.amazonaws.com	S3 API endpoint for writing Iceberg data files. Optional for AWS S3 but required for S3-compatible storage like MinIO.
aws_access_key	XXX	AWS access key ID for authentication. Optional if using IAM roles or instance profiles. If using IAM: Docker Compose: add the required IAM environment variables under x-envs in your compose file. See Service Environment Variables for setup details. Kubernetes: set up pod IAM in Cloud IAM Integration.
aws_secret_key	XXX	AWS secret access key for authentication. Optional if using IAM roles or instance profiles. If using IAM: Docker Compose: add the required IAM environment variables under x-envs in your compose file. See Service Environment Variables for setup details. Kubernetes: set up pod IAM in Cloud IAM Integration.
arrow_writes	false/true	Writes data and delete files using Apache Arrow based writer and registers them in Iceberg.
token	abc...xyz	Specifies the Bearer token sent in the Authorization header for authenticating with the REST catalog service.
oauth2_uri	https://auth.server.com/oauth/token	URL of the REST catalog’s OAuth2 token endpoint. OLake Go uses this with REST Credential to request an access token during OAuth2 client-credentials authentication.
credential	<client_id>:<client_secret>	Specifies the client ID and secret for OAuth2, formatted as client_id:client_secret. Used with REST Auth URI when OLake Go requests an access token from the catalog’s auth service. Not needed when using REST Token or other auth types.
scope	PRINCIPAL_ROLE:ALL	Required for OAuth2 client-credentials authentication along with REST Auth URI and REST Credential. Tells the OAuth2 server what level of access to grant, space-separated if requesting more than one. Values are catalog-specific. Not needed when using REST Token or other auth types.
rest_signing_name	s3tables	AWS SigV4 service name used to sign REST catalog API requests. Required when REST Auth Type is sigv4 or REST Enable Signature V4 is turned on.
rest_signing_region	us-east-1	AWS region used for SigV4 signing of REST catalog API requests. Required when REST Auth Type is sigv4 or REST Enable Signature V4 is turned on. Must match your catalog region (usually the same as AWS Region).
rest_signing_v_4	true/false	Enable AWS Signature V4 signing.
no_identifier_fields	true/false	Needed to set true for Databricks Unity Catalog as it doesn't support identifier fields

Catalog Name Supported for v0.3.5 and above

For the catalog name, OLake Go only supports lowercase letters and underscores. Spaces and special characters are not supported.