S3 Tables

OLake supports S3 table (as REST catalog destination) bucket with AWS signing V4.

Configuration

Create a json for destination config (destination.json)

{
  "type": "ICEBERG",
  "writer": {
    "catalog_type": "rest",
    "rest_catalog_url": "https://s3tables.us-east-1.amazonaws.com/iceberg",
    "iceberg_s3_path": "arn:aws:s3tables:<REGION>:<ACCOUNT_ID>:bucket/<BUCKET_NAME>",
    "iceberg_db": "<NAMESPACE>",
    "aws_access_key": "",
    "aws_secret_key": "",
    "aws_region": "<REGION>",
    "rest_signing_name": "s3tables",
    "rest_signing_region": "<REGION>",
    "rest_signing_v_4": true
  }
}

Configuration Key Details

Change the following placeholders in the above configuration with:

• REGION -> Region for AWS bucket and catalog
• NAMESPACE -> This will be your s3 table bucket namespace
• ACCOUNT_ID -> AWS account identifier
• BUCKET_NAME -> Table Bucket Name

Authentication Fields (optional):

• token - Bearer token for token-based authentication
• oauth2_uri - OAuth2 server URI for OAuth2 authentication
• rest_auth_type - Authentication type (e.g., "oauth2")
• credential - Client secret or credential for OAuth2 (Usually id:secret)
• scope - OAuth2 scopes (space-separated)
• rest_signing_name - Service name for AWS Signature V4 (e.g., "s3tables")
• rest_signing_region - Region for AWS Signature V4 signing
• rest_signing_v_4 - Enable AWS Signature V4 signing (boolean)

More details on how to configure and use the S3 writer will be available soon. In the meantime, you can refer to the Iceberg writer documentation for general guidance on Iceberg integration, as the principles are similar across different storage backends.