Last updated:|... min read
source.json
{
"bucket_name": "my-data-warehouse",
"region": "us-east-1",
"path_prefix": "data/",
"access_key_id": "<YOUR_AWS_ACCESS_KEY_ID>",
"secret_access_key": "<YOUR_AWS_SECRET_ACCESS_KEY>",
"file_format": "parquet",
"max_threads": 10,
"retry_count": 3
}
Alternative: Use environment variables instead of hardcoding credentials:
export AWS_ACCESS_KEY_ID="your_access_key"
export AWS_SECRET_ACCESS_KEY="your_secret_key"
Then omit credentials from the config file - the driver will use environment variables automatically.