Parquet local writer

info

Please read the getting started instructions before proceeding with this section.

Using Dockerized OLake

destination.json

{
  "type": "PARQUET",
  "writer": {
    "local_path": "/mnt/config"
  }
}

Build OLake

destination.json

{
  "type": "PARQUET",
  "writer": {
    "local_path": "./mnt/config"
  }
}

Key	Data Type	Example Value	Description & Possible Values
type	string	"PARQUET"	Specifies the output file format. Currently, only the Parquet format is supported.
writer.local_path	string	"./mnt/config"	The directory in user's local machine where Parquet files will be stored.

Note: This configuration enables the Parquet local writer. For more details, check out the README section.

Data Partitioning

Refer to S3 docs for more details.