Ingesting Files from S3 with OLake: AWS, MinIO, LocalStack
Ingest files from Amazon S3 and S3-compatible storage with OLake: stream discovery, formats, incremental sync and tips for AWS, MinIO and LocalStack.
Posts tagged “OLake”
Blogs on the topic OLake
View All Tags
Making OLake's MOR Iceberg Tables Work in Databricks
Make OLake's Merge-on-Read Iceberg tables work in Databricks with an automated MOR to Copy-on-Write script that gives accurate analytics queries.
OLake — now an Arrow-based Iceberg Ingestion Tool
OLake Go's Arrow-based Iceberg writer: how the rolling writer and equality delete writer work, and why a full load runs about 1.75x faster.
Kafka as a Source in OLake Go: Sync, Concurrency, Partitions
OLake Go's Kafka source: schema discovery, custom group balancing, partition-aware concurrency and incremental sync to Iceberg with exactly-once semantics.
Postgres to Iceberg to Doris: A Lakehouse with OLake
Build a lakehouse with PostgreSQL, Apache Iceberg and Apache Doris for real-time analytics, with OLake Go handling ingestion. Step-by-step guide.
Serverless Iceberg Lakehouse with OLake and Bauplan
How OLake Go replicates Postgres to Apache Iceberg and Bauplan runs serverless, Git-style versioned transformations on top of those tables.