Conflict-Free CDC into Apache Iceberg for Autonomous Agents
Conflict-free CDC into Apache Iceberg for AI agents: build temporal memory with OLake and ClickHouse and get past the Iceberg read amplification wall.
Blogs on OLake
Engineering posts from the OLake team on database replication, CDC, Apache Iceberg, lakehouse tooling and OLake release news.
Apache Iceberg Table Maintenance Made Easier with OLake Fusion
OLake Fusion handles Apache Iceberg table maintenance for CDC tables: tiered compaction for small and delete files, health metrics and lower Spark cost.
50% Cheaper Iceberg Compaction: OLake Fusion vs Spark
We benchmark Spark rewrite_data_files against OLake Fusion compaction on Apache Iceberg by running a full TPCH lineitem load from Postgres to GCP, applying 200k-record CDC batches every 2 minutes, and tracking TPC-H Query 6 performance, runtime, resource usage, and infrastructure cost.
The Architect’s Guide to CDC with Apache Iceberg
Learn how to design reliable CDC pipelines into Apache Iceberg, covering ingestion patterns, delete handling, and architecture best practices.
Iceberg Compaction: How Much Faster Are TPC-H Queries?
We ran TPC-H queries on Iceberg tables with many small files, then compacted them and ran the same queries again. Here's how much faster compaction made them.
Apache Iceberg Observability: Monitoring Metadata and Metrics
Apache Iceberg observability: use metadata tables to monitor file counts, table growth and schema changes, then add alerts and metrics reporters.