OLake Go (v0.8.0 - v0.8.2)

June 29, 2026 – July 11, 2026

🎯 What's New

Sources

1. 2PC support for Kafka -
   Adds two-phase commit support for Kafka by persisting consumer group ID, partition, and offset information in destination metadata, enabling reliable recovery so Kafka consumption can resume correctly from the last committed state.

Destinations

1. Single JVM per process for Iceberg writes -
   Previously, every Iceberg writer spawned its own JVM per stream/chunk, causing excessive memory usage and OOM risk on large concurrent syncs. Now a single shared JVM handles all streams, with ThreadSession based isolation preserving per-stream state and cutting memory overhead.

🔧 Bug Fixes & Stability

1. Refactored MySQL chunking helpers for testability -
   Split the MySQL GetOrSplitChunks flow into smaller helper functions for chunk sizing, column selection, bound selection, and split strategy routing, enabling independent unit tests without changing existing chunking behavior.

2. Placeholder for unavailable TOAST values in Postgres CDC -
   Added an olake_unavailable_value placeholder to mark unchanged TOAST columns whose values aren't available in PostgreSQL logical replication events.

3. Fixed discover failure for schema/table names containing "." -
   Tables with a . in their schema or name (e.g. user1.test_table) broke discover, since namespace and name were joined into one string and later re-split on ., aborting discovery entirely. Fixed by carrying a types.StreamID{Namespace, Name} struct through the driver contract across all 8 drivers, avoiding the round-trip.

4. Fixed nil pointer panic in DropStreams -
   destination.DropStreams panicked when a writer returned no shutdown callback, breaking clear-destination and any path relying on DropStreams. Fixed by guarding against a nil shutdown callback before invoking it.

5. Fixed CDC insert failure on GCS via S3-compat -
   CDC syncs to Iceberg on GCS failed at commit because deleting an unwritten equality-delete file returned 404 on GCS, crashing the sync. Fixed by wrapping the S3 client so a delete on a missing key is treated as success, restoring AWS-like delete behavior only when a custom s3_endpoint is configured.

6. Fixed partition spec column casing mismatch in Iceberg -
   Table creation failed when use_source_column_names was disabled, since destination schema columns were normalized to lowercase (e.g. ID → id) but the partition spec payload still carried the original source column name, causing the Java writer to fail building the PartitionSpec against the lowercased schema. Fixed by using the resolved destination column name when populating the partition spec.