OLake (v0.3.5 - v0.3.8)

November 29 – December 28, 2025

🎯 What's New

Sources

1. Filter parsing for special-character column names (Only for CLI)-
   Enhanced filter parsing to support quoted column names with double quotes and backticks, including names containing hyphens, spaces, dots, and other special characters.

2. Oracle source integration tests -
   Added comprehensive integration tests for Oracle 23c as source, covering full-load and incremental syncs plus primary/secondary cursor field validation for all drivers.

3. MariaDB CDC binlog support -
   Added MariaDB CDC source support by fixing binlog position detection: correctly identifies MariaDB flavor for MySQL, uses SHOW MASTER STATUS query with 4-column scanning, and resolves syntax/column count errors that prevented CDC from MariaDB 10.11+

4. SSH tunnel support for MongoDB/Oracle -
   Extended SSH tunnel connectivity to MongoDB and Oracle drivers.

5. Incremental sync and partitioning integration tests -
   Added integration tests covering full load and incremental sync workflows combined with partitioning validation for comprehensive end-to-end testing.

6. Kafka SSL/TLS configuration support -
   Added SSL/TLS configuration options for Kafka connections, including plain TLS (no SASL), mutual TLS with CA and client certificates/keys, and documentation for configuring SSL authentication with Kafka.

Catalogs

1. Custom catalog name support -
   Replaced hardcoded olake_iceberg catalog name with configurable catalog name from destination settings, allowing users to specify and sync under their preferred catalog name.

Destinations

1. Parquet writer integration tests -
   Added comprehensive integration tests for Parquet writer from PostgreSQL, MySQL, and MongoDB sources, covering schema evolution for both Parquet and Iceberg destinations plus edge cases for robust validation.

2. Arrow-based Iceberg writer support -
   Added an Apache Arrow–based Iceberg writer that writes data and delete files to object storage and registers them via the Iceberg Java API, supporting both full-refresh and CDC via equality deletes.

🔧 Bug Fixes & Stability

1. MongoDB cursor ObjectId fix -
   Fixed MongoDB cursor saving ObjectId values as ObjectId(number) strings, which caused id <= string comparisons to fail and skip all records. Now saves ObjectId cursors as hex strings for proper comparison, and resolved timestamp parsing by supporting MongoDB's native timestamp format to prevent epoch-start defaults.

2. Clear destination state reset on manual clear -
   When manual clear destination is triggered from the UI, OLake now sets the state to nil in the database and sends an empty {} state to clear destination, ensuring the next sync always runs a full refresh without relying on the previous global-pointer check.

3. MySQL DECIMAL/NUMERIC precision fix -
   Fixed precision loss for MySQL DECIMAL and NUMERIC values in Iceberg (legacy and Arrow) and Parquet writers so values like 157640.87 are preserved correctly instead of becoming 157640.875.

4. S3 upload size limit increase -
   Switched from single PUT requests (5GB limit) to multipart uploads to support larger file payloads to S3.