OLake Go (v0.9.0 - v0.9.2)

July 12, 2026 – Aug 13, 2026

🎯 What's New

Sources

1. S3 object storage abstraction -
   Refactored the S3 source driver to use a storage-agnostic object store interface instead of directly depending on the AWS SDK. This simplifies testing and lays the groundwork for supporting additional object stores such as GCS and Azure Blob, while preserving existing S3 behavior.

Platform Features

1. OLake Go version in startup logs -
   Added OLake Go version logging during application startup so the running version is immediately visible in the logs. This makes it easier to verify deployments, troubleshoot issues, and confirm the exact OLake Go version running in an environment.

2. Faster integration test startup -
   Updated the OLake Go Docker image to use a shared custom base image across all drivers, reducing the time required to spawn fresh driver containers during integration tests. This speeds up test execution and improves the development workflow.

3. Simplified local development and testing -
   Added new make targets to simplify local development and testing. Contributors can now start test databases, build drivers, and run integration, two-phase commit, and unit tests using the same workflow as CI, making it easier to set up a development environment and ensure consistent test execution.

4. Enhanced sync statistics -
   Expanded stats.json to report both the number of records and the total source bytes processed during a sync, providing better visibility into data movement and sync performance. Metrics are now accurately tracked across retries to prevent overcounting, and CPU usage is included to help monitor sync performance.

5. Source bytes in telemetry -
   Added source bytes read to OLake Go telemetry, extending the existing sync statistics to report the amount of data processed through telemetry.

6. Decoupled integration test dependencies -
   Removed the remaining shared Go dependencies between the integration test suites and the OLake Go product module, allowing the tests to build and run independently. This also keeps test-only dependencies from affecting OLake Go driver builds and simplifies dependency management across the project.

7. Integration tests against shipped driver images -
   Updated integration tests to run against the actual OLake driver Docker images, matching the environment and entrypoint used by deployed drivers. This ensures CI validates the shipped driver artifacts, including their runtime dependencies and entrypoint configuration, and catches image-specific regressions that were previously missed.

Catalogs

1. Google BigLake catalog support -
   Upgraded Apache Iceberg to v1.10.2, adding support for the Google BigLake catalog. Now we can configure BigLake catalogs directly from OLake Go, including the required Google authentication and catalog settings.

2. Google BigLake Multi-Bucket catalog support -
   Extended BigLake catalog support to enable OLake Go to write data to BigLake multi-bucket catalogs. Added the required GCP project configuration to ensure requests are correctly routed through the BigLake catalog.

Destinations

1. Rolling file writes for the S3 destination -
   Added rolling file writes for the S3 destination, allowing large partitions to be written as multiple size-bounded files instead of a single output file. This reduces memory usage during syncs and produces more manageable output files for large datasets.

Monitoring and Observability

1. Metrics Endpoint for Prometheus -
   Added a Prometheus /metrics endpoint to the OLake UI, exposing sync metrics such as sync status, duration, records ingested, bytes read, CPU usage, and memory usage. This enables users to monitor OLake Go syncs metrics through Prometheus.

🔧 Bug Fixes & Stability

1. Preserved record order during concurrent filtering -
   Fixed an issue where records could be processed out of order during concurrent filtering, potentially resulting in out-of-order CDC events. Filtering now preserves the original record order while retaining the performance benefits of concurrent evaluation.

2. Improved integration test reliability -
   Fixed an issue where stale Iceberg writer processes could prevent subsequent integration tests from starting, causing port conflicts between test runs. Test environments now clean up leaked processes correctly, ensuring reliable sequential test execution.

3. Decoupled integration test framework -
   Moved the integration test framework into a separate module, decoupling it from the main OLake Go codebase. This simplifies driver dependencies and allows integration tests to be built and run independently.

4. S3 Parquet data type compatibility -
   Fixed data type conversions in the S3 Parquet parser to ensure schemas and values are consistent with OLake Go's database drivers. This fixes issues with timestamp precision, numeric types, unsigned integers, UUIDs, and legacy timestamps, preventing data loss, overflow, and incorrect destination schemas.

5. Java dependency vulnerability scanning -
   Added Trivy-based vulnerability scanning for the Java Iceberg writer to detect known HIGH and CRITICAL security vulnerabilities during CI. This improves security coverage for the Java dependencies used by OLake Go.

6. Lint coverage for all driver modules -
   Expanded CI linting to cover all OLake Go driver modules, ensuring code quality issues are detected across the entire product codebase. Existing lint findings across the drivers have also been resolved, keeping driver builds consistent with the project's linting standards.

7. JVM trust store synchronization -
   Fixed an issue where custom CA certificates added to the OS trust store were not available to the JVM, causing TLS connections to internal REST catalogs, Hive metastores, and self-signed S3 endpoints to fail. The JVM trust store now stays synchronized with the OS trust store, ensuring consistent certificate handling across OLake Go components.

8. PostgreSQL CDC support for read replicas -
   Fixed an issue that prevented PostgreSQL CDC from running against read replicas because the driver attempted to read the current WAL position using a primary-only function. The driver now uses the replica's WAL replay position when running in recovery mode, enabling CDC on PostgreSQL read replicas while preserving existing behavior on primary instances.

9. Improved Iceberg destination configuration -
   Updated the Iceberg destination to provide catalog-specific configuration fields for Lakekeeper, Nessie, S3 Tables, Unity, Polaris, and BigLake. Authentication options and related settings are now shown based on the selected catalog, with catalog-specific configurations automatically applied to simplify destination setup.

10. Size-based gRPC batching -
    Added size-based batching for gRPC requests, allowing records to be grouped based on batch size rather than using a fixed number of records. This provides more consistent control over the amount of data sent in each batch.

11. Fixed Apache HttpClient security vulnerability -
    Updated the Apache HttpClient dependency used by the Iceberg writer to address CVE-2026-54399, a denial of service vulnerability caused by excessive HTTP headers. This ensures the Iceberg writer uses a patched version of the underlying HTTP client library.

12. Fixed last-batch flush handling -
    Fixed an issue where errors during the final batch flush could prevent the commit from being stopped correctly and could be overwritten by subsequent errors. This ensures failures such as schema evolution errors are properly surfaced and prevents records from being silently missed.

13. MongoDB inline TLS certificate support -
    Fixed MongoDB TLS configuration for UI and Docker deployments by allowing certificates to be provided directly through the connector configuration instead of requiring filesystem paths. This enables secure MongoDB connections in ephemeral environments while preserving support for existing CLI configurations using mounted certificate files.