Skip to main content

1. OLake native features

  1. Open data formats – OLake writes raw Parquet and fully ACID snapshots in Apache Iceberg so your lakehouse stays engine-agnostic.
  2. Iceberg writer – The dedicated Iceberg Java Writer (migrating to Iceberg Go writer) produces exactly-once, rollback-ready Iceberg v2 tables.
  3. Smart partitioning – We support both Iceberg partitioning rules andAWS S3 partitioning for Parquet for fast scans and efficient querying.
  4. Parallelised chunking – OLake splits big tables into smaller chunks, slashing total faster processing and parallel execution.
  5. Change Data Capture – We capture WALs for Postgres, binlogs for MongoDB and oplogs for MySQL in near real-time to keep the lake fresh without reloads.
  6. Schema evolution & datatype changes – Column adds, drops and type promotions are auto-detected and written per Iceberg v2 spec, so pipelines never break.
  7. Stateful, resumable syncs – If a job crashes (or is paused), OLake resumes from the last committed checkpoint—no manual fixes needed.
  8. Back-off & retries – OLake supports backoff retry count, meaning, if a sync fails, it will retry the sync after a certain period of time.
  9. Synchronization Modes: OLake supports both full, CDC (Change Data Capture) and strict CDC (Tracks only new changes from the current position in the MongoDB change stream, without performing an initial backfill) synchronization modes. Incremental sync is a work in progress and will be released soon.
  10. Level-1 JSON flattening – Nested JSON fields are optionally expanded into top-level columns for easier SQL.
  11. Airflow-first orchestration – Drop our Docker images into your DAGs (EC2 or Kubernetes) and drive syncs via Airflow.
  12. Developer playground – A one-click OLake + Iceberg sandbox lets you experiment locally with Trino, Spark, Flink or Snowflake readers.

2. Source-level features

  1. PostgreSQL – Full loads and WAL-based CDC for RDS, Aurora, Supabase, etc. See the Postgres connector.

  2. MySQL – Full loads plus binlog CDC for MySQL RDS, Aurora and older community versions. See the MySQL connector.

  3. MongoDB – High-throughput oplog capture for sharded or replica-set clusters. See the MongoDB connector.

  4. Optimised chunking strategies

    • MongoDBSplit-Vector, Bucket-Auto & Timestamp; details in the blog What Makes OLake Fast.
    • MySQL – Range splits driven by LIMIT/OFFSET next-query logic.
    • PostgresCTID ranges, batch-size column splits or next-query paging.
  5. Work-in-progress connectors

  6. We map each of database datatypes to respective Iceberg datatypes so your source schema remains more or less unaffected by the type conversion from a source database to Iceberg. See for:

3. Destination-level features

  1. Apache Iceberg – Primary target format, can write to AWS S3, Azure, GCS: see Iceberg Writer docs.

  2. Catalog options:

  3. Plain Parquet – Write partitioned Parquet to S3 or Google Cloud Storage (GCS) with the Parquet Writer.

  4. Query engines – Any Iceberg v2-aware tool (Trino, Spark, Flink, Snowflake, etc.) can query OLake outputs immediately. See here for more

4. Upcoming features

  1. Telemetry hooks – Segment IO & Mixpanel integration (PR #290).
  2. Universal MySQL CDC – MariaDB, Percona & TiDB support (PR #359).
  3. Incremental sync – Rolling out for MongoDB PR #268, then Postgres and MySQL.
  4. Roadmap tracker – See the live OLake roadmap.

Need Assistance?

If you have any questions or uncertainties about setting up OLake, contributing to the project, or troubleshooting any issues, we’re here to help. You can:

  • Email Support: Reach out to our team at hello@olake.io for prompt assistance.
  • Join our Slack Community: where we discuss future roadmaps, discuss bugs, help folks to debug issues they are facing and more.
  • Schedule a Call: If you prefer a one-on-one conversation, schedule a call with our CTO and team.

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!