# OLake

> Fastest open-source data replication and Apache Iceberg table maintenance platform, moving data from databases and Kafka to data lakehouses.

OLake is a high-performance, open-source data platform with two products. OLake Go is an EL (Extract and Load) engine that replicates data from sources like PostgreSQL, MySQL, MongoDB, Oracle, MSSQL, IBM Db2, Apache Kafka, and S3 into Apache Iceberg tables or Parquet files on object storage, supporting full refresh, incremental sync, and Change Data Capture (CDC) with stateful, resumable syncs. OLake Fusion automates Apache Iceberg table maintenance, including compaction of small files, and benchmarks around 2x faster and roughly 50% cheaper than Spark for Iceberg compaction. OLake Go benchmarks 5-500x faster than traditional replication tools. Both products are open source under the Apache 2.0 license.

OLake is built for data engineers, data architects, and developers building or scaling data infrastructure. Key use cases include setting up CDC pipelines into Apache Iceberg, migrating databases to cloud object storage, keeping Iceberg tables compacted and query-ready, and enabling real-time analytics on fresh data. Its focus on open table formats avoids vendor lock-in: OLake writes standard Iceberg tables compatible with catalogs such as AWS Glue, Hive Metastore, and REST catalogs (Nessie, Polaris, Unity Catalog), and queryable from engines including Trino, Spark, Snowflake, Databricks, and ClickHouse.

## Products & Core Documentation

- [OLake Go Overview](https://olake.io/docs): Getting started with OLake Go, the open-source EL engine for replicating databases and Kafka to Apache Iceberg, with supported sources, destinations, and capabilities.
- [OLake Fusion Overview](https://olake.io/docs/fusion/getting-started/overview/): Getting started with OLake Fusion for automated Apache Iceberg table maintenance and compaction.
- [OLake Quickstart Guide](https://olake.io/docs/getting-started/quickstart): Start using OLake Go with a single Docker command and access the UI.
- [Configure Your First Ingestion Job](https://olake.io/docs/getting-started/creating-first-pipeline): Step-by-step guide to creating a first OLake Go replication job.
- [OLake Playground](https://olake.io/docs/getting-started/playground): Try OLake Go in a sandboxed playground environment.
- [OLake Go vs Fivetran, Debezium, Estuary & Airbyte: PostgreSQL Benchmark](https://olake.io/docs/benchmarks/ingestion/?tab=postgres): Ingestion throughput benchmark comparing OLake Go against Fivetran, Debezium, Estuary, and Airbyte Cloud for PostgreSQL to Apache Iceberg full-load and CDC sync.
- [OLake Go vs Fivetran, Debezium, Estuary & Airbyte: MongoDB Benchmark](https://olake.io/docs/benchmarks/ingestion/?tab=mongodb): Ingestion throughput benchmark comparing OLake Go against Fivetran, Debezium, Estuary, and Airbyte Cloud for MongoDB to Apache Iceberg sync.
- [OLake Go vs Fivetran, Debezium, Estuary & Airbyte: MySQL Benchmark](https://olake.io/docs/benchmarks/ingestion/?tab=mysql): Ingestion throughput benchmark comparing OLake Go against Fivetran, Debezium, Estuary, and Airbyte Cloud for MySQL to Apache Iceberg sync.
- [OLake Go vs Fivetran, Debezium, Estuary & Airbyte: Oracle Benchmark](https://olake.io/docs/benchmarks/ingestion/?tab=oracle): Ingestion throughput benchmark comparing OLake Go against Fivetran, Debezium, Estuary, and Airbyte Cloud for Oracle to Apache Iceberg sync.
- [OLake Go vs Apache Flink: Kafka Benchmark](https://olake.io/docs/benchmarks/ingestion/?tab=kafka): Ingestion throughput benchmark comparing OLake Go against Apache Flink for Kafka to Apache Iceberg sync.
- [OLake Go vs Fivetran, Debezium, Estuary & Airbyte: MSSQL Benchmark](https://olake.io/docs/benchmarks/ingestion/?tab=mssql): Ingestion throughput benchmark comparing OLake Go against Fivetran, Debezium, Estuary, and Airbyte Cloud for MSSQL to Apache Iceberg sync.

## Install & Configure

- [Install with Docker Compose (UI)](https://olake.io/docs/install/olake-ui): Deploy OLake Go with its web UI using Docker Compose.
- [Install with Docker (CLI)](https://olake.io/docs/install/docker-cli): Run OLake Go from the command line using Docker.
- [Install on Kubernetes with Helm](https://olake.io/docs/install/kubernetes): Deploy OLake Go on Kubernetes using Helm charts.

## Sources, Destinations & Core Concepts

- [PostgreSQL Source Connector](https://olake.io/docs/connectors/postgres): Configuration guide for the OLake PostgreSQL source connector, covering full refresh, incremental, and CDC sync modes.
- [MongoDB Source Connector](https://olake.io/docs/connectors/mongodb/): Configuration guide for the OLake MongoDB source connector, covering full refresh, incremental, and CDC sync modes.
- [MySQL Source Connector](https://olake.io/docs/connectors/mysql/): Configuration guide for the OLake MySQL (and MariaDB) source connector, covering full refresh and CDC sync modes.
- [Oracle Source Connector](https://olake.io/docs/connectors/oracle/): Configuration guide for the OLake Oracle source connector, covering full refresh and incremental sync modes.
- [Kafka Source Connector](https://olake.io/docs/connectors/kafka/): Configuration guide for the OLake Kafka source connector, covering real-time message sync from topics.
- [Db2 Source Connector](https://olake.io/docs/connectors/db2/): Configuration guide for the OLake IBM Db2 LUW source connector, covering full refresh and incremental sync modes.
- [S3 Source Connector](https://olake.io/docs/connectors/s3/): Configuration guide for the OLake S3 source connector, covering CSV, JSON, and Parquet ingestion from AWS S3, MinIO, and LocalStack.
- [MSSQL Source Connector](https://olake.io/docs/connectors/mssql/): Configuration guide for the OLake MSSQL source connector, covering full refresh, incremental, and CDC sync modes.
- [Iceberg Destination and Catalogs (AWS Glue and more)](https://olake.io/docs/writers/iceberg/catalog/glue): Configuring Apache Iceberg destinations across supported catalog integrations.
- [Iceberg Partitioning in OLake](https://olake.io/docs/writers/iceberg/partitioning): How OLake handles metadata-driven Iceberg partitioning.
- [OLake Architecture](https://olake.io/docs/core/architecture): Core architecture of the OLake platform.
- [Compatibility with Iceberg Catalogs](https://olake.io/docs/understanding/compatibility-catalogs): Supported Iceberg catalog integrations including Glue, Hive Metastore, REST, and JDBC catalogs.
- [Compatibility with Query Engines](https://olake.io/docs/understanding/compatibility-engines): Query engines that can read OLake-written Iceberg tables.
- [Schema Evolution in OLake](https://olake.io/docs/features/schema): How OLake handles schema changes and data type changes without breaking downstream jobs.
- [OLake Use Cases](https://olake.io/docs/core/use-cases): Common use cases for OLake across replication and lakehouse workloads.
- [OLake UI API](https://olake.io/docs/api/olake-ui-api): API documentation for the OLake UI.
- [Release Notes](https://olake.io/docs/release/ingestion/overview): Version history and release notes for OLake ingestion.

## Key Features & Concepts

- [OLake Managed Offering Interest Form](https://olake.io/contact): Form for teams interested in OLake's managed offering.
- [Apache Iceberg Hub](https://olake.io/iceberg): Curated Apache Iceberg blogs and resources.
- [Iceberg Query Engine](https://olake.io/iceberg/query-engine): Information on query engines for data stored in Apache Iceberg format.

## Blog Articles

- [Debezium Kafka Challenges & How OLake Solves Them](https://olake.io/blog/issues-debezium-kafka): Discusses common issues when using Debezium with Kafka for CDC.
- [OLake Architecture](https://olake.io/blog/olake-architecture): Deep dive into the architecture and design of OLake.
- [OLake Architecture Deep Dive](https://olake.io/blog/olake-architecture-deep-dive): In-depth technical explanation of OLake's internal architecture and data replication.
- [Apache Iceberg Features and Benefits](https://olake.io/blog/apache-iceberg-features-benefits/): Explains the key features and advantages of using Apache Iceberg.
- [Apache Iceberg vs Hive Comparison](https://olake.io/blog/apache-iceberg-hive-comparison/): Compares Apache Iceberg with Hive for data lake table formats.
- [Iceberg Lakehouse Observability and Metadata Monitoring](https://olake.io/blog/apache-iceberg-lakehouse-observability-metadata-monitoring/): Monitoring and observability in Iceberg lakehouses using metadata.
- [Apache Iceberg Metadata Explained](https://olake.io/blog/apache-iceberg-metadata-explained/): Detailed explanation of snapshots and manifests in Apache Iceberg tables.
- [Apache Iceberg Row Lineage](https://olake.io/blog/apache-iceberg-row-lineage/): Understanding row-level lineage capabilities in Apache Iceberg v3.
- [Iceberg Table Maintenance with OLake Fusion](https://olake.io/blog/apache-iceberg-table-maintenance-olake-fusion/): How OLake Fusion helps maintain Apache Iceberg tables efficiently.
- [Apache Iceberg vs Delta Lake Guide](https://olake.io/blog/apache-iceberg-vs-delta-lake-guide/): A comparative guide to Apache Iceberg and Delta Lake table formats.
- [Apache Polaris Lakehouse](https://olake.io/blog/apache-polaris-lakehouse/): Building a scalable lakehouse with Iceberg, Trino, OLake, and Apache Polaris.
- [Architect's Guide: CDC to Apache Iceberg](https://olake.io/blog/architect-guide-cdc-apache-iceberg/): Architectural guidance for implementing CDC pipelines into Apache Iceberg.
- [Binlogs Explained](https://olake.io/blog/binlogs/): Explanation of MySQL binary logs (binlogs) and their use in replication and data recovery.
- [Build Data Lakehouse with Iceberg, ClickHouse, and OLake](https://olake.io/blog/build-data-lakehouse-iceberg-clickhouse-olake/): Guide on building a data lakehouse using Iceberg, ClickHouse, and OLake.
- [Modern Data Lakehouse with OLake, Iceberg, Lakekeeper, Trino](https://olake.io/blog/building-modern-data-lakehouse-with-olake-iceberg-lakekeeper-trino/): Building a modern lakehouse stack with OLake, Iceberg, Lakekeeper, and Trino.
- [Open Data Lakehouse with OLake and Presto](https://olake.io/blog/building-open-data-lakehouse-with-olake-presto/): Steps to build an open data lakehouse using OLake, PrestoDB, and MinIO.
- [Conflict-Free CDC into Apache Iceberg](https://olake.io/blog/conflict-free-cdc-into-apache-iceberg/): Techniques for achieving conflict-free CDC pipelines into Iceberg.
- [Create OLake Job with Docker CLI](https://olake.io/blog/creating-job-olake-docker-cli/): How to create and manage OLake replication jobs (Postgres to Iceberg) using Docker and the CLI.
- [Data Lake vs Data Lakehouse: Modern Stack](https://olake.io/blog/data-lake-vs-data-lakehouse-modern-stack/): Comparing data lakes and data lakehouses in modern data architectures.
- [Data Lake vs Delta Lake](https://olake.io/blog/data-lake-vs-delta-lake/): Understanding the differences between data lakes and Delta Lake.
- [Data Warehouse vs Lakehouse](https://olake.io/blog/data-warehouse-vs-lakehouse/): Comparing traditional data warehouses with modern data lakehouses.
- [Debezium vs OLake](https://olake.io/blog/debezium-vs-olake/): A comparison of Debezium + Kafka and OLake for CDC performance and ease of use.
- [Deploy OLake on Kubernetes with Helm](https://olake.io/blog/deploying-olake-on-kubernetes-helm/): Guide for deploying OLake on Kubernetes using Helm charts.
- [How OLake Guarantees Exactly-Once Delivery to Apache Iceberg](https://olake.io/blog/exactly-once-delivery-iceberg/): How OLake atomically commits data files and progress markers, then reconciles against a checkpoint on restart to avoid duplicating or losing records.
- [Flatten Array Data](https://olake.io/blog/flatten-array/): How to flatten object types and query arrays in semi-structured data.
- [7x Faster Iceberg Writes: How We Rebuilt OLake's Destination Pipeline](https://olake.io/blog/how-olake-becomes-7x-faster/): Explains the technical reasons behind OLake's high-speed Iceberg writes.
- [Set Up MongoDB to Apache Iceberg](https://olake.io/blog/how-to-set-up-mongodb-apache-iceberg/): Step-by-step guide for replicating MongoDB data to Apache Iceberg.
- [Set Up Postgres to Apache Iceberg](https://olake.io/blog/how-to-set-up-postgres-apache-iceberg/): Guide for replicating PostgreSQL data into Apache Iceberg tables.
- [Set Up PostgreSQL CDC on AWS RDS (OLake Guide)](https://olake.io/blog/how-to-set-up-postgresql-cdc-on-aws-rds/): Step-by-step OLake guide for PostgreSQL CDC setup on AWS RDS.
- [Set Up PostgreSQL CDC on AWS RDS](https://olake.io/blog/how-to-set-up-postgresql-cdc-on-aws-rds-2025/): Guide for configuring PostgreSQL Change Data Capture on AWS RDS.
- [Sync IBM Db2 LUW to Lakehouse with OLake](https://olake.io/blog/ibm-db2-luw-to-lakehouse-sync-apache-iceberg-olake/): Replicating IBM Db2 LUW data to a lakehouse using OLake and Iceberg.
- [Iceberg Compaction: Spark vs Fusion Benchmark](https://olake.io/blog/iceberg-compaction-spark-vs-fusion-benchmark/): Benchmark showing OLake Fusion running Iceberg compaction around 2x faster and roughly 50% cheaper than Spark.
- [Iceberg Compaction TPC-H Benchmark](https://olake.io/blog/iceberg-compaction-tpch-benchmark/): TPC-H benchmark results showing how much faster queries run after Iceberg compaction.
- [Iceberg vs Delta Lake Delete Methods](https://olake.io/blog/iceberg-delta-lake-delete-methods-comparison/): Comparison of delete operation methods in Apache Iceberg and Delta Lake.
- [Iceberg Variant and Geospatial Types](https://olake.io/blog/iceberg-variant-geospatial-types/): Variant and geospatial data types in Apache Iceberg v3.
- [Iceberg vs Parquet: Table vs File Format](https://olake.io/blog/iceberg-vs-parquet-table-format-vs-file-format/): Clarifies the difference between Iceberg (table format) and Parquet (file format).
- [Ingest S3 Files with OLake as Streams](https://olake.io/blog/ingesting-files-from-s3-with-olake-turn-buckets-into-reliable-streams/): Using OLake to ingest files from S3 (AWS, MinIO, LocalStack) and treat buckets as reliable streams.
- [JSON vs BSON vs JSONB Comparison](https://olake.io/blog/json-vs-bson-vs-jsonb/): Compares different JSON-like data formats: JSON, BSON, and JSONB.
- [MongoDB CDC with Debezium and Kafka](https://olake.io/blog/mongodb-cdc-using-debezium-and-kafka/): Guide on implementing MongoDB CDC using Debezium and Kafka.
- [MongoDB Synchronization Strategies](https://olake.io/blog/mongodb-synchronization-strategies/): Exploring different strategies for synchronizing MongoDB data.
- [MySQL to Apache Iceberg Replication](https://olake.io/blog/mysql-apache-iceberg-replication/): Steps for replicating MySQL data into Apache Iceberg tables.
- [OLake with Airflow on EC2](https://olake.io/blog/olake-airflow-on-ec2/): Guide for running OLake syncs with Apache Airflow on AWS EC2 instances.
- [OLake and Airflow on Kubernetes](https://olake.io/blog/olake-airflow/): How to run OLake syncs on Kubernetes using Apache Airflow.
- [Compact Iceberg Tables with Apache Amoro](https://olake.io/blog/olake-amoro-iceberg-lakehouse/): Handling small files and automating Iceberg compaction with Apache Amoro.
- [OLake Arrow-Based Iceberg Ingestion](https://olake.io/blog/olake-arrow-based-iceberg-ingestion/): How OLake's Arrow-based architecture delivers faster Iceberg ingestion.
- [OLake and Bauplan for a Serverless Iceberg Lakehouse](https://olake.io/blog/olake-bauplan-iceberg-lakehouse/): Building a serverless Iceberg lakehouse combining OLake's speed with Bauplan's Git workflows.
- [OLake Ingestion Filters Explained](https://olake.io/blog/olake-ingestion-filters-explained/): How to use and configure SQL-style ingestion filters in OLake.
- [OLake Kafka to Iceberg](https://olake.io/blog/olake-kafka-iceberg/): Deep dive into Kafka as a source in OLake, covering sync, concurrency, and partition handling.
- [OLake MOR/CoW with Databricks](https://olake.io/blog/olake-mor-cow-databricks/): Making OLake's Merge-on-Read Iceberg tables compatible with Databricks' query engine.
- [AWS DMS vs OLake](https://olake.io/blog/olake-vs-aws-dms/): Choosing the right tool for your Iceberg pipeline: AWS Database Migration Service (DMS) compared with OLake.
- [Postgres, Iceberg, Doris Lakehouse with OLake](https://olake.io/blog/postgres-iceberg-doris-lakehouse-olake/): Building a lakehouse with Postgres, Iceberg, and Apache Doris using OLake.
- [How OLake Handles Schema Evolution Without Breaking Your Pipeline](https://olake.io/blog/schema-evolution-without-breaking-pipelines/): How OLake uses Apache Iceberg's field ID-based column tracking to safely handle added, dropped, renamed, and widened columns without data loss or pipeline failures.
- [Sync MSSQL to Lakehouse with OLake](https://olake.io/blog/sync-mssql-to-your-lakehouse-with-olake/): Guide for synchronizing Microsoft SQL Server data to a lakehouse.
- [What Makes OLake Fast?](https://olake.io/blog/what-makes-olake-fast/): Explains the core optimizations behind OLake's high-throughput data replication.

## Customer Stories

- [OLake Customer Stories](https://olake.io/customer-stories): Case studies from teams using OLake to sync data and build data lakehouses.
- [Xeno: Zero Pipeline Failures, 50% Faster Loads](https://olake.io/customer-stories/xeno-aws-dms-alternative-mysql-cdc): How Xeno, an AI-powered customer engagement platform for retailers, replaced AWS DMS with OLake for MySQL CDC on Kubernetes, cutting full-load time nearly 50% and eliminating schema-change pipeline breaks.
- [LendingKart: 100x Compressed Lake Ingestion Volume](https://olake.io/customer-stories/lendingkart-Improved-Data-Correctness): How LendingKart reduced daily MongoDB data movement from gigabytes to megabytes and completed an 11-year historical backfill their Debezium + Spark setup could not deliver reliably.
- [PhysicsWallah: MongoDB CDC into an Iceberg Lakehouse](https://olake.io/customer-stories/physicswallah-mongodb-cdc-iceberg): How PhysicsWallah's data engineering team evaluated OLake for large-scale MongoDB CDC ingestion into their lakehouse platform.
- [Bitespeed: From 40-Minute to Sub-Minute Segmentation Queries](https://olake.io/customer-stories/bitespeed-segmentation-queries): How Bitespeed rebuilt its customer segmentation engine on OLake and Apache Iceberg while staying within budget.
- [Cordial: Multi-Cluster MongoDB Ingestion for an AI-Ready Lakehouse](https://olake.io/customer-stories/cordial-real-time-data-sync): How Cordial is unifying thousands of MongoDB collections into a single Apache Iceberg lakehouse to power AI agents.
- [Astrotalk: Replacing Google Datastream for Databricks Migration](https://olake.io/customer-stories/astro-talk-lakehouse-transformation): How Astrotalk used OLake as its primary CDC engine for large-scale PostgreSQL and MySQL replication to S3 during its move to Databricks.
- [Customer Stories Archive](https://olake.io/customer-stories/archive/): Archive of all customer stories.

## Community & Engagement

- [OLake Webinars & Events](https://olake.io/webinar): Information and registration for upcoming OLake webinars and events.
- [OLake Community](https://olake.io/community): Overview of OLake community resources and forums.
- [OLake Contributors](https://olake.io/community/contributors): Top contributors to the OLake project.
- [OLake Contributor Program](https://olake.io/community/contributor-program): Details about the program for OLake project contributors.
- [OLake Google Summer of Code](https://olake.io/community/gsoc): OLake's participation in Google Summer of Code.
- [Community Ideas](https://olake.io/community/ideas): Platform for community members to share and discuss ideas.
- [Community Proposal Guidelines](https://olake.io/community/proposal-guidelines): Guidelines for submitting proposals to the OLake community.
- [Community Proposal Template](https://olake.io/community/proposal-template): Template for submitting project proposals.
- [Contribute to OLake](https://olake.io/docs/community/contributing): Contribution guide for OLake Go, including development environment setup.
- [3rd Community Meetup](https://olake.io/community/3rd-community-meetup/): Recap covering the Apache Iceberg Writer and MongoDB connector, community contributions, and upcoming UI/API work.
- [4th Community Meetup](https://olake.io/community/4th-community-meetup/): Recap covering the improved Parquet writer, performance stats file, CDC strategies, Iceberg catalog support, and schema evolution.
- [5th Community Meetup](https://olake.io/community/5th-community-meetup/): Recap covering Apache Iceberg as a destination for S3 and local setups, a live Postgres-to-Iceberg sync demo, and upcoming MySQL/Postgres features.
- [6th Community Meetup](https://olake.io/community/6th-community-meetup/): Recap featuring PhysicsWallah's production migration from Redshift to an Iceberg lakehouse using OLake, plus OLake's Golang + Java architecture and upcoming transformation and UI features.
- [8th Community Meetup](https://olake.io/community/8th-community-meetup/): Recap covering Oracle CDC, filtering capabilities, incremental sync, and Helm deployment on Kubernetes.
- [9th Community Meetup](https://olake.io/community/9th-community-meetup/): Recap covering Kafka-powered CDC pipelines, sync management features, destination handling, and IAM-based security for MongoDB.
- [10th Community Meetup](https://olake.io/community/10th-community-meetup/): Recap covering the new S3, MSSQL, and DB2 source connectors, the S3 connector architecture, Merge-on-Read to Copy-on-Write conversion, and Social Winter of Code contributor spotlights.
- [OLake Slack Community](https://olake.io/slack): Join the OLake community on Slack for real-time discussions.
- [OLake on GitHub](https://github.com/datazip-inc/olake): OLake's open-source repository under the Apache 2.0 license.

## Blog Archives, Authors & Tags

- [OLake Blog](https://olake.io/blog): Main blog page with articles on data engineering, Apache Iceberg, and lakehouses.
- [Blog Archive](https://olake.io/blog/archive/): Archive of all blog posts, organized by date.
- [Blog Authors](https://olake.io/blog/authors/): List of authors who contribute to the OLake blog.
- [Blog Tags](https://olake.io/blog/tags/): Index of all tags used in OLake blog posts, including Apache Iceberg, CDC, compaction, lakehouse, benchmarks, and connectors.

## Company & Legal

- [OLake Homepage](https://olake.io/): Homepage for OLake, the fastest open-source database and Kafka replication tool for S3 and Apache Iceberg.
- [About OLake](https://olake.io/about-us): Information about the OLake team and mission.
- [Contact OLake](https://olake.io/contact): Contact information for OLake support, partnerships, and inquiries.
- [OLake Branding Guidelines](https://olake.io/branding): Guidelines for using OLake's brand assets.
- [OLake Terms of Use](https://olake.io/terms-of-use): Legal terms and conditions for using OLake services.
- [OLake Privacy Policy](https://olake.io/privacy-policy): Information on how OLake handles user data and privacy.