OLake makes data replication faster by parallelizing full loads, leveraging change streams for real-time sync, and pulling data in database native (e.g BSON format) for efficient ingestion.
Quick Results
For Enterprise solutions
Control your cloud
OLakeGet Early Access.
Faster Parallel & Full Load
Full load performance is improved by splitting large collections into smaller virtual chunks, processed in parallel.
CDC Cursor Preservation
When you add new big tables after a long time of setting up the ETL, we do full load for it, in parallel to already running incremental sync. So CDC cursors are never lost. We manage overhead of data ingestion order and deduplication.
Optimized Data Pull
Instead of directly transforming data from Databases during extraction, we first pull it in its native format (BSON, MongoDB's native data format, which stores JSON-like documents more efficiently). Once we have the data, we decode it on the ETL side
Efficient Incremental Sync
Using Databases change stream logs (binglogs for MySQL, oplogs for mongoDB, WAL logs for Postgres), OLake enables parallel updates for each collection. This method facilitates rapid synchronisation and ensures that data is consistently updated with near real-time updates.
