OLake Fusion
Keep your Iceberg tables fast with less compute
OLake Fusion helps keep your Iceberg tables performant, by automating compaction and reducing manual overhead.
Problem Statement
Small files distracting from the bigger objective?
- Problem ISmall files pile up faster than you notice
- Problem IIQueries get slower every month
- Problem IIIProblems aren’t visible until they’re expensive
- Problem IVCompaction jobs become debugging issues
Features
Built to keep your lakehouse performant
Tiered Compaction
Run compaction on trigger tiers instead of one whole job. Lite every 20 minutes, Medium every 40 minutes, and Full as a periodic deep-clean for much larger datasets like terabyte-scale tables.
Omit Decay
Small files, delete files, and excess metadata pile up as tables evolve. Fusion resolves them periodically so query performance never degrades.
Easy Configuration
One target-size parameter replaces the seven Spark rewrite_data_files needs. Same result, far less to manage.
Self-Hosted
Open-source and deployable on Docker or Kubernetes so you pay only for the compute and storage you provision.
Benchmarks
Engineered for performance
OLake Fusion compacts Apache Iceberg tables 2.06× faster than Apache Spark and at roughly half the cost.
1 Parameter, 1.8 billion rows compacted in 27 minutes.
| Metrics | OLake FusionOpen Source | Spark Compaction |
|---|---|---|
| Total Compaction Time | 27m 2s2.06x faster | 55m 47s |
| Compaction Cost/ Job | $1.0652% less cost | $2.19 |
| Config Parameters | 110x simpler | 10+ |
How OLake Fusion does this
In order to outperform Spark, Fusion runs compaction on a per-table schedule you define. Rather than treating every table the same, it offers three tiers;
- Lite for frequent lightweight cleanup
- Medium for regular small-file and delete resolution
- Full for occasional deep rewrites
Each tier is scheduled independently, with higher tiers automatically overriding lower ones when schedules overlap.
To improve read efficiency, Fusion converts expensive equality deletes into position deletes, reducing the work query engines do at read time.
FAQs
You might have some questions
OLake Fusion is a self-hosted open-source software that keeps your lakehouse tables efficient, compact, and query-ready as your data continuously grows. As Iceberg tables evolve through ingestion, updates, and deletes, they accumulate small files, delete files, and excess metadata, all of which degrade query performance and increase storage overhead over time. Fusion manages that for you automatically.
Six situations call for it: frequent data ingestion or updates, accumulation of small files, presence of delete files, high partition cardinality, degrading query performance, and growing table size over time. If your tables are under continuous CDC pressure, you're in all six.
Yes, that's the intended path. Iceberg Maintenance ships as a maintenance module inside the OLake UI from v0.4.0, so Go handles ingestion and Fusion handles maintenance from the same place. New users can start with a combined Ingestion + Maintenance setup, and existing OLake Go users just upgrade the UI (Docker or Helm/Kubernetes) to unlock the module, no separate tool to adopt. Our own compaction benchmark ran exactly this way: OLake Go ingesting the TPC-H lineitem table while Fusion compacted it.
Yes, Fusion maintains Apache Iceberg tables through your Iceberg catalog, so it operates on the tables themselves rather than on OLake's ingestion pipeline.
On a CDC-like TPC-H workload (300 GB, ~1.8 billion rows in lineitem), Fusion compacted in 27 minutes 2 seconds versus Spark rewrite_data_files at 55 minutes 47 seconds; 2.06× faster on identical infrastructure.
Yes. On the same $2.36/hour infrastructure, the benchmark job cost $1.06 with Fusion and $2.19 with Spark, roughly half, because the job finishes in half the time.
No. In the benchmark, compaction ran for two hours while CDC-style updates (~200,000 rows every 2 minutes) and repeated TPC-H Query 6 executions continued in parallel, specifically to measure how concurrent compaction affects query latency and stability.
Fusion uses tiered triggers rather than one blunt job: Lite every 20 minutes and Medium every 40 minutes in the benchmark, plus Full as a periodic deep-clean for much larger datasets, terabyte-scale tables where long-term small-file buildup is higher. At the benchmark's sub-100 GB destination size, Full wasn't needed.
Fusion is open-source and is deployed on Docker or Kubernetes. You pay only for the compute and storage you provision.
One parameter: target-size (512 MB in the benchmark). For comparison, the Spark rewrite_data_files job in the same test needed seven: strategy, target/max/min file size, concurrent rewrites, partial-progress, and delete-file threshold.