Last updated:|... min read
Roadmap
New Connectors
- Kafka
- AWS S3
Observability & Monitoring Module
- Real-time metrics and pipeline insights
- Improved alerting and diagnostics
Upcoming Features
- Heartbeat mechanism for sync health tracking
- Incremental Sync Mode
- Filtering Data in case of Full Load
- SMTP’s for CDC
- Helm deployments
- Partition evolution
Detailed roadmap can be found on GitHub OLake Roadmap 2024-25
Source Connector Level Functionalities Supported
Additional Planned sources - AWS S3 |
Kafka
Destination Level Functionalities Supported
Features/Functionality | ![]() | ![]() | ![]() | ![]() | ![]() |
---|---|---|---|---|---|
Flattening & Normalization (L1) | ✅ | ✅ | ✅ | ✅ | ✅ |
Partitioning | ✅ | ✅ | ✅ | ✅ | ✅ |
Schema Changes | ✅ | ✅ | WIP | WIP | WIP |
Schema Evolution | ✅ | ✅ | ✅ | ✅ | ✅ |
Catalogue Support
Catalog type | Docs / example | Comments | |
---|---|---|---|
![]() | REST – Lakekeeper | LINK | [Officially Supported] Rust-native catalog with optimistic locking; Helm chart available for K8s. |
![]() | REST - Unity | LINK | [Supported and tested] Unity Catalog (Databricks) with Personal Access Token authentication. |
REST – Gravitino | LINK | [Supported, Yet to be tested] Uses standard Iceberg REST; Gravitino adds multi-cloud routing. | |
![]() | REST – Nessie | LINK | [Supported, Yet to be tested] Time-travel branches/tags; supply nessie.endpoint in destination.json . |
![]() | REST – Polaris | LINK | [Supported, Yet to be tested] |
![]() | Hive Metastore | Hive catalog config | Classic HMS; good fit for on-prem Hadoop or EMR. |
![]() | JDBC Catalog (Postgres/MySQL) | JDBC catalog sample | Stores Iceberg metadata in an RDBMS(Postgres); easiest to spin up locally with Postgres. |
AWS Glue Catalog | Glue catalog IAM & config | Best choice on AWS; lets Athena, EMR, and Redshift query the same tables instantly. | |
![]() | Azure Purview | Not Planned, submit a request | |
![]() | BigLake Metastore | Not Planned, submit a request |
See OLake Roadmap GitHub Discussion for more details on the kind of features we are planning to add in the future and see the current progress of those feature implementation for more details.