Oracle to Databricks: Incremental Sync via Iceberg
Oracle to Databricks usually means GoldenGate, a Kafka pipeline, or a nightly export picked up by Auto Loader. This guide takes a different route.
OLake Go reads the Oracle tables over a plain Java Database Connectivity (JDBC) connection and writes them once as Apache Iceberg tables into Unity Catalog. A cursor column keeps them fresh.
The tables land as Unity Catalog managed Iceberg tables, with the files in your own cloud storage. Nothing else is in the path: no GoldenGate, no Kafka, no Databricks ingestion pipeline. DuckDB and Spark read the same tables through the same catalog.
Key takeaways
- OLake Go replicates Oracle into Databricks as Iceberg tables: a full historical snapshot first, then incremental syncs on a cursor column such as
UPDATED_AT, with no GoldenGate or Kafka in between. - The tables are Unity Catalog managed Iceberg tables. You query them from the Databricks SQL editor or a notebook right away, with no external table and no refresh.
- The same tables, written once by OLake Go, are also queryable from DuckDB and Spark through Unity Catalog. In the run on this page all three engines returned identical rows.
The short answer. To replicate Oracle to Databricks without redo-log access, run OLake Go with a read-only Oracle user as the source and Unity Catalog as the Iceberg destination, with Databricks as a target query engine. The first sync copies each table in full (the historical snapshot).
Every later sync fetches the rows whose cursor column moved and upserts them with deletion vectors. Databricks, DuckDB and Spark then query the same Iceberg tables, with no second copy.
How Oracle data reaches Databricks here
The first sync reads each table in parallel chunks, writes Parquet data files, and records the highest cursor value it saw: that is the historical load. Every later sync selects rows above that value and writes them as new data files. A deletion vector, a small bitmap per data file, retires the old version of each updated row.
Each sync ends with one commit to Unity Catalog. Databricks sees the new snapshot on its next query, and DuckDB and Spark see it on their next catalog load.
The cursor is the whole contract. A row that changes without its updated_at moving is never picked up, and a deleted row stays in the Iceberg table. Set the column in every write path, and soft-delete if deletes matter.
What you need
- Oracle Database 11g or later reachable over JDBC, with a read-only user and a cursor column on each table. The tables on this page ran on Oracle Database 23ai Free in Docker.
- A Databricks workspace with Unity Catalog, a catalog with managed storage, and an admin who can turn on external data access and grant
EXTERNAL USE SCHEMA. The run on this page used a workspace on Google Cloud. - OLake UI from the Docker Compose install. The run on this page used OLake Go v1.1.0.
- Docker, and about 30 minutes.
Step 1: Prepare Oracle for incremental sync
Most readers point OLake Go at tables they already have. If you want to follow along with the exact data from this page, create these two tables first:
The two sample tables used on this page
CREATE TABLE shop.customers (
id NUMBER(10) GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
name VARCHAR2(100), email VARCHAR2(200), city VARCHAR2(100), tier VARCHAR2(20),
created_at TIMESTAMP DEFAULT SYSTIMESTAMP,
updated_at TIMESTAMP DEFAULT SYSTIMESTAMP);
CREATE TABLE shop.orders (
id NUMBER(10) GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
customer_id NUMBER(10) REFERENCES shop.customers(id),
product VARCHAR2(100), amount NUMBER(10,2), status VARCHAR2(20),
created_at TIMESTAMP DEFAULT SYSTIMESTAMP,
updated_at TIMESTAMP DEFAULT SYSTIMESTAMP);
-- Then insert whatever you like. This page started with 8 customers and 40 orders.
OLake Go reads Oracle tables over a plain Java Database Connectivity (JDBC) connection: a parallel full snapshot first, then incremental syncs that fetch only rows whose cursor column moved past the last value seen. There is no log-based change data capture (CDC) for Oracle in OLake Go, so three things must be in place before the first sync.
| Setting | Value | Why it matters |
|---|---|---|
| A read-only user | CREATE SESSION plus SELECT on each table | The snapshot and every incremental sync read the tables directly. |
| A cursor column | a column that grows with every change, such as updated_at | Incremental syncs select WHERE cursor > last value. Rows updated without touching the cursor are not picked up. |
| A primary key | on every table | OLake Go uses it to upsert changed rows into the Iceberg table. Without one, changed rows are appended. |
-- 1. A user for OLake Go
CREATE USER olake IDENTIFIED BY "<password>";
GRANT CREATE SESSION TO olake;
-- 2. Read access to the tables you sync
GRANT SELECT ON shop.customers TO olake;
GRANT SELECT ON shop.orders TO olake;
GRANT SELECT ANY DICTIONARY TO olake; -- lets discovery read the catalog views
-- 3. A cursor column that your application always sets, for example
-- updated_at TIMESTAMP DEFAULT SYSTIMESTAMP, updated by a trigger or by the application on every change
Check the result before moving on:
SELECT username FROM all_users WHERE username = 'OLAKE';
SELECT table_name FROM all_tables WHERE owner = 'SHOP';
The full list of privileges, with the reason for each, is on the generic Oracle setup guide.
A deleted row has no cursor value to read, so it stays in the Iceberg table. If deletes matter, soft-delete in Oracle (a deleted_at column that also bumps updated_at) or schedule an occasional full refresh.
Step 2: Set up Unity Catalog in Databricks
Databricks reads Iceberg tables through Unity Catalog, the catalog built into every Databricks workspace. OLake Go writes straight into it, so the tables become Unity Catalog managed Iceberg tables. They show up in Catalog Explorer and the SQL editor the moment a sync commits.
The files sit in the catalog's managed storage, which is backed by an external location in your own cloud account. DuckDB and Spark reach the same tables through the same Unity Catalog endpoint.
| Catalog | OLake Go writes | Databricks reads | Verified on this page |
|---|---|---|---|
| Unity Catalog | Yes | Yes: native: Unity Catalog managed Iceberg tables | Yes, end to end |
- Unity Catalog
Unity Catalog is the catalog built into every Databricks workspace. OLake Go writes to it through Unity Catalog's Iceberg REST (representational state transfer) endpoint at https://<workspace-url>/api/2.1/unity-catalog/iceberg-rest, and the tables it creates are Unity Catalog managed Iceberg tables. Databricks allows writes through this endpoint for managed Iceberg tables only (Databricks: access tables from Iceberg clients, checked 2026-10-30).
Four things must be in place in the workspace, once:
| Setting | Where | Why it matters |
|---|---|---|
| External data access on the metastore | Catalog, the gear icon, Metastore, then External data access | Without it the Iceberg REST endpoint refuses every client that is not Databricks itself. |
| A catalog with managed storage | Catalog, Create catalog, with a storage location backed by an external location | OLake Go creates one schema per job inside it. The catalog name is the warehouse value every client uses. |
EXTERNAL USE SCHEMA for the writing principal | On the catalog, so it covers the schemas OLake Go creates | Unity Catalog checks this privilege, on top of the usual ones, before it serves a table to an outside engine. |
| A personal access token or a service principal | Settings, Developer, Access tokens | OLake Go sends it as a bearer token on every catalog call. |
-- The catalog OLake Go writes into. Its managed storage holds the Iceberg files.
CREATE CATALOG IF NOT EXISTS lakehouse MANAGED LOCATION '<external location URL>/lakehouse';
-- What the writing principal needs: browse and create schemas, then read and write tables in them
GRANT USE CATALOG, CREATE SCHEMA ON CATALOG lakehouse TO `olake@example.com`;
GRANT USE SCHEMA, CREATE TABLE, SELECT, MODIFY ON CATALOG lakehouse TO `olake@example.com`;
GRANT EXTERNAL USE SCHEMA ON CATALOG lakehouse TO `olake@example.com`;
Granting on the catalog makes the privileges inherit to every schema OLake Go creates later. EXTERNAL USE SCHEMA is not part of ALL PRIVILEGES, and only the catalog owner can grant it (Databricks: Unity Catalog privileges, checked 2026-10-30). A service principal with OAuth machine-to-machine credentials is the better choice in production. The other authentication methods are on the Unity Catalog destination page.
Unity Catalog accepts updates to a managed Iceberg table only as copy-on-write files or as Iceberg v3 deletion vectors. It rejects equality delete files with Can not create a Path from a null string [ErrorCode: 2015]. Set Databricks as a target query engine on the job, and OLake Go writes deletion vectors from the first sync.
Step 3: Connect Oracle to OLake Go
- OLake UI
- OLake CLI
- In the OLake UI, open Sources and click Create Source. Pick Oracle as the connector, give the source a name and keep the OLake Go version on the latest release.
- Fill in the host and port, choose Service Name (or SID) as the connection type and enter it, then the username and password from step 1. Type the username the way Oracle stores it, which is upper case unless you created it in double quotes: OLake Go quotes the name when it connects, so
olakeandOLAKEare different users. - Click Set up source. OLake Go tests the connection and saves the source.
Here is the source form from the run on this page. Oracle ran in another container, so the host is host.docker.internal. The connection type is Service Name (FREEPDB1) rather than a system identifier (SID), and the user is the one from step 1.
The full field reference, including SSL and SSH tunnels, is on the Oracle connector page.
Create source.json in your config folder with either service_name or sid:
{
"host": "oracle-host",
"username": "oracle-user",
"password": "oracle-password",
"service_name": "oracle-service-name",
"sid": "oracle-sid",
"port": 1521,
"max_threads": 10,
"backoff_retry_count": 3,
"jdbc_url_params": {},
"ssl": {
"mode": "disable"
}
}
Then confirm OLake Go can reach the database:
docker run --pull=always \
-v "$PWD/config:/mnt/config" \
olakego/source-oracle:latest \
check --config /mnt/config/source.json
A working connection returns "status": "SUCCEEDED". Every field is described on the Oracle connector page.
Step 4: Configure the Iceberg destination
- Unity Catalog
- OLake UI
- OLake CLI
- Open Destinations and click Create Destination.
- Pick Apache Iceberg as the connector and Unity as the catalog type.
- Choose Personal Access Token as the authentication type and fill in the fields below. The catalog name from the setup goes into the catalog field.
- Click Create. OLake Go calls the Unity Catalog endpoint with the token and saves the destination.

The run on this page wrote into a catalog called datazip, with a personal access token that expires after seven days. The snippets below use lakehouse; replace it with your catalog name.
Unity Configuration Parameters
| Parameter | Sample Value | Description |
|---|---|---|
REST Catalog URLrequired | https://1234567890123456.1.gcp.databricks.com/ | Databricks workspace URL with Unity Catalog REST API endpoint. Use your actual workspace URL. |
Authentication Typerequired | Personal Access Token | Authentication method OLake Go uses for Unity Catalog REST API requests. Set to match your workspace's authentication configuration. |
Unity Catalog Namerequired | workspace | Name of the catalog in Unity. |
Tokenrequired | dapi123456789... | Databricks Personal Access Token |
| Catalog Name | olake_iceberg | Name of the Iceberg catalog OLake Go registers tables under. Defaults to olake_iceberg if left empty. |
| Enable Arrow Writes | false | Writes data and delete files using Apache Arrow based writer and registers them in Iceberg. |
Create destination.json next to source.json:
{
"type": "ICEBERG",
"writer": {
"catalog_type": "unity",
"catalog_name": "olake_iceberg",
"rest_catalog_url": "https://<workspace-url>/api/2.1/unity-catalog/iceberg-rest",
"iceberg_s3_path": "lakehouse",
"rest_auth_type": "personal access token",
"token": "<personal access token>",
"arrow_writes": false
}
}
iceberg_s3_path holds the Unity Catalog catalog name, not a bucket path: Unity Catalog decides where the files go.
Unity Configuration Parameters
| Parameter | Sample Value | Description |
|---|---|---|
catalog_typerequired | unity | Defines the catalog type used by the writer. |
rest_catalog_urlrequired | https://1234567890123456.1.gcp.databricks.com/ | Databricks workspace URL with Unity Catalog REST API endpoint. Use your actual workspace URL. |
iceberg_s3_pathrequired | workspace | Name of the catalog in Unity. |
rest_auth_typerequired | Personal Access Token | Authentication method OLake Go uses for Unity Catalog REST API requests. Set to match your workspace's authentication configuration. |
tokenrequired | dapi123456789... | Databricks Personal Access Token |
| catalog_name | olake_iceberg | Name of the Iceberg catalog OLake Go registers tables under. Defaults to olake_iceberg if left empty. |
| arrow_writes | false | Writes data and delete files using Apache Arrow based writer and registers them in Iceberg. |
Step 5: Create the job and run the first sync
- OLake UI
- OLake CLI
- Open Jobs and click Create Job. Give it a name and a schedule. Every Day at a fixed hour, or a cron expression, both work; the schedule is how often the change stream is drained, not how often a full copy runs.
- Select the source and the destination you created.
- Under Advanced Settings, set Target Query Engines to Databricks, plus DuckDB and Apache Spark if they will read the tables too. OLake Go then resolves an Iceberg delete format every one of those engines can read, so you never pick one by hand.
- Click Next. OLake Go discovers the tables and opens the Streams page.
- Tick the tables to replicate. For each one, set the sync mode to Full Refresh + Incremental with
UPDATED_ATas the cursor column and the ingestion mode to Upsert. Leave the Upsert Type as OLake Go resolved it (Deletion Vector in the run on this page). - Click Create Job, then open the job's menu and choose Sync Now for the first run.
Three target query engines were set on this job, and Databricks narrows the delete formats to one: deletion vectors on Iceberg v3 tables. Because the Oracle connector has no change data capture (CDC) mode, each stream runs as Full Refresh + Incremental with UPDATED_AT as the cursor.

Discover the tables, choose the streams, then sync:
docker run --pull=always \
-v "$PWD/config:/mnt/config" \
olakego/source-oracle:latest \
discover --config /mnt/config/source.json
discover writes two files: available_streams.json (every table it found, do not edit) and selected_streams.json (the tables you want and their settings). In selected_streams.json, set sync_mode to {props.cliSyncMode ?? 'cdc'} for each table and pick the delete format your readers can open:
{
"selected_streams": {
"SHOP": [
{
"stream_name": "CUSTOMERS",
"sync_mode": "incremental",
"cursor_field": "UPDATED_AT",
"normalization": true,
"append_mode": false,
"update_type": "dv"
}
]
}
}
update_type accepts eq (equality deletes, the default), pos (positional deletes) and dv (deletion vectors). Positional deletes and deletion vectors keep a small index between runs, so give the container a persistent directory for it through OLAKE_INDEX_DB_DIR. The first sync runs without a state file; OLake Go writes one at the end of the run:
docker run --pull=always \
-v "$PWD/config:/mnt/config" \
-e OLAKE_INDEX_DB_DIR=/mnt/config/index-db \
olakego/source-oracle:latest \
sync \
--config /mnt/config/source.json \
--selected-streams /mnt/config/selected_streams.json \
--available-streams /mnt/config/available_streams.json \
--destination /mnt/config/destination.json
Every later run adds --state /mnt/config/state.json to the same command, which is how OLake Go knows to replay only the changes since the last run. The full selected_streams.json reference is on the Docker CLI page; the index directory is described under index DB setup.
On this page the first run copied 8 customers and 40 orders (48 records, 2.24 KB) and finished in 22 seconds, container start included. OLake Go names the Unity Catalog schema after the job and the source schema, lower-cased, so oracle_to_databricks on schema SHOP produced oracle_to_databricks_shop. The Iceberg DB field at the top of the Streams page changes it before the first sync.
Step 6: Query the tables in Databricks
- Unity Catalog
There is no registration step. The schema and tables OLake Go committed through Unity Catalog are managed Iceberg tables in the catalog you configured, and the Databricks SQL editor or a notebook reads them by their three-part name.
-- What OLake Go created (schema = <job name>_<source database>[_<source schema>])
SHOW TABLES IN lakehouse.oracle_to_databricks_shop;
SELECT id, name, city, tier
FROM lakehouse.oracle_to_databricks_shop.customers
ORDER BY id;
SELECT status, count(*) AS n, round(sum(amount), 2) AS total
FROM lakehouse.oracle_to_databricks_shop.orders
GROUP BY status ORDER BY status;
Any SQL warehouse or cluster on a recent Databricks Runtime reads them; a 2X-Small serverless warehouse is enough for checks. DESCRIBE EXTENDED on a table shows the Iceberg provider and the deletion vectors in its properties.
OLake Go adds four columns to every table: _olake_id (a hash of the primary key), _op_type, _olake_timestamp and _cdc_timestamp. _op_type is r for a row from the first snapshot, c for an insert, u for an update and d for a delete. Filter them out of dashboards, or use them to see when a row last changed.
Step 7: Change rows in Oracle and sync again
To see the incremental path work, the run on this page made 12 changes in Oracle after the first sync. Every update also sets updated_at, which is what the cursor sees:
INSERT INTO customers (name, email, city, tier) VALUES ('Maya Patel', 'maya@example.com', 'Pune', 'silver');
INSERT INTO customers (name, email, city, tier) VALUES ('Lucas Silva', 'lucas@example.com', 'São Paulo', 'bronze');
INSERT INTO orders (customer_id, product, amount, status) VALUES (9, 'Laptop', 1299.00, 'placed');
INSERT INTO orders (customer_id, product, amount, status) VALUES (9, 'Mouse', 25.50, 'placed');
INSERT INTO orders (customer_id, product, amount, status) VALUES (10, 'Monitor', 349.99, 'placed');
INSERT INTO orders (customer_id, product, amount, status) VALUES (3, 'Keyboard', 89.00, 'placed');
INSERT INTO orders (customer_id, product, amount, status) VALUES (7, 'Webcam', 79.90, 'placed');
UPDATE customers SET tier = 'gold', updated_at = SYSTIMESTAMP WHERE name = 'Liam Carter';
UPDATE customers SET city = 'Abu Dhabi', updated_at = SYSTIMESTAMP WHERE name = 'Zara Ali';
UPDATE orders SET status = 'delivered', updated_at = SYSTIMESTAMP WHERE id IN (1, 4, 7);
COMMIT;
The second sync fetched 12 rows (598 bytes) above the saved cursor and finished in 18 seconds end to end. For customers that meant one new data file with 4 rows and one deletion vector marking the 2 old versions. OLake Go's commit report shows addedDataFiles=1, addedRecords=4, addedDVs=1 and addedPositionalDeletes=2, and orders the same shape with 8 rows and 3 deleted positions.

Databricks shows the result on the next query, with no refresh. Liam Carter is gold, Zara Ali moved to Abu Dhabi, and the two new customers are there. In incremental mode OLake Go cannot tell an insert from an update, so the 4 rows the cursor picked up carry _op_type = u and the other 6 keep r.
Schema changes. When a column is added in Oracle, OLake Go adds it to the Iceberg table on the next sync and commits the new schema to Unity Catalog. Widening changes are applied; narrowing or incompatible type changes fail the sync rather than corrupt the table. Full rules are on the schema evolution page.
Query the same tables from DuckDB and Spark
Nothing is exported, copied or refreshed. Both engines read the files OLake Go wrote, through the same Unity Catalog endpoint, with storage credentials vended by Databricks.
- DuckDB
- Apache Spark
DuckDB attaches the Unity Catalog catalog as an Iceberg REST catalog with the same token. With vended_credentials, Unity Catalog hands DuckDB short-lived storage credentials per table, so no cloud keys are configured on the DuckDB side.
INSTALL iceberg; LOAD iceberg;
INSTALL httpfs; LOAD httpfs;
CREATE SECRET unity (TYPE iceberg, TOKEN '<personal access token>');
ATTACH 'lakehouse' AS uc (
TYPE iceberg,
ENDPOINT 'https://<workspace-url>/api/2.1/unity-catalog/iceberg-rest',
SECRET unity,
ACCESS_DELEGATION_MODE 'vended_credentials'
);
SELECT id, name, city, tier
FROM uc.oracle_to_databricks_shop.customers
ORDER BY id;
DuckDB reads Iceberg v3 deletion vectors from duckdb-iceberg 1.5.3, so the rows match Databricks after every sync. The principal behind the token needs EXTERNAL USE SCHEMA, the same as OLake Go.
DuckDB 1.5.5 returned the same 10 customers and the same orders roll-up as Databricks in the run on this page.
Open source Spark reaches Unity Catalog as a plain Iceberg REST catalog. The warehouse is the catalog name, the token is a personal access token, and the X-Iceberg-Access-Delegation header asks Unity Catalog for short-lived storage credentials.
spark-sql \
--packages org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.0,org.apache.iceberg:iceberg-gcp-bundle:1.10.0 \
--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \
--conf spark.sql.catalog.uc=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.uc.type=rest \
--conf spark.sql.catalog.uc.uri=https://<workspace-url>/api/2.1/unity-catalog/iceberg-rest \
--conf spark.sql.catalog.uc.warehouse=lakehouse \
--conf spark.sql.catalog.uc.token=<personal access token> \
--conf spark.sql.catalog.uc.header.X-Iceberg-Access-Delegation=vended-credentials
SELECT id, name, city, tier FROM uc.oracle_to_databricks_shop.customers ORDER BY id;
Iceberg 1.8 or later reads the v3 deletion vectors OLake Go writes for Databricks; this page was tested with 1.10.0 on Spark 4.0.1. Swap iceberg-gcp-bundle for iceberg-aws-bundle or iceberg-azure-bundle to match the cloud the workspace runs on.
Spark 4.0.1 with Iceberg 1.10.0 returned the same 10 customers and the same orders roll-up as Databricks and DuckDB in the run on this page.
An orders roll-up matched Oracle to the cent in all three engines. It gave 16 delivered orders for 8659.68, 18 placed for 10522.26, 11 shipped for 6397.25.
Data types: Oracle to Iceberg to Databricks
OLake Go maps Oracle types to Iceberg types, and Databricks maps Iceberg types to its own. Numbers are the group to watch.
| Oracle | Iceberg (written by OLake Go) | Databricks shows | Notes |
|---|---|---|---|
NUMBER(n,0), n up to 9 | int | INT | |
NUMBER(n,0), n above 9 | long | BIGINT | The NUMBER(10) keys on this page arrived as BIGINT. |
NUMBER(n,m) with a scale, FLOAT, BINARY_DOUBLE | double | DOUBLE | The orders.amount sums on this page matched Oracle to the cent; round at query time when you need exact decimal arithmetic. |
BINARY_FLOAT | float | FLOAT | |
DATE, TIMESTAMP, TIMESTAMP WITH TIME ZONE, TIMESTAMP WITH LOCAL TIME ZONE | timestamptz (UTC) | TIMESTAMP | Oracle DATE carries a time; it arrives as a UTC instant. |
VARCHAR2, NVARCHAR2, CHAR, CLOB, NCLOB, LONG | string | STRING | |
RAW, LONG RAW, BLOB, BFILE, INTERVAL types | string | STRING | Binary values arrive as text; keep large binaries out of the sync. |
Oracle types in the table come from the Oracle connector reference; the Databricks side comes from DESCRIBE on the tables of this run.
Cost and maintenance
- Storage is your cloud bucket behind the catalog's managed location. You pay your cloud's storage rates for the Parquet data files and deletion vectors.
- Compute. OLake Go runs wherever you run Docker and does not use a Databricks warehouse or cluster to write. Databricks compute is spent only when you query; the checks on this page ran on a 2X-Small serverless SQL warehouse.
- Maintenance. Each sync adds a small data file, plus a deletion vector when rows change. Plan compaction from the writer side: OLake Fusion does this for the catalogs OLake Go writes to, and Spark's
rewrite_data_filesworks through the same Unity Catalog endpoint. - Token rotation. A personal access token expires on the date set when it was created, and an expired token fails the next sync at the first catalog call. Use a service principal with OAuth machine-to-machine credentials for production jobs.
- Oracle load. Each incremental sync is one indexed range scan per table on the cursor column; index
updated_atso the scan stays cheap as the table grows.
Troubleshooting
1. The source test fails to connect
ORA-12514: Cannot connect to database. Service FREEPDB1 is not registered with the listener
The service name is wrong or the listener has not finished registering it. Run lsnrctl status on the server, or connect with the SID instead. On a fresh container the service appears a minute or two after startup.
2. The source test fails with ORA-01017 although sqlplus logs in
failed to ping database: ORA-01017: invalid credential or not authorized; logon denied
OLake Go quotes the username when it connects, so Oracle compares it case-sensitively. A user created as CREATE USER olake is stored as OLAKE; enter it in upper case. The run on this page hit this on its first attempt with olake in lower case.
3. Discovery sees no tables
The OLake Go user can log in but has no SELECT on the tables, or the tables live in another schema. Grant SELECT per table (step 1) and check all_tables as that user.
4. Updated rows are missing after an incremental sync
The update did not move the cursor column. OLake Go only fetches rows whose cursor value is above the last one it saw, so an UPDATE that leaves updated_at untouched is invisible. Set the column in every write path, or add a trigger that sets it on UPDATE.
5. The destination test fails with a permission error
Three settings cover nearly every failure. External data access may be off on the metastore, or the principal may lack EXTERNAL USE SCHEMA on the catalog. The third is a wrong catalog name in the destination.
ALL PRIVILEGES does not include EXTERNAL USE SCHEMA, so a catalog owner still has to grant it explicitly. Run the grants from step 2 again for the principal that owns the token.
6. Databricks rejects a commit with "Can not create a Path from a null string"
Can not create a Path from a null string [ErrorCode: 2015]
Unity Catalog returns this when a client commits equality delete files to a managed Iceberg table, per Databricks: access tables from Iceberg clients (checked Oct 2026). Set Databricks as a target query engine on the job, so OLake Go writes deletion vectors, and start any table that was written with another delete format again from a full refresh.
How this compares with other ways to move Oracle data to Databricks
| Method | Freshness | Updates and deletes | Where the data lands | What you run |
|---|---|---|---|---|
| Lakeflow Connect | Not available for Oracle as a CDC connector; the overview lists MySQL, PostgreSQL and SQL Server (checked Oct 2026) | Not applicable | Not applicable | Not applicable |
| Debezium's Oracle connector, Kafka and Databricks Structured Streaming (checked Oct 2026) | Seconds, from LogMiner | Yes, with your own merge logic | Kafka topics, then tables written by a Databricks streaming job | Kafka, Kafka Connect, Debezium and a running Databricks job |
| Export to files, then Auto Loader (checked Oct 2026) | Hours to daily | No; full reloads or your own merge | Tables written by a Databricks pipeline | An export script, a bucket and a Databricks job |
| OLake Go to Iceberg on Unity Catalog (this guide) | Incremental on a schedule down to one minute | Inserts and updates that move the cursor; deletes need a soft-delete column | Apache Iceberg tables that Databricks, DuckDB, Spark and Trino all read | One Docker Compose stack, open source; your storage plus your own compute |
Log-based capture through Debezium fits when you need hard deletes and seconds of latency, and already run Kafka. Writing Iceberg once avoids a second copy as soon as another engine needs the same rows.
When not to use this
- You need hard deletes replicated. Incremental sync cannot see a deleted row. Soft-delete in Oracle, or use a log-based tool.
- Tables have no cursor column and cannot get one. Without a column that grows with every change, only Full Refresh works.
- Every query must run on Databricks-native tables. OLake Go writes managed Iceberg tables; if a pipeline needs native Delta Lake features on these rows, load them into a Delta table downstream.
FAQs
How do I replicate Oracle to Databricks without GoldenGate?
Run OLake Go (open source, Apache 2.0) against the Oracle database with a read-only user, and point it at Unity Catalog's Iceberg REST endpoint. The first sync copies each table in parallel chunks; every later sync fetches only rows whose cursor column (for example UPDATED_AT) moved, and upserts them. The tables appear in Databricks as Unity Catalog managed Iceberg tables. GoldenGate, Kafka and a Databricks ingestion pipeline are not needed.
Does Lakeflow Connect have an Oracle connector?
Not among its change data capture database connectors. The Lakeflow Connect overview lists MySQL, PostgreSQL and SQL Server for CDC ingestion (checked Oct 2026). OLake Go reads Oracle over JDBC with a cursor column instead, so no redo-log access or Oracle license option is involved.
Which Iceberg delete format does OLake Go write for Databricks?
Deletion vectors, on Iceberg v3 tables. Under Unity Catalog, Databricks reads deletion vectors but not Iceberg v2 positional or equality deletes. Setting Databricks as a target query engine on the job makes OLake Go pick deletion vectors from the first sync.
Does OLake Go do change data capture (CDC) from Oracle redo logs?
No. The Oracle connector runs Full Refresh and Full Refresh + Incremental; it does not read redo logs or LogMiner. Incremental syncs select rows where a cursor column is greater than the last value seen, which captures inserts and updates that bump the cursor but not deletes.
Which Oracle data types change on the way to Databricks?
NUMBER with a scale, FLOAT and BINARY_DOUBLE become Iceberg double and show up in Databricks as DOUBLE, so round at query time when you need exact decimal arithmetic. Whole-number NUMBER columns become INT or BIGINT. DATE and every TIMESTAMP variant become TIMESTAMP in UTC. CLOB, RAW, BLOB and the INTERVAL types arrive as STRING.
Can DuckDB and Spark read the same Databricks tables?
Yes. Both attach Unity Catalog as an Iceberg REST catalog with a token and receive vended storage credentials. In the run on this page DuckDB 1.5.5 and Spark 4.0.1 with Iceberg 1.10.0 returned the same 10 customers and the same orders roll-up as Databricks.
Related guides
Deeper reading on this site: