Skip to main content

Postgres to MinIO AIStor: CDC into Iceberg Tables

Postgres into MinIO as Iceberg usually means two systems next to the object store: a catalog such as Polaris or a Hive Metastore, and a pipeline such as Debezium and Kafka. MinIO AIStor removes the first one. Its AIStor Tables feature serves the Iceberg REST (Representational State Transfer) catalog from the object store itself, and OLake Go removes the second by reading the Postgres write-ahead log (WAL) directly.

In the run on this page, OLake Go wrote both tables into an AIStor Tables warehouse. Apache Spark, DuckDB and Trino read them through the same catalog endpoint and returned the same rows.

By Shubham Satish BaldavaUpdated November 4, 2026

Key takeaways

  • OLake Go replicates Postgres into Iceberg tables in MinIO AIStor: a full historical snapshot first, then change data capture (CDC) from the WAL.
  • The catalog lives inside AIStor at /_iceberg, so there is no catalog server, metastore or database to run next to the object store.
  • Spark, DuckDB and Trino read the same tables with Signature Version 4 (SigV4) signing, and returned identical rows and totals in the run on this page.

The short answer. To replicate Postgres to MinIO AIStor with change data capture (CDC), run OLake Go with a Postgres replication slot as the source and AIStor Tables as a Generic REST catalog with SigV4 signing. The first sync copies each table in full; every later sync replays inserts, updates and deletes from the WAL.

Spark, DuckDB and Trino then query the tables in the AIStor warehouse.

How Postgres CDC reaches MinIO AIStor here​

Two things happen in order, and OLake Go does both on its own. The first sync records the current WAL position through the replication slot, then reads each table in parallel chunks and writes Parquet data files. Every later sync opens the slot at the saved position and replays the inserts, updates and deletes since.

OLake Go talks to AIStor twice on every sync, on the same port. It uploads the files through the S3 API, then commits the new snapshot through the catalog at /_iceberg, with both requests signed by the same access key.

An update needs the old version of the row retired. With positional deletes, OLake Go keeps an index of where each primary key sits (data file and row number) and writes a small delete file that points at those positions. Readers skip the retired rows without comparing keys.

One thing to keep an eye on: the replication slot pins WAL on the Postgres server until OLake Go has read it. A job that is paused for days makes the WAL grow by everything written since. Keep the job on a schedule, and alert on pg_replication_slots lag if disk is tight.

What you need​

  • Postgres 12 or later with wal_level = logical and a user that can read the WAL. The tables on this page ran on Postgres 16. Hosted Postgres on Amazon Relational Database Service (RDS), Aurora, Google Cloud SQL, Azure, Neon or Supabase works the same way; step 1 links the setup guides.
  • MinIO AIStor with Tables, reachable from OLake Go and from your query engines, and a license file. The run on this page used AIStor RELEASE.2026-09-19T17-05-25Z, one node in Docker, on the free tier license.
  • At least one engine that signs Iceberg REST requests with SigV4. This page shows Apache Spark 3.5.6 with Iceberg 1.10.0, DuckDB 1.5.5 and Trino 484.
  • OLake UI from the Docker Compose install. The run on this page used OLake Go v1.1.1 for the destination.
  • Docker, and about 40 minutes.

Step 1: Prepare Postgres for CDC​

Most readers point OLake Go at tables they already have. If you want to follow along with the exact data from this page, create these two tables first:

The two sample tables used on this page
Sample schema
CREATE TABLE customers (
  id SERIAL PRIMARY KEY,
  name VARCHAR(100), email VARCHAR(200), city VARCHAR(100), tier VARCHAR(20),
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW());

CREATE TABLE orders (
  id SERIAL PRIMARY KEY,
  customer_id INT REFERENCES customers(id), product VARCHAR(100), amount NUMERIC(10,2), status VARCHAR(20),
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW());

-- Then insert whatever you like. This page started with 8 customers and 40 orders.

OLake Go reads changes through logical replication with the built-in pgoutput plugin (Postgres 10 and later). Four things must be in place on the database before the first sync.

SettingValueWhy it matters
wal_levellogicalWithout it Postgres writes no row images to the WAL (write-ahead log), so there is nothing to replicate. Changing it needs a restart.
Replication roleLOGIN REPLICATION plus SELECT on the tablesThe first snapshot reads the tables; the change stream opens a replication connection.
Publicationone per OLake Go job, covering the synced tablesTells Postgres which tables' changes to publish.
Replication slotone per OLake Go job, plugin pgoutputKeeps WAL segments until OLake Go has read them.
On the source database
-- 1. Logical replication. wal_level needs a restart (a parameter-group change on managed Postgres).
ALTER SYSTEM SET wal_level = 'logical';
ALTER SYSTEM SET max_replication_slots = 10;
ALTER SYSTEM SET max_wal_senders = 10;
ALTER SYSTEM SET wal_sender_timeout = 0;   -- so a long first snapshot is not cut off

-- 2. A role for OLake Go
CREATE ROLE olake WITH LOGIN REPLICATION PASSWORD '<password>';
GRANT USAGE ON SCHEMA public TO olake;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO olake;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO olake;

-- 3. A publication for the tables you sync, and a pgoutput slot
CREATE PUBLICATION olake_pub FOR TABLE customers, orders;
SELECT pg_create_logical_replication_slot('olake_slot', 'pgoutput');

Tables with a primary key need nothing more. A table without one needs ALTER TABLE <table> REPLICA IDENTITY FULL; so that its updates and deletes carry the old row.

Check the result before moving on:

sql
SHOW wal_level;                                        -- logical
SELECT slot_name, plugin, active FROM pg_replication_slots;
SELECT pubname, puballtables FROM pg_publication;

Managed Postgres sets wal_level through a parameter group or server setting rather than ALTER SYSTEM. Each flavour has its own guide:

Where Postgres runsGuide
Amazon RDSRDS Postgres CDC setup
Amazon AuroraAurora Postgres CDC setup
Azure Database for PostgreSQLAzure Postgres CDC setup
Google Cloud SQLCloud SQL Postgres CDC setup
Your own server or VMGeneric Postgres CDC setup
Docker, for a local testLocal Postgres with Docker Compose
One slot and one publication per job

Two OLake Go jobs sharing a replication slot read each other's changes and both lose data. Create a separate slot and publication for every job.

Step 2: Set up AIStor Tables​

An Iceberg table is a set of files plus a catalog entry that says which metadata file is current. In MinIO AIStor, both live in the object store: AIStor Tables holds the entries in a warehouse and keeps the files next to them. OLake Go writes through the catalog, and every engine on this page reads through it.

The AIStor Tables catalog page lists the other engines that read it. If you run the open source MinIO server instead, the Postgres to Apache Iceberg guide pairs it with a Polaris catalog.

CatalogOLake Go writesApache Iceberg readsVerified on this page
MinIO AIStor TablesYesYes: Iceberg REST catalog built into the object storeYes, end to end

MinIO AIStor is MinIO's commercial object store. Its AIStor Tables feature puts an Iceberg REST catalog inside the store itself, at the /_iceberg path on the S3 port, so there is no Polaris, Glue or metastore to run next to it. OLake Go writes through that catalog with Signature Version 4 (SigV4) signing, and Spark, Trino and DuckDB read from the same endpoint.

Two things need to exist before OLake Go can write: an AIStor server with Tables enabled, and a warehouse in it. The run on this page used one AIStor node in Docker with a free tier license, which MinIO limits to a single compute resource (MinIO: deploy AIStor as a container, checked Nov 2026).

AIStor, single node in Docker
docker run -dt \
  -p 9000:9000 -p 9001:9001 \
  -v $HOME/minio/data:/mnt/data \
  -v $HOME/minio/minio.license:/minio.license \
  -e MINIO_ROOT_USER=<root-user> -e MINIO_ROOT_PASSWORD=<root-password> \
  --name aistor-server \
  quay.io/minio/aistor/minio:latest minio server /mnt/data \
  --console-address ":9001" --license /minio.license

Port 9000 serves both the S3 API and the Iceberg REST catalog; port 9001 is the AIStor console. The license file comes from MinIO's pricing page (the free tier) or from SUBNET.

Then create the warehouse with the MinIO client. Tables needs AIStor RELEASE.2026-02-02T23-40-11Z or later and mc RELEASE.2026-02-03T00-12-26Z or later (MinIO: AIStor Tables, checked Nov 2026).

Warehouse, once
mc alias set aistor http://<aistor-host>:9000 <root-user> <root-password>
mc table warehouse create aistor olake
mc table warehouse list aistor

What each part is for:

  • The warehouse name is what OLake Go and every engine use as the catalog's warehouse. It is a plain name (olake), never an s3:// path.
  • The catalog signs requests with SigV4 under the service name s3tables. The region is required by SigV4 but not checked by AIStor, so any non-empty value works.
  • OLake Go needs an access key that may create namespaces and tables in the warehouse and write its files. The run on this page used the root user in a lab. In production, create a dedicated user with mc admin user add and attach a policy scoped to the warehouse (MinIO: AIStor Tables).

You can check the catalog before OLake Go touches it: mc table warehouse list returns the warehouse, and a REST client pointed at http://<aistor-host>:9000/_iceberg with the same keys lists its namespaces.

Step 3: Connect Postgres to OLake Go​

  1. In the OLake UI, open Sources and click Create Source. Pick Postgres as the connector, give the source a name and keep the OLake Go version on the latest release.
  2. Under Change data capture (CDC), leave Enable CDC on and enter the Replication Slot and Publication you created in step 1. The default Initial Wait Time is fine.
  3. Under Connection details, fill in the host, port, database name, username and password. Leave Schemas empty to discover every non-system schema, or list the ones you want.
  4. Click Set up source. OLake Go tests the connection, then checks the four CDC prerequisites (wal_level, the replication privilege, the slot and the publication) and shows the current and recommended value for anything that fails.

Here is the source form from the run on this page. Postgres ran in another container, so the host is host.docker.internal; the user, slot and publication are the ones from step 1.

OLake UI Edit source form for Postgres with Enable CDC on, replication slot olake_slot, publication olake_pub, host host.docker.internal, database shop and user olake
The Postgres source in OLake UI: Enable CDC on, with the slot, publication and replication user from step 1.

Set up source runs the connection test and the CDC checks (wal_level, the slot, the publication) before anything is saved. When one fails, the dialog names the setting and its recommended value.

The full field reference, including SSL modes and SSH tunnels, is on the Postgres connector page.

Step 4: Configure the AIStor destination​

  1. Open Destinations and click Create Destination.
  2. Pick Apache Iceberg as the connector and Generic REST as the catalog type.
  3. Set REST Catalog URI to the AIStor catalog (http://<aistor-host>:9000/_iceberg), S3 Path to the warehouse name (olake), S3 Endpoint to the AIStor S3 address (http://<aistor-host>:9000), and REST Auth Type to sigv4.
  4. Fill in AWS Access Key and AWS Secret Key with the AIStor access key, AWS Region with us-east-1, REST Signing Name with s3tables and REST Signing Region with us-east-1, and switch on REST Enable Signature V4.
  5. Click Create. OLake Go signs a request to the catalog, creates a test table in the warehouse, writes a file to AIStor and drops the table again.

Here is the destination form from the run on this page. AIStor ran in another container with its S3 port mapped to 9200 on the host, so both addresses use host.docker.internal:9200; the keys are masked.

OLake UI Edit destination form for Apache Iceberg with catalog type Generic REST, sigv4 auth, REST catalog URI ending in /_iceberg, S3 path olake, signing name s3tables and Signature V4 on
The AIStor destination in OLake UI: Generic REST with SigV4, the warehouse name as S3 Path, and s3tables as the signing name.

Step 5: Create the job and run the first sync​

  1. Open Jobs and click Create Job. Give it a name and a schedule. Every Day at a fixed hour, or a cron expression, both work; the schedule is how often the change stream is drained, not how often a full copy runs.
  2. Select the source and the destination you created.
  3. Under Advanced Settings, set Target Query Engines to Apache Spark, DuckDB and Trino, or whichever engines will read the tables. OLake Go then resolves an Iceberg delete format every one of those engines can read, so you never pick one by hand.
  4. Click Next. OLake Go discovers the tables and opens the Streams page.
  5. Tick the tables to replicate. For each one, set the sync mode to Full Refresh + CDC and the ingestion mode to Upsert. Leave the Upsert Type as OLake Go resolved it (Positional Deletes in the run on this page).
  6. Click Create Job, then open the job's menu and choose Sync Now for the first run.

Three target query engines were set on this job: Apache Spark, DuckDB and Trino. All three read every Iceberg delete format, and the run on this page used positional deletes on Iceberg format version 2 for every stream.

OLake UI Streams Selection page for postgres_to_aistor with customers and orders selected, Full Refresh + CDC, Upsert and Positional Deletes on Iceberg format version V2
Each stream: Full Refresh + CDC, Upsert, and Positional Deletes on Iceberg v2. The Iceberg DB name at the top is editable.
OLake UI Job Logs and History for postgres_to_aistor showing two completed syncs of 12 seconds each
Job history: the 12 s historical snapshot, then the 12 s change-only sync.

On this page the first run copied 8 customers and 40 orders (48 records, 2.23 KB) and finished in 12 seconds, container start included. OLake Go names the namespace after the job, the source database and the schema, so postgres_to_aistor on database shop produced postgres_to_aistor_shop_public in the olake warehouse.

Step 6: Query the tables from DuckDB​

DuckDB is the quickest way to check the result from a laptop: one S3 secret, one ATTACH, and the whole warehouse is visible.

DuckDB attaches AIStor Tables as an Iceberg REST catalog and signs every catalog request with SigV4. One S3 secret does both jobs: it signs the catalog calls under the s3tables service and reads the data files from AIStor.

DuckDB 1.5.5
INSTALL iceberg; LOAD iceberg;

CREATE SECRET aistor_s3 (
TYPE S3,
KEY_ID '<access-key>', SECRET '<secret-key>',
ENDPOINT '<aistor-host>:9000',   -- host and port only, no scheme
URL_STYLE 'path', USE_SSL 0, REGION 'local'
);

ATTACH 'olake' AS aistor (
TYPE ICEBERG,
ENDPOINT 'http://<aistor-host>:9000/_iceberg',
AUTHORIZATION_TYPE 'sigv4',
SIGV4_SERVICE 's3tables',
SIGV4_REGION 'local',
SECRET aistor_s3
);

SHOW ALL TABLES;
SELECT id, name, city, tier FROM aistor.postgres_to_aistor_shop_public.customers ORDER BY id;

Set USE_SSL 1 when AIStor serves HTTPS. MinIO documents DuckDB 1.5 or later for AIStor Tables and asks for single-level namespaces in warehouses DuckDB reads, which is what OLake Go creates.

DuckDB 1.5.5 shell querying aistor.postgres_to_aistor_shop_public.customers and returning 10 rows with id, name, city and tier
DuckDB 1.5.5 reading the customers table from AIStor Tables after the change sync: 10 rows, one per key.

Step 7: Change rows in Postgres and sync again​

To see the change path work, the run on this page made 12 changes in Postgres after the first sync:

Changes made in Postgres between the two syncs
INSERT INTO customers (name, email, city, tier) VALUES
  ('Maya Patel', 'maya@example.com', 'Pune', 'silver'),
  ('Lucas Silva', 'lucas@example.com', 'São Paulo', 'bronze');
INSERT INTO orders (customer_id, product, amount, status) VALUES
  (9, 'Laptop', 1299.00, 'placed'), (9, 'Mouse', 25.50, 'placed'),
  (10, 'Monitor', 349.99, 'placed'), (3, 'Keyboard', 89.00, 'placed'), (7, 'Webcam', 79.90, 'placed');
UPDATE customers SET tier = 'gold', updated_at = NOW()      WHERE name = 'Liam Carter';
UPDATE customers SET city = 'Abu Dhabi', updated_at = NOW() WHERE name = 'Zara Ali';
UPDATE orders    SET status = 'delivered', updated_at = NOW() WHERE id IN (1, 4, 7);

The second sync read 12 records (594 bytes) from the slot and finished in 12 seconds end to end. For customers that meant one new data file with 4 rows: 2 inserts plus the new versions of 2 updated rows. One positional delete file retires the 2 old versions by file and row number.

OLake Go's commit report in the sync log shows it directly: addedDataFiles=1, addedRecords=4, addedPositionalDeleteFiles=1, addedPositionalDeletes=2, and no equality delete files. The orders table has the same shape with 8 rows and 3 deletes.

OLake UI sync log filtered to the Iceberg commit report for aistor.postgres_to_aistor_shop_public.customers, showing addedDataFiles 1, addedPositionalDeleteFiles 1, addedRecords 4 and addedPositionalDeletes 2
The change-only sync in the OLake UI log: one data file and one positional delete file per table, committed to AIStor Tables.

Every engine shows the result on its next query, with no refresh. Liam Carter is gold, Zara Ali moved to Abu Dhabi, and the two new customers are there. The _op_type column shows how each row arrived: r for the snapshot, c for the inserts and u for the updates.

Schema changes. When a column is added in Postgres, OLake Go adds it to the Iceberg table on the next sync, and readers see it the next time they load the table. Narrowing or incompatible type changes fail the sync rather than corrupt the table; full rules are on the schema evolution page.

Query the same tables from Trino and Spark​

Nothing is exported or copied for a second engine. Trino and Spark read the files OLake Go wrote, through the same AIStor Tables endpoint.

Trino reaches AIStor Tables through its Iceberg connector in REST mode, with SigV4 signing under the s3tables service. One catalog properties file does it; the same keys sign catalog requests and read the files.

etc/catalog/aistor.properties
connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=http://<aistor-host>:9000/_iceberg
iceberg.rest-catalog.warehouse=olake
iceberg.rest-catalog.security=SIGV4
iceberg.rest-catalog.signing-name=s3tables
iceberg.unique-table-location=true
fs.native-s3.enabled=true
s3.region=local
s3.endpoint=http://<aistor-host>:9000
s3.path-style-access=true
s3.aws-access-key=<access-key>
s3.aws-secret-key=<secret-key>
trino
SHOW SCHEMAS FROM aistor;
SELECT id, name, city, tier FROM aistor.postgres_to_aistor_shop_public.customers ORDER BY id;

Trino 477 and later take iceberg.rest-catalog.security=SIGV4; on Trino 476 and earlier use iceberg.rest-catalog.sigv4-enabled=true instead. The run on this page used Trino 484.

Trino 484 CLI querying customers in the aistor catalog and returning the same 10 rows as DuckDB
Trino 484 reading the same table from AIStor Tables: the same 10 rows.

All three engines agreed on the counts and the money: 10 customers, 45 orders and 25579.19 in total. Spark 3.5.6, DuckDB 1.5.5 and Trino 484 each returned those numbers from the same tables.

Data types: Postgres to Iceberg on AIStor​

OLake Go maps Postgres types to Iceberg types; AIStor stores the files and does not change them. Most columns pass through unchanged, and three groups are worth knowing about.

PostgresIceberg (written by OLake Go)Notes
integer, serialint
bigint, bigseriallong
numeric(p,s), realdoubleOLake Go stores decimals as doubles. The orders.amount totals on this page matched Postgres to the cent; round at query time when you need exact decimal arithmetic.
timestamptz, timestamptimestamptz (UTC)timestamp has no zone in Postgres; it is read as UTC.
varchar, text, uuid, json, jsonbstringParse JSON at query time.
booleanboolean
byteabinary

Postgres types in the table come from the Postgres connector reference. OLake Go adds _cdc_lsn on Postgres tables, the WAL position each change came from.

Cost and maintenance​

  • License and storage. AIStor needs a license file. The free tier covers one compute resource; larger clusters need a paid AIStor license from MinIO. The files sit on your own drives or servers.
  • Compute. OLake Go runs wherever you run Docker. Each engine spends compute only when it queries, and none of them keeps its own copy.
  • Maintenance. OLake Go does not compact the tables. A CDC table that gets a new small file and delete file per sync wants compaction; Spark's rewrite_data_files works through the same AIStor Tables endpoint.
  • WAL disk. The replication slot keeps WAL until OLake Go reads it. A job paused for a weekend on a busy server can hold tens of gigabytes. Drop the slot if you retire a job.
  • Key rotation. The AIStor access key signs both the catalog and the file requests. Rotate it in AIStor and in the OLake Go destination together; an old key fails the next sync at the first signed request, before any data moves.

Troubleshooting​

1. The CLI destination test fails before it reaches AIStor​

invalid destination type has been passed []

destination.json holds only the writer settings. OLake Go expects them wrapped as {"type": "ICEBERG", "writer": {...}}, as in the CLI tab of step 4. The run on this page hit this when it first sent the bare writer block.

2. Catalog requests are rejected while file writes work​

The signing settings are incomplete. AIStor Tables only accepts catalog requests signed under the s3tables service, so check that REST Enable Signature V4 is on, REST Signing Name is s3tables and REST Signing Region is not empty. The S3 file path uses the same keys but a different service, which is why file access can work while the catalog does not.

3. The warehouse is not found​

S3 Path holds the warehouse name, such as olake, not an s3:// URL: MinIO's docs say the warehouse is the plain name, never prefixed with s3:// or s3a://. Run mc table warehouse list to see the exact name.

4. DuckDB or Trino sees the catalog but not the tables​

Each engine signs the catalog requests itself. DuckDB needs SIGV4_SERVICE 's3tables' on the ATTACH; Trino 477 and later need iceberg.rest-catalog.security=SIGV4 and signing-name=s3tables, and Trino 476 and earlier use iceberg.rest-catalog.sigv4-enabled=true instead.

How this compares with other ways to move Postgres data to Iceberg on MinIO
MethodCatalogUpdates and deletesWhat you run
Debezium, Kafka and Kafka Connect with an Iceberg sinkA separate REST or Hive catalogAppended as change events, with a separate merge job for the current stateDebezium, Kafka, Kafka Connect and a catalog server
Flink CDC with the Iceberg Flink sinkA separate catalog, or AIStor Tables through RESTUpserts through equality deletes on tables with a primary keyA Flink cluster with checkpoint storage
Open source MinIO with Polaris and OLake GoApache Polaris next to MinIOUpserts through positional deletes, equality deletes or deletion vectorsMinIO, Polaris and OLake Go; see the Postgres to Iceberg guide
OLake Go to MinIO AIStor (this guide)AIStor Tables, inside the object storeUpserts through positional deletes, equality deletes or deletion vectors, matched to the target enginesAIStor and one Docker Compose stack for OLake

Debezium and Kafka make sense when Kafka already carries the change events for other consumers. Flink wins when you need joins, enrichment or second-level latency on the way in.

Open source MinIO with Polaris avoids the AIStor license at the cost of one more service to run. AIStor keeps the catalog, the files and the access keys in one system.

When not to use this​

  • You need sub-minute freshness. OLake Go runs on a schedule, down to one minute between syncs. A streaming pipeline is the better fit for second-level latency.
  • You run the open source MinIO server. AIStor Tables is part of the licensed AIStor product. Pair open source MinIO with Polaris or Lakekeeper instead.
  • Tables have no primary key. Inserts replicate; updates and deletes cannot be matched to a row and are not applied. Add a primary key or set REPLICA IDENTITY FULL first.

FAQs​

How do I replicate Postgres to MinIO without Kafka or Debezium?

Run OLake Go (open source, Apache 2.0) against a Postgres database with logical replication on, and point it at the Iceberg REST catalog built into MinIO AIStor. The first sync copies the tables in full, and every later sync replays only the changes from the write-ahead log. Each sync lands as one Iceberg commit in an AIStor Tables warehouse, with no Kafka, Debezium or Spark job in the path.

What is AIStor Tables?

AIStor Tables is the Apache Iceberg REST catalog that MinIO runs inside the AIStor object store, at the /_iceberg path on the S3 port. Tables live in warehouses, and engines reach them with Signature Version 4 signing under the s3tables service name. There is no separate catalog server, metastore or database to run.

Does this work with the open source MinIO server?

No. AIStor Tables is part of MinIO AIStor, which needs a license file to run. The free tier license covers one compute resource, which is what the run on this page used. With the open source MinIO server, pair it with a separate catalog such as Apache Polaris, as in the Postgres to Apache Iceberg guide.

How does OLake Go authenticate to AIStor Tables?

With SigV4. In OLake Go the destination is a Generic REST catalog with rest_auth_type sigv4, rest_signing_name s3tables, a signing region and the AIStor access key and secret key. The same keys write the data files to AIStor's S3 endpoint.

Which Iceberg delete format does OLake Go write here?

Spark, DuckDB and Trino all read equality deletes, positional deletes and deletion vectors, so every format was open to this job. The run on this page used positional deletes on Iceberg format version 2. MinIO documents AIStor Tables as supporting Iceberg v3, including deletion vectors, if your engines need them.

What does OLake Go need on the Postgres side?

wal_level set to logical, a replication slot and a publication for the tables, and a user with the REPLICATION attribute and SELECT on the tables. Hosted Postgres on RDS, Aurora, Cloud SQL, Azure, Neon or Supabase exposes the same settings through a parameter group or dashboard toggle.

Postgres to SnowflakeSame source, read from Snowflake.Postgres to ClickHouseSame source, read from ClickHouse.Postgres to DuckDBSame source, read from DuckDB.Postgres to BigQuerySame source, read from BigQuery.Postgres to Amazon RedshiftSame source, read from Amazon Redshift.Postgres to DatabricksSame source, read from Databricks.MySQL to Apache IcebergSame engine, replicated from MySQL.MongoDB to Apache IcebergSame engine, replicated from MongoDB.Oracle to Apache IcebergSame engine, replicated from Oracle.SQL Server to Apache IcebergSame engine, replicated from SQL Server.Kafka to Apache IcebergSame engine, replicated from Kafka.Postgres connector referenceEvery field, sync mode and troubleshooting note for the source.Query engine compatibilityWhich engines read which catalogs and delete formats.All integrationsEvery source and engine pair with a tested guide.

Deeper reading on this site: