What Is Change Data Capture? A Plain Guide (2026)
25 min read
Key takeaways
Change data capture (CDC) means spotting the rows that were added, changed or deleted in a database and sending only those changes to another system. A common method is log-based CDC, which reads the database's own change log. Log-based and trigger-based methods see real deletes.
Timestamp queries can't, and comparing copies can only infer them.
If you can change database settings and you need deletes, pick log-based. If you can't get log access, a timestamp column is the simplest fallback.
CDC is not always real time. The sources here document at-least-once delivery, so duplicates can happen. It also costs something to run.
- IBM defines CDC as detecting changes made to data in a database and sending those changes to downstream systems. IBM lists three common methods and Databricks adds a fourth, and deletes are what separate them.
- Log-based CDC has real limits. It needs database setup, it does not carry table structure changes in PostgreSQL (per Debezium), and a stalled reader can make the database keep log files, which use up storage (PostgreSQL docs).
- The sources here document "at least once" delivery, so your destination has to cope with repeats after a crash.
Vendor facts come from each vendor's own pages, read in October 2026. Where a source gives a condition, we give it too. We did not run the other tools.
What is change data capture, in plain words?
Picture a customer table with 10 million rows, and 300 of them changed today. A batch job copies all 10 million rows again tonight. CDC copies the 300.
CDC asks "what happened since I last looked?" Each change arrives as an event: a row was inserted, a row was updated, or a row was deleted.
People use the term for any tool or technique that does this. Typical sources are databases such as PostgreSQL, MySQL, SQL Server, MongoDB or Oracle.
Why CDC exists: batch versus incremental
A batch job runs on a schedule, reads whole tables, and replaces the copy. Confluent notes that batch syncing means data isn't synced right away, and that production databases can be slowed down while the job runs.
CDC is incremental. It sends only the changes. That matters most for big tables that change a little at a time.
CDC isn't always the right answer. Our read: if you only need yesterday's numbers and your tables are small, a nightly batch job has fewer moving parts. The extra setup pays off for large tables, deletes that must show up, or freshness.
What teams actually use CDC for
IBM (International Business Machines) names four benefits: faster decisions from fresh operational data, zero-downtime migrations, better ETL (extract, transform, load) processes, and fresher data for AI models. Its listed use cases include fraud detection, Internet of Things (IoT) devices, inventory and supply chain, and compliance reporting.
IBM says ETL is commonly batch-oriented at scheduled intervals, and that CDC "complements ETL pipelines by transmitting incremental updates after the initial data load".
Confluent also lists microservice integration as a use case.
AWS's Database Migration Service (DMS) docs describe two task types for ongoing replication. In a "full load plus CDC" task, the existing data is copied first and then the source's changes are applied. With "CDC only", the destination is assumed to hold the data already.
The four CDC methods
IBM lists log-based, timestamp-based and trigger-based as among the most common, and Databricks adds comparing snapshots for sources that offer nothing else.
| Method | How it finds changes | Sees deletes? | Main strength | Main limit |
|---|---|---|---|---|
| Log-based | Reads the database's transaction log | Yes, with limits (see below) | No queries against your tables, ordered changes | Needs database setup and log access, log formats differ |
| Trigger-based | A trigger writes each change to a side table | Yes | Works inside the database, no log access needed | Triggers fire on each transaction and can tax the source |
| Timestamp (query) based | Queries a "last modified" column | Only soft deletes | Simple to build, needs no log access | Needs a column, scans rows, misses hard deletes |
| Snapshot comparison | Compares full copies taken at different times | Only by inference (a row missing from the newer copy) | Works when the source offers no change feed | Changes are inferred by comparison, and it is only as fresh as the last copy |
Another split is who does the work. Confluent describes push as the source database sending updates to downstream systems, and pull as downstream systems polling the source at fixed intervals.
Log-based CDC
A transaction log is the record a database keeps of the changes made to it, and log-based CDC reads that record. IBM explains that each change in the log has an ordered position, such as a log sequence number (LSN), which tells the tool exactly where it is.
Confluent says log-based CDC adds no computational overhead to the source database, captures inserts, updates and deletes, and needs no schema change. That is Confluent's own claim.
The same page lists the catches. Log formats are not standardized, and changes that were rolled back must be filtered out.
Debezium's FAQ adds a practical point. Most databases have to be configured first, and running a connector can still put some extra load on the source. Our read: reading the log avoids repeated table scans, but it doesn't make the database free of work.
Trigger-based CDC
A trigger is a small piece of code that a database runs automatically when a row is inserted, updated or deleted. In trigger-based CDC, the trigger copies the change into a separate table, often called a shadow table. Confluent says you need one trigger per operation per table.
According to IBM, this approach can tax the source because triggers fire on every transaction. On the plus side, triggers run on inserts, updates and deletes, so unlike timestamp queries they see a real delete. Confluent adds that it requires changes to the source database schema and can become complex to manage with a large number of triggers.
Timestamp (query) based CDC
This one is also called query-based CDC. Each table needs a column that records when a row last changed, and a tool runs a query such as "give me rows changed since my last run". IBM describes it as needing columns that record the date and time of each change.
Confluent says it can only handle soft deletes (where the row is kept and flagged as deleted), not real deletes. It also adds load because rows must be scanned, and it requires changes to the database schema. Our read: for small tables where deletes are rare or flagged, it may be enough.
Snapshot comparison
Sometimes a source offers nothing but periodic full exports. Databricks says that when a change feed is unavailable, snapshot-based ingestion is the only option, and that finding the changes means comparing records across snapshots to infer inserts, updates and deletes.
The same page lists why teams skip CDC feeds: added load on production, performance worries, legacy systems, and not owning the source database. Our read: use comparison as the fallback.
Which CDC method fits your situation
| Your situation | Best fit | Why | Watch out for |
|---|---|---|---|
| You need deletes and low load on the source, and you can change database settings | Log-based | Reads the log, not your tables, and sees deletes | Setup, log retention, stalled readers, no DDL in PostgreSQL |
| You cannot get log access, and the table has a reliable "last modified" column | Timestamp (query) | Simple and needs no special database feature | Hard deletes are invisible, and rows are scanned each run |
| You can add triggers, writes are modest, and you need deletes without log access | Trigger-based | Captures all three operations inside the database | Triggers fire on every transaction (IBM) and can get complex to manage (Confluent) |
| You do not own the source, or it only provides exports | Snapshot comparison | The only option when no change feed exists (Databricks) | Changes are inferred by comparing copies, and freshness equals your export schedule |
| You need yesterday's numbers only, on small tables | Plain batch load | Fewer moving parts (our read) | Data is only as fresh as the last run (Confluent) |
How log-based CDC works in each database
Databases keep logs of changes for their own needs, such as recovery and replication, and log-based CDC tools read them. The name and the way to read it differ by database. PostgreSQL calls its log the write-ahead log (WAL).
| Database | The log | How CDC reads it | Worth knowing |
|---|---|---|---|
| PostgreSQL | write-ahead log (WAL) | Logical decoding through a replication slot | The docs say a slot keeps required WAL even when nothing is reading it |
| MySQL and MariaDB | Binary log (binlog) | A tool reads the row-level binlog | Debezium's FAQ says a MySQL server needs the row-level binlog and a user allowed to read it |
| SQL Server | Transaction log | Built-in CDC copies changes into change tables | Microsoft says it uses the SQL Server Agent |
| MongoDB | Oplog (operations log) | Change streams | Change streams are available for replica sets and sharded clusters (MongoDB docs) |
| Oracle | Redo logs | LogMiner, an Oracle utility that queries them with SQL | Debezium's FAQ says its Oracle support is based on LogMiner and XStream |
PostgreSQL
PostgreSQL's logical decoding works by decoding the write-ahead log into a form a program can read. A replication slot is a bookmark that tracks how far one consumer has read. The docs say slots persist across crashes and know nothing about the state of their consumers.
If the reader stops and the slot isn't dropped, PostgreSQL keeps the log files the slot still needs. The docs say this consumes storage and, in extreme cases, could cause the database to shut down.
PostgreSQL's documentation for the pg_replication_slots view describes the columns of each slot. A query such as SELECT slot_name, active, wal_status, safe_wal_size FROM pg_replication_slots; lists the columns the docs describe. We did not run it for this guide, and the output below is made up to show you how to read it.
slot_name | active | wal_status | safe_wal_size | What to check |
|---|---|---|---|---|
cdc_slot_a | true | reserved | 4 GB | Reader is keeping up |
cdc_slot_b | false | extended | empty | Reader stopped, log is piling up |
cdc_slot_c | false | lost | empty | Changes are gone, start over |
The docs define wal_status with four values:
reserved: the log the slot needs is withinmax_wal_size.extended: that limit is exceeded, but the files are still kept.unreserved: the slot no longer holds the files it needs, and some will be removed at the next checkpoint.lost: the slot is no longer usable.
Our read: extended is a sign to check why the reader is behind. unreserved is a warning, but the docs say it can return to reserved or extended. The docs say a lost slot is no longer usable, so plan on a new slot and a new first copy.
safe_wal_size is the number of bytes that can still be written to the log before the slot is in danger of being lost. The docs say it is empty for lost slots, and when max_slot_wal_keep_size is -1.
Debezium's FAQ says monitoring PostgreSQL needs a logical decoding plug-in, and that pgoutput ships with PostgreSQL 10 and later. Whether it can be enabled depends on the particular database's configuration.
The setting that flips the switch is wal_level. PostgreSQL's documentation for that setting says the logical level adds the information needed for logical decoding, that it can only be set at server start, and that it increases WAL volume, particularly if many tables use REPLICA IDENTITY FULL and there are many updates and deletes.
PostgreSQL documentation, wal_level setting, October 2026.
PostgreSQL documentation, logical decoding concepts, October 2026.
MySQL and MariaDB
MySQL's change log is the binary log, or binlog. Debezium's FAQ says the server must be set to use the row-level binlog, and the tool needs a user with permission to read it.
Row-level means the log records what happened to each row. MariaDB also keeps a binary log, and MariaDB's documentation describes it as recording data and structure changes for replication and recovery.
SQL Server
SQL Server ships a CDC feature. Microsoft's documentation says the source of change data is the transaction log, and that a capture process copies changes from it into change tables. You switch it on per database and then per table.
Microsoft also says it uses the SQL Server Agent, the job scheduler, so the Agent has to be running. For tools that read SQL Server changes, see our SQL Server CDC tools comparison.
Microsoft's SQL Server change data capture documentation, October 2026.
MongoDB
MongoDB's change streams let an application subscribe to changes on a collection, a database or a whole deployment. MongoDB says this avoids the complexity and risk of tailing the oplog by hand, which is MongoDB's own claim. Change streams are available for replica sets and sharded clusters.
Streams can pick up after an interruption with a resume token, and a start point in the past must fall within the time range of the oplog. A collection drop or rename ends the stream. From version 6.0, you can ask for the document before and after a change, if you enable that setting on the collection.
MongoDB documentation, change streams with pre- and post-images, October 2026.
Oracle
Oracle's documentation describes LogMiner as a feature of the database that lets you query the online and archived redo log files through a SQL interface. Debezium's FAQ says its Oracle support is based on LogMiner and XStream.
What CDC captures, and what it misses
It is easy to assume CDC "captures every change". The documentation lists exceptions.
| Topic | What the sources say |
|---|---|
| Deletes | Timestamp-based CDC handles only soft deletes (Confluent). In MySQL, deletes caused by a cascade (a delete that automatically removes related rows) are not in the binlog and cannot be captured (Debezium FAQ) |
| Schema changes | Debezium says PostgreSQL logical decoding does not support data definition language (DDL) changes, so the tool cannot report them. DDL means commands that change table structure, such as adding a column |
| Truncate | A truncate command empties a whole table. Debezium's PostgreSQL docs say a truncate event has no message key, so for topics with multiple partitions there is no ordering guarantee for the change events or the truncate events of a table |
| Initial data | PostgreSQL purges old WAL, so Debezium first takes a consistent snapshot, then streams from the exact point of that snapshot |
| Old values | In PostgreSQL, the REPLICA IDENTITY setting controls how much of the previous row appears on updates and deletes, and a table without a primary key emits only create events under the default setting. In MongoDB, a delete has no post-image |
| Ordering | Debezium's FAQ says applications see events in the order they occurred, and that most connectors put one table's events in one topic, where the order is kept |
| Duplicates | After a crash, PostgreSQL may resend recent changes (PostgreSQL docs). Debezium's PostgreSQL docs say it provides at least once delivery of change events |
The schema change, snapshot, truncate, old-values and delivery lines come from the Debezium PostgreSQL connector docs. The delete and ordering lines come from Confluent's CDC guide and the Debezium FAQ.
Exactly once versus at least once
"At least once" means every change is delivered, but a retry may deliver some twice. "Exactly once" means that, whatever retries happen, each change has its effect on the destination one time only.
The sources cited in this guide document at-least-once delivery after a failure. Whether you get exactly-once effects depends on the whole pipeline, including how the destination applies changes.
The PostgreSQL docs say a logical slot sends each change once in normal operation, but after a crash it can return to an earlier position. They add that clients are responsible for handling the same message more than once.
Debezium's FAQ makes a related point: even if events are duplicated during failures, the end result after applying all of them stays the same. Our read: that holds only if the destination applies changes by key, and ignores a change that is older than what it already holds. Exactly-once effects come from the pipeline, built on top of at-least-once delivery.
PostgreSQL documentation, replication slots, October 2026.
A worked example: one change, start to finish
Here is one change followed all the way through, using the example from the Databricks documentation. An employee table in Oracle has 50 rows, and one employee's job title changes. The CDC feed holds a single update record for that employee. A full copy would carry all 50 rows.
Each record carries three things: the operation (insert, update or delete), the data values, and a sequence number or timestamp so changes can be put in order. Databricks says the sequence number lets late or out-of-order arrivals be applied correctly.
Now the destination has a choice to make. It can keep only the latest row per employee, so the title simply changes. Or it can keep every version, so you can see the title was once "Owner" and is now "Manager".
Databricks calls these slowly changing dimensions (SCD) Type 1 and Type 2. Which one you want depends on whether you need history for audits or trends.
We made up this illustration of why the order number matters.
| Order sent | Sequence | Change for employee 7 | Destination after it |
|---|---|---|---|
| 1 | 10 | Update title to "Owner" | Owner |
| 2 | 12 | Update title to "Manager" | Manager |
| 3 | 14 | Delete the row | No row |
| 4 (replay after a restart) | 12 | Update title to "Manager" | Row is back, which is wrong |
A destination that matches only on the employee key lets the replay of change 12 bring the row back. One that remembers the delete at sequence 14, and skips anything older, keeps the row gone. Our read: the destination has to remember deletes as well as rows.
Where CDC data can land
CDC is only the capture step. After that, a tool has to carry the changes somewhere. IBM lists data lakes, data warehouses, streaming platforms such as Apache Kafka, stream processing engines such as Apache Spark, and data pipelines (IBM names extract-transform-load and extract-load-transform pipelines, tools that move and reshape data).
Debezium also has a Server mode. Its docs say it streams change events to messaging and storage systems without requiring Apache Kafka, and it includes a sink that writes to Apache Iceberg tables. Apache Iceberg is an open table format for large analytic tables.
Landing CDC data in a lake adds one more job. The Apache Iceberg table spec describes tables built on immutable files, so an update or a delete can't simply edit a row in place.
The Iceberg specification (read in October 2026) handles this with delete files, which record deleted rows next to the data files. Format version 2 adds row-level deletes of two kinds: position deletes, which name a data file and a row position, and equality deletes, which mark rows by column values such as id = 5. Version 3 uses deletion vectors for position deletes.
So a writer can record a CDC delete as a small delete file next to the data instead of rewriting the data files. Whether a given tool does that, or rewrites files, is the tool's own choice. Our read: ask which it uses, and how it applies updates, before you trust the lake table to match the source. For tools that do both the capture and the landing, see our best change data capture tools comparison, and for copying whole databases into a lake or warehouse, our data replication tools comparison.
What practitioners say
We read one Hacker News thread, "Listen to Database Changes Through the Postgres WAL", in October 2026. Two comments follow.
One commenter who builds on PostgreSQL logical replication called it "a great tool, but it is also very low-level and under-documented". The same comment adds: "In most cases, replication slots cannot be preserved when upgrading Postgres major versions." Another commenter, who works with MySQL, said "Even with Debezium I have run into myriad issues".
Before you switch CDC on: a checklist
- Check the setup the database needs. For example, MySQL needs the row-level binlog and a user who can read it, and PostgreSQL needs a logical decoding plug-in (Debezium FAQ).
- Plan the first copy and the start point. Log-based tools start with a snapshot because old log files are purged, as Debezium's PostgreSQL docs explain. AWS warns that for Oracle, starting from a system change number or timestamp misses open transactions, meaning ones that began before the start point and committed after it.
- Watch the log the database keeps for you. PostgreSQL's docs say a slot consumes storage, so check each slot's
wal_statusand the disk. Our read: dropping a slot that is still needed gives up the changes it held, so you would need a new first copy. - Make writes safe to repeat. PostgreSQL's docs suggest recording the last
LSNyou saw and skipping repeated data. Our read: also apply changes by row key, so a duplicate cannot create a second row. - Decide how deletes, truncates and table changes are handled. Write the answer down, because the tool may not do it for you.
- Measure latency on your own system. AWS says there are no SLAs for CDC latency in DMS.
Common myths about CDC
Myth: CDC needs Kafka. IBM lists Apache Kafka as one place to send changes, and Debezium documents a Server mode that runs without it. For a route from a database to Iceberg without Kafka, see our comparison.
Myth: CDC is real time. IBM says CDC enables near real-time or real-time synchronization. AWS says its DMS does not provide real-time replication for CDC, that latency varies with workload and network, and that there are no latency service guarantees. So treat "real time" as a goal and measure it on your own system.
AWS DMS documentation, ongoing replication, October 2026.
Myth: CDC is free. The software may be open source. PostgreSQL's docs say a slot that holds log files consumes storage, and Debezium's FAQ says a connector may place extra load on the source database. Someone also has to run the pipeline and watch it. Some CDC tools are paid products, and we quote no prices here because they differ by vendor.
Where OLake Go fits
OLake Go is an open-source tool that copies databases into Apache Iceberg tables or Parquet files. For Postgres, MySQL and MariaDB, MongoDB and SQL Server, our docs list four sync modes: full refresh, full refresh plus incremental, full refresh plus CDC (an initial load, then log-based CDC), and CDC only (see the sources and sync modes table). CDC is available for these four databases.
Our docs say Postgres CDC uses the pgoutput plug-in with a replication slot and a publication, and each job needs its own slot. MySQL needs the row-format binlog with binlog_row_image=FULL, MongoDB needs a replica set with the oplog enabled, and SQL Server uses its native CDC, whose change tables keep events for 3 days by default.
Iceberg tables can be registered in AWS Glue, a Hive Metastore, a JDBC (Java Database Connectivity) catalog, or a REST (web API) catalog such as Nessie, Polaris, Unity, Lakekeeper, BigLake, Amazon S3 Tables or Snowflake Horizon (v0.11.0 and later). See our catalog compatibility page.
Short glossary
- CDC: change data capture, finding and sending only the changes.
- Transaction log: the file where a database records changes for recovery.
- WAL: PostgreSQL's write-ahead log.
- Binlog: MySQL's binary log of data changes.
- Oplog: MongoDB's operations log.
- Replication slot: a PostgreSQL bookmark that keeps log files until a reader has consumed them.
LSN: log sequence number, the position of a change in the log.- Snapshot: a full copy of a table at one moment.
- DDL: commands that change table structure, such as adding a column.
- Soft delete: marking a row as deleted instead of removing it.
- At-least-once: every change arrives, but repeats are possible.
Conclusion
Pick the method by what you must see (deletes especially) and what access you have. Before you switch log-based CDC on, check its setup, log retention and blind spots. Then make the destination safe to repeat, because sooner or later a change will arrive twice.
Test your CDC basics
Question 1 of 5
A customer table has 10 million rows and 300 changed today. What does change data capture (CDC) send to the other system?
