MongoDB and Atlas CDC Setup

This guide covers setting up Change Data Capture (CDC) for both self-hosted MongoDB and MongoDB Atlas. Most steps apply to both, with notes where Atlas differs because certain configurations are pre-enabled.

1. Enable Replica Set Mode

• Self-Hosted MongoDB: Start MongoDB with replica set enabled to activate the oplog.

  • Inside Docker:

    docker run -d --name mongodb  -p 27017:27017  mongo:6.0 --replSet rs0

  • Outside Docker:

    mongod --replSet rs0 --bind_ip_all

• MongoDB Atlas: Replica set is pre-configured; skip this step.

2. Initialize the Replica Set (Self-Hosted Only)

• Self-Hosted MongoDB:

  1. After MongoDB starts with replica set mode, initialize it:

     docker exec -it mongodb mongosh

  2. Then run:

     rs.initiate()

  3. You should see:

     { "ok" : 1 }

  ✅ Now your MongoDB has a replica set and oplog enabled.

• MongoDB Atlas: Initialization is already done; skip this step.

3. Create a User

Applicable for both MongoDB (Self-Hosted) and Atlas.

Olake needs a user that can:

• Read/write your application database (to ingest data).
• Read from the local database (where oplog is stored).

Create the user in the admin database:

use admin
db.createUser({
  user: "olake_user",
  pwd: "your_password",
  roles: [
    { role: "readWrite", db: "your_db" },
    { role: "read", db: "local" }
  ]
})

4. Verify oplog

*Applicable for both Self-Hosted and Atlas.

Check that the oplog collection exists:

use local
show collections
db.oplog.rs.findOne()

If you see an entry, ✅ oplog is working.

---

info

• Atlas users can skip Steps 1 and 2 as these are pre-configured.
• Follow Steps 3 and 4 for both environments.

Once CDC setup is complete, proceed with the MongoDB Source Setup