Salesforce CDC (Change Data Capture): The Ultimate Guide
Explore this step-by-step guide to Salesforce CDC (change data capture) using Airbyte in 3 Easy steps that will revolutionize your business.
In today’s data-driven world, change data capture enables fast integration of Salesforce data with external systems by tracking real-time changes—create, update, delete, and undelete—in Salesforce records and publishing events for external systems to consume, ensuring efficient synchronization, reducing downtime, and supporting timely, accurate decisions.
Salesforce CDC at a glance:
Why Implement CDC in Salesforce?
Cloud CRM systems like Salesforce face integration challenges with internal data systems, causing data discrepancies. Change Data Capture (CDC) ensures continuous synchronization of new and updated data, maintaining long-term data integrity. CDC efficiently manages Salesforce record changes without overloading system resources, enabling real-time notifications and allowing users to efficiently integrate Salesforce data with external systems.
What Are the Key Stages of Salesforce CDC Data Replication?
- Initial Dataset Replication: The entire dataset is replicated in an external system as the first batch.
- Ongoing Synchronization: Changed records, including newly added or updated data, are continuously synchronized with the system. This ensures that corresponding records in the external data store are accurately updated based on the changes occurring in Salesforce.
- Duplicate Record Management: Managing and rectifying record duplication in the database is crucial. Utilizing a robust data integration system like Airbyte ensures corresponding updates are executed efficiently.
How Do You Set Up Salesforce CDC With Airbyte?
Grant permissions in Workbench to allow external systems to receive Salesforce CDC data. Accept requested permissions when logging in. Airbyte simplifies configuring platform events on the Lightning Platform for real-time data synchronization. A technical lead can follow these steps to set up the integration.
Prerequisites:
- Salesforce account with Enterprise access or API quota.
- Optional: Create a read-only user for Airbyte to maintain data integrity.
Creating a Read-Only User for Airbyte:
- Log in to Salesforce as an admin and go to Setup.
- Navigate to Administration > Users > Profiles.
- Create a new profile or edit an existing one to grant read permissions for Standard and Custom Objects.
- Assign this profile to the Airbyte user.
Airbyte’s Salesforce connector supports both Standard and Custom Objects, allowing import and individual handling of all available objects.
Step 1: Configure the Source
Log in to your Airbyte Cloud account, select the Salesforce connector under Sources, and configure it for data synchronization.
For example, you can configure the Salesforce connector to sync employee data from an HR system with Salesforce, ensuring that any updates in the HR system are reflected in Salesforce in real-time.
Step 2: Configure the Destination
Choose and set up your destination, where data will be stored securely, such as an external data store.
Step 3: Configure the Connection
Create a connection between source and destination, setting replication frequency to synchronize your Salesforce CDC data. It is important to consider transaction-based logic when setting replication frequency to ensure efficient handling of resource-intensive business processes.
How Do You Set Up Salesforce CDC Manually?
Salesforce’s streaming events API supports CDC for real-time synchronization between Salesforce and external systems.
Step 1: Enable CDC
In Setup, enable Change Data Capture for desired standard objects or custom objects (e.g., “product_item”) to sync Salesforce records in real time.
Step 2: Subscribe to Event Channel
Use Salesforce’s EMP Connector (on GitHub) to subscribe to CDC event channels and allow subscribers to receive change event messages. Replay past events with a replayID cursor to recover from outages.
Step 3: Build and Run EMP Connector
Build and run the EMP Connector jar to listen for events and verify integration.
It is crucial to ensure that the logic used in setting up Platform Events is correctly implemented to facilitate seamless communication and avoid issues such as recursion.
Step 4: Access Salesforce Events
Subscribe to the /data/ChangeEvents URL for all objects or use the format /data/objectChangeEvent for custom objects (e.g., /data/product_itemChangeEvent).
This setup enables accurate, real-time Salesforce data synchronization with external systems. It is crucial to handle encrypted data properly to ensure secure synchronization.
/data/ChangeEvent
For accessing change data events on custom objects the below format is to be used.
/data/__ChangeEvent
In our case, since our object product_item is a custom object, we will use the below URL
/data/product_item__ChangeEvent
Step 5: Subscribe to Events
To subscribe to events in the provided URL, open a shell and use the following command along with the JAR file.
java -jar target/emp-connector-0.0.1-SNAPSHOT-phat.jar /data/product_item__ChangeEvent
The user credentials for this sign-in are the username and password. An additional authentication option is to use an OAuth-based token. Please keep the command window open for the above shell to take additional actions, such as creating, updating, and deleting records.
Step 6: Create and Explore Object
After subscribing, create a new “product_item” object by filling in the “Product Name” and “Product Description” fields and saving. This ensures data stays synchronized, like syncing employee records with an HR system using CDC. Additionally, the synchronization process handles records that are created, updated, or deleted, ensuring comprehensive data consistency.
Step 7: Monitor Event Messages
Go to the terminal where the employee connector is running. You will find messages in the following format:
The changeType shows the change: CREATE, UPDATE, DELETE, or UNDELETE. Events can be replayed up to three days via replayID for data accuracy.
What Are the Benefits of Using a CDC Solution?
Change Data Capture (CDC) streamlines operations by syncing Salesforce data with external systems in real time, ensuring consistent and up-to-date records. Streamlined operations keep data current through timely change events, while Airbyte integration enables smooth data flow with custom connectors. CDC offers detailed change notifications for creates, updates, deletes, and undeletes, and customizable replication lets you select specific data sets or the entire database. Together, Salesforce CDC and Airbyte enhance data management and security with Shield Platform Encryption.
What Are the Limitations of the Custom-Code Approach?
Salesforce Data Capture is a powerful solution that utilizes change data capture to enhance data usage for businesses. Implementing Salesforce Change Data Capture (CDC) through custom code presents several challenges.
It requires deep Salesforce knowledge, custom programming for data sync, and faces limits like 1000 concurrent CometD clients and daily API rate caps. Airbyte’s Incremental Sync helps mitigate these issues. Users should be cautious with custom implementations to avoid potential issues.
How Does Salesforce CDC Compare to Platform Events?
Both CDC and Platform Events use streaming product technology for reliable messaging, but differ in key ways:
- CDC tracks changes in standard and custom Salesforce record fields automatically, while Platform Events require manual publishing via Apex or REST API.
- CDC events include detailed change data like record ID, change type, and modified fields; Platform Events contain only custom fields you define.
- CDC events are ordered and replayable for up to 24 hours; Platform Events are unordered and retained for 72 hours.
Frequently Asked Questions
What is Salesforce Change Data Capture (CDC)?
Salesforce CDC publishes change events whenever records are created, updated, deleted, or undeleted, letting external systems receive those changes in near real time without repeatedly polling the API.
How do you enable CDC in Salesforce?
In Salesforce Setup, go to Change Data Capture and select the objects you want to track. Salesforce then publishes change events for those objects to the event bus for subscribers to consume.
What is the difference between Salesforce CDC and Platform Events?
CDC automatically captures changes to Salesforce records, while Platform Events are custom, developer-defined events published explicitly by your apps. CDC is best for data replication, and Platform Events for custom app messaging.
How do you sync Salesforce CDC data to a warehouse?
Use a data integration tool like Airbyte to subscribe to Salesforce change events and load them into a destination such as Snowflake, BigQuery, or Redshift, without building and maintaining a custom EMP connector.
Suggested Reads:
Real-Time Sync Starts Here
Salesforce Change Data Capture empowers your team to stay ahead of every update — no polling, no guesswork. From pipeline updates to inventory shifts, CDC ensures your systems stay in sync with what’s happening in Salesforce, minute by minute. Completing the integration setup is crucial for achieving real-time synchronization.
With Airbyte’s 700+ pre-built connectors, you can automate CDC workflows across your entire stack. Set up incremental syncs, manage custom objects, and integrate with your data warehouse — all with a developer-friendly platform designed for scale.
Ready to unlock real-time Salesforce data? Start building with Airbyte today.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.