MySQL to Snowflake: How to Replicate Your Data

Moving data from MySQL to Snowflake means getting operational rows into a warehouse built for analytics, then keeping them current without hammering the production database. A mysqldump handles the first copy. It does not handle the second one, it re-reads everything each time, and it never tells you what was deleted.

This guide covers the managed path: replicating MySQL tables into Snowflake with Airbyte's certified connectors, using change data capture so inserts, updates and deletes all arrive. It also covers the one constraint that causes most of the trouble people report, which is a database setting rather than anything in Airbyte.

MySQL to Snowflake at a glance:

Why move data from MySQL to Snowflake?

Two situations account for most of these pipelines.

The first is analytical load. MySQL is serving the application well, and then the analytics team starts running joins across the largest tables during business hours. Replicating into Snowflake moves that work onto separate compute, so a slow dashboard query stops being an incident for the people using your product. It also gives analysts the window functions and the query performance that a transactional database was never built to provide.

The second is consolidation. MySQL is one of several systems that need to be joined together, alongside a CRM, a payment processor and an events pipeline. Snowflake becomes the place where those meet, and MySQL becomes one source among many. That framing matters when you choose sync modes, because a table nobody will join is a table you can sync less often.

What do you need before you start?

The MySQL source connector has prerequisites that stop the pipeline at setup rather than at runtime, and one that stops it a week later:

Binary logging enabled. Change data capture reads the MySQL binary log, so logging has to be on. On a self-managed server this is a configuration file change, and most cloud providers expose it as a single option.

Enough binlog retention. Around ten days, expressed as 864000 seconds, is the figure commonly used for an Airbyte pipeline. On Amazon RDS this is a different setting entirely, called binlog retention hours, and it defaults to zero, which purges logs immediately and makes CDC impossible until you change it.

A dedicated read-only user. Give it replication permissions rather than reusing an application account, so access can be revoked without collateral damage and so the source of a query is obvious in the process list.

Encryption. The connector requires it by default and the connection fails if the source will not encrypt. Verifying the certificate authority is the stricter option and worth using where you control the certificates. Full details are in the MySQL source documentation.

On the Snowflake side you need a warehouse, a database, a schema and a role with permission to create tables in that schema. If your organisation restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list on the MySQL side before you begin.

How do you build a MySQL to Snowflake pipeline in Airbyte?

Step 1: Prepare the MySQL server

Enable binary logging, set retention generously, and create the read-only replication user. Do this before anything else, because a connection configured against a server with no binlog will pass its credential test and then fail in a way that looks like a connector problem. On RDS, remember that the retention setting is applied through an RDS-specific stored procedure rather than a parameter group.

Step 2: Configure the MySQL source

In Airbyte, click Sources in the left navigation, then New Source, and select MySQL. Supply the host, port, database, credentials and SSL mode, and choose Read Changes using Binary Log as the replication method. Airbyte tests the connection immediately and then takes you to the new connection page. Full setup steps are covered in adding a source.

Step 3: Configure the Snowflake destination

Click Destinations, then New Destination, and select Snowflake. Supply the account identifier, warehouse, database, schema and role, along with your authentication method. Airbyte creates one table per stream and adds its own metadata columns alongside your data, so tell whoever reviews the schema that those are expected. The same flow is described in adding a destination.

Step 4: Create the connection and choose sync modes

Click Connections, then New connection, and pick the source and destination you just created. Airbyte fetches the schema, and you select the tables you want and a sync mode per stream. Incremental append and deduped mirrors current state in MySQL, which is what most entity tables want. Plain incremental append keeps every version Airbyte ever saw, which is what you want when someone has to prove what a row looked like six months ago. Set a replication frequency, save, and trigger the first sync.

One scheduling note that belongs in a calendar rather than a checklist. The initial snapshot assumes no schema changes happen while it runs, so coordinate with whoever ships migrations before starting a large first sync. It is also worth setting the connection to require manual approval of schema changes, so a new table cannot enter a running sync without the snapshot it needs.

Why does my connection keep running full refreshes?

This is the symptom that brings most people to the troubleshooting docs. Normally the first sync is a full snapshot and every sync after it is incremental. If you are seeing repeated full refreshes, open the logs and look for a message saying the saved offset is no longer present on the server and that Airbyte is going to trigger a sync from scratch. MySQL purged the binlog past the point Airbyte needed, so there was nothing to resume from.

There are two causes and they want different fixes. The first is a busy database generating binlog faster than your retention window covers the gap between syncs, which you solve by syncing more often, extending retention, or both. Syncing more often is usually cheaper, because retained binlog costs disk on the source.

The second is specific to large databases and considerably nastier. If the initial snapshot itself takes longer than your retention period, the binlog position recorded at the start has already expired by the time it finishes. The connector cannot resume, so it starts over, takes just as long, and fails identically. Nothing about that loop looks like an error, because each run reports as a successful full refresh. Size retention against your snapshot duration, not against your sync interval.

When is CDC the wrong choice?

There is a second replication method, a cursor-based standard mode that reads a column such as updated_at, and Airbyte recommends against it for good reasons. A cursor cannot see deletions. It cannot see an update that leaves the cursor column untouched. And it depends on your application maintaining that column on every write path, including ones written years ago by someone who has left.

It earns its place in two situations. If your server does not expose the binlog at all, cursor mode is the only option available. And if the dataset is small and append-only, a cursor avoids the retention configuration and the DBA conversation entirely, at no real cost, because there are no deletes to miss.

One more consideration specific to MyISAM. The snapshot avoids table locks on InnoDB, but MyISAM tables are still locked while they are read. Legacy schemas often retain a few, and they tend to be exactly the logging or audit tables something writes to constantly. Find out before you start rather than during.

Frequently asked questions

Can I replicate MySQL to Snowflake in real time?

Not strictly in real time, but close. Airbyte reads the binary log to capture inserts, updates and deletes incrementally, and you control how often the connection runs. Latency is a function of your sync frequency rather than of re-scanning tables.

What binlog retention should I set on Amazon RDS?

At least 24 hours, applied through the RDS-specific configuration procedure rather than a parameter group. The default is zero, which purges binary logs immediately and makes CDC impossible.

Are deletes propagated to Snowflake?

Yes, when the stream uses CDC. Deleted rows arrive carrying a deletion timestamp, and in append and deduped mode they stop appearing in the final table. Cursor-based replication cannot capture deletions at all.

Does the initial snapshot lock my tables?

Not for InnoDB tables, where writes from other clients continue normally. Tables using the MyISAM engine are locked, so check whether your schema still contains any.

Can I do this without writing code?

Yes. The setup above is entirely UI-driven, from creating the source through to scheduling the connection. If you would rather manage pipelines as infrastructure, the same source, destination and connection can be provisioned from configuration files instead.

Get your MySQL data into Snowflake

You can have a MySQL to Snowflake connection running before the end of the day, with CDC, deletes and schema handling covered. Set binlog retention generously first, because that single setting accounts for most of the trouble people report, and freeze schema changes while the initial snapshot runs.

Airbyte's connector catalog includes 600+ pre-built connectors, so the same MySQL source can fan out to a lakehouse or a streaming platform later without rebuilding anything. For the equivalent decisions on the other major open source database, see PostgreSQL to Databricks, and if Snowflake sits alongside another warehouse, BigQuery to Snowflake covers moving between them.

Start syncing now →