Singlestore to Snowflake: How to Move Your Data
Move SingleStore into Snowflake with Airbyte. Why the lack of CDC matters on operational data, choosing cursor columns, and protecting the cluster during loads.
Moving SingleStore into Snowflake is an unusual request on its face, because SingleStore already runs analytical queries quickly. The reasons people do it are cost and company: memory-optimised storage is expensive to fill with history, and your other sources are not in SingleStore.
This guide covers the managed path with Airbyte. One connector limitation shapes the whole design and matters more here than on most databases, because SingleStore is usually holding data that changes constantly rather than an append-only log.
SingleStore to Snowflake at a glance:
Why move data from SingleStore to Snowflake?
Two situations account for most of these pipelines, and neither is that Snowflake queries faster.
The first is tiering by cost. SingleStore earns its price by keeping working data in memory, which is exactly the wrong economics for three years of history nobody queries weekly. Moving the cold tail to a warehouse lets the cluster stay sized for what it is good at.
The second is joining. Your CRM, your finance system and your support platform are not in SingleStore, and the questions people actually ask span all of them. A warehouse is where that meeting happens, and it is not a judgement on the database you are reading from.
What do you need before you start?
Short list, and the cursor question is the one that decides whether the result is trustworthy:
A dedicated read-only user. With access to every table you intend to replicate. Creating one for the pipeline rather than reusing an application account means its queries are identifiable when somebody asks what is consuming the cluster. The SingleStore source documentation lists the requirements.
A cursor column per table. There is no change data capture, so incremental sync depends on a column that only increases and is set on every write. An updated timestamp maintained by your application is the usual answer, and tables without one are limited to full refresh.
Network access, with SSL or an SSH tunnel. The connector supports both, which matters because a SingleStore cluster is usually inside a private network serving an application rather than exposed to the internet.
Snowflake objects and a role. A warehouse, database, schema and a role able to create tables in that schema. Size the warehouse generously for the initial load and scale it back afterwards, since the first sync is by far the heaviest.
Note also that this is a Marketplace connector at an early version, so alert on sync failures rather than assuming silence means success.
How do you build a SingleStore to Snowflake pipeline in Airbyte?
Step 1: Find your cursors and decide about deletes
Go through the tables you intend to replicate and identify a suitable cursor column on each. At the same time, ask whether rows are ever deleted or hard-updated in those tables, because the answer determines how much work the section below creates for you.
Step 2: Configure the SingleStore source
Click Sources in the left navigation, then New Source, and select SingleStore, following adding a source. Supply the host, port, database and read-only credentials, then configure SSL mode or an SSH tunnel according to how your cluster is reachable.
Step 3: Configure the Snowflake destination
Click Destinations, then New Destination, and select Snowflake, following adding a destination. Supply the account identifier, warehouse, database, schema and role with your authentication method. Airbyte creates one table per stream with its own metadata columns alongside your data.
Step 4: Nominate cursors and schedule off-peak
Click Connections, then New connection, select your tables, nominate a cursor field for each and choose an incremental sync mode. Schedule it when the cluster is least busy, because this is another client competing with whatever application SingleStore is serving.
Split your heaviest tables into their own connection so they do not hold up everything else, and so you can give them a different schedule if they turn out to need one.
Why does the missing CDC matter here specifically?
Because of what SingleStore is usually doing. The connector reads with a cursor rather than from a log, so a query filtered on a timestamp cannot observe an absence, and deletions never reach Snowflake. That is true of many sources and it bites harder on this one.
A warehouse-to-warehouse pipeline usually reads tables that transformation jobs rebuild wholesale, where deletion barely applies. SingleStore is typically holding live operational state behind an application, where rows are updated and removed constantly, so the gap between source and destination is real rather than theoretical.
Two remedies, and the first is much better. Ask your application team to soft-delete, marking rows with a flag and an updated timestamp rather than removing them, which makes deletions visible to a cursor. Where that is not possible, schedule a periodic full refresh on those specific tables and accept the load it costs.
How do you avoid competing with the application?
By remembering what the cluster is for. SingleStore is generally chosen because something needs fast reads and writes at the same time, which means a replication query pulling a large table is competing with the workload that justified the database in the first place.
So treat the initial load as a separate exercise from the ongoing sync. Run it during a genuinely quiet period, table by table if the volumes warrant it, rather than pointing everything at the cluster at once and discovering the effect on your application's response times.
Afterwards the ongoing syncs are far lighter, provided your cursors are doing their job. That is another reason to get them right at the start, since a table falling back to full refresh is a full read of operational data on every run rather than once.
Frequently asked questions
Does the SingleStore source support change data capture?
No. Incremental sync is cursor-based and append only, so deletions are not captured and updates are only seen when they advance the cursor column.
How do I get deletions into Snowflake?
Ask your application team to soft-delete with a flag and an updated timestamp, which a cursor can see. Otherwise schedule a periodic full refresh on the affected tables.
Which column should I use as the cursor?
A column that only increases and is set on every write, usually an updated timestamp your application maintains. Tables without one are limited to full refresh.
Can I connect to a private cluster?
Yes. The connector supports both SSL and SSH tunnel connections, which covers most SingleStore deployments since they usually sit inside a private network.
Can I do this without writing code?
Yes. The setup above is entirely UI-driven. The only code you might want is a soft-delete change in your application, which is a schema decision rather than pipeline work.
Get your SingleStore data into Snowflake
Find reliable cursor columns first, and settle the deletion question with your application team, because soft deletes are far cheaper than repeated full refreshes on operational tables. Then run the initial load when the cluster is quiet, split heavy tables into their own connection, and alert on failures.
Airbyte's connector catalog includes 700+ pre-built connectors, so operational data can sit beside the commercial sources that explain it. For a similar relational source, see MySQL to Snowflake, and for a warehouse to warehouse move, Amazon Redshift to Snowflake.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.