Firebase Realtime Database to Snowflake: How to Move Your Data

Move Firebase Realtime Database to Snowflake with Airbyte. The key and value output shape, scoping the node path, service accounts, and modelling JSON in VARIANT.

Summarize with AI:

Moving Firebase to Snowflake takes application data out of a JSON tree built for mobile clients and puts it in a warehouse where analysts can query it with SQL. Firebase Realtime Database is excellent at serving an app and unhelpful for analytics, which is why this pipeline exists. Airbyte handles the extraction with no code.

One fact about the output shape determines how much work happens after the sync, and it is better known now than discovered later. The connector does not produce typed columns. It produces a key and a value, where the value is the node's contents serialised as JSON.

CapabilitySupportedWhat it means for this pipeline
Output shapeKey and valueThe record body arrives as JSON, not as typed columns
Node pathConfigurableDefaults to the root node, which is rarely what you want
AuthenticationService accountJSON keys only, with the Database Viewer role
Buffer sizeConfigurableControls how many records are fetched at a time
Snowflake VARIANT128 MB capOversized values are nulled and recorded in row metadata

What a record looks like

The Firebase Realtime Database source emits two fields per record. One is the key, the node's identifier in the tree. The other is the value, holding that node's contents as JSON.

So a node holding user records produces one row per user with the identifier in one column and everything else, every field the app writes, in the second. Firebase Realtime Database has no schema for the connector to read, so there is nothing from which to infer typed columns.

Snowflake handles this well, which is the good news. Its VARIANT type and the dot and colon notation for accessing JSON paths are genuinely pleasant to work with, and a view that extracts the fields you query into typed columns is straightforward to write. Do write it, though. Treat the landed table as raw and put a model between it and your dashboards, so that the day the app starts writing a new field you change one definition rather than every query.

Two Snowflake limits are worth knowing given that everything arrives as one payload: VARCHAR caps at 16 MB and VARIANT at 128 MB, with values beyond those nulled and the change recorded in the row's metadata column. A deeply nested Firebase node is unlikely to approach that, but a node holding an entire user's activity history might.

Scoping the node path

The Node Path setting selects which part of the tree to sync and defaults to the root node. That default is almost never right on a real application database.

Firebase trees usually hold everything the app touches: session state, presence data, caches, and denormalised copies of the same records stored several times over for read performance. Syncing from the root pulls all of it into Snowflake, where you pay to store and scan data nobody will query. Point the connector at the specific subtree carrying the entity you want, and configure separate connections for separate entities rather than one broad sync you filter afterwards.

A Buffer Size setting controls how many records are fetched at a time, which is the knob to reach for if syncs on a large node are struggling.

Credentials and Snowflake setup

You need a Google Cloud service account with the Firebase Realtime Database Viewer role. Airbyte recommends creating one exclusively for Airbyte, for permissioning and auditing. Create the service account, add it as a member in your project, grant the role, then create a key choosing JSON, since JSON keys are the only supported format. Download it at once, because Google shows the contents only then, and delete the file from your machine after configuring the source.

On the Snowflake destination, create dedicated entities rather than reusing existing ones: a role, user, warehouse, database, and schema, with the role holding OWNERSHIP on the database. Airbyte publishes a setup script. Use an X-Small warehouse with a sixty-second auto-suspend dedicated to syncs, since Snowflake bills per second and resumes the warehouse on every load.

The Decimal Data Type setting matters less here than on a typed source, because numbers arrive inside the JSON payload rather than as numeric columns. It still applies to anything Airbyte types as a number, so if you extract monetary values in a downstream model, cast them explicitly rather than relying on the default.

Frequently asked questions

Why is all my data in one column?

That is the connector's output shape: a key and a value, where the value holds the node's contents as JSON. Extract the fields you need into typed columns in a Snowflake view or model.

How do I sync only part of my database?

Set the Node Path to the subtree you want. It defaults to the root node, which pulls in session state, caches, and denormalised copies you probably do not want to store.

Which credentials does the connector need?

A Google Cloud service account with the Firebase Realtime Database Viewer role, and a key in JSON format. Other key formats are not supported.

What happens to a very large node?

Snowflake caps VARIANT at 128 MB and VARCHAR at 16 MB. Values beyond those are nulled and the change is recorded in the row's metadata column.

Get your Firebase data into Snowflake

Scope the node path narrowly, use a dedicated service account with a JSON key, and build an extraction model over the value column rather than querying JSON everywhere. For the same source into a different database, see our guide to Firebase Realtime Database to MySQL. For a relational source into the same warehouse, see PostgreSQL to Snowflake.

Start syncing now →

Integrate with 600+ apps using Airbyte

Move data from 600+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.