Firebase Realtime Database to DynamoDB: How to Move Your Data
Move Firebase Realtime Database into DynamoDB with Airbyte. Reconciling a JSON tree with a key-value model, why key design is irreversible, and handling deletes.
Moving Firebase Realtime Database into DynamoDB is usually a migration rather than an integration. Both are schemaless stores built for fetching data by key at scale, so the question is rarely what the data will look like and usually how you get from one cloud's model to another's.
This guide covers the managed path with Airbyte. Two shapes have to be reconciled: Firebase is one deep JSON tree addressed by path, while DynamoDB is a flat collection of items addressed by a partition key you choose in advance and cannot change.
Firebase Realtime Database to DynamoDB at a glance:
Why move data from Firebase to DynamoDB?
Two situations account for most of these pipelines, and neither is analytics.
The first is a migration in progress. An application is moving from Firebase onto AWS, and the pipeline keeps DynamoDB current while traffic shifts across, so the two can run side by side rather than requiring a single cutover.
The second is serving an AWS-hosted service with data your mobile app already writes to Firebase, without giving that service Firebase credentials. If you want to analyse the data rather than serve it, send Firebase to a warehouse or lakehouse instead.
What do you need before you start?
Two of these are decisions that cannot be revisited once the table exists:
A decision about which node to sync. Realtime Database is one deeply nested JSON tree, and the node path you give the connector determines what each record contains. Choosing the wrong level produces either enormous items or fragments that mean nothing on their own.
A decision about your key design. DynamoDB items are addressed by a partition key and optionally a sort key, and those are what your application looks up by. Work out that access pattern before creating the table, because it is not something you change later.
A Google service account and AWS credentials. A service account with read access to the Firebase database, and either AWS keys or an IAM role with write access to the target table. On self-managed Airbyte running in AWS, a role avoids long-lived keys in configuration.
Raised write capacity for the initial load. The default provisioned capacity is low, and a bulk load will exhaust it and be throttled, which presents as a slow sync rather than a clear error. Raise it before the first sync or use on-demand capacity.
Finally, know what your application writes into that node, because neither system will tell you. On a migration this matters twice over, since both the old and new shapes have to be understood.
How do you build a Firebase to DynamoDB pipeline in Airbyte?
Step 1: Reconcile the two shapes
Decide which Firebase node becomes your items, then decide which field within those items becomes the partition key. Those two choices together determine everything the table can do, and they are the whole design of this pipeline rather than a preliminary to it.
Step 2: Configure the Firebase source
Click Sources in the left navigation, then New Source, and select Firebase Realtime Database, following adding a source. Supply the database name, your service account credentials and the node path you decided on.
Step 3: Configure the DynamoDB destination
Click Destinations, then New Destination, and select DynamoDB, following adding a destination. Supply the region, table name prefix and credentials. Confirm the capacity settings on the target table now rather than during the first load.
Step 4: Choose a sync mode that is safe for a live table
Click Connections, then New connection, select your stream and a sync mode. Overwrite is destructive, so if an application is reading the table while a sync runs it will see an incomplete picture. Where you need a full reload, write to a new table, verify it, then repoint the application.
On a migration that pattern is doubly useful, because it gives you a tested table to switch traffic onto and an untouched previous version to fall back to.
Why is the key design the whole design?
Because Firebase and DynamoDB are both schemaless and address data completely differently. In Firebase you navigate a path down a tree; in DynamoDB you supply a partition key and get an item. Nothing about the source tells you what that key should be, so it is a judgement you make.
The judgement is constrained by how records arrive. The destination writes each record as a JSON document rather than mapping fields onto DynamoDB attributes individually, so a global secondary index, which needs an attribute, cannot be built on a field buried inside that document.
So every query your application needs has to be served by the partition and sort keys, because there is no indexing your way out afterwards. On a migration that means enumerating the reads your Firebase code performs today and checking each one can be satisfied by the key design before you create the table.
What happens to deleted records?
They stay in DynamoDB. The connector has no change data capture, so a record your application removed from Firebase remains in the destination table indefinitely, and nothing reports the discrepancy.
During a migration that matters more than it might elsewhere, because the two stores are meant to agree and users may be served by either. A table quietly accumulating records that no longer exist in the source is a divergence that grows every day and is invisible until somebody compares counts.
The cleanest fix is in your application: write a deleted flag rather than removing the node, and filter on it when reading. Where you cannot change the application, a periodic full refresh using the load-then-repoint pattern rebuilds the table and reflects removals, at the cost of a complete re-read each time.
Frequently asked questions
How do I decide what an item should be?
By the node path you give the connector. Pick the node whose children are the entities your application fetches, so each child becomes one item.
Can I build a secondary index on a field from Firebase?
No. Records land as a JSON document rather than individual attributes, and a global secondary index needs an attribute. Satisfy the access pattern through your key design instead.
Are deletions in Firebase reflected in DynamoDB?
No. There is no change data capture, so use a deleted flag in your application and filter on it, or rebuild the table periodically.
Why is my initial load so slow?
Almost certainly write capacity throttling. The default provisioned capacity is low for a bulk load, so raise it before the first sync or use on-demand capacity.
Can I do this without writing code?
The Airbyte setup is entirely UI-driven. Your application code will need to read the JSON structure rather than individual attributes, which is worth knowing before anyone builds against it.
Get your Firebase data into DynamoDB
Settle the node path and the key design together, because between them they decide everything the table can do and neither can be changed afterwards. Raise write capacity before the first load, use load-then-repoint for rebuilds, and handle deletions with a flag rather than hoping the pipeline notices them.
Airbyte's connector catalog includes 600+ pre-built connectors, so application data can reach a key-value store and an analytical one at once. For the analytical version, see Firebase Realtime Database to BigQuery, and for another source feeding the same destination, PostgreSQL to DynamoDB.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.