How To Use dbt Defer To Optimize Your Data Workflow
Summarize this article with:
✨ AI Generated Summary
Long dbt runs usually come from one simple reality. Your project DAG is big, and every small change wants to pull half the graph along for the ride. That slows down local development, makes CI noisy, and burns warehouse compute.
dbt defer is built for this exact problem. It lets you run and test only the models you changed, while dbt resolves upstream ref() calls using artifacts from another environment, usually production.
TL;DR: How to Use dbt Defer at a Glance
- dbt defer speeds up development and CI by letting you run only the models you changed while resolving upstream dependencies from another environment, usually production.
- It works by pointing dbt at a production
manifest.json, soref()calls for upstream models use existing production relations instead of rebuilding them. - Defer is ideal for large DAGs where rebuilding upstream models slows local iteration and makes CI noisy and expensive.
- A typical workflow uses
--select state:modified+ --defer --state path/to/prod_stateto test changes without rebuilding the world. - Defer is best for fast iteration and CI; dbt clone is a better fit when you need physical tables in dev for downstream tools or deployment workflows.
What is dbt Defer?
Defer is a dbt feature you use with commands like dbt run, dbt test, or dbt build. When you defer, dbt can point upstream references to relations from a different environment, based on that environment’s manifest.json state artifacts.
This is most useful when your dev schema does not have every upstream model built, but production does.
What do You Need Before You Start?
You need a recent state directory from the environment you want to defer to. In practice, that means you need the artifacts from a successful production job, especially manifest.json. dbt uses that state for both state selection and deferral behavior.
At minimum, plan for:
- A path that contains
manifest.jsonfrom production - A dev target schema where you will build only the models you are changing
How to Use dbt Defer Step by Step?
Here’s the step-by-step process:
1. Capture Production State Artifacts
In dbt Cloud, your production job artifacts are already stored per run. In dbt Core, save the target/ artifacts from your production run somewhere your CI or developers can access.
What matters is that you can point --state at a folder that contains the production manifest.json.
2. Run Only what You Changed, and Defer Everything Else
A common workflow is:
- Select the models that changed
- Defer upstream parents to production
Example:
dbt build --select state:modified+ --defer --state path/to/prod_state
Why this works:
--select state:modified+tells dbt to run changed nodes plus their downstream children--defer --state ...tells dbt to resolve upstream refs against the environment described by the state manifest
3. Use Defer in CI for “slim” Runs
In CI, the usual goal is to test what changed without rebuilding the world. Defer is specifically well suited for CI workflows, while clone is often better for CD patterns.
A typical CI pattern looks like:
dbt build --select state:modified+ --defer --state path/to/prod_state
If you only want tests:
dbt test --select state:modified --defer --state path/to/prod_state
Note that state-based test selection can include tests that are new or modified, and tests that depend on modified resources.
4. Confirm What dbt Is Going to Run before You Run It
Before you spend compute, preview the selection:
dbt ls --select state:modified+ --state path/to/prod_state
This helps you catch “why is this included” moments early, especially because state comparison has real caveats depending on what changed and where.
5. Keep the State Fresh
Defer is only as accurate as the state artifacts you point to. If your production manifest is old, you can end up deferring to relations that no longer match what production actually has. That can lead to confusing behavior.
Make it a habit to:
- Fetch the latest production artifacts at the start of CI jobs
- Update the state location every time production deploys
dbt Cloud’s “defer to production” workflow is designed around always using production artifacts for comparison and reference resolution.
When Should You Use dbt clone Instead?
If you need actual physical objects in your dev schema so downstream tools can query them outside dbt, dbt clone may be the better fit. The dbt docs call out that clone creates additional warehouse objects, but enables use cases deferral cannot.
Rule of thumb from dbt:
- Defer is often the simpler fit for CI and fast iteration
- Clone is often the fit for CD patterns and blue green style deployments
What Usually Breaks, and How to Avoid It?
When working with dbt state and deferral, most issues come from a small set of predictable misconfigurations that are easy to overlook.
Most of these issues trace back to one thing: the state of your production data. If upstream sources are inconsistent or late, dbt defer will inherit those problems and make them harder to diagnose.
For teams managing many sources across multiple environments, talk to sales to see how Airbyte keeps ingestion reliable with CDC-based pipelines and predictable, capacity-based pricing.
Where Airbyte Fits
Defer makes dbt iteration faster, but it assumes production tables are trustworthy. That starts upstream, with reliable ingestion into your warehouse.
If you are rebuilding too much because upstream data is inconsistent or late, it is worth tightening the ingestion layer first. Airbyte is built for this part, with 600+ connectors and deployment options that let you keep data movement predictable as your stack grows.
Try Airbyte if you want to get clean source data into your warehouse fast, then let dbt defer speed up your dev and CI loop.
Frequently Asked Questions
What does dbt defer actually do?
dbt defer allows you to run only the models you changed while resolving upstream ref() calls against another environment, usually production. Instead of rebuilding upstream models in dev or CI, dbt uses metadata from a production manifest.json to reference existing relations.
Can I use dbt defer with incremental models?
Yes. Incremental models work with defer as long as the upstream relations exist in the deferred environment. dbt will still apply incremental logic to the models you run locally, while upstream dependencies are resolved from production.
Does dbt defer skip tests as well?
Defer does not automatically skip tests. Tests follow the same selection logic as models. If a test depends on a deferred upstream model, dbt will reference the production relation, but the test itself will still run unless you explicitly exclude it.
What happens if my production state file is outdated?
If the state artifacts are stale, dbt may defer to relations that no longer match production reality. This can lead to confusing behavior or false confidence in results. Always use the most recent production manifest.json, especially in CI workflows.
.webp)
