Datadog to Snowflake: How to Move Your Data
Move Datadog data into Snowflake with Airbyte. Warehouse cost settings, the number type default that affects cloud cost data, and the application key trap.
Moving Datadog to Snowflake is mostly a retention play. Datadog charges by how long it holds your logs and metrics, and those windows expire long before most audit and capacity-planning questions get asked. Snowflake stores the same data for a fraction of the cost and lets you join it against everything else in the warehouse.
The pipeline is straightforward. What deserves attention is Snowflake's billing model, which punishes careless scheduling, and one numeric default that will quietly approximate any values you care about.
What can you pull from Datadog?
The Datadog source exposes twelve fixed streams covering Logs, AuditLogs, Metrics, Series, Monitors, Incidents, IncidentTeams, ServiceLevelObjectives, SyntheticTests, Dashboards, Downtimes, and Users. Beyond those, the Queries setting lets you define custom streams against the metrics, cloud_cost, logs, and rum data sources, each becoming its own table.
Two behaviours to know. The query, start date, and end date filters only apply to Logs and AuditLogs, so they do nothing for the other streams. And the connector does not support namespaces, meaning everything lands in the schema you nominate. Our Datadog to BigQuery guide breaks down what each stream holds if you want the full inventory.
One credential detail causes more outages than anything else here. Datadog needs both an API key and an application key, and the application key inherits the permissions of whoever created it. A role change or an offboarded employee breaks the connection with an authorization error that looks nothing like a permissions problem. Create it under a service account.
Which Snowflake settings drive your bill?
Observability data arrives in high volume on a schedule, which makes this pipeline unusually sensitive to how the warehouse is configured. Four settings do most of the work.
The Snowflake destination docs include a setup script that creates a dedicated role, user, warehouse, database, and schema. Run it from a worksheet with the ACCOUNTADMIN role. If you want to avoid Fail-safe storage charges on log tables entirely, create a dedicated transient database for Airbyte rather than converting tables afterwards.
Why does the number type default matter?
Snowflake's Decimal Data Type option controls how the Airbyte number type is stored, and it defaults to FLOAT, an approximate binary floating-point type with about 15 digits of precision. The alternative, NUMBER(38,9), is exact.
For most observability metrics FLOAT is genuinely fine, since a latency percentile does not need exact decimal representation. The exception is cloud cost data. If you are pulling cloud_cost through a custom query and joining it to finance tables, approximate floats are the wrong representation and the discrepancies will be small enough to go unnoticed and large enough to matter. Set NUMBER(38,9) before the first sync, because converting an existing column can null or truncate values.
Snowflake also caps VARCHAR at 16 MB and VARIANT at 128 MB. Values over the limit are nulled and recorded in the _airbyte_meta column, which is worth querying occasionally as a data-quality check on verbose log payloads.
How do you set it up?
Create the Datadog API and application keys, then configure the source with any custom queries you need. On the Snowflake side, run the setup script, then choose authentication. Key pair is stronger than a password: generate an unencrypted PKCS#8 key with openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt and register the public key on the user.
Snowflake accepts connections from any IP unless an administrator has set a network policy. If one exists, add the Airbyte Cloud IP addresses to the allowlist. Then select streams, set a frequency, and run.
Scope your log queries before that first run. Datadog holds a great deal of log data, and a broad query with no start date will move far more than you intended and bill you on both sides for the privilege.
Frequently asked questions
Which Snowflake warehouse size should I use?
X-Small with a 60-second auto-suspend, as the setup script provisions. Snowflake bills per second and resumes the warehouse on every load, so minimising idle time matters far more than raw compute size for this workload.
Should I change the number type from FLOAT?
For latency and throughput metrics, FLOAT is fine. For cloud cost data joined against finance tables, switch to NUMBER(38,9) before the first sync, since converting an existing column risks nulling or truncating values.
Why did my connection suddenly fail with an authorization error?
Check who owns the Datadog application key. It carries that person's permissions, so a role change or an offboarding breaks the connection. Recreate it under a service account with a stable role.
Can I get Datadog cloud cost data into Snowflake?
Yes, through a custom query using cloud_cost as the data source. It is not one of the twelve fixed streams. Pair it with the exact number type so the figures survive the trip.
Why is my date filter ignored on some streams?
Those filters only apply to Logs and AuditLogs. Other streams ignore them, so control volume through stream selection instead.
Get your Datadog data into Snowflake
Size the warehouse small, keep Time Travel short on log tables, and scope your queries before the first run. Consolidating onto Google's warehouse instead? See our guide to Datadog to BigQuery. Replicating a database into the same warehouse? See PostgreSQL to Snowflake.
Integrate with 600+ apps using Airbyte
Move data from 600+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.