TL;DR Short answer: Datadog data splits into two jobs. Monitors, SLOs, incidents, metrics queries and cloud costs are small enough to sync through the API with an ELT tool. High-volume logs should leave through Datadog's own archives or a telemetry pipeline, because the API is too slow and rate-limited for bulk export. Seven options:
Airbyte Datadog Log Archives Datadog Observability Pipelines Vector Cribl Stream Steampipe Custom code with the Datadog API The Datadog constraint: logs are only kept for your indexed retention period and metrics roll up to coarser granularity over time, so anything you need long-term must be exported before Datadog discards or aggregates it. Datadog ETL tools move observability data, such as monitors, incidents, SLOs, metrics, logs and cloud costs, out of Datadog and into a warehouse, lake or SIEM. Teams do it to keep history beyond Datadog's retention, to cut log costs, and to join reliability and cost data with product, finance and customer data.
Why move Datadog data into a warehouse? Datadog is built for real-time monitoring, not long-term analysis. Its retention windows are short and priced by volume, and its data is hard to join with the rest of the business. Exporting it answers questions like: how did uptime and SLO attainment trend over the year, which services drive cloud cost growth, how do incidents correlate with churn or support tickets, and what must be kept for compliance after Datadog's retention ends.
How do the Datadog ETL tools compare? Tool Best for Data covered Deployment Pricing model Airbyte Warehouse analytics Monitors, SLOs, incidents, metrics, logs, RUM, cloud cost Cloud, self-hosted, Enterprise Flex Free self-hosted; Cloud volume-based; Pro and Flex capacity-based Log Archives Bulk log retention All ingested logs Native to Datadog No Datadog charge; cloud storage costs Observability Pipelines Pre-ingestion log routing Logs before Datadog Workers in your environment Datadog subscription Vector Open-source routing Logs and metrics Self-hosted Free, open source Cribl Stream Vendor-neutral routing Logs, metrics and traces Cloud or self-hosted Free tier, then data volume Steampipe SQL queries over the API Configuration and API objects Self-hosted CLI Free, open source Custom code Specific endpoints Anything the API exposes Wherever you run it Free, engineering time
Which Datadog ETL tools should you consider? Seven options, from API-based ELT to log pipelines:
1. Airbyte Airbyte is an open-source ELT platform with 700+ connectors. Its Datadog source syncs twelve streams, including monitors, service level objectives, incidents, dashboards, downtimes, synthetic tests, users, logs and audit logs. You can also define custom query streams over metrics, cloud cost, logs and RUM data in Datadog's own query syntax, and incremental sync keeps runs small.
It is the right fit for reliability and cost analytics in a warehouse, where Datadog data sits next to product, finance and customer data. For multi-terabyte log export, pair it with Log Archives rather than pulling logs through the API.
Pros Cons Twelve streams plus custom queries Not suited to bulk log export through the API Cloud cost and RUM data alongside metrics Bound by Datadog API rate limits Self-hosted, Cloud or Enterprise Flex Audit logs require the right application key access
2. Datadog Log Archives Log Archives forward every log Datadog ingests, including logs you choose not to index, into your own S3, GCS or Azure Blob storage as compressed JSON files. It is the most reliable way to keep complete, long-term log history at storage prices, and archived logs can be loaded into a warehouse or lake, or rehydrated back into Datadog when needed.
Pros Cons Complete log history at storage prices Logs only, with no metrics or configuration Includes logs that were never indexed Files need a loader to reach a warehouse Rehydration back into Datadog when needed Archive format is Datadog-specific JSON
3. Datadog Observability Pipelines Observability Pipelines runs workers in your own environment that collect, filter, transform and route logs before they reach Datadog. Teams use it to redact sensitive data, drop noisy logs, and send copies to cheaper storage or other tools, which reduces Datadog cost as well as moving data out.
Pros Cons Control logs before ingestion and billing Requires a Datadog subscription Redaction and filtering in your environment Workers to deploy and operate Route to multiple destinations at once Focused on logs rather than analytics data
4. Vector Vector is an open-source observability data pipeline maintained by Datadog. It collects, transforms and routes logs and metrics to a wide range of sinks, including object storage, Kafka, ClickHouse and Elasticsearch, and it is known for high throughput and a small footprint.
Pros Cons Free, open source and very fast You deploy, scale and monitor it Many sources and sinks Configuration as code, with no UI Powerful transformation language Needs engineering time to tune
5. Cribl Stream Cribl Stream is a vendor-neutral telemetry pipeline that can receive data from the Datadog Agent, then reduce, enrich and route it to object storage, data lakes, SIEMs or back to Datadog. Security and platform teams use it to control observability costs and keep long-term copies outside any single vendor.
Pros Cons Vendor-neutral routing and reduction Commercial beyond the free tier Strong UI for building pipelines Another platform to operate Many destinations, including lakes and SIEMs Best value at larger data volumes
6. Steampipe Steampipe is an open-source tool that exposes APIs as SQL tables, and its Datadog plugin lets you query monitors, dashboards, users, logs and other objects with SQL. It suits audits, inventory reports and lightweight exports, rather than scheduled, high-volume pipelines.
Pros Cons Query Datadog with plain SQL Not a scheduled pipeline by itself Free and open source Limited by API rate limits Joins Datadog with other cloud APIs Best for small, ad hoc extracts
7. Custom code with the Datadog API Datadog's official client libraries for Python, Go, Java, TypeScript and Ruby cover every API endpoint. Custom code fits narrow needs, such as exporting a specific metric at a set granularity, but you own pagination, rate limits, retries and maintenance as the API evolves.
Pros Cons Full control over endpoints and granularity You own rate limits, retries and scheduling Official clients in several languages Ongoing maintenance No licence cost Slow for bulk log export
What should you check before exporting Datadog data? Datadog exports fail most often on keys, regions and volume. Check these first:
Site and region: Datadog runs separate sites, such as US1, US3, US5, EU1 and AP1. The tool must point at your site, and a mismatch looks like an authentication error.API key and application key: the API key identifies your organisation, while the application key carries the permissions of whoever created it. Create the application key under a service account so it does not break when someone leaves.Log volume: the logs API is paginated and rate-limited, which makes bulk export slow and fragile. Use Log Archives or a telemetry pipeline for full log history, and the API for targeted queries.Retention and granularity: logs are only kept for your indexed retention period, and metric data rolls up to coarser intervals over time. Export what you need on a schedule, before it expires or loses detail.Sensitive data: logs often contain personal data, tokens and internal identifiers. Redact them in the pipeline or restrict access in the destination.Cost: exporting is cheaper than retaining in Datadog, but query costs, storage and egress still apply. Narrow queries and archive filters keep exports focused.Datadog's API and export features cover several kinds of data:
Monitors, downtimes and incidents: alert definitions, states and incident history, used for reliability reporting. Service level objectives: SLO definitions and attainment over time. Metrics: timeseries queried at chosen intervals, from infrastructure, APM and custom metrics. Logs and audit logs: application, infrastructure and security logs, plus a record of changes in Datadog itself. Real user monitoring: front-end performance and session data from RUM. Cloud cost: cost and usage data from Datadog Cloud Cost Management, useful for FinOps reporting. How do you start pulling data from Datadog? If you decide to test Airbyte, setting up a Datadog pipeline takes three steps.
Step 1: Set up Datadog as a source In Datadog, create an API key and an application key, ideally under a service account. In Airbyte, open Sources, choose New source and select Datadog , enter both keys, choose your Datadog site, and optionally add custom queries for the metrics, logs, RUM or cloud cost data you need.
Step 2: Set up a destination Pick where your Datadog data should land: a warehouse such as Snowflake, BigQuery or Databricks, a database, or another Airbyte destination.
Step 3: Configure the connection Select the streams you need, such as monitors, SLOs, incidents and your custom queries, use incremental sync where supported, and schedule runs to stay within Datadog's API rate limits. The process is the same whether you deploy Airbyte Open Source or start with Airbyte Cloud .
Which Datadog ETL tool should you choose? Most teams need two routes. For reliability, cost and usage analytics in a warehouse, use Airbyte to sync monitors, SLOs, incidents, metrics queries and cloud cost data next to the rest of the business. For complete log history, turn on Log Archives, or put Observability Pipelines, Vector or Cribl Stream in front of Datadog if you also want to cut ingestion costs. Steampipe and custom code cover audits and narrow, specific exports.
For a wider view of the market, see our guide to the top data integration tools .
Sync Datadog to your warehouse with Airbyte Replicate Datadog monitors, SLOs, incidents, metrics, RUM and cloud cost queries incrementally, next to 700+ other sources, self-hosted, on Airbyte Cloud or with Enterprise Flex.
Try Airbyte Cloud free for 14 days or talk to our team .
Frequently Asked Questions (FAQs) 1. How do you export Datadog logs for long-term retention? Turn on Datadog Log Archives, which forward all ingested logs to your own S3, GCS or Azure Blob bucket. From there you can load them into a warehouse or lake, or rehydrate them into Datadog when needed.
2. Why does my Datadog connection fail authentication with valid keys? Usually because the connection points at the wrong Datadog site. Datadog runs separate regional sites, so set the site that matches your account, and supply both an API key and an application key.
3. Can you export Datadog metrics to Snowflake or BigQuery? Yes. ELT tools such as Airbyte query Datadog metrics through the API at the interval you choose and load them into Snowflake, BigQuery or another warehouse. Export regularly, because older metrics roll up to coarser granularity.
4. How long does Datadog keep logs and metrics? Logs are kept for the indexed retention period on your plan, often days or weeks, while metrics are kept for longer at progressively coarser granularity. Export anything you need beyond that window.
Suggested reads Data Observability Tools
Elasticsearch ETL Tools
Kafka ETL Tools
Data Lake Tools
.webp)