PostHog to Databricks: How to Move Your Data
Move PostHog into Databricks with Airbyte. Tuning events_time_step, organisation-wide rate limits, and landing nested event properties as a bronze layer.
Moving PostHog into Databricks puts product behaviour where it can be modelled rather than only reported. PostHog answers what users did extremely well and knows nothing about revenue, support cost or contract value, and the questions worth asking combine both.
This guide covers the managed path with Airbyte. The friction is entirely on the way out of PostHog, where rate limits apply across your whole organisation and only one stream syncs incrementally, and one setting decides whether your first sync finishes.
PostHog to Databricks at a glance:
Why move data from PostHog to Databricks?
Two situations account for most of these pipelines.
The first is modelling rather than reporting. Lifetime value prediction, churn modelling and cohort work that needs notebooks and Spark rather than a reporting interface. A lakehouse is where that happens, and event data is exactly the shape it handles well.
The second is joining product usage to revenue and support data under one governance model. If your questions are funnels and retention curves, PostHog already answers those and this pipeline earns you little.
What do you need before you start?
Short list, with one item that is a conversation rather than a credential:
A PostHog personal API key and a start date. If you self-host PostHog, change the base URL from the default to your instance. Self-hosted instances are not subject to PostHog Cloud's rate limits, though your own infrastructure becomes the constraint instead. The PostHog source documentation lists every field.
An agreed share of the rate limit. PostHog applies its limits across the organisation rather than per key, so your sync competes with every other tool and script on the account. Find out what else talks to PostHog before choosing a schedule.
An honest estimate of event volume. PostHog recommends batch exports for large or regular exports of events. If you are moving hundreds of millions of events, use those. If you are moving a few million plus cohorts, flags and persons, this connector is the right tool.
Unity Catalog access. Each stream becomes a table in Unity Catalog and the connector needs permission to create Volumes for Avro staging. Agree the catalog and schema before the first sync.
If your organisation restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list before you begin.
How do you build a PostHog to Databricks pipeline in Airbyte?
Step 1: Decide which streams you actually need
Only events is incremental. Projects, annotations, cohorts, feature flags, insights and persons are re-read in full on every sync. For the configuration objects that is cheap, since they number in the hundreds. Persons is the exception, and on a consumer product with a large identified user base it is genuinely large and re-read every time.
Step 2: Configure the PostHog source
Click Sources in the left navigation, then New Source, and select PostHog, following adding a source. Supply the API key, start date and base URL. Set events_time_step deliberately: it controls how many days of events each request covers, and it is the difference between a first sync that completes and one that dies partway through.
Step 3: Configure the Databricks destination
Click Destinations, then New Destination, and select Databricks, following adding a destination. Supply your workspace details, the catalog and schema, and authentication. Event properties are nested and the connector preserves that structure, which suits this destination because working with nested data in Spark is ordinary rather than awkward.
Step 4: Create the connection and schedule it
Click Connections, then New connection, select your streams and a sync mode for each. Use incremental on events. Run the backfill with a short events_time_step even if steady state would tolerate more, because the historical period is where the limits bite.
If the sync is slow after tuning events_time_step, look at persons next. Splitting it into its own connection on a slower schedule is a legitimate fix and usually the right one.
Why does the rate limit affect your whole organisation?
Because PostHog applies its limits at the organisation level rather than per API key. The analytics endpoints are capped at 240 requests per minute and 1,200 per hour, and every tool, script and personal key on the account draws from the same budget.
The practical effect is that a colleague running a script against the persons API can throttle your sync, and your backfill can throttle them. That makes the schedule an organisational decision rather than a technical one, and a large initial backfill something to plan rather than start on a whim.
Many teams end up running both approaches: PostHog batch exports for the event firehose, and this connector for cohorts, feature flags, insights and persons. That is a sensible split rather than a compromise, because the two mechanisms are good at different things.
How should you model event data in a lakehouse?
Treat the landed events table as a bronze layer. PostHog event properties are nested and the connector preserves that structure, which is an advantage here rather than a problem, because Spark works with nested data natively and you lose nothing by landing it as it arrived.
Build a silver table extracting the properties your team queries into typed columns, and keep bronze so you can reprocess history when the product team adds a property. On a source where new properties appear without warning, that raw layer is what saves you starting from the day you noticed.
Then think about partitioning, because events accumulate indefinitely and a table that is never partitioned becomes expensive to scan. Partitioning on the event date is the obvious choice and worth establishing while there is one model to change rather than a set of notebooks.
Frequently asked questions
Why is only the events stream incremental?
The other streams describe configuration rather than an append-only log, and re-reading them in full is cheap enough that incremental support was never added. Persons is the one to watch on large instances.
Does the PostHog rate limit apply per API key?
No. It is applied across your entire organisation, so other tools and scripts using different personal keys share the same budget as your sync.
My backfill keeps failing partway through. What should I change?
Reduce events_time_step so each request covers fewer days. Smaller slices keep individual requests within what the API returns and let the sync checkpoint more often.
Should I use this connector or PostHog's batch exports?
Batch exports for very high event volumes, which is PostHog's own recommendation. This connector for cohorts, feature flags, insights and persons, and for moderate event volumes.
Can I do this without writing code?
The Airbyte setup is entirely UI-driven. You will want Spark or SQL models extracting event properties and joining to your revenue data, which is where the analysis lives.
Get your PostHog data into Databricks
Tune events_time_step before anything else, agree your share of the organisation's rate limit, watch persons on large instances, and land events as bronze so the nested properties stay available for reprocessing when the product team adds a field.
Airbyte's connector catalog includes 600+ pre-built connectors, so product events can sit beside the revenue and support data that explain them. For the same source in a columnar engine, see PostHog to ClickHouse, and for another product analytics source in a lakehouse, Mixpanel to BigQuery.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.