Iterable to BigQuery: How to Move Your Data

Moving Iterable into BigQuery lets you judge campaigns against outcomes rather than engagement. Opens and clicks are available in Iterable; whether a flow produced customers who bought again needs revenue data Iterable has never seen, and that join only happens in a warehouse.

This guide covers the managed path with Airbyte. Two things shape the build: one API sits behind a limit far tighter than the rest and governs your sync time, and the connector makes two deliberate choices on your behalf that are worth understanding.

Iterable to BigQuery at a glance:

Why move data from Iterable to BigQuery?

Two situations account for most of these pipelines.

The first is attribution you own. Iterable will report what it believes a campaign achieved, and checking that against your order and retention data is a separate exercise that needs both datasets in one place with definitions you control.

The second is keeping event history beyond what the interface makes convenient, and joining users to product usage. If your questions are heavy aggregations over very large event volumes where query latency is the point, a column store answers those better than a warehouse will.

What do you need before you start?

Four things, and the second causes a failure that looks like something else entirely:

An Iterable API key. Created in your Iterable project. The Iterable source documentation covers the setup and is unusually candid about the connector's behaviour.

The region that matches your key. The connector calls a different base address per region, and a mismatch presents as an authentication error, which sends people rotating a perfectly good key.

Patience for the event streams. Iterable's export API allows four requests per minute per project, and every event stream depends on it, which makes a large backfill a matter of days rather than hours.

A BigQuery dataset in the right location. Location is fixed at creation and BigQuery will not join across locations, so put this where your order and product data already live.

If your network restricts traffic by IP, add the Airbyte Cloud IP addresses to the relevant allow list before you begin.

How do you build an Iterable to BigQuery pipeline in Airbyte?

Step 1: Match the region to the key that issued it

Confirm which Iterable data centre issued your API key and set the region accordingly, because this is the mistake that wastes an afternoon. A mismatch produces authentication errors rather than anything mentioning regions, so the natural response is to assume the key is wrong and generate another, which fails identically. It is also worth rechecking if a previously working configuration suddenly starts failing this way.

Step 2: Configure the Iterable source

Click Sources in the left navigation, then New Source, and select Iterable, following adding a source. Supply the API key, region and start date. Select streams knowing that the event streams behave quite differently from the rest, and that the lookback window is adjustable from its default of five minutes.

Step 3: Configure the BigQuery destination

Click Destinations, then New Destination, and select BigQuery, following adding a destination. Supply the project identifier, dataset and service account credentials. Event volumes can be large on an active sending account, which is one of the cases where Cloud Storage staging is worth considering over batched inserts.

Step 4: Create the connection and check what arrived

Click Connections, then New connection, select your streams and a sync mode. Then compare the list membership counts that arrived against what Iterable reports, because a successful sync does not mean a complete one here, for reasons covered below.

Then have reporting read views that filter on the extraction timestamp partition, since BigQuery bills on bytes scanned and event tables grow quickly.

Why is the export API the whole timeline?

Because the two APIs behind this connector are metered nothing like each other. Most Iterable endpoints allow a hundred requests per second per project, which is generous enough to ignore. The export API allows four per minute, and every event stream depends on it: email, push, SMS, in-app, web push, inbox, purchase and custom events all come through that door.

The connector handles this as well as anything could, adapting the size of its date range slices to stay within the limit rather than hammering the API and backing off. That is the right behaviour and it does not change the arithmetic: on a project with years of sending history, the event backfill is measured in days and there is no setting that shortens it.

So plan around it rather than against it. Set a start date covering the history you genuinely need rather than everything ever sent, start the pipeline well before the deadline that prompted it, and tell whoever is waiting that the configuration streams will populate almost immediately while the event streams take considerably longer. Steady state is comfortable; it is the first pass that costs.

What does the connector decide on your behalf?

Two things, and they pull in opposite directions. The first protects you: Iterable's export API is eventually consistent, so recent events may not appear immediately, and the connector subtracts a lookback window from the end of each sync window to avoid missing them, then removes the duplicates that overlap creates. Five minutes by default, and adjustable if your project runs further behind.

The second keeps the sync alive at a cost. When fetching users per list, a list that returns a server error is skipped and processing continues, so one misbehaving list cannot block everything else. That is a sensible trade and it means a sync can report success while quietly missing the membership of whichever lists failed that day.

Which is why the check matters. Compare list membership counts in BigQuery against what Iterable shows, at least after the first sync and ideally on a schedule, because this is the rare pipeline where completeness and success are genuinely different questions. A gap that appears one day and fills the next is the expected behaviour rather than a fault, and knowing that is the difference between a shrug and an investigation.

Frequently asked questions

My credentials are correct but authentication fails.

Check the region, since the connector calls a different base address per data centre and a mismatch surfaces as an authentication error rather than a regional one.

Why are event streams so much slower than the rest?

They use the export API, which allows four requests per minute per project, while other endpoints allow a hundred per second. The connector adapts its slice sizes and cannot change the limit.

Some list memberships look incomplete.

Lists returning server errors are skipped so the sync can continue. Compare counts against Iterable, and expect gaps that appear one day and fill the next.

What is the lookback window for?

The export API is eventually consistent, so the connector holds back a few minutes to avoid missing recent events, then filters the duplicates that overlap produces.

Can I do this without writing code?

The pipeline, yes. The views joining campaign activity to your own revenue data, and the completeness checks, are SQL worth writing early.

Get your Iterable data into BigQuery

Match the region to the data centre that issued your key, because the failure looks like bad credentials and sends people down the wrong path. Expect the event streams to dominate your timeline, since the export API allows four requests per minute and nothing changes that. Then check list membership counts against Iterable, because skipped lists keep the sync running and a successful run here is not the same as a complete one.

Airbyte's connector catalog includes 600+ pre-built connectors, so messaging can be judged against the revenue it claims. For another messaging platform into the same destination, see Customer Io to BigQuery, and for product analytics into the same destination, Mixpanel to BigQuery.

Start syncing now →