Looker to BigQuery: How to Move Your Data

Moving Looker into BigQuery gives you a record of your reporting estate and, unusually for a business intelligence connector, the output of specific reports. Most of what arrives describes how Looker is configured; one stream actually runs something.

This guide covers the managed path with Airbyte. Two things shape the build: that results stream is opt-in and named by identifier, and everything it returns arrives as text regardless of what it actually is.

Looker to BigQuery at a glance:

Why move data from Looker to BigQuery?

Two situations account for most of these pipelines.

The first is governing the reporting estate, since dashboards and Looks accumulate and nobody can say which are maintained. The second is capturing the output of particular reports so a number somebody quotes has a record behind it.

What this is not is a replacement for modelling your underlying data, since a Look's output is already aggregated. If several systems need to react to Looker activity rather than analyse it, Looker to Kafka covers that shape instead.

What do you need before you start?

Four things, and the second is the decision that distinguishes this connector:

An API3 key and your domain. The client identifier and secret are generated per Looker user, so the key carries that person's access. The Looker source documentation explains how to create one.

A list of Look identifiers, if you want results. The Run Look stream executes only the Looks you name, so this is where you decide whether this pipeline carries report output at all.

A BigQuery dataset in the right location. Location is fixed at creation and BigQuery will not join across locations, so put this where the data you intend to compare against already lives.

A plan for casting. Everything a Look returns arrives as text, so any arithmetic on those figures needs a view standing between the table and the analyst.

If your Looker instance restricts traffic by IP, add the Airbyte Cloud IP addresses to the allow list before you begin.

How do you build a Looker to BigQuery pipeline in Airbyte?

Step 1: Decide which Looks are worth running

Pick the handful of Looks whose output somebody genuinely needs recorded, and collect their identifiers, because naming them is the only way their results reach this pipeline. Reports quoted in board packs or used as an agreed figure are good candidates. Everything else is better rebuilt from the underlying data, since a Look's output is a summary somebody chose rather than something you can take apart.

Step 2: Configure the Looker source

Click Sources in the left navigation, then New Source, and select Looker, following adding a source. Supply the domain, client identifier and secret, plus any Look identifiers you want run. Check those identifiers exist before saving, since a wrong one is not caught gracefully.

Step 3: Configure the BigQuery destination

Click Destinations, then New Destination, and select BigQuery, following adding a destination. Supply the project identifier, dataset and service account credentials. Volumes are modest, so batched standard inserts are ample and staging is unnecessary.

Step 4: Create the connection and choose append

Click Connections, then New connection, select your streams and a sync mode. Only full refresh is available, and appending is what turns repeated runs into a record of how a figure moved rather than only what it says today.

Then write the casting views, because nothing useful happens with these figures while they are text.

Why does one stream behave differently?

Because it does something the others do not: it runs a query. The bulk of this connector describes your Looker instance, listing dashboards, Looks, folders, users, permissions and scheduled plans, which is a catalogue in the same way most business intelligence connectors are.

The Run Look stream is the exception, executing the Looks whose identifiers you supply and returning their output. That is unusual and genuinely useful, because it means a report's numbers can be captured rather than only its definition, which most tools in this category will not do.

It is also opt-in and deliberately narrow. You are naming reports by identifier, which means somebody has to maintain that list as Looks are renamed, rebuilt or retired, and a Look deleted in Looker becomes a stream that stops returning anything. Treat the identifier list as configuration with an owner rather than as something set once.

Why is every result column text?

Because the types are not available to take. Looker's API returns the values a Look produced without describing what they are, so the connector treats every column as a string rather than guessing, which is the honest choice and an inconvenient one.

The consequence is that a revenue figure, a count and a date all arrive as text, and nothing about the table hints otherwise. Somebody summing a column gets an error or, worse, a comparison that sorts lexicographically and looks plausible, which is the kind of wrong answer that survives review.

So put a view between the table and everybody else, casting each column to what it actually is and naming it clearly. Use safe casting rather than a plain conversion, since a Look can return a placeholder or a formatted value where you expected a number, and have reporting read the view rather than the landing table so the rules live in one place.

Frequently asked questions

Can I get the data behind my dashboards?

Only through the Run Look stream, for Looks you name by identifier. Everything else describes how Looker is configured rather than what it returns.

Why can I not sum a column?

Look results arrive as strings, because types are not available from the API. Cast them in a view, using safe casting for anything that might be formatted.

What happens if a Look identifier is wrong?

It is not handled gracefully, so confirm each identifier exists before saving the source rather than after the schema page misbehaves.

Why is part of my instance missing?

The API3 key is specific to a Looker user, so the dataset reflects that person's access. Use an account whose visibility matches the question you are answering.

Can I do this without writing code?

The pipeline, yes. The casting views are SQL and they are what makes report output usable rather than merely stored.

Get your Looker data into BigQuery

Decide which Looks are worth running and collect their identifiers, since that list is the only route to report output and it needs an owner as Looks change. Note whose API key you used, because its access bounds the dataset. Then append rather than overwrite so figures gain a history, and cast every result column in a view before anybody tries arithmetic on text.

Airbyte's connector catalog includes 600+ pre-built connectors, so a reporting estate can be governed alongside the numbers it produces. For the same source into a warehouse with different controls, see Looker to Snowflake, and for another business intelligence catalogue into the same destination, Metabase to BigQuery.

Start syncing now →