ServiceNow to BigQuery: How to Move Your Data

Moving ServiceNow into BigQuery puts your IT estate where it can be counted and joined to everything else the business spends money on. ServiceNow holds the authoritative record of what hardware you run and what software you are licensed for, and neither reconciles against procurement without leaving the platform.

One thing to establish immediately, because it decides whether this pipeline is what you want. The connector covers the Configuration Management Database and Software Asset Management applications. It does not carry incidents, changes, problems or requests, so if you came here for ticket analytics, the section below on scope will save you an afternoon.

ServiceNow to BigQuery at a glance:

Why move data from ServiceNow to BigQuery?

Two situations fit this connector well, and both are about assets rather than tickets.

The first is reconciling what you own against what you pay for. Licence entitlements in ServiceNow, invoices in your finance system and cloud spend in billing exports only meet in a warehouse, and that comparison is usually where the savings are found.

The second is infrastructure history. The CMDB describes your estate as it is now, and a warehouse accumulating dated snapshots tells you how it changed, which servers appeared during a migration and what quietly persisted after a decommissioning project was declared finished.

What do you need before you start?

The first item involves a commercial conversation rather than a configuration screen:

An Airbyte enterprise agreement covering this connector. ServiceNow is a premium enterprise connector available at an additional cost, so it is not something you switch on from the catalogue. The ServiceNow source documentation sets out what it covers.

Your instance base URL and a service account. The connector authenticates with a username and password, and that account must have access to every table you want included. Its permissions define your dataset, so create one for the pipeline rather than borrowing an administrator's.

Confirmation that the streams answer your question. Check the published stream list against what you actually need before committing, because the coverage is narrower than the ServiceNow name suggests and the connector is still incubating at an early version.

A BigQuery service account and a decided dataset location. With the BigQuery User and BigQuery Data Editor roles. Queries can only reference datasets in the same physical location and that cannot be changed after creation, so match it to wherever your billing and procurement data already lives.

If your organisation restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list before you begin.

How do you build a ServiceNow to BigQuery pipeline in Airbyte?

Step 1: Match the streams to your question

Write down what somebody asked you for, then read the stream list. If the answer involves servers, network devices, printers or licence entitlements, this connector covers it. If it involves tickets, approvals or resolution times, it does not, and no configuration changes that.

Step 2: Configure the ServiceNow source

Click Sources in the left navigation, then New Source, and select ServiceNow, following adding a source. Enter your instance as the base URL, then the username and password of the service account. Run the connection test and review which streams appear.

Step 3: Configure the BigQuery destination

Click Destinations, then New Destination, and select BigQuery, following adding a destination. Supply the project, dataset and service account key. Batched standard inserts is the default and is ample, because an estate inventory is thousands of rows rather than millions.

Step 4: Choose append if you want history

Click Connections, then New connection, and choose a sync mode. Only full refresh is available, so the real decision is overwrite for a mirror of the current estate, or append to accumulate a snapshot per sync. Daily is generous for an inventory that changes when somebody provisions a server.

If you choose append, the extraction timestamp becomes both your snapshot date and the partitioning column, which is a useful coincidence covered below.

What is actually in these streams?

Two application areas. The Configuration Management Database streams describe infrastructure as configuration items: Linux, Windows, AIX and Solaris servers, ESX hosts, routers, switches, firewalls, load balancers, printers, scanners, storage switches and chassis hardware. The Software Asset Management streams cover licences, product lifecycle records and model categories.

What is absent is the half of ServiceNow most people picture. There are no incident records, no change requests, no problems and no service catalogue items, so questions about mean time to resolution, change failure rate or ticket volume by team cannot be answered from this dataset.

If those are your questions, Connector Builder against ServiceNow's Table API is the route, since the API exposes far more than this connector currently wraps. That is more work than selecting a connector and considerably less than discovering the gap after promising somebody a dashboard.

Why does partitioning matter on an inventory this small?

Because append mode changes the arithmetic. A single snapshot of a few thousand configuration items is trivial, and a daily snapshot kept for three years is a thousand copies of it, which is the point at which BigQuery billing on bytes scanned starts to notice.

The convenient part is that output tables are partitioned daily on the extraction timestamp, and with full refresh plus append that timestamp is exactly your snapshot date. So the column you naturally filter on to get a particular day's estate is already the partitioning column, and the pruning happens without you designing for it.

Make that explicit for whoever queries the tables. A question about the current estate should filter to the latest snapshot rather than scanning every day ever collected, and saying so once in your documentation prevents a year of unnecessarily expensive queries.

Frequently asked questions

Can I sync incidents and change requests?

Not with this connector. It covers Configuration Management Database and Software Asset Management streams only. For ITSM records, build against ServiceNow's Table API in Connector Builder.

Is the ServiceNow connector included in my plan?

It is a premium enterprise connector available at an additional cost, so it needs a commercial conversation rather than a configuration change.

Can I change the dataset location later?

No. It is fixed at creation, and queries can only reference datasets in the same physical location, so match it to the billing and procurement data you intend to join.

How do I keep query costs down with daily snapshots?

Filter on the extraction timestamp, which is the partitioning column and also your snapshot date. A question about the current estate should read the latest snapshot rather than every day collected.

Can I do this without writing code?

Yes. The setup above is entirely UI-driven. You will want BigQuery views joining assets to procurement and billing data, which is where the reconciliation lives.

Get your ServiceNow data into BigQuery

Check the streams against your question before anything else, because the coverage is narrower than the name suggests. Create the dataset in the right location since that is permanent, choose append so you accumulate an estate history, and tell your analysts to filter on the partitioning column.

Airbyte's connector catalog includes 700+ pre-built connectors, so asset data can sit beside the identity and infrastructure sources that complete the picture. For the same source in a lakehouse, see ServiceNow to Databricks, and for software estate on the workspace side, Google Workspace Admin to BigQuery.

Start syncing now →