Google Search Console to MySQL: How to Move Your Data
Move Google Search Console into MySQL with Airbyte. Why fresh data and incremental sync conflict, site URL formats, and how dimensions decide your row counts.
Moving Google Search Console into MySQL puts organic search performance where an application or an internal tool can query it. A client reporting portal, a content dashboard or an SEO tool that needs impressions and rankings is far simpler to build against a database it already connects to than against Google's API.
This guide covers the managed path with Airbyte. Two settings interact in a way that decides whether your stored figures are ever correct, and the combination that causes trouble is the one people reach for when they want fresher data.
Google Search Console to MySQL at a glance:
Why move data from Google Search Console to MySQL?
Two situations account for most of these pipelines.
The first is powering a reporting application. An agency portal showing clients their organic performance, or an internal content tool, reads from a database it already connects to rather than calling Google on every page load and managing quota of its own.
The second is keeping history past Search Console's own window, which is limited, and joining it to your content records. If you want multi-year analysis across many properties, a warehouse is the better destination for reasons of volume.
What do you need before you start?
One of these is a decision rather than a credential, and it is the one that matters:
Your site URLs, in the right format. Search Console distinguishes URL prefix properties, written as a full address, from domain properties, written with an sc-domain prefix. Supplying the wrong form for your property returns nothing. The Google Search Console source documentation shows both.
OAuth credentials or a service account. Authenticate directly on Airbyte Cloud, or supply a client identifier, secret and refresh token. Whichever account you use must already have access to the properties in Search Console.
A decision about data freshness. The default of final returns only data Google has processed and confirmed. The alternative includes fresh figures that are not yet finalised and may change, which interacts badly with incremental sync for the reason below.
A MySQL user with write access. On the target schema, ideally one dedicated to synced data. Airbyte adds its own metadata columns, so plan to expose views to your application rather than the raw tables.
If your organisation restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list on the MySQL side before you begin.
How do you build a Search Console to MySQL pipeline in Airbyte?
Step 1: Decide your reporting grain
Work out which dimensions your application actually displays, because that decides both your custom reports and your row counts. Query and page together across a large site produces an enormous number of rows, and on a relational destination with a volume guideline that is the choice that matters most.
Step 2: Configure the Search Console source
Click Sources in the left navigation, then New Source, and select Google Search Console, following adding a source. Supply your site URLs, authenticate, set a start date, and leave data freshness on final. Define any custom reports here with a name and the dimensions you settled in step 1.
Step 3: Configure the MySQL destination
Click Destinations, then New Destination, and select MySQL, following adding a destination. Supply the host, port, database and credentials. Watch identifier lengths, since MySQL allows 64 characters and custom report names combined with dimension names can produce long table and column names.
Step 4: Create the connection and tune threads if needed
Click Connections, then New connection, select your streams and an incremental sync mode. The connector defaults to forty worker threads, and if you sync a large number of site URLs or start seeing rate limit errors, reducing that number is the documented remedy.
If a search analytics stream fails with a 400 error mentioning an unsupported aggregation type, there is a setting that forces the automatic aggregation type, which is the documented fix rather than something to debug.
Why should fresh data and incremental sync not be combined?
Because they make opposite assumptions. Fresh data is Google's provisional view, not yet processed and verified, and it changes as the figures settle. Incremental sync reads what is new since last time and does not revisit, which is exactly the wrong behaviour for numbers that are still moving.
Put them together and your database stores the provisional version of every day permanently. Google eventually finalises those figures and your tables never learn about it, so your application shows numbers that disagree with Search Console and nothing in the pipeline suggests why. Airbyte's documentation recommends leaving freshness on final for precisely this reason.
The temptation is real, because final data lags and somebody always wants yesterday's numbers. The honest answer is that yesterday's numbers do not exist yet in a form worth storing, and a portal showing a recent figure that later changes is worse than one showing nothing for a day or two.
How do custom reports decide your row counts?
By the dimensions you choose, and with one addition you do not control. Date is included in every query whether you specify it or not, so your grain is always at least daily, and listing it explicitly only changes the order results are grouped in.
Each additional dimension multiplies the rows. Date alone is small, date plus country is manageable, and date plus query plus page on a site with substantial organic traffic produces a table that will test the volume guidance for a relational destination within months.
So build the reports your application displays rather than everything that might be interesting. The primary key is formed from your dimensions plus date, the site URL and the search type, which is worth knowing when you write the views your product reads from.
Frequently asked questions
Should I enable fresh data to get more recent figures?
Not with incremental sync. Fresh data is provisional and changes, while incremental does not revisit, so you would store the unfinalised version permanently. Leave it on final.
My site URL returns nothing. What is wrong?
Probably the format. A URL prefix property is written as a full address, while a domain property needs the sc-domain prefix, and they are not interchangeable.
I am getting rate limit errors. What should I change?
Reduce the number of concurrent threads, which defaults to forty. That is the documented remedy when syncing many site URLs or hitting limits.
Do I need to include date in my custom report dimensions?
No. Date is included whether you specify it or not, and listing it only changes the grouping order of the results.
Can I do this without writing code?
Yes. The setup above is entirely UI-driven. You will want MySQL views hiding the metadata columns so your application reads a stable interface.
Get your Search Console data into MySQL
Leave data freshness on final if you sync incrementally, because storing provisional figures permanently is the failure that damages trust in a reporting product. Get the site URL format right, choose dimensions against what your application displays, and expose views rather than raw tables.
Airbyte's connector catalog includes 600+ pre-built connectors, so organic search can sit beside the paid channels you compare it against. For paid search, see Google Ads to BigQuery, and for organic video, YouTube Analytics to BigQuery.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.