REST API to MySQL: How to Move Your Data

Getting data from a REST API into MySQL is the most common request in data engineering that has no single answer, because a REST API is not a system. It is a shape. What you build depends entirely on how the specific API handles authentication, pagination, and the question of what changed since last time.

One clarification first, because it costs people an afternoon. Airbyte has a connector named Public APIs, and it does not mean any public API. It syncs the catalogue at publicapis.org, a directory listing of free APIs, with two streams and no pagination. It is a useful toy and not what you want here. This guide covers the two real options for an arbitrary API, and how to land the result in MySQL.

REST API to MySQL at a glance:

Why move data from a REST API to MySQL?

Two situations account for most of these pipelines.

The first is a vendor or internal service with no maintained connector. You need its data alongside everything else, and the alternative is a scheduled script that somebody owns until they do not. A declaratively defined connector removes that ownership problem, because it lives in Airbyte alongside your other pipelines and behaves the same way.

The second is giving an existing application something it can read. Plenty of internal tools and admin panels speak MySQL and nothing else, so the API data has to land somewhere they can query. That is a different job from analytics, and it is why MySQL rather than a warehouse is the destination here.

What do you need before you start?

Most of this is reading the API's documentation carefully, which is the harder skill:

An authentication method you understand. Bearer token, API key in a header or query parameter, basic auth, or OAuth 2.0 with a refresh flow. Connector Builder supports these directly. If the API does something genuinely unusual, establish that now rather than halfway through.

The pagination scheme, named precisely. Page number, offset, or a cursor token returned in the response body or a Link header. Getting this wrong is the single most common reason a custom connector returns exactly one page and looks like it worked.

An answer on incremental filtering. Does any endpoint accept an updated_since parameter or an equivalent? If yes, you can configure a datetime cursor. If no, you are doing full refresh, and it is better to know that going in than to discover it when the vendor bills you for a daily full scan.

A MySQL user with write access. A database and a user able to create and write tables in the target schema. Airbyte adds its own metadata columns alongside your data, so decide whether the consuming application tolerates them or reads a view that excludes them.

If your organisation restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list on the MySQL side before you begin.

How do you build a REST API to MySQL pipeline in Airbyte?

Step 1: Check whether a connector already exists

Before building anything, search the catalogue for the service you are integrating. With 600+ connectors available, a maintained one covering your source is a real possibility, and a maintained connector handles the rate limits, schema changes and pagination quirks you would otherwise be discovering yourself. Only build when nothing fits.

Step 2: Explore with the HTTP Request source, or go straight to Builder

Airbyte ships a generic HTTP Request source. Point it at a URL, give it credentials, and it retrieves the response, which is the fastest way to answer whether the data is even useful. It is deliberately unsophisticated: no stream modelling, no managed schema, and incremental behaviour is whatever you construct around it. Use it to explore, not to run a pipeline for two years.

Step 3: Define the connector in Connector Builder

Connector Builder is the low-code interface for defining a real source connector. You describe the API declaratively and Airbyte generates something that behaves like any catalogue connector, including in scheduling and state management. Configure authentication, then pagination, then your streams, pointing Builder at the record array inside each response since most APIs wrap results in an envelope like data or results. The testing panel runs live requests as you configure, so validate pagination against real responses before publishing.

Step 4: Configure the MySQL destination and connect

Click Destinations, then New Destination, and select MySQL, following adding a destination. Then create the connection, select your streams and a sync mode for each, and set a frequency that respects whatever rate limit the API applies.

How do nested API responses land in MySQL?

API responses nest, and Airbyte preserves that structure rather than guessing at a flattening. In MySQL nested objects and arrays land as JSON rather than as columns, which is correct behaviour and mildly awkward to query.

Do the flattening in a downstream view rather than fighting the connector. Extract the specific fields your application reads into real columns, point it at the view, and keep the raw table. That way, when the vendor adds a field, you can reprocess history rather than starting from the day you noticed.

Watch identifier lengths while you do it. MySQL caps identifiers at 64 characters, and a flattened path through two levels of nesting reaches that faster than you expect. Alias columns to something short in your view rather than letting a generated name collide.

What happens when the API changes?

This is the question that decides whether building your own connector was the right call, and it deserves an honest answer: you own it. A maintained catalogue connector gets updated when a vendor changes their API. One you built in Connector Builder gets updated when you notice.

Two things reduce the pain. Because the connector is declarative rather than code, changes are usually a field edit in the Builder interface rather than a development task, which means someone other than its original author can fix it. And because it lives in your workspace alongside everything else, its failures surface in the same place as your other pipelines rather than in a cron log nobody reads.

If the connector turns out to be broadly useful, it can be contributed back to the Airbyte catalogue, at which point maintenance stops being solely yours. Worth considering for anything pointed at a well-known public API rather than an internal service.

Frequently asked questions

Is the Public APIs connector what I want?

Only if you specifically want data from publicapis.org. For your own API or a third party's, use Connector Builder.

Why does my custom connector return only one page?

Pagination is misconfigured. Check whether the API uses page numbers, offsets or a cursor token, and whether that token comes back in the response body or a Link header.

What if the API has no incremental filter?

Then full refresh is the honest answer. Reduce the frequency, and check whether a subset of endpoints can run incrementally even if the others cannot.

Who maintains a connector I build myself?

You do. Because it is declarative rather than code, fixes are usually a field edit rather than a development task, and useful connectors can be contributed back to the Airbyte catalogue.

Can I do this without writing code?

Yes. Connector Builder is a user interface, and the whole setup above is UI-driven. What you need is the ability to read the API's documentation, which is the harder skill.

Get your REST API data into MySQL

Check the catalogue first, because a maintained connector beats one you own. If you do have to build, the work is not the loading, it is reading the documentation closely enough to know the pagination scheme and whether the API will tell you what changed. Answer those two questions and Connector Builder handles the rest.

Airbyte's connector catalog includes 600+ pre-built connectors, so it is worth looking before you build. For the same approach pointed at other destinations, see REST API to PostgreSQL and REST API to BigQuery.

Start syncing now →