Airbyte’s Official API and Terraform Provider now in Open Source

In 2021, Airbyte released the documentation for an internal API known as the Configuration API, which was designed for communications between different Airbyte components. This API can be complex & confusing for users, as it was originally designed for internal use by the Airbyte team.

To address these issues, we built the Airbyte API, which provides a better developer experience and is specifically designed for the use cases of Airbyte users. The Airbyte API provides a polished & curated view of the Airbyte platform, making it easier to use and navigate. Earlier this year, we walked through some of the key benefits of the Airbyte API, which included releasing our Airbyte Terraform Provider. But until now, both API and Terraform Provider were only available on Airbyte Cloud. 

In May 2022, we also released an unofficial alpha Octavia CLI, as this was very much requested by the community. Eventually, the community started building their own unofficial Terraform providers, which were downloaded more than 15,000 times by the community. But all of these are not really maintained nor official.

All this changes today! We’re excited to announce that both our official Airbyte API and Terraform Provider are now available on Airbyte Open Source and Airbyte Enterprise, this makes them available to all Airbyte users. 

How to transition from the Configuration API to the Airbyte API

For users that are currently using the Configuration API, migrating to the Airbyte API should be relatively easy as many of the requests have a very similar set of required fields. The major difference between the Configuration API and the Airbyte API is that the Airbyte API provides a REST interface instead of the RPC style found in the Configuration API. Please consult the documentation for the Airbyte API here for more information on how to use the Airbyte API.

Sunsetting the Configuration API

We will be sunsetting official Airbyte support & official documentation for the Configuration API by early 2024. For any remaining API endpoints that still need to be replicated to the new Airbyte API, then this will happen between now and then. If there are Configuration API endpoints your team relies on, that you are not seeing equivalents for in the Airbyte API, we would love to hear from you. You can let us know by emailing riley@airbyte.io. 

What happens to Octavia CLI?

Similarly, we will most likely eventually sunset the unofficial alpha Octavia CLI too, in favor of the Terraform Provider. 

However, there is still some work that needs to be done to import resources from the API nicely. We will be working on those by the end of the year.

If you’re curious about the benefits of the Terraform Provider, here’s an article detailing them. It enables engineering teams to interface with Airbyte from the development tools they use daily. It's designed to streamline configuration changes and foster effortless collaboration, harnessing the power of "Infrastructure-as-Code" to save you time and minimize errors. Programmatically create new Airbyte connections or quickly test configuration changes. Airbyte resource settings & configurations can be version-controlled, reviewed, and tested, enabling teams to increase development speed and improve reliability, consistency, and scalability of their infrastructure.

What’s Next

If you are curious about the API or the Terraform Provider, you can join our team for a live demo on 08/31.

The Airbyte API and Terraform Provider provide ease of use and a great developer experience. They are quickly becoming a go-to tool for data teams thanks to their ability to manage large Airbyte deployments, their integration with other tools in the Modern Data Stack, and their ability to streamline ELT workflows. Many of our Cloud customers have already successfully integrated the Airbyte API into their DAG workflows, as well as their Powered-by-Airbyte workflows. 

Get started by checking the Airbyte API and Terraform Provider docs!

‍