Why Airbyte Made Alpha and Beta Connectors Free
Today, Airbyte launched the Free Connector Program, making all alpha and beta connectors on Airbyte Cloud free of charge. Explicitly, this means any data pipeline with an alpha or beta source or destination will be free to use.
We did this because we believe that our users should only pay for connectors with the highest reliability and to increase the usage of data connectors in order to increase the velocity of improvements. This blog post is covering these points in detail.
But first, let’s go over what alpha, beta and GA mean for Airbyte connectors.
The different status of Airbyte Connectors
Airbyte defines Generally Available (GA) connector reliability as a >99% sync success rate with more than 50 users. It also has to spend less than 24 hours in a failed state, among several other criterias.
You might wonder about reliability for alpha and beta connectors. If they are free, will they be stable enough? To date, alpha connectors have a 90% sync success rate, and beta connectors have a 93% sync success rate. And with our low-code CDK, these rates are increasing each month due to ease of use with our CDK and its templatized code.
So why would Airbyte give access to these connectors for free? It goes back to our mission of commoditizing data integration.
Why giving free access to alpha and beta connectors
Addressing the long tail through our community and better tooling.
We understood early on that it would be easier to build a long tail of connectors than to maintain them and keep them at a GA state with >99% of successful syncs. So we switched our focus to building better tooling for our community so the new connectors contributed would have easier maintenance.
This is how the low-code config-based Connector Development Kit project was born. Making it a config-based kit enables maintenance to be much easier. Any API change will take you 2 minutes to fix.
Combining the better tooling with a growing community of contributors will bring more and more connectors and get us to cover the long tail of potential sources and destinations. That is why the community has always been a first-class citizen at Airbyte. In the end, we doubled the number of contributors from 300 to 600 in 2022. And, in the last year, the number of connectors almost doubled from 170 to 300+. The % of connectors contributed by the community grew from 50% to 65%.
But tooling and community will only bring a long tail of alpha connectors. How do we get them all to GA?
How free helps on bringing reliability
The main thing missing to bring them all to GA is usage. Indeed, there will always be edge cases to implement. The best way to identify and manage them is to have a lot of usage by many users with different use cases. This is where the Free Connector Program comes in. More users for all those alpha and beta connectors means we can increase the improvement velocity.
Also, we need 50+ users of all connectors to bring them to GA too!
This is how Airbyte will be able to provide thousands of GA connectors and to commoditize data integration overall.
But the Free Connector Program is more than this!
Creating a reliability standard across the industry
The Free Connector Program is a message to our community of users that we are strongly incentivized to bring reliability to all the connectors they use. It is about showing them how important reliability is to us. Airbyte will only thrive as a business if reliability is at the forefront of everything we do.
But it goes even beyond this. The Free Connector Program is Airbyte’s statement that any connector with a <99% success rate shouldn’t be charged for. Data pipelines are at the bottom of the data infrastructure Maslow pyramid along with data storage. If data pipelines break, it prevents the whole organization from leveraging this data, should it be for analytics, operational use case, or data activation (reverse-ETL). ELT pipelines are the beginning of anything data related in the organization.
Conclusion
So to summarize, Airbyte’s mission to commoditize data integration by bringing reliability at scale to a long tail of connectors can only be achieved by getting 3 steps of a flywheel right:
- Community: a great community of data engineering contributors to help build the connectors they need
- CDK: some great tooling to help with building and maintaining connectors
- Free Connector Program: usage across all connectors to help identify edge cases and bring those connectors to GA
If this resonates with you, sign in the Free Connector Program, use our alpha / beta connectors for free, and file any issues you see. This is how you can contribute to our mission to commoditize data integration once and for all! Obviously, we hope the connectors will work seamlessly for your use case, so Airbyte keeps providing value and simplifying your life.
We also thought about this program as a way to give back to our community during a difficult economical context.
