Article

Airbyte October Update

•

November 3, 2021

•

5 min read

Last month, we were heads down on launching Airbyte Cloud and we got a lot of other things done along the way. Let's go over what we've shipped in the past month!

Incremental Normalization

Airbyte now only applies basic normalization to new data synced with incremental sync. This will massively reduce costs and processing time, making basic normalization much more feasible to run in production environments.

Keep in mind that you may need to delete your old normalized tables to run the new version of normalization! Shout outs to users olivermeyer, mohammad-bolt, and andresbravog for supplying their implementation ideas/prototypes while developing this feature.

Airbyte Cloud

This month, we launched Airbyte Cloud into its Private Beta stage, where we are looking to perfect our offering and before releasing it to the public. Check out our pricing calculator here to see how much you can save by moving away from volume-based pricing!

OAuth

In Airbyte Cloud, we’ve made it possible to configure credentials for certain connectors with OAuth, making the setup process much easier.

Airbyte Hacktoberfest

Over the past month, we launched the first edition of our Hacktoberfest event, which brought the community together to create connectors for Airbyte open-source. Through this event, we have a total of 42 connectors (and counting!) being worked on by the community, which is an awesome step forward in solving the long tail of integrations.

Content

ETL Pipelines with Airflow

Ari Bajo Rouvinen, our content strategist, wrote an awesome article about the concerns with using Airflow for ETL/ELT, which made it to the front page of Hacker News and generated a lot of discussion about data engineering practices.

How to Build ETL Sources in Under 30 Minutes

Abhi Vaidyanatha, our developer advocate, made a video on how you can use our Python CDK to create an HTTP source very quickly. Don’t believe it? Check it out here.

Replicate Postgres data to Snowflake

We recently also published a recipe detailing how to transfer your OLTP data into Snowflake. Check it out here to learn how easy it is to migrate and how we can cover some special cases.

Community Content

Partition and cluster BigQuery tables with Airbyte and dbt - by Tuan Nguyen
Replicate Salesforce and Zendesk data to Keen for unified analytics - a collaboration with the Chargify team
Airbyte’s explosive growth through community-led approach - by Shalini Tekwani and Pramod Rao

Slack

We’re continuing our massive community growth and have gained about 500 new members in our community Slack over the past month, placing us at about 3600 members currently.

GitHub

Most notably, we have over 30 new PR contributors this month, bringing our unique contributor count to 154. We are up to 4.4k stars, from 4.1k earlier on in the month, continuing our steady growth.

Hiring

We are currently hiring for a ton of positions, including:
‍

Head of Product Marketing
Head of Community
Software Engineers (Connectors and Platform)
Community Managers
Developer Advocates
Technical Writers
Technical Product Managers
and more...

Up Next

This month, we are focusing on stabilizing Airbyte Cloud and implementing more OAuth workflows for connector configuration. Additionally, we are looking to add the functionality for automatic adaptation to source schema changes.

If you haven't already, join the Slack to stay up to date on our more frequent updates!

Limitless data movement with free Alpha and Beta connectors

Introducing: our Free Connector Program ->

The data movement infrastructure for the modern data teams.

Try a 14-day free trial