About Arnaud Coutin
Arnaud Coutin is the growth lead at Chance. He tests data extraction tools to find the most efficient and cost-effective options in the market. Arnaud has worked on projects requiring expertise in back-end development, product management, and growth in the past four years. Before joining Chance, he was a pre-sales product manager at Orange France.
Our conversation with Arnaud highlights how his company developed the most advanced career guidance system while managing an extensive data pipeline. He and his team tested various tools for data extraction and migration like Stitch in the past but found them ineffective for their needs. Airbyte has empowered his team to be more productive and data-driven by streamlining data movement from multiple CRM platforms into a centralized data warehouse.
About Chance
Chance has created a digital coaching program for users to choose their career path, climb the ladder of professional success, and achieve social mobility. Through virtual coaching, real-life mentoring, and collective dynamics, our company helps users discover what they love doing. We have grown significantly since our launch seven years ago and today use several B2B and B2C CRM databases to power our business processes.
What was the problem?
Consolidating data sources in a single data warehouse
Our business relies on various data sources, with Pipedrive being our primary B2C and B2B CRM system. Together with Typeform and Calendly, our team can track user journeys and maintain our community of coaches. To follow the user's actions on the website or within the product for event transmission, we use Segment. Additionally, we use Facebook and Google Advertising to run our ad campaigns. So, to streamline our business operations, it was vital to bring data from various sources into a central warehouse.
After trying different data tools, we realized that our old architecture was complicated, inefficient, and rigid. Some of the significant problems with it were:
Low quality and unsupported data integrations
Initially, we integrated Stitch into our technical architecture for extracting and synchronizing data from multiple data warehouses. However, Stitch did not offer native support for Typeform tap integration, and its support team could not ensure the quality or stability of the integration.
"When we tried to use Stitch to solve our problem, we found that Stitch maintains some integration taps but not all. Stitch did not maintain the Typeform tap integration, so the quality of the integration was quite low and didn't meet our requirements."
Single destination data transfers
Many of our use cases required data transfers to multiple data warehouse locations. However, Stitch only allows data transfers to a single destination, limiting the scalability of our use cases.
"We wanted to synchronize data from our warehouse to other databases, but with Stitch, you can only have one destination. So we can only push in one data warehouse. That's a problem because we don't want to always push data into the same warehouse."
Unreliable and slow data synchronization
Since Stitch couldn’t meet our data integration needs, our team decided to plug in Singer and use its webhooks to synchronize data in real-time. However, it was time-consuming to move all our data into the data warehouse and reconciling it using the webhook data. Additionally, since the webhook had a heavy data payload, there were multiple gaps in our data transfers. For example, the webhook transmitted data from 3:00 to 3:30 but did not transmit data from 3:30 to 4:00 when the synchronization period was every half hour.
"We tried using Singer's webhook integrations, but we ended up having to synchronize data every 8 hours, and even then, we were losing half the information. Due to this, there were gaps in data, and Singer’s support team couldn't help us debug this issue."
How did we discover Airbyte?
An increasing customer base and demand for coaching services forced us to rethink our infrastructure. In searching for a tool to help streamline data extraction and migration, we discovered Airbyte as a solution. There were initial concerns about adding yet another technology to our stack — our team wanted to minimize the risk of having too many data extraction technologies. Nevertheless, we gave Airbyte a try and have not looked back since.
How was the problem solved?
Painless set-up process
It was straightforward to set up and utilize Airbyte. In contrast to Stitch, we can now set up the Typeform integration in just minutes and sync data to multiple destinations. In addition, Airbyte has enabled us to streamline our data extraction processes and maximize efficiency.
"At some point, we decided to use Airbyte. We did a Docker pull, and it worked perfectly out of the box. Then, on Amazon EC2, we followed the script, and everything was running with Airbyte within 15 minutes. It's almost like magic. It syncs data every day, and we don't encounter any issues."
Frictionless open source innovation
Our team can advance digital innovation with minimal friction and maximum transparency, thanks to Airbyte's open-source model. We also don't have to pay for every row of data synced using Airbyte, opening more cases. For example, you can easily create your self-hosted Airbyte environment for a marketing project, analyze data, and get creative without worrying about incurring additional costs.
"We conducted a market study on Fivetran, Stitch, and Airbyte and fell in love with Airbyte's vision and community. Using Airbyte gives you the freedom to self-host and do whatever you want with it - and for free! It's so exciting to see that you're shipping stuff every week, and it's also exciting to see what is getting done. Watching the constant stream of updates on the Slack channel motivates us to try and implement the newly available integrations."
How do we feel about using Airbyte?
Our company uses proprietary augmented skills assessment technology to develop a digital coaching journey that sets everyone on the path to a fulfilling career. Airbyte is a critical component within our technical architecture, used to extract and process information from different data sources. As we grow, we are confident in Airbyte's capabilities to power various use cases. For example, Airbyte was initially used only for Typeform integration but can also work for use cases such as PostgreSQL to PostgreSQL synchronization.
"As we continue to grow daily, we need new, custom connectors to help us enrich our marketing automation flow. For instance, we may need connectors for Livestorm, the browser-based online web conferencing software, to extract customer profile data from the platform and push it back to Pipedrive to help manage our mailing list using Airbyte connectors."
