About Apostol Tegko
Apostol Tegko is a data analyst from London, tasked with identifying, analyzing, and interpreting trends or patterns in Fnatic’s complex data sets. He and his team solve complex customer-facing projects in collaboration with the company’s data engineering, data analytics, and machine learning teams. Before joining Fnatic in 2020, he worked in the data science space with Chip and OYO UK companies.
Apostol reveals how Fnatic efficiently streamlines its data integration processes to manage the data pipeline. In addition, he also shares the challenges encountered using external third-party resources and how the search for a solution led his team to discover Airbyte.
About Fnatic
A leader in esports with a history of more tier 1 winnings than any other team in history, Fnatic has been around since 2004. With a growing fanbase and a strategy to advance esports performance, we were the third most-watched esports brand in 2020. Fast Company ranked us 6th in its Most Innovative Gaming Companies 2021 list.
Our company has an engaged follower base across our social media platforms, and hundreds of millions of people watch our teams compete. Like most large-scale organizations, we employ many third-party applications to cater to the different sections of our business.
What was the problem?
Dependence on external infrastructure
My team ran into issues that relied on external infrastructure when using various tools and homegrown software to manage our data pipelines. We needed a change and wanted a way to reclaim our data pipeline from outside the VPC.
"A lot of our legacy infrastructure was handled externally, and we are looking to build it internally to reduce the reliance on our external agency and improve management of the pipelines. For example, our current Airbyte setup has a single destination, PostgreSQL, with Freshdesk, Google Sheets, and SFTP successfully syncing data in."
One of the significant limitations in our existing technology stack was:
Limited data transfers
Our data team’s original use case was the SFTP connector. Unfortunately, we used a machine learning tool with no API, and the data collected could be sent only to SFTP servers. Because of this limitation, our team struggled to find a data pipeline solution that could address this use case efficiently and cost-effectively.
"Initially, we tested this using the SFTP connector, but now we have connected Freshdesk, which we had previously used Stitch for. However, we wanted to save time when consolidating our disparate data, and Stitch did not perform well."
How did we discover Airbyte?
Given the staggering amount of third-party APIs in data silos, consolidating this data proved challenging in many cases. Hence, we were motivated to handle data integration internally and discovered Airbyte as a solution with its flexible, custom-built connectors.
How did Airbyte solve the problem?
Flexible and customizable connectors
Previously, whenever there was a requirement for a new connector, an external third party would handle the request and develop it. But with Airbyte's simple setup process and open-source model, we created the connectors internally. Additionally, the ability to customize our connectors saved about two to three hours a week due to the efficiency improvements.
"Where the Stitch connector failed to work, the Airbyte connector did."
We also found that using the Airbyte Connector Developer Kit (CDK) to develop a custom connector was uncomplicated.
"We first tried Airbyte with the publicly available Exchange Rate API, following the tutorial. Afterward, we used the CDK to build an SFTP connection to our Postgres database. It was very straightforward."
Easy data consolidation
After searching and trying out many external options, we found Airbyte's APIs to be the perfect solution to sync multiple data streams together, including Freshdesk, Google Sheets, SFTP, and more. Additionally, with Airbyte's APIs, our team could consolidate many of our UTM parameters generated from both influencer traffic and the programming team into one place.
How do we feel about using Airbyte?
Thanks to the advanced data connectors and the responsive team of experts at Airbyte, we have saved a lot of time and effort in optimizing our data integration procedures. With Airbyte's help and consistent support, our team is more than confident in taking on the world of performance esports!
"Airbyte is an up-and-coming solution for the single point of data integration. We continuously migrate our connectors from our existing solution to Airbyte as they become available on Airbyte, and look forward to the release of Airbyte Cloud!"
