Ink-credible Data People: Airbyte OSS Contributor Daniel Diamond

September 28, 2022

6 min read

Summarize with AI:

To continue our community appreciation blog series, we're bringing you our next ink-credible data person: Daniel Diamond, an Airbyte open source contributor based out of Melbourne Australia, who is currently leading a multidisciplinary data platform team at 99designs and fostering a data-driven culture across the organization. Keep reading for some tips on submitting your first PR, a few book recommendations and a pic of his favorite watch.

How/when did you first discover Airbyte? What was your Aha moment?

I try to keep up with the ever-evolving data space and in particular, the open source community. I first heard about Airbyte through the dbt slack community at the end of 2020. This was roughly the same time that I launched a large-scale initiative to integrate the data platform from one acquired company into another. We needed a DMS alternative, we needed connectors for 3rd party data sources and we had to move a lot of data. There were various ELT tools out there, however Airbyte’s open-source strategy, long-term vision and particularly their interest in the CDC approach was sufficient to consider.

What is your top tip for using Airbyte that others may not know?

Airbyte leverages debezium under the hood, which means you can implement CDC incremental syncing capabilities. A far more efficient approach than the standard approach.

What have you worked on in the past six months that you’re particularly proud of?

Specifically relating to Airbyte, I’ve helped extend the Octavia CLI project, which allows users to leverage IaC when working with Airbyte. Instead of updating your Airbyte connections manually via the UI, users can now import all existing sources, destinations and connections and update them programmatically. Users can deploy version-controlled yaml definitions of their Airbyte instances through CI/CD.

Favorite Airbyte slack channel:

Probably #good-reads. The data space has been evolving rapidly and so having a tether to the latest and greatest ideas or projects helps one feel like it’s not entirely impossible to keep up.

We can’t thank you enough for your contributions. Why did you decide to contribute? Why is open source important to you?

Contributing to Airbyte has been a win-win strategy. As I run into additional requirements or desirable features, I have the freedom to explore and extend the product and have it reviewed by the team.

Open source fosters a community of like-minded and passionate individuals. For the individual, you benefit from exposure to other ways of thinking and coding styles. For the product, you benefit from continuous engagement and user feedback from a diverse set of individuals not to mention the actual code contributions.

Tip for first time contributors?

Keep your contributions small in scope and get your feet wet. Don’t worry about breaking things, you should feel comfortable to rely on Airbyte’s process: PR review, CICD and testing frameworks.

Looking to the distant future, what is the technology that you’re most excited about and that you think holds a lot of promise?

The Modern Data Stack has turned into quite a controversial topic. Every day more tools arise whilst adjacent to this, enterprise players (Snowflake, Databricks etc.) are extending their capabilities e.g. snowflake x salesforce. At some point I anticipate consolidation and I’m curious if 3rd party applications will get involved. Related, we still haven’t unlocked source application-led event-streaming capabilities across the data space.

Most used emoji?‍

🚀

Spring or Fall?

‍Autumn. The water’s warm, the days aren’t 40C and the streets are beautiful.

Airbyte has an active book club, so we’re always curious what are some good reads. What is a favorite book you’ve read this year?

It’s mostly been work-related book clubs and thus been pretty technical/managerial. Turn The Ship Around is a great one if you’re interested in building leaders. A more recent read was American Prometheus, a thorough biography into Oppenheimer’s life. Highly recommended if you’re interested in this complex individual or the making of the atomic bomb. Also a timely recommendation with the upcoming Nolan film 🎥.

On your website, you listed data, watches, music and travel as interests. What's is your fav watch?

The halios tropik B rounds out the collection nicely. Made by an awesome microbrand out of Canada, this clean automatic bronze diver watch develops a beautiful patina over time. Or you can rogue, pull out the ammonia and force patina the thing.