.png)
Many of you liked our Staff Picks for Data Podcasts this year, so of course we wanted to expand further to help you discover all the great data content out there making sense of the vast data landscape. However we wanted more data to help make more confident picks, so in October we launched the State of Data Engineering survey, to hear from you!
Since #SpotifyWrapped is so in vogue this time of year, I figured we could also present our results in meme form:
.png)
The full results will be released following our Move(data) Conference next week (sign up now! See an incredible array of lightning talks from all the top names in data, and win raffle prizes!) but we wanted to release some results early to tease them.
Methodology
We surveyed ~900 Data Engineers on a range of questions, from their demographics, to compensation, to their data stack (95 data points in total! We are extremely grateful for everyone who stuck through the survey). But we are also hellbent on understanding and serving the Data Engineering community everywhere they feel at home, so we made sure to ask for their takes on the best Newsletters, Podcasts, YouTubes, and Communities in a non-exclusive multiple choice list with an option for write-ins.
The world of newsletters is pretty hard to categorize, since people can write whatever they want. However we soon discovered there was a basic line between Ecosystem-aggregator newsletters and Individual-writer newsletters:
- Individual Writers publish long form original essays and explainers.
- Ecosystem Aggregator newsletters curate the news and the best work they find elsewhere, including the work of Individual Writers. They may also be a collective magazine that accepts multiple authors, like Towards Data Science.
As a rule, Aggregators tend to be more popular, since they represent a discovery layer over individual work. But we wanted to highlight and acknowledge the incredible work that individual writers perform as well, so we are separating them into different categories.
Here goes!
The Best Individual Data Newsletters of 2022
#1: Seattle Data Guy (https://seattledataguy.substack.com/)
Former Facebook Data Engineer Ben Rogojan, better known as the Seattle Data Guy, got an early start on Substack in Nov 2019 and has been a content machine ever since (we count 91 posts in the past 3 years), also expanding into a very successful YouTube channel. Almost one year ago he quit Facebook to go fulltime into consulting, and it looks like the strategy has paid off.
Highlighted posts:
- Data is the How, Business is the Why
- Let’s Move Fast and Get Rid of Data Engineers
- Airbyte - Is Open Source the Way Forward For Data Automation?
#2: Benn Stancil (https://benn.substack.com/)
Benn cofounded Mode Analytics 9 years ago, but in 2021 started what has evolved into the “Friday Night Fights” of the Data Community, with consistently stellar writing, a willingness to say the things people only whisper behind closed doors, and a historical perspective that can only come from being involved in data engineering since before the job title existed.
Highlighted posts:
- His How dbt fails, How Fivetran fails, How Snowflake fails thought exercises
- Do data-driven companies actually win?
- The end of Big Data and the case for consolidation
#3: Metadata Weekly (https://metadataweekly.substack.com/)
Prukalpa Sankar cofounded Atlan 3 years ago and also got into the content game last year. Compared to the other generalist newsletters, Metadata Weekly tends to be a great deal more focused on problems related to data catalogs (linear, semantic layer, active metadata, dataops), yet still retains a remarkably personal tone that makes it a joy to read.
Highlighted posts:
- The Rise of DataOps
- The past, present and future of the semantic layer
- The Great Data Debate: Unbundling or Bundling?
#4: Technically (https://technically.substack.com/)
Justin Gage formerly worked on Data at Digital Ocean and Growth at Retool, but his side hustle flowered into one of the biggest tech newsletters at Substack (35k subscribers and counting) and he now works on Developer Marketing at Amplify Partners. Technically started out a much more generalist tech newsletter (“What’s an API?”) but since “What’s a data warehouse?” Justin started digging into his data roots.
Highlighted data posts:
#5: Learn Analytics Engineering (https://madisonmae.substack.com/)
Madison Schott takes her extensive experience as a data engineer at Capital One and analytics engineer at Winc and maps out the data ecosystem in her wonderfully approachable, and always deeply technically informed style. Fun fact, she is one of the most frequent guest writers on the Airbyte blog!
Highlighted data posts:
- Is Fivetran Worth the Investment? Vs Airbyte Tutorial
- Aspiring Analytics Engineers, Start Here
- Sorry, I Hate Airflow
For anyone keeping count, yes, Substack seems to have a 100% market share in data newsletters by individuals!
The Best Data Ecosystem Newsletters of 2022
#1: Towards Data Science (https://towardsdatascience.com/)
TDS was the undisputed top vote in our survey and is probably the oldest still-running dedicated data publication, starting in 2013 but going all-in on Medium in 2016. Today it is a Canadian company running with a staff of 4 editors and 15 editorial associates, covering topics across Python, Marketing, Machine Learning, and Startups.
Highlighted data engineering posts:
- Why Python is not the programming language of the future
- Top 10 Databases to use in 2021
- Why Data Quality is Harder than Code Quality (by our very own Ari Bajo!)
#2: Data Engineering Weekly (https://www.dataengineeringweekly.com/)
This Substack by Bazaarvoice/Slack/Zendesk principal engineer Ananth Packkildurai started in 2020 and, like it says on the tin, has put out an issue every week for 109 weeks and counting. This is a “links roundup” newsletter but with excellent commentary, always coming with a one paragraph summary. Occasionally, he also does longer form essays as well, making this a blend of individual and aggregator newsletter.
Highlighted posts:
- Data Engineering Weekly #100
- Yes, You Need To Implement a Data Contract System
- Omicron Paradigm: Architectural patterns for the Infinite Data Logistic
#3: The Analytics Engineering Roundup (https://roundup.getdbt.com/)
dbt's Roundup by Tristan Handy and Anna Filippova is the #1 newsletter to keep up with dbt and the broader Analytics ecosystem, including a wonderful podcast with great guests. Tristan started the Data Science Roundup in 2015, but renamed to the current iteration as dbt and analytics engineering took the world by storm.
Highlighted posts:
- Analytics isn't for analysts
- In Search of New Standards
- The Data Diaspora shouting out our wonderful Thalia Barrera!
#4: Blef.fr Data News (https://www.blef.fr/blog/)
This blog-and-newsletter by former Head of Data and now data consultant Christophe Blefari is the rare gem - an independently run data newsletter not published on Substack or Medium! This was another “mixed” newsletter with links roundups and personal commentary, but we loved the deep reflection and well groomed news links. All links are made available in The Explorer, which also serves as an exclusive data community.
Highlighted posts:
- How to learn data engineering
- Why are data engineers bad at meeting deadlines?
- Must-read articles of 2021
#5: All Hands on Data (https://allhandsondata.substack.com/)
This newsletter by Shipyard cofounders Blake Burch and Steven Johnson is just 6 months old, but already making waves as a consistent aggregator of the data world, receiving just enough votes to make it into the top 10.
It’s too early to highlight posts right now but we love the progress!
The Best Write-ins
Last but not least, we left plenty of room for people to write in newsletters that we regrettably may have overlooked (we’ll certainly fix this next year!). Newsletters don’t have to be a popularity contest - if they have value to you, then that’s plenty! Write-ins are high compliments - these questions were asked at the end of an already very long survey, so we appreciate all respondents who wrote in their favorites unprompted!
In no particular order:
Obligatory Call to Action
What we released here was the very first teaser of the State of Data Engineering survey results. Join us at the Move(Data) Conference next week for more insights on the Modern Data Stack!
About the Author
About the Author
%201.svg){kind=small}