All
Data insights
Engineering
University
Community
Company
Article

The State of Data 2023

The field of data is constantly evolving, and it is essential to understand the current landscape to stay informed and competitive. The State of Data offers valuable insights into the latest trends, tools, and practices in data engineering.
John Lafleur
•
May 25, 2023
•
5 min read

Top Stories

Article
Data Modeling: The Unsung Hero of Data Engineering: Architecture Pattern, Tools and the Future (Part 3)
Simon Späti
•
May 26, 2023
•
25 min read
Article
Pandas 2.0 and its Ecosystem (Arrow, Polars, DuckDB)
Simon Späti
•
March 6, 2023
•
7 min
Article
BigQuery 101: A Beginner's Guide to Google's Cloud Data Warehouse
Thalia Barrera
•
January 12, 2023
•
15 min read
Article
The Evolution of The Data Engineer: A Look at The Past, Present & Future
Thalia Barrera
•
October 19, 2022
•
15 min read
Article
The Rise of the Semantic Layer: Metrics On-The-Fly
Simon Späti
•
September 29, 2022
•
20 min read

Join 20,000+ data engineers

getting fresh data insights weekly from our newsletter.

Data insights

Thought leadership articles, notes, videos and podcast episodes about data engineering and infrastructure.
See all
Article

Top Azure Data Services Overview: Integration, Storage and Analytics

Edgar Cervantes De Los Rios
•
April 26, 2320
•
10 min read
Article
Testing Data Pipelines with dbt-expectations: A Beginner's Guide
Madison Schott
•
June 7, 2023
•
10 min read
Article
Data Lineage: The Unseen Lifeline of Data-Driven Organizations
Thalia Barrera
•
May 30, 2023
•
12 min read
Article
Data Modeling: The Unsung Hero of Data Engineering: Architecture Pattern, Tools and the Future (Part 3)
Simon Späti
•
May 26, 2023
•
25 min read
Article
Data Engineering to Analytics Engineering: How to Successfully Transition
Madison Schott
•
May 9, 2023
•
10 min read
Article
DataNews.filter() - Navigating Entity-Centric Modeling
Simon Späti
•
May 4, 2023
•
10 min read
Article
Data Modeling – The Unsung Hero of Data Engineering: Modeling Approaches and Techniques (Part 2)
Simon Späti
•
May 3, 2023
•
15 min read
Article
The Art of Abstraction in ETL: Keeping The Good Things Going
Emily Riederer
•
May 3, 2023
•
8 min read
Article
Top Azure Data Services Overview: Relational Databases
Edgar Cervantes De Los Rios
•
April 17, 2023
•
10 min read
Video
Empowering Data Teams: Let Them Choose Their Own Tools
Chris Sean
•
April 14, 2023
•
12 min read
Video
Bring Your Own Infra
Davin Chia
•
April 13, 2023
•
10 min read

Data news

Our reaction to the news in the data industry.
See all
No items found.
No items found.

Engineering

Our engineering team shares their learnings along our journey.
See all
Article

Airbyte Schema Propagation: Keeping your replicated catalog up to date

Discover the new schema propagation feature in both Airbyte Open Source and Airbyte Cloud that simplifies data integration and reduces operational load.
Malik Diarra
•
June 7, 2023
•
3 min read
Article
Airbyte Column Selection: Control over the exact data to sync
Malik Diarra
•
June 6, 2023
•
4 min read
Article
Airbyte Checkpointing: Ensuring Uninterrupted Data Syncs
Evan Tahler
•
June 1, 2023
•
5 min read
Note
An Easier Way to Understand Airbyte Synchronization through Events
Benoit Moriceau
•
May 31, 2023
•
2 min read
Note
Supercharging e2e Testing with Cypress and Airbyte’s Config API
Teal Larson
•
May 31, 2023
•
2 min read
Note
Using the Airbyte API to make an iOS App
Brian Leonard
•
May 25, 2023
•
2 min read

University

Tutorials, courses (soon), and glossary terms to help you improve your knowledge in data, and soon to get you full certifications.

Tutorials

See all
Sync MySQL CDC to Kafka using Change Data Capture
•
June 6, 2023
•
15 minutes
Migrate your data from Redshift to Snowflake
•
June 6, 2023
•
15 minutes
Replicate Salesforce and Zendesk data to Keen for unified analytics
•
June 6, 2023
•
25 minutes
Orchestrate data ingestion and transformation pipelines with Dagster
•
June 6, 2023
•
15 minutes

Glossary

See all
Data Contract
Data Contracts are API-like agreements between software/data engineers who own services and data consumers that understand how the business works.
DuckDB
DuckDB is an in-process SQL OLAP database management system. It has strong support for SQL. DuckDB is borrowing the SQLite shell implementation.
Data Engineering Concepts
Some core concepts we are going to explore include data warehouses, data lakes, storage layers, and more.
Modern Data Stack
The Modern Data Stack (MDS) is a heap of open-source tools to achieve end-to-end analytics from ingestion to transformation to ML over to a columnar data warehouse or lake solution with an analytics BI dashboard backend.
Open Data Stack
The open data stack is a better term for the Modern Data Stack but focuses on solutions built on open-source and open standards covering the Data Engineering Lifecycle.

Data engineering resources

See all
Columnar Storage Formats: Benefits and Use Cases in Data Engineering
Aditi Prakash
•
May 31, 2023
•
10 min read
Data Warehouse vs Data Mart: Understanding the Differences and Use Cases
Aditi Prakash
•
May 31, 2023
•
14 min read
Operational Data Stores (ODS): An Overview and Use Cases in Data Engineering
Aditi Prakash
•
May 24, 2023
•
12 min read
PostgreSQL vs Redshift: A Comparison for Data Engineers
Aditi Prakash
•
May 18, 2023
•
15 min read
Data Warehouse vs Database: Key Differences and Considerations
Aditi Prakash
•
May 12, 2023
•
15 min read
Data Lake vs Data Warehouse: Comparing Data Storage Solutions and Their Applications
Aditi Prakash
•
May 11, 2023
•
15 min read
Snowflake vs. BigQuery: Navigating the Data Warehouse Landscape
Thalia Barrera
•
May 10, 2023
•
15 min read
SQL vs NoSQL: A Comparison of Database Technologies for Data Engineers
Aditi Prakash
•
May 5, 2023
•
15 min read
Exploring Different Types of Databases: A Guide for Data Engineers
Aditi Prakash
•
May 5, 2023
•
15 min read
PostgreSQL vs MySQL: A Detailed Comparison for Data Engineers
Aditi Prakash
•
May 5, 2023
•
15 min read

Company

Get to know our story and keep track what we’ve been shipping.
See all
Article
Announcing Airbyte 0.50: Checkpointing, Column Selection, and Schema Propagation
John Lafleur
•
June 8, 2023
•
4 min read
Article
Building Connectors with No-Code | The Drip May 2023 Edition
Justin Chau
•
June 1, 2023
•
Article
Introducing our new content hub
John Lafleur
•
May 30, 2023
•
2 min read
Article
Launch of Airbyte API and More Community Support | April 2023 Airbyte Product Updates
Justin Chau
•
May 1, 2023
•
Article
Better supporting our contributors and active users
John Lafleur
•
April 26, 2023
•
4 min read
Article
Official Launch of Airbyte API
Riley Brook
•
April 24, 2023
•
Article
The Drip | February 2023 Airbyte Product Updates
Justin Chau
•
March 1, 2023
•
Article
The Drip | January 2023 Airbyte Product Updates
Justin Chau
•
February 1, 2023
•

Community

See some of our community’s updates and success stories.
See all
Article
The State of Data 2023
John Lafleur
•
May 25, 2023
•
5 min read
Video
Co-Founders Q&A | A Retrospective 1-2 Years After Raising $150M
Chris Sean
•
May 16, 2023
•
46 min watch
Article
Upgrading our Community Pull Requests Experience
Evan Tahler
•
April 28, 2023
•
Video
Are Building Custom ETL Pipelines Outdated?
Chris Sean
•
April 28, 2023
•
8 min read

Our shorts

Have a few minutes to spare for some quick education tips?
See all
Video Short
Persisting Data with Docker
Justin Chau
•
April 26, 2023
•
Video Short
Learning SQL with Airbyte | Part 1
Justin Chau
•
April 20, 2023
•
Video Short
How to Add PGAdmin to Docker
Justin Chau
•
April 18, 2023
•
Video Short
Why use Docker to Spin Up Postgres
Justin Chau
•
April 12, 2023
•
Video Short
How to use Postgres Without Installing It Locally
Justin Chau
•
April 11, 2023
•