Home Data Insights Tutorials Engineering Company Community

All Data Insights articles

Some thought leadership articles about data engineering and infrastructure.

The Data Ecosystem Is Ready for ETL To Be Dead

Charles Giardina on Mar 24, 2023

3 Techniques to Write Highly Optimized Queries For BigQuery

Kelvin Gakuo on Mar 23, 2023

The Art of Abstraction in ETL: Dodging Data Extraction Errors

Emily Riederer on Mar 21, 2023

The Snowflake Effect: From Data Warehouse to Data Cloud

Thalia Barrera on Mar 13, 2023

ETL vs ELT: The Key Differences

John Lafleur on Mar 07, 2023

Pandas 2.0 and its Ecosystem (Arrow, Polars, DuckDB)

Simon Späti on Mar 06, 2023

Amazon S3: Best Practices for Managing and Optimizing it

Faithful Adeda on Mar 06, 2023

The Art and Science of Measuring Data Teams Value

Thalia Barrera on Feb 28, 2023

Maximizing Snowflake Storage: Understanding Views and Table Types

Madison Schott on Feb 20, 2023

The Benefits of Open-Source ELT

Simon Späti on Feb 13, 2023

Free Tier isn’t Free: Why Developers Should Insist on Open Source

John Lafleur on Jan 31, 2023

5 Signs Analytics Engineering Might Be the Right Career For You

Madison Schott on Jan 30, 2023

Snowflake security best practices: access control, data masking, and governance

Madison Schott on Jan 18, 2023

BigQuery 101: A Beginner's Guide to Google's Cloud Data Warehouse

Thalia Barrera on Jan 12, 2023

You have collected unstructured data! Now what?

Alex Marquardt on Jan 11, 2023

Modern Data Stack: The Struggle of Enterprise Adoption

Simon Späti on Jan 09, 2023

The Open (aka Modern) Data Stack Distilled into Four Core Tools - Part I

Simon Späti on Jan 03, 2023

Into the Fediverse: the Data Engineer's Guide to Mastodon

swyx on Dec 21, 2022

How to Build Software Products Faster by Thinking Like a Data Engineer

Evan Tahler on Dec 19, 2022

Data Warehouse vs. Operational Database! What? How? Which One?

Alex Marquardt on Dec 16, 2022

4 ways to optimize your BigQuery tables for faster queries

Kelvin Gakuo on Dec 15, 2022

12 Things You Need to Know to Become a Better Data Engineer in 2023

Thalia Barrera on Dec 09, 2022

Best Data Newsletters in 2022: State of Data Engineering Survey results

swyx on Dec 02, 2022

Redshift Turns 10: The Evolution of Amazon’s Cloud Data Warehouse

Thalia Barrera on Nov 28, 2022

What is an ELT data pipeline?

Alex Marquardt on Nov 18, 2022

How to optimize Redshift performance and reduce costs

Offisong Emmanuel on Nov 18, 2022

We forced a bot to understand the Data Nets debate so you don't have to (nobody does)

swyx on Oct 20, 2022

The Evolution of The Data Engineer: A Look at The Past, Present & Future

Thalia Barrera on Oct 19, 2022

Will Rust Take over Data Engineering? 🦀

Simon Späti on Oct 19, 2022

The Rise of the Semantic Layer: Metrics On-The-Fly

Simon Späti on Sep 29, 2022

Why is data quality harder than code quality?

Ari Bajo Rouvinen on Aug 31, 2022

4 questions data security experts ask before moving data

Patsy Bailin on Aug 30, 2022

Data Lake / Lakehouse Guide: Powered by Data Lake Table Formats (Delta Lake, Iceberg, Hudi)

Simon Späti on Aug 25, 2022

Reverse ETL Explained: Concepts, Use Cases & Where It Fits In Your Data Stack

Thalia Barrera on Aug 25, 2022

Best practices for data modeling with SQL and dbt

Madison Schott on Aug 23, 2022

Best Data Podcasts in 2022: Airbyte Staff Picks

swyx on Aug 22, 2022

Data News: Dagster 1.0 Launch Recap

Simon Späti on Aug 11, 2022

6 ways to reduce Snowflake costs

Madison Schott on Jul 26, 2022

Data Orchestration Trends: The Shift From Data Pipelines to Data Products

Simon Späti on Jun 14, 2022

Data Integration Guide: Techniques, Technologies, and Tools

Alex Marquardt on May 19, 2022

Understanding Change Data Capture (CDC): Definition, Methods and Benefits

Thalia Barrera on May 12, 2022

Best Practices to Design a Data Ingestion Pipeline

Madison Schott on May 10, 2022

Best Practices for Snowflake Users, Roles, and Permissions

Madison Schott on Apr 19, 2022

Snowflake Data Warehouse Architecture: How to Organize Databases, Schemas and Tables

Madison Schott on Mar 22, 2022

SQL vs Python for Data Analysis

Richard Pelgrim on Mar 14, 2022

Data Replication: Examples, Techniques & How to Solve Challenges

Thalia Barrera on Feb 23, 2022

Best Practices for your dbt Style Guide

Madison Schott on Feb 15, 2022

Using SQL String Functions to Clean Raw Data

Madison Schott on Jan 24, 2022

How to Collect Behavioral Data? A Guide for Data Engineers and Analysts

Arpit Choudhury on Jan 14, 2022

Using an ETL Framework vs Writing Yet Another ETL Script

Charles Giardina on Dec 16, 2021

ETL Pipelines with Airflow: the Good, the Bad and the Ugly

Ari Bajo Rouvinen on Oct 08, 2021

Why ETL Needs Open Source to Address the Long Tail of Integrations

Sherif Nada on Jul 01, 2021

How Open-source Can Disrupt Build vs. Buy Considerations

John Lafleur on Jan 21, 2021

How We Leveraged Singer for Our MVP

Charles Giardina on Nov 23, 2020

Why You Should NOT Build Your Data Pipeline on Top of Singer

Charles Giardina on Nov 19, 2020

Why the Future of ETL Is Not ELT, But EL(T)

John Lafleur on Nov 03, 2020

How to Build Thousands of Connectors

Michel Tricot on Oct 30, 2020

The State of Open-Source Data Integration and ETL

John Lafleur on Oct 18, 2020

Airbyte vs. Singer: Why Airbyte Is Not Built on Top of Singer

John Lafleur on Oct 11, 2020

Open-Source vs. Commercial Software: How to Better Solve Data Integration

Michel Tricot on Oct 08, 2020

How We Can Commoditize Data Integration Pipelines

John Lafleur on Sep 22, 2020

How to Get Your Engineering Team Involved in Product Efforts

John Lafleur on Aug 11, 2020

Loading more...