PyAirbyte
Airbyte's 700+ connectors, as a Python library.
PyAirbyte is an open-source Python library that runs Airbyte connectors in your code. Pull data from APIs, databases, and files into DuckDB, Postgres, Snowflake, or BigQuery, then work with it in pandas, SQL, or your AI framework.
pip install airbytePython 3.10 to 3.12. Using uv? Run uv add airbyte.
Data ingestion in a few lines of Python
Pick a connector, select streams, and read. PyAirbyte installs the connector, tracks incremental state, and caches the records locally so you can query them right away.
35,000+
25,000+
Open source
Read GitHub issues into a local DuckDB cache, then hand them to pandas. Run it again and incremental streams fetch only what's new.
import airbyte as ab
source = ab.get_source(
"source-github",
config={
"repositories": ["airbytehq/PyAirbyte"],
"credentials": {
"personal_access_token": ab.get_secret("GITHUB_PERSONAL_ACCESS_TOKEN"),
},
},
)
source.check()
source.select_streams(["issues"])
result = source.read() # writes to a local DuckDB cache by default
issues = result["issues"].to_pandas()How does PyAirbyte work?
PyAirbyte runs the same connectors that power Airbyte Cloud and Airbyte Core (open source), on your laptop, in a notebook, or wherever your Python code runs. It's built for Python developers who want to build data pipelines in their own code.
Data flow diagram: a source connector reads from an API, database, or file, PyAirbyte loads the records into a SQL cache or an Airbyte destination, and your code queries them with pandas, SQL, or AI frameworks.
Source connector
APIs, databases, files
PyAirbyte
Runs the connector from your code.
SQL cache
DuckDB, Postgres, Snowflake, BigQuery, or an Airbyte destination
Your code
pandas, SQL, AI frameworks
What you can build with PyAirbyte
Prototype a pipeline in a notebook
Test a source in Jupyter or Google Colab before you commit to a connection. Read a few streams, check record counts, and shape the data in pandas.
Replace custom ETL scripts
Swap one-off API scripts for Airbyte connectors that already handle pagination, rate limits, and incremental state.
Run pipelines as code
Apply data engineering best practices to your pipelines. Keep the logic in version control, review it in pull requests, test it in CI, and run it from Airflow, Dagster, or any orchestrator that runs Python.
Feed analysis and AI apps
Load records into pandas or query them with SQL. Then pass them to LangChain, LlamaIndex, or whatever framework runs your LLM app.
Build and test connectors
Run a connector from a local directory or a Connector Builder YAML manifest, and check it against real data before you publish it.
Explore data without a warehouse
Read a source into the local DuckDB cache and query it with SQL on your laptop. No destination to configure, and no warehouse account to wait on.
PyAirbyte MCP: run Airbyte from your AI client
Airbyte Cloud and Airbyte Flex
Connect your MCP client to
https://mcp.airbyte.com/mcpand sign in with your Airbyte account.Airbyte Core (open source)
Run the server yourself with
uvx --from=airbyte@latest airbyte-mcpand point it at your instance.
What PyAirbyte handles for you
700+ connectors
Use the same connector catalog as Airbyte Cloud. Python and YAML connectors run natively, and the rest run in Docker automatically.
SQL caches
Records land in a local DuckDB file by default. Swap in Postgres, Snowflake, or BigQuery by passing a different cache to
read().Destinations
Call
destination.write(source)to send data to an Airbyte destination, with incremental state carried between runs.Incremental syncs
PyAirbyte stores stream state alongside your data, so each run picks up where the last one stopped.
Isolated installs
Each connector installs into its own virtual environment with uv. Set a different Python version per connector with
use_python.Airbyte Cloud from Python
Deploy sources, destinations, and connections, set schedules, run syncs, and read results from Snowflake and BigQuery destinations.
Start with the library. Run production on Airbyte.
PyAirbyte
Free and open source. Connectors in your code, local caches, and full control over where it runs.
pip install airbyteAirbyte Cloud
Scheduling, monitoring, and alerting, with SSO, RBAC, and audit logging on the plans that include them. Try it free for 14 days.
Airbyte Flex
Airbyte's data plane runs in your cloud, VPC, or on-prem environment, and Airbyte manages the control plane. Your data never touches our servers.
Frequently asked questions
What is PyAirbyte?
Does PyAirbyte replace Airbyte?
What is the PyAirbyte cache, and can PyAirbyte write to destinations?
Can I build ETL and ELT pipelines with PyAirbyte?
Can I move millions of rows or terabytes of data with PyAirbyte?
What can I build with PyAirbyte?
How does PyAirbyte handle schema changes?
Where does PyAirbyte store state for incremental syncs?
Can I manage and run Airbyte Cloud connections from PyAirbyte?
Does PyAirbyte work with Airflow, Dagster, and other orchestrators?
Can I use PyAirbyte to develop and test my own connectors?
How is PyAirbyte related to the Airbyte MCP?