Move Hugging Face - Datasets data anywhere it matters.

Send Hugging Face - Datasets rows and splits to any warehouse, lake or AI app.

Sync Hugging Face - Datasets free Read the docs

Source

Hugging Face - Datasets

Snowflake
BigQuery
Postgres
Databricks
Claude

Streams: 2
Destinations: 50+
Standard support: 0.0.68

Overview

What is the Airbyte Hugging Face - Datasets connector?

The Airbyte Hugging Face - Datasets connector extracts data from Hugging Face - Datasets and loads it into 50+ destinations such as Snowflake, BigQuery, Databricks or Postgres, with full refresh. It publishes 2 streams you can select per connection.

01
Sync to your warehouse
Land 2 Hugging Face - Datasets tables in 50+ destinations on a schedule you control.
02
One authorization
Authenticate Hugging Face - Datasets once and Airbyte keeps every scheduled sync running on it.
03
Cloud or self-hosted
Run the Hugging Face - Datasets connector on Cloud, OSS, Hybrid.

2 streams

Every table you can sync from Hugging Face - Datasets

rows
Pagination DefaultPaginator
Sync modes FullIncr.
splits
Pagination No pagination
Sync modes FullIncr.

rows
Pagination	DefaultPaginator
Sync modes	FullIncr.

splits
Pagination	No pagination
Sync modes	FullIncr.

Setup

Live in three steps

Get your Hugging Face - Datasets credentials from the vendor dashboard or developer settings.
Configure the source in Airbyte with the required connection fields.
Pick a destination, choose streams, set full refresh or incremental, and schedule syncs.

Configuration fields

Hugging Face - Datasets configuration fields
Field	Type	Description
dataset_name*	string	Not documented.
api_token	string	Hugging Face user access token (see https://huggingface.co/settings/tokens). Optional for public datasets, but required for private or gated datasets and strongly recommended to avoid anonymous per-IP rate limits (see https://huggingface.co/docs/hub/en/rate-limits).
dataset_splits	array	Splits to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configs_and_splits for more details)
dataset_subsets	array	Dataset Subsets to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configs_and_splits for more details)

Hybrid deployment

Sovereign by architecture, not by policy.

The data plane runs in your environment. Records, credentials and keys stay with you; the control plane only receives configuration and job metadata.

Cloud
OSS
Hybrid

Your environment

DataConnectorsCredentialsKeys

↕ metadata only

Airbyte control plane

OrchestrationMonitoringUI

FAQ

Hugging Face - Datasets connector questions

What data can I sync from Hugging Face - Datasets with Airbyte?

2 streams: rows, splits.

Does the Hugging Face - Datasets connector support incremental sync?

The Hugging Face - Datasets connector supports full refresh sync. Check the stream list on this page for incremental support per table.

Where can I run the Hugging Face - Datasets connector?

Run the Hugging Face - Datasets connector on Cloud, OSS, Hybrid. Hybrid deployments keep the data plane in your environment while the control plane orchestrates syncs.

Didn't find your answer? Talk to sales