Connect AWS Datalake data with any data source.

Move data from any source into AWS Datalake with Airbyte.

Load data into AWS Datalake Read the docs

Sources: 590+
Standard support: 0.1.58

Overview

What is the Airbyte AWS Datalake connector?

The Airbyte AWS Datalake connector loads data from 590+ Airbyte sources into AWS Datalake, with full refresh and incremental sync on a schedule you control.

01
Load from any source
Move data into AWS Datalake from 590+ Airbyte sources on a schedule you control.
02
Incremental syncs
Pull only the records that changed since the last run instead of reloading everything.
03
Cloud or self-hosted
Run the AWS Datalake connector on Cloud, OSS, Hybrid.

Setup

Live in three steps

Confirm your AWS Datalake destination credentials and permissions.
Configure the destination in Airbyte with the required connection settings.
Choose a source, map streams, and schedule syncs into AWS Datalake.

Configuration fields

AWS Datalake configuration fields
Field	Type	Description
bucket_name*	string	The name of the S3 bucket. Read more here.
credentials*	object	Choose How to Authenticate to AWS.
lakeformation_database_name*	string	The default database this destination will use to create tables in per stream. Can be changed per connection by customizing the namespace.
region*	string	The region of the S3 bucket. See here for all region codes.
aws_account_id	string	target aws account id
bucket_prefix	string	S3 prefix
format	object	Format of the data output.
glue_catalog_float_as_decimal	boolean	Cast float/double as decimal(38,18). This can help achieve higher accuracy and represent numbers correctly as received from the source.

Hybrid deployment

Sovereign by architecture, not by policy.

The data plane runs in your environment. Records, credentials and keys stay with you; the control plane only receives configuration and job metadata.

Cloud
OSS
Hybrid

Your environment

DataConnectorsCredentialsKeys

↕ metadata only

Airbyte control plane

OrchestrationMonitoringUI

FAQ

AWS Datalake connector questions

What data can I load into AWS Datalake with Airbyte?

Airbyte moves data from any supported source into AWS Datalake on a schedule you control, with full refresh and incremental sync where the destination supports it.

Does the AWS Datalake connector support incremental sync?

Yes. The AWS Datalake destination supports full refresh and incremental sync, so Airbyte can load only the records that changed since the last sync.

Where can I run the AWS Datalake connector?

Run the AWS Datalake connector on Cloud, OSS, Hybrid. Hybrid deployments keep the data plane in your environment while the control plane orchestrates syncs.

Didn't find your answer? Talk to sales