FTP to Amazon Redshift: How to Move Your Data
Move FTP files into Amazon Redshift with Airbyte. Why there is no plain FTP connector, using SFTP Bulk, and why the pipeline needs an S3 staging bucket.
Moving files from an FTP server into Amazon Redshift turns a folder somebody drops files into every night into rows your analysts can query. Partner feeds and vendor extracts arrive this way constantly, and they stay invisible to everybody until something loads them.
Two things shape this build. There is no plain FTP connector, because the one you want is SFTP Bulk and SFTP is a different protocol running over SSH. And Redshift cannot be loaded without a staging bucket, so an AWS prerequisite sits in front of a file server pipeline.
FTP to Amazon Redshift at a glance:
Why move data from FTP to Amazon Redshift?
Two situations account for most of these pipelines.
The first is automating a manual step. A partner drops files onto a server every night and somebody currently downloads them, opens them to check nothing looks odd, and uploads them somewhere else. That person is a pipeline nobody has written yet, and they are doing it on holiday too.
The second is joining a file feed to warehouse data, such as matching a vendor's product list against your own catalogue. File transfer is how a great deal of business data still moves between organisations, and a warehouse is where it becomes useful rather than archived.
What do you need before you start?
One item can stop the project and one is an AWS prerequisite you cannot skip:
A server that genuinely speaks SFTP. Ask whoever runs it whether SSH-based transfer is enabled, usually on port 22, rather than assuming from the word people use. The SFTP Bulk source documentation covers the connector, and the section below covers what to do if the answer is no.
Credentials, and a private key as a file if you use one. A username with either a password or a key pair. If you authenticate with a key, Airbyte requires the key to be uploaded as a file rather than pasted into the field, which catches people who are used to pasting secrets.
An S3 staging bucket and IAM permissions. The Redshift destination writes data to S3 as files with a manifest, then issues a COPY command, which is Redshift's own recommended approach. You need put, get, delete and list on the bucket, in the same region as the cluster.
A folder path, a glob pattern and Redshift entities. The pattern selects which files belong to the stream. On the warehouse side you need a database, a schema and a user able to write data and manage staging operations, and Airbyte writes into two schemas rather than one.
Note also that SFTP Bulk is a Marketplace connector at an early release stage, so alert on sync failures rather than assuming silence means success.
How do you build an FTP to Amazon Redshift pipeline in Airbyte?
Step 1: Confirm the protocol, then the delivery method
Establish that the server supports SFTP before planning anything else. Then choose the delivery method: replicate records parses your files into rows, which is what a warehouse wants, while copy raw files transfers them without parsing and is for archival cases.
Step 2: Configure the SFTP Bulk source
Click Sources in the left navigation, then New Source, and select SFTP Bulk, following adding a source. Supply the host, port and credentials, uploading the private key as a file if you use one, then set the folder path, file format and glob pattern.
Step 3: Configure the Redshift destination
Click Destinations, then New Destination, and select Redshift, following adding a destination. Supply the cluster host, database, schema and credentials, along with your S3 bucket and IAM key. Staging files are removed after a successful COPY by default, and can be retained if you need them for debugging.
Step 4: Create the connection and set your keys
Click Connections, then New connection, select your stream and an incremental sync mode so new files are picked up without re-reading the ones already processed. Set the frequency against how often files actually arrive, and set sort keys on whatever date column your queries filter by.
Then build views over the raw tables so anything consuming this reads a stable interface rather than Airbyte's metadata columns, and check the result after the first sync if your files are CSV.
What if your server only speaks plain FTP?
Then this connector will not reach it, and no configuration changes that. FTP and SFTP are unrelated protocols that happen to share three letters: FTP is the older unencrypted protocol on port 21, while SFTP runs inside an SSH session on port 22. A client for one cannot talk to the other.
In practice most servers people call FTP have supported SFTP for years, because sending business data unencrypted is no longer acceptable to anybody's security team. So the first move is simply to ask, and the answer is usually yes even when the habit of saying FTP persists.
Where it genuinely is plain FTP only, ask whoever runs the server to enable SFTP, or have the files landed in S3 instead and use the S3 source. That second route is worth considering here regardless, since you already need a bucket for staging and your estate is on AWS anyway.
Why does a file server pipeline need a bucket?
Because the requirement comes from the destination rather than the source. Redshift is designed to be loaded in bulk from S3, so the connector writes files with a manifest and then issues a COPY command pointing at them. That is Redshift's own recommended path and there is no direct insert alternative.
So files move from a server, through a bucket, into a warehouse, which is one more hop than people picture when they think about loading a CSV. Budget for a bucket in the same region as your cluster, IAM credentials with four permissions on it, and whatever approval your organisation requires for new storage.
It also raises a question worth asking once. If the files are going to pass through S3 anyway, could your partner write them there directly and remove the server from the picture entirely? That is not always possible, and when it is, the resulting pipeline is simpler and the protocol question disappears with it.
Frequently asked questions
Is there an FTP connector in Airbyte?
Not for plain FTP. The connector is SFTP Bulk, which speaks SFTP over SSH. Confirm your server supports that before planning the pipeline.
Do I need an S3 bucket to load into Redshift?
Yes. The destination stages data in S3 and loads it with COPY, which is Redshift's recommended approach. Put the bucket in the same region as your cluster.
My private key is rejected. What is wrong?
Airbyte requires the key to be uploaded as a file rather than pasted into the field as text. Save it and upload it.
Why did the same rows arrive twice?
A file was rewritten in place, giving it a new modification time. Use a deduplicating sync mode where your data has a natural key, or a view that picks the latest version.
Can I do this without writing code?
Yes. The setup above is entirely UI-driven. You will want Redshift views so consumers read a stable interface rather than the raw synced tables.
Get your FTP data into Amazon Redshift
Confirm the server speaks SFTP before anything else, then provision the staging bucket in your cluster's region. Upload your private key as a file, write a specific glob pattern, ask whether files are ever rewritten in place, and consider whether the files could land in S3 directly instead.
Airbyte's connector catalog includes 700+ pre-built connectors, so partner files can reach a warehouse alongside your own systems. For the same source in another warehouse, see FTP to Snowflake, and for files already in object storage, Amazon S3 to Amazon Redshift.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.