Shopify to Amazon Redshift: How to Move Your Data
Move Shopify into Amazon Redshift with Airbyte. Why a missing scope produces no error, the S3 staging requirement, and checking streams after the first sync.
Moving Shopify into Amazon Redshift puts commerce data where it can be judged against everything that produced it. Shopify knows what was ordered and by whom, and nothing about what those customers cost to acquire or how much support they needed afterwards.
Two things shape this build. A missing permission produces a missing stream rather than an error, and Redshift cannot be loaded without a staging bucket, which adds an AWS prerequisite to a commerce pipeline.
Shopify to Amazon Redshift at a glance:
Why move data from Shopify to Amazon Redshift?
Two situations account for most of these pipelines.
The first is joining orders to acquisition cost. What a customer cost to acquire, whether they came back, and what they were worth over two years are questions your shop cannot answer alone, and a warehouse is where advertising and support data meet the transactions.
The second is joining commerce data to the rest of an AWS estate. If your marketing sources, cost reports and support records already live there, bringing orders across puts everything in reach of one set of queries.
What do you need before you start?
The scopes are where most of the trouble starts, and the bucket is a prerequisite you cannot skip:
A custom app with every scope you need, granted together. Created in your Shopify admin, or authenticate with OAuth on Airbyte Cloud. Grant read access for each resource you intend to sync in one pass, because the cost of missing one is covered below. The Shopify source documentation lists which scope each stream needs.
Your shop name in the right form. The subdomain only, so a store at example.myshopify.com has a shop name of example rather than the full address.
An S3 staging bucket and IAM permissions. The Redshift destination writes data to S3 as files with a manifest, then issues a COPY command, which is Redshift's own recommended approach. You need put, get, delete and list on the bucket, in the same region as the cluster.
A start date and Redshift entities. Commerce questions often reach back several years. On the warehouse side you need a database, a schema and a user able to write data and manage staging operations, and Airbyte writes into two schemas rather than one.
Note also that an SSH tunnel covers the SQL connection only. Staging traffic reaches S3 over HTTPS regardless, so a network design assuming everything passes through a bastion is wrong.
How do you build a Shopify to Amazon Redshift pipeline in Airbyte?
Step 1: Grant every scope in one pass
Work out which streams answer your questions, then grant the corresponding read scopes together and write down what you granted. That record is the thing you will check when a stream turns out to be missing, and having it saves an afternoon of guessing.
Step 2: Configure the Shopify source
Click Sources in the left navigation, then New Source, and select Shopify, following adding a source. Enter the shop name as the subdomain, your credentials and a start date. Then refresh the source schema and compare the available streams against the scopes you granted.
Step 3: Configure the Redshift destination
Click Destinations, then New Destination, and select Redshift, following adding a destination. Supply the cluster host, database, schema and credentials, along with your S3 bucket and IAM key. Staging files are removed after a successful COPY by default, and can be retained if you need them for debugging.
Step 4: Create the connection and set your keys
Click Connections, then New connection, select your streams and a deduplicating incremental sync mode. Set sort keys on the order date, since commerce reporting is almost always filtered by period, and check every selected stream produced rows after the first sync.
Rate limit warnings in the logs are normal and not failures, because the connector backs off and continues rather than giving up.
Why does a missing scope produce no error?
Because Shopify grants access per resource, and a stream your credentials cannot reach is simply absent from the catalogue rather than present and failing. From the connector's point of view there is nothing to report, so the sync succeeds and the table never appears.
That is a reasonable security model and an awkward failure mode, because an absent table looks like a configuration choice somebody made rather than a permission nobody granted. The usual sequence is that a report comes out short, and the investigation starts with the warehouse rather than the Shopify app.
So keep the record of what you granted and check every expected stream produced rows after the first sync. Using an app dedicated to this pipeline also means somebody adjusting permissions for a different integration cannot quietly change what your warehouse receives.
Why does a commerce source need an AWS bucket?
Because the requirement comes from the destination rather than the source. Redshift is designed to be loaded in bulk from S3, so the connector writes files with a manifest and then issues a COPY command pointing at them. That is Redshift's own recommended path and there is no direct insert alternative.
So a pipeline from a hosted shop platform acquires an AWS prerequisite, which surprises teams who assumed the Shopify side was the only setup work. Budget for a bucket in the same region as your cluster, IAM credentials with four permissions on it, and whatever approval your organisation requires for new storage.
Commerce volumes are modest by warehouse standards, so this rarely becomes a performance question. It is a planning question, and the time it costs lands at the start of the project rather than during the sync.
Frequently asked questions
Why is a stream missing from my catalogue?
Almost always a missing scope, since Shopify grants access per resource and an unreachable stream is absent rather than failing. Add the scope and refresh the source schema.
Do I need an S3 bucket to load into Redshift?
Yes. The destination stages data in S3 and loads it with COPY, which is Redshift's recommended approach. Put the bucket in the same region as your cluster.
What should I enter as the shop name?
The subdomain only. A store at example.myshopify.com has a shop name of example rather than the full address.
My logs are full of rate limit warnings. Is that a problem?
No. The connector backs off and continues, so those warnings are normal operation rather than failures.
Can I do this without writing code?
Yes. The setup above is entirely UI-driven. You will want Redshift views joining orders to your advertising and support data, which is where the analysis lives.
Get your Shopify data into Amazon Redshift
Grant every scope in one pass and record what you granted, because a missing one produces an absent table rather than an error. Provision the staging bucket in your cluster's region, sort on the order date, and check every expected stream produced rows before anyone builds on the data.
Airbyte's connector catalog includes 700+ pre-built connectors, so commerce data can sit beside the marketing and support sources that explain it. For the same source in a lakehouse, see Shopify to Databricks, and for payment data alongside it, Stripe to Amazon Redshift.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.