Netsuite to ClickHouse: How to Move Your Data
Move NetSuite into ClickHouse with Airbyte. The enterprise connector tier, assembling token based authentication, and why financial totals need FINAL.
Moving NetSuite into ClickHouse puts financial data behind an engine fast enough to sit under a live dashboard. NetSuite holds the authoritative record of what was sold, invoiced and paid, and asking it to answer an aggregate query while running your business is not what it was built for.
Two things shape this build. The credential chain needs an administrator and four values from two places, and ClickHouse collapses duplicate rows on its own schedule rather than yours, which matters when the figures are financial.
NetSuite to ClickHouse at a glance:
Why move data from NetSuite to ClickHouse?
Two situations account for most of these pipelines.
The first is serving a dashboard people refresh constantly. A revenue or orders view that somebody checks hourly is a poor use of NetSuite's capacity and an excellent use of a columnar engine, which answers the same aggregate in milliseconds.
The second is joining financial records to high-volume data you already hold. Matching invoices against product events or support activity is a join, and doing it in the engine that holds the heavy side avoids moving anything twice. For governed, long-term reporting, a warehouse suits better.
What do you need before you start?
One commercial conversation, and a credential chain that needs an administrator:
An Airbyte agreement covering this connector. NetSuite is a premium enterprise connector available at an additional cost, so it is not something you switch on from the catalogue. The NetSuite source documentation covers what it includes.
Token based authentication enabled, and four credential values. Switched on under setup, company and enable features, on the SuiteCloud tab. An integration record gives you a consumer key and secret, and an access token gives you a token ID and secret. Each is displayed once.
Your realm, which is the account ID. Found in your NetSuite account settings. Sandbox accounts carry a suffix, so a realm copied from a test environment will not work against production.
A ClickHouse instance and credentials. With permission to create tables in the target database. The destination writes through the native protocol into typed columns, which suits financial figures better than storing everything as JSON.
If your ClickHouse instance restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list before you begin.
How do you build a NetSuite to ClickHouse pipeline in Airbyte?
Step 1: Get an administrator and scope the role
Book time with whoever administers NetSuite, because the whole credential chain needs them. While you have their attention, decide what the integration role can see, since that role defines the boundary of your dataset and widening it later means going back to them.
Step 2: Configure the NetSuite source
Click Sources in the left navigation, then New Source, and select NetSuite, following adding a source. Supply the realm, consumer key and secret, token ID and secret, and a start date. Then refresh the source schema and look at what appeared, because a NetSuite catalogue reflects that specific account's configuration.
Step 3: Configure the ClickHouse destination
Click Destinations, then New Destination, and select ClickHouse, following adding a destination. Supply host, port, database and credentials. Choose your sort order deliberately, usually on the transaction date, since on a columnar engine that affects query performance more than anything in the pipeline.
Step 4: Select records and deduplicate
Click Connections, then New connection, select the streams your dashboard needs and choose incremental append and deduped as the sync mode. Take what you report on rather than everything available, since an ERP holds a great deal and most of it will never reach a dashboard.
Then put any query producing a financial total behind a view using FINAL, for the reason set out below.
Why does no two NetSuite accounts look alike?
Because NetSuite is configured rather than merely used. Custom fields, custom record types and renamed standard fields are normal practice, and an implementation partner will have spent months shaping the account around how that business works.
So the catalogue you see is specific to your account, and generic guidance about which streams to take is worth less here than almost anywhere else. The person who knows what a custom field actually means is whoever configured it, and that conversation is part of the build rather than optional background.
On this destination that conversation also shapes your sort order, because knowing which fields people filter and group by is what makes a columnar table fast. Getting it from the person who configured the account beats guessing from column names.
Why do financial totals need FINAL?
Because ReplacingMergeTree collapses duplicate rows during background merges rather than on write. A query run shortly after a sync can legitimately return rows that the same query returns cleanly an hour later, which is expected behaviour rather than a fault.
On most sources that is a minor annoyance. On financial data it is worse, because a revenue figure that differs depending on when somebody looked is the kind of discrepancy that ends in a meeting rather than a bug report. Finance teams reasonably expect a total to be a total.
So put anything summing money behind a view using FINAL, and tell your analysts which tables are safe to query directly. The cost is a slower query on the few places exactness matters, which is a trade worth making when the alternative is explaining why yesterday's number moved.
Frequently asked questions
Is the NetSuite connector included in my plan?
It is a premium enterprise connector available at an additional cost, so it needs a commercial conversation rather than a configuration change.
Why does my revenue total change between queries?
ReplacingMergeTree removes duplicates during background merges rather than immediately. Query with FINAL wherever exactness matters, which on financial figures is most places.
Why will the connector not authenticate?
Check token based authentication is enabled in NetSuite, then check the realm, since sandbox account IDs carry a suffix and will not work against production.
Why does my catalogue look different from the documentation?
Because NetSuite accounts are heavily customised. Custom record types and fields are normal, so your catalogue reflects your own configuration rather than a standard shape.
Can I do this without writing code?
Yes. The setup is UI-driven on both sides, though the NetSuite half needs an administrator. You will want ClickHouse views using FINAL for financial totals.
Get your NetSuite data into ClickHouse
Book the administrator early because the credential chain depends on them, take only the records your dashboard needs, and choose a sort order with the person who configured the account. Then put financial totals behind views using FINAL, because a number that moves between queries is a credibility problem.
Airbyte's connector catalog includes 700+ pre-built connectors, so ERP data can sit beside the events that explain it. For the same source in a warehouse, see NetSuite to Snowflake, and for payment data in the same engine, Stripe to ClickHouse.
Integrate with 700+ apps using Airbyte
Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.