Air-Gapped AI: Running Models Where Data Cannot Leave

Most teams treat this as a security ranking with the air gap at the top. Ranking models that way hides the decision that matters: whether the controls survive contact with production. Sovereignty in regulated AI rests on durable control over every boundary crossing, and that control has to hold through routine updates, failures, audits, and changes in agent behavior. A design that looks isolated at launch develops dependencies on external systems within a year of operating it. Pick the model whose controls your organization can sustain, not the one that sounds most restrictive, whatever a vendor calls it.

TL;DR

• Four isolation models sit on the sovereignty spectrum, and hybrid in-boundary architecture covers most of the regulated workloads that assume they need a disconnected one.
• Almost no regulation categorically requires disconnection. DoD IL6 is the strongest case, and it turns on your authorization boundary rather than on the rulebook.
• Model weights and enterprise data are two supply chains crossing into the same boundary, and both need checksums, signatures, lineage headers, and approval gates.
• Agent context assembled at query time is a third crossing that no approval gate has touched. Pre-materialize it in-boundary instead.
• Tokenizer caches and hidden egress dependencies break after launch rather than during validation, so run a default-deny egress test before you order hardware.

Try Airbyte Flex

Which Isolation Model Should You Choose for Your AI Workload?

Choose the least restrictive model that satisfies your sovereignty, access, and legal requirements. Virtual Private Cloud (VPC) isolation, a sovereign cloud region, confidential computing, and a true air gap differ in who can reach your data, who can compel access, and whether a network path remains at all.

A hybrid architecture keeps your data plane in-boundary while a cloud control plane handles orchestration. Data, credentials, and compute stay inside your environment, but the control plane still needs an outbound path, and removing that path is exactly what a true air gap does. The air-gapped deployment choice therefore sits beyond hybrid on the sovereignty spectrum rather than being a stricter version of it. A true air gap is the only model that removes the network path to connected external systems, including the public internet. Your model weights, prompts, embeddings, retrieved context, outputs, and telemetry then live on an isolated network, and every artifact enters through a controlled crossing.

Sovereign regions from US-headquartered providers can satisfy residency requirements while leaving jurisdictional access risk in place. CLOUD Act exposure follows the chain of corporate control even when the servers sit in another country, so operational separation alone does not resolve the jurisdictional question. Residency and jurisdiction are separate tests, and a procurement checklist that asks only where the racks are will pass an architecture that fails the second one.

Confidential computing narrows access to the customer, but only when an independent party verifies attestation. A recent hardware-attested inference design integrates Intel TDX CPU trusted execution environments with confidential NVIDIA H200 GPUs and attestation at both the virtual machine and application layers. If the provider both operates your trusted execution environment (TEE) and verifies its own attestation, you are still trusting the provider. An independent verifier such as Intel's Trust Authority breaks that circularity, and without one, confidential computing is a strong encryption story rather than a sovereignty story.

The four models differ most in who can reach the data and who can compel access.

Each step down that table buys a narrower access surface and charges for it in operational overhead. The bill comes due in capacity planning long before it shows up in an audit finding.

When Does Your In-Boundary Capacity Beat On-Demand Infrastructure?

In-boundary capacity beats on-demand infrastructure when you sustain high utilization and can absorb the staffing and facilities costs. It loses when capacity sits idle or when demand changes faster than you can provision hardware.

Compared with lower-priced external services, an arXiv cost-benefit analysis finds that infrastructure break-even can extend to several years. The tradeoff works when privacy, sovereignty, predictable capacity, or freedom from a single vendor justifies the cost difference.

An on-premise TCO calculation includes staffing and facilities well beyond the hardware line. Check utilization before anything else, because idle capacity can cost more than the on-demand infrastructure it replaced, and sovereignty then has to justify the whole difference rather than part of it. That is a defensible position to hold in front of a CFO, but only if a regulation actually forces the architecture.

Which Regulations Require You to Consider an Air Gap?

Most regulations governing these workloads stop short of requiring you to disconnect the network. DoD IL6 presents the strongest case for a disconnected or classified-network architecture, and even there you validate the requirement against the controlling contract, the data classification, and the authorization boundary.

Most Rules Let You Keep a Controlled Network Path

The Health Insurance Portability and Accountability Act (HIPAA) accepts a Business Associate Agreement (BAA) with the cloud provider under HHS cloud guidance where that relationship applies. A BAA forms one part of HIPAA compliance, and the rule imposes no physical air gap of its own.

International Traffic in Arms Regulations (ITAR) gets overstated more often than any other rule on this list. The ITAR carve-out, effective March 25, 2020 and codified at 22 CFR § 120.54, applies only to unclassified technical data and turns on end-to-end FIPS 140 encryption that stays unbroken in transit. Classified ITAR data has separate handling rules, so you must distinguish the data classification before choosing an architecture. Classification, encryption, access, and destination together determine export control compliance, and no single deployment model applies universally.

Defense Rules Depend on Scope

Cybersecurity Maturity Model Certification (CMMC) Level 2 scopes controls to a Controlled Unclassified Information (CUI) enclave under 32 CFR Part 170, and organization-wide scope is not automatic. Enclave scoping lets you design a dedicated authorization boundary, though it does not by itself establish compliance or satisfy every required control, and FedRAMP equivalency may apply on top of it.

DoD IL5 can use dedicated infrastructure, US-citizen personnel restrictions, and FedRAMP High with Committee on National Security Systems Instruction (CNSSI) 1253 overlays. IL6 handles information classified up to SECRET and may involve Secret Internet Protocol Router Network (SIPRNet) connectivity and facility clearance requirements that make ordinary commercial connectivity unsuitable. Your authorizing authority evaluates those requirements and determines whether the architecture has to run disconnected, which is why IL6 is the one case on this list where the answer is assessed directly rather than read off a rule.

European Rules Emphasize Governance and Concentration

Regulators actively enforce DORA, whose infrastructure requirements address concentration risk rather than connectivity. On November 18, 2025, the European Supervisory Authorities designated 19 CTPPs, or Critical ICT Third-Party Providers, including Amazon Web Services, Google Cloud, Microsoft, Oracle, and SAP. If your firm depends on them, you document and manage that concentration; nothing in DORA tells you to disconnect.

The EU AI Act follows a staged implementation timeline, with Article 50 transparency obligations applying from August 2, 2026 and other requirements taking effect earlier or later. Article 10 drives infrastructure, since its data-governance requirements include audit trails for training-data provenance across training, validation, and test sets. GDPR behaves the same way: it sets no categorical air-gap requirement and instead makes data location, control, transfer mechanism, and potential compelled access four separate risks to assess.

Your data sovereignty systems have to distinguish where data sits, who controls it, and which legal authority can compel access. Once you have settled which model the rules actually demand, the harder engineering question is how anything gets into the boundary you just drew.

How Do You Move Data Into Your Boundary Without Breaking It?

Govern model artifacts and enterprise data as two supply chains crossing into the same boundary. Both need checksums, signatures, lineage headers, and approval gates so an assessor can trace an answer back to the weight file and the source record behind it.

Model Weights Are a Supply Chain Crossing

A weight file carries a license, a provenance history, and its own dependencies, so treat every crossing into an air gap as a supply chain decision. Record a checksum for every weight file, because inside the gap that checksum is the only provenance evidence you have.

Package each release as a bundle holding the model artifact, container image digest, tokenizer files, evaluation report, and model card. A release specification can store the bundle in an Open Container Initiative (OCI) registry with MLflow for metadata and MinIO for immutable artifacts. Cosign signs the bundle so the receiving environment verifies integrity without reaching back to the internet.

Enterprise Data Is the Second Crossing

Stage records from Enterprise Resource Planning (ERP), Electronic Health Record (EHR), and SaaS systems through CDC. Approve them, transfer them, then reconcile against the source once the transfer lands. Connected extraction design for hybrid ETL pipelines keeps that movement controlled without requiring physical media, and a cloud-orchestrated hybrid control plane can schedule it wherever an outbound path exists. A true air gap also runs the scheduler inside the boundary.

Attach lineage headers at ingestion, before the system writes vectors to the index. According to CSO Online's guidance, store source tags and access permissions alongside the vector metadata. Chunking is where lineage most often breaks, since one parent record becomes several child chunks and each child has to inherit the parent's permission attributes explicitly. Deletes and permission changes follow the same capture path, and freshness stays visible from source commit through retrieval.

Agent Context Assembled at Runtime Is an Unapproved Egress Path

If your agent pulls Salesforce, Zendesk, and Stripe records at query time, the request and the returned records create a bidirectional boundary crossing that no approval gate, checksum, or lineage header has touched. An in-boundary agent stack retrieves against a local index instead, assembling context from data your organization replicated in advance and resolving tool calls against services inside the boundary. Vendor APIs stay outside the runtime path.

Context pre-materialized through the enterprise-data crossing inherits that crossing's provenance chain, and runtime assembly inherits none. Agent write actions belong on the same footing: route them through approved in-boundary services with explicit authorization, audit logging, and lineage for every change. Getting the crossings right at design time still leaves the dependencies nobody wrote down.

What Breaks for You After Day One in a Disconnected AI Stack?

Run your current stack inside a network namespace with a default-deny egress rule before you buy hardware. The test surfaces hidden egress dependencies, and each one maps to a capability the disconnected version has to provide locally.

Tokenizer Caches and Hidden Egress Dependencies Fail After Launch

Many machine learning libraries download tokenizer files from Hugging Face on first use. An artifact that passed validation at launch fails later, when a worker pod restarts and the cached tokenizer is gone.

Pre-stage tokenizer files in the local registry first, then container images, Python dependencies, and evaluation datasets, and point every component at the local path through configuration. The default-deny test surfaces the rest of the list, including external evaluation dependencies, vector database telemetry, and the Slack webhook your evaluation pipeline pings on regression.

Patching Needs a Signed Out-of-Band Path

Red Hat's mirror-factory pattern keeps a controlled environment that retrieves, validates, and prepares platform dependencies before anything transfers into the isolated network. Add an emergency path to the same physical-media workflow to keep patching lag bounded. Your team can certify a signed bundle faster than the standard cadence, and a critical Common Vulnerabilities and Exposures (CVE) fix then avoids waiting for the next scheduled crossing.

Keep monitoring container images after they reach production, including rescanning them after publication. Plan triage against a mirrored vulnerability feed, since live National Vulnerability Database (NVD) access is unavailable inside the gap.

Observability Has to Stay Local

Keep metrics, traces, and drift statistics inside your boundary, and let only redacted diagnostic packs leave, through the same approved crossing the artifacts used to enter. Local logging and lineage provide the evidence an assessor asks to see, and a hybrid deployment produces the same evidence trail without the media-transfer overhead.

Use a readiness test to confirm your team can complete each required task within defined times and without unplanned connectivity. The test covers importing, verifying, and promoting a critical fix, reconstructing an existing release, exporting a redacted diagnostic pack, and rolling back. Whichever model survives that test, the data still has to arrive.

How Does Airbyte Flex Keep AI Data Inside Your Boundary?

Airbyte Flex runs the enterprise-data crossing as a hybrid deployment. Airbyte operates the control plane while the data plane, along with your records, credentials, keys, and compute runs in your environment, though some metadata such as cursor and primary-key values sits in the control plane. Its 700+ connectors move records from ERP, EHR, and SaaS sources into your in-boundary data plane without routing them through Airbyte's cloud, and its open-source foundation lets your security team review the connector code before it touches regulated data.

Airbyte moves 26 billion records daily for 7,000+ companies, including 18% of the Fortune 500. For AI workloads, the same replicated records are what an in-boundary retrieval stack indexes, so agent reads inherit the boundary and access controls the pipeline already enforces.

Airbyte works with orchestration frameworks such as LangChain, CrewAI, and LlamaIndex rather than replacing them, and Flex sits at the hybrid point on the sovereignty spectrum, which is where most of the workloads described above actually land.

Where Should You Start?

Start with the default-deny egress test on the stack you already operate, before ordering any hardware. The dependency list it produces is the real scope of your air gap, and for most teams that list is short enough that a hybrid deployment or confidential computing covers the requirement. A classified authorization is the case that pushes past both.

Airbyte supports that path with role-based access control, organization-level audit logging on supported paid tiers for events such as connection, permission, and source changes, and CDC replication into a data plane you operate. Airbyte Flex is the deployment model for teams that need the full connector catalog running inside their own boundary.

Get a demo to see how Flex runs in your environment.

Frequently Asked Questions

How Do You Keep Retrieval Data Fresh When Your Boundary Blocks Continuous Sync?

Retrieval answers are only as current as the last approved crossing, so the honest move is to expose that age rather than hide it. Lineage headers carried from source commit through chunk to retrieval let you state an answer's age at query time. Reconcile after each scheduled transfer so you can prove the transfer omitted nothing between source and index.

Which Open-Weight Licenses Should You Check Before You Deploy?

Check attribution, use restrictions, distribution terms, and commercial thresholds for each artifact version, and record the license and provenance history for every weight file that crosses into the boundary. The Llama Community License adds a 700 million monthly-active-user threshold and requires "Built with Meta Llama 3" attribution. Qwen3 and gpt-oss ship under Apache 2.0 and DeepSeek under MIT, while Mistral splits its catalog between Apache 2.0 and a non-production license that restricts commercial use.

Does an Air Gap Remove Your Insider and Removable-Media Risk?

No. Infected removable media routinely crosses an air gap, and delayed patching inside the gap increases lateral-movement exposure once an attacker is already in. Many nominally air-gapped systems also communicate with a connected IT network, which means they lack the complete isolation their classification implies.

How Many New PHI Stores Does Your Hospital AI Feature Create?

In most cases, more than the original architecture review recorded. A retrieval index, trace log, evaluation set, and their backups each hold protected health information (PHI), and each rarely appears on the first inventory. Every one belongs on the list an assessor will ask for.