📅 Book a 30-min Demo📞 Call/text (571) 293-0242
advanced 14 min read

How to Deploy Air-Gapped AI

Running useful AI on a network with no route to the internet — model choice, hardware, updates across the boundary, and how to verify the gap actually holds

On ibl.ai you own all the code and the data, run it model-agnostic across any LLM, and pay with no per-seat pricing — so you can deploy anywhere, from your own cloud to a fully air-gapped network.

Last updated:

How do you deploy Air-Gapped AI?

An air gap makes data exfiltration a topology problem rather than a policy problem. There is no outbound route, so nothing can leave regardless of what any component tries to do.

That constraint rules out every hosted model API by definition and forces three design decisions most AI deployments never confront: which open-weight models to host, how much GPU memory the workload actually needs, and how updates cross the boundary without becoming the weakest link.

This guide covers the sequence that works, and the verification step most teams skip — proving the gap holds rather than assuming it.

Prerequisites

An isolated network segment

A network with no route to the internet, ideally already used for the data the AI will process, so the AI does not become the reason data moves between zones.

GPU capacity sized to the model

Memory is the binding constraint, not compute. A 32GB card comfortably serves a 27–34B model in 4-bit quantization, which covers most enterprise workloads.

Full source access to the platform

Without the source you cannot verify that no component calls out, and cannot patch one that does. Air-gapped operation and code ownership are linked requirements.

A controlled media transfer procedure

A reviewed process for moving model weights and platform updates across the boundary, with provenance verification and a rollback path already agreed.

1

Inventory every outbound dependency before you start

Most platforms assume connectivity in places nobody documents: telemetry, licence checks, font and icon CDNs, package registries at runtime, error reporting, and remote configuration.

List every component and its network behaviour

Read the source; do not rely on documentation.

Identify licence checks that phone home

These fail closed and will take the system down after install.

Find CDN references in the web interface

Fonts, icons and scripts loaded from a public CDN break the UI silently.

Check for runtime package or model downloads
Confirm error reporting can be disabled or redirected internally
Warnings
  • A component that fails open is worse than one that fails closed — it will appear to work while silently attempting egress.
2

Choose open-weight models and pin their versions

Only models whose weights you can download qualify. Pin each by hash so the deployed state is verifiable and reproducible, which is also what makes later audits answerable.

Select models sized to your available VRAM
Record the model hash alongside the version

An endpoint name is not a version; a hash is.

Read the licence for commercial-use conditions

Open-weight licences are bespoke and often carry restrictions.

Validate quality on a held-out set from your own domain
Tips
  • Quantization is usually the difference between fitting and not fitting. Test 4-bit quality on your own evaluation set rather than trusting general benchmarks.
3

Size hardware from measured tokens, not from vendor guidance

Estimate concurrent users, tokens per request and acceptable latency from a representative sample of real work, then size for the peak you actually need to serve.

Measure tokens per request on real documents
Establish peak concurrency, not average
Size VRAM for model plus KV cache at that concurrency
Plan for a second node before you need it

Procurement into an isolated environment is slow.

4

Deploy the platform inside the boundary

Install the full stack — orchestration, retrieval, vector store, agent runtime, admin interface and logging — inside the isolated segment, with no component depending on an external service.

Mirror all package dependencies internally
Host fonts, icons and assets locally
Point logging and metrics at internal storage
Configure authentication against the internal directory
5

Establish the update procedure across the boundary

Model and platform updates arrive on reviewed physical media. Treat cadence, provenance verification and rollback as designed procedures rather than as incidents to improvise through.

Define who reviews and approves media transfers
Verify hashes on arrival against a separately communicated value
Stage updates in a test environment inside the boundary first
Keep the previous model and platform version available for rollback
Warnings
  • Transfer media is the one route into the environment, which makes it the highest-value target. Treat it with the same rigour as the gap itself.
6

Verify the gap holds, then verify it again after every change

This is the step teams skip. An air gap is a claim until it is tested, and a routine platform update can reintroduce an outbound dependency that nobody notices.

Monitor egress attempts at the boundary

You want zero, and you want to know if that changes.

Run the platform with a deliberate deny-all rule and confirm nothing breaks
Re-verify after every platform or model update
Document the verification result as part of the change record

Key Considerations

organizational

Update cadence becomes a deliberate decision

Slower, reviewed updates are a security property rather than a limitation, but the organization must consciously choose the cadence rather than letting it drift to never.

technical

Model capability is bounded by the hardware you bought

You cannot burst to a larger model when a hard task appears. Size for the workload you actually have, and route work the local model cannot handle to a human.

compliance

The audit trail is entirely yours

Logs are written and read inside the boundary in a schema you control, which is a stronger evidentiary position than any hosted deployment can offer.

budget

Capacity is bought up front

Air-gapped deployment converts variable per-token cost into fixed capacity cost, which is worse at low utilization and better as request volume grows.

Success Metrics

Zero, sustained

Outbound connection attempts

Boundary egress monitoring, reviewed after every change

Meets the threshold set before deployment

Model response quality on domain evaluation set

Held-out set drawn from real internal work, re-run after each model update

Within the window your policy specifies

Time to apply a security update

Change records from media transfer to production

Within the agreed service level

Peak-hour latency

Internal metrics at measured peak concurrency

Common Mistakes to Avoid

Assuming on-premise means air-gapped

Consequence: The hardware is yours while the system still calls a hosted API or ships telemetry, so the gap never existed.

Prevention: Test with a deny-all egress rule and confirm the system still works before declaring it air-gapped.

Selecting a model before measuring the workload

Consequence: Hardware is bought for a model that is too large for the budget or too small for the task, and procurement into an isolated environment is slow to correct.

Prevention: Measure tokens per request and peak concurrency on real work first, then choose the model that fits.

Treating updates as an afterthought

Consequence: The environment drifts years behind on security patches because no procedure was agreed while attention was on the initial deployment.

Prevention: Design and rehearse the media transfer procedure during deployment, not after the first vulnerability disclosure.

Never re-verifying the gap

Consequence: A routine update reintroduces an outbound dependency and nobody notices until an audit or an incident.

Prevention: Make egress verification a mandatory step in the change record for every platform and model update.

Can you do this on infrastructure you own?

ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.

1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.

ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.

Frequently Asked Questions

Related Resources

Ready to transform your institution with AI?

See how ibl.ai deploys AI agents you own and control—on your infrastructure, integrated with your systems.