The Short Answer
Researchers at USENIX Security 2025 reconstructed a local LLM's output text from CPU cache access patterns alone, at a 5.2% average edit distance, using unprivileged code on the same machine. Air-gapping closes the network boundary, not the host boundary. With ibl.ai you own all the code and the data, so you control tenancy, host access and process isolation — the controls that actually decide this.
Running a model on your own servers and running it where no other process can observe it are two different guarantees, and only the first one comes for free.
What did the USENIX Security 2025 cache side-channel paper actually recover?
Both the input and the output text of a locally-running LLM, at high fidelity.
The paper is I Know What You Said: Unveiling Hardware Cache Side-Channels in Local Large Language Model Inference, by Zibo Gao and seven co-authors at the Institute of Information Engineering, Chinese Academy of Sciences.
It was posted to arXiv on 10 May 2025 and presented at the 34th USENIX Security Symposium that August.
Two leaks are chained together. Token values come from cache access patterns in the token-embedding lookup: a cache hit on row j of the embedding weight matrix indicates that the token with index j was processed.
Token positions come from the timing of the autoregressive decoding steps.
A reconstruction algorithm fuses the two signals, and a fine-tuned model restores fluent text from the recovered token list.
The reported accuracy is specific. Restored output text had an average edit distance of 5.2% from ground truth and restored input text 17.3%, with average cosine similarity above 98% for both.
The evaluation covered Llama, Gemma, Falcon, Mistral and Phi across ten inference frameworks.
One correction to how this circulates: it is not a discovery from this week. The core result has been public for sixteen months and peer-reviewed for over a year.
Does the cache side-channel attack on local LLMs require privileged access or co-residency?
Co-residency yes, privilege no. That combination is the entire security story.
The attacker has to execute unprivileged code on the same physical machine as the inference process, sharing a last-level cache with it. It never interacts with the victim's LLM — no prompt, no API call, no network path to the model at all.
It also needs the model weights to be physically shared memory. Mainstream frameworks memory-map model files for zero-copy loading, so attacker and victim touch the same physical pages.
OS page deduplication is the fallback route to the same condition, and the measurement primitive is Flush+Reload.
One condition matters a great deal for defenders: the token-embedding lookup has to run on the CPU. That held in nine of the ten frameworks tested — only HuggingFace Transformers kept the embedding on the GPU, and that was the single configuration in which the attack failed.
The authors state their own limits clearly. Input reconstruction degrades as inputs get longer, GPU-side embedding is out of reach without a separate GPU cache attack, and the attack assumes the tokenizer and model file format are known.
They also report having disclosed the issues to the affected software developers.
Why doesn't air-gapping an on-premise LLM prevent a cache side-channel attack?
Because air-gapping is a statement about the network boundary, and this attack lives entirely inside the host boundary.
The framing worth discarding is that air-gapping fails here against physics. Nothing is defeated by physical law. The attack needs an adversary's process scheduled on the same silicon as the inference process, which is an access-control fact about the host, not an inevitability.
That distinction is what makes it actionable. An air-gapped host running one workload, with no untrusted code and no interactive access for anyone outside the admin group, does not satisfy the paper's threat model.
An internet-isolated Kubernetes cluster that schedules the inference pod onto a shared node pool alongside arbitrary tenant workloads satisfies it precisely — inside the covered boundary, behind every firewall.
The boundary also keeps moving. SLAC, posted in August 2026 and accepted to ACM CCS 2026, extends the idea to the system-level cache that CPU and GPU share on Apple silicon, again from an unprivileged co-resident process.
It recovers input keywords at up to 94.8% and response tokens at up to 88.9% — but those headline numbers come from its GPU-priming variant, and its LLM results are on TinyLlama-1.1B and GPT-2 Medium.
The authors state the remaining conditions plainly: a restricted vocabulary of the 3,000 most frequent tokens, domain dictionaries of 100 to 1,000 keywords, and results averaged over 1,000 to 4,000 runs on an M1.
They propose no concrete mitigation beyond cache-hierarchy redesign.
Which controls actually reduce cache side-channel exposure on an inference host?
Four, and none of them is a network control.
Tenancy. Dedicated inference hosts rather than a shared scheduling pool. This is the control that does the most work, because it removes the co-residency the attack depends on.
Who can execute code on the host. Containers are not a cache boundary. Namespaces and cgroups partition filesystems, processes and CPU shares; they do not partition a last-level cache.
A container scheduled beside the inference container is co-resident in the sense that matters here.
How model weights are loaded. Disabling zero-copy memory-mapped loading removes the shared-page condition, at a cost the authors measured as 17% slower model loading, and about 32% extra memory once two or more instances run concurrently.
Restricting page sharing to designated inference sessions is the lower-cost version of the same idea.
Hardware isolation. Cache partitioning through Intel Cache Allocation Technology isolates the last-level cache, though the paper notes it is absent from the consumer-grade hardware where many local deployments run.
Confidential computing goes further and costs more. One 2025 measurement of H100 confidential-computing mode reports non-CC latency 20–30% lower and non-CC throughput 45–70% higher than confidential mode, in a single-GPU batch-inference configuration with model swapping.
Where the framework allows it, moving the embedding lookup onto the GPU also removes the specific signal this attack reads.
Does this mean self-hosting an LLM is less safe than using a managed AI API?
No, and the paper does not support that reading.
Every control above is a decision about a host you operate, and a managed API buyer cannot make any of them. They cannot specify tenancy, enumerate what else runs on the inference node, pin workloads, or verify any of it after the fact.
The provider may run a well-isolated fleet. The customer's evidence for that is an attestation document.
The asymmetry is not that self-hosting is automatically safer. It is that self-hosting is the only arrangement in which the question has an answer you can check.
That is the same argument that makes an air-gapped clinical AI platform survive a third HIPAA review, applied one layer below the network diagram.
It comes with an obligation. Ownership of the host is what makes host-level isolation possible, and it is also what makes it your responsibility.
A self-hosted deployment on a shared, multi-tenant node pool with broad developer shell access has the ownership and none of the isolation.
How does ibl.ai isolate inference hosts for regulated deployments?
By putting the whole stack on infrastructure the institution controls, so tenancy and host access are configuration rather than a vendor policy.
With ibl.ai you own all the code and the data.
The platform self-hosts inside your own perimeter with full source code access — so the loading path, the scheduling and the isolation posture are all inspectable rather than asserted — is model-agnostic across any LLM and switchable at any time, is usage-based with no per-seat pricing, and can deploy anywhere from your own cloud to on-premise, GovCloud, or a fully air-gapped network.
For a security reviewer, the practical consequence is that the questions in this post are answerable.
You can state which hosts run inference, what else is scheduled on them, who can execute code there, and how weights are loaded — then verify each answer against a system you operate.
1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.
ibl.ai is family-owned and operated from New York, NY.
Related reading: air-gapped clinical AI platform — what the network boundary does buy a HIPAA-covered health system, and where it stops.
Sources: the attack, threat model, accuracy figures and mitigations from Gao et al., USENIX Security 2025; the CPU-to-GPU extension and its stated conditions from SLAC; the confidential-computing overhead measurement from Martínez Ibarra et al..