The Short Answer
Healthcare AI's largest security exposure is not prompt injection — it is the default cloud architecture that sends protected health information outside the hospital's perimeter on every request, by design. A business associate agreement allocates liability; it is not a data-location control. On ibl.ai you own all the code and the data, self-hosted on-premise, in your VPC, or fully air-gapped, so PHI never leaves your covered environment.
That is an architecture claim, not a vendor claim, and HIPAA's own text is where it gets settled. The sections below read the regulation — 45 CFR 164.504(e)(2), 164.306, 164.312 — clause by clause against one question: does any of this keep PHI inside the hospital?
Is the biggest healthcare AI security risk prompt injection or the deployment architecture?
Ask a health system's security team to rank AI risks and the list usually opens with prompt injection, model jailbreaks, hallucinated clinical content, and over-broad access. That ranking is not invented.
OWASP's Top 10 for LLM Applications places prompt injection at LLM01 — the number-one entry on its list of ten — in the 2026 edition published in August 2026 as in the 2025 edition before it.
Those risks are real and belong on the register. But a threat model is a ranking, and the ranking is wrong when it leaves the largest exposure off the list entirely.
The exposure it leaves off is the data path. In a cloud AI deployment, every clinician prompt — a symptom description, a lab value, a draft note — crosses the hospital's network boundary and is processed on infrastructure the hospital does not operate.
That is not a vulnerability. It is the product working as designed.
The U.S. Department of Health and Human Services published its own read of where healthcare loses data in the January 6, 2025 proposed Security Rule update (90 FR 898).
Between 2018 and 2023, the Department reported, breaches of unsecured PHI reported to it rose 100 percent and the number of individuals affected rose 950 percent, with hacking up 260 percent and ransomware up 264 percent.
In 2022, the same document states, "approximately three-fourths of the breaches of unsecured PHI affecting 500 or more individuals were the result of hacking of electronic equipment or a network server." In 2023, over 160 million individuals were affected by 500-plus-individual breaches — a new record.
A network server. The breach record is dominated by infrastructure the data was sitting on, not by a model being talked out of its instructions.
Does a business associate agreement keep PHI inside the hospital's network?
No. A business associate agreement is a contract, and the regulation specifies exactly what that contract must contain. The implementation specifications at 45 CFR 164.504(e)(2) enumerate, exhaustively, what that contract must do.
Read them against one question — does this clause keep PHI inside my perimeter? — and the answer is the same every time.
| Required BAA provision (45 CFR 164.504(e)(2)) | What it actually does | Keeps PHI inside your perimeter? |
|---|---|---|
| (i) Establish permitted and required uses and disclosures | Defines what the vendor may do with PHI it already holds | ✗ |
| (ii)(A) Not use or further disclose other than as permitted | A promise about behavior, enforced after the fact | ✗ |
| (ii)(B) Use appropriate safeguards, comply with subpart C | Obliges the vendor to secure its own environment | ✗ |
| (ii)(C) Report unpermitted uses, disclosures and breaches | Notification after the loss has occurred | ✗ |
| (ii)(D) Flow the same terms down to subcontractors | Extends the contract chain — and the data path with it | ✗ |
| (ii)(E)–(G) Make PHI available for access, amendment, accounting | Patient-rights plumbing, routed through the vendor | ✗ |
| (ii)(H) Comply with obligations it carries out for you | Transfers duties, not data location | ✗ |
| (ii)(I) Make books and records available to the Secretary | Audit access for OCR, on the vendor's premises | ✗ |
| (ii)(J) Return or destroy all PHI at termination, if feasible | Presumes the PHI is already in the vendor's custody — and says "if feasible" | ✗ |
| (iii) Authorize termination for material breach | The enforcement mechanism: cancel the contract after you find out | ✗ |
Every required provision, and not one data-location control among them. Every one of them presupposes that PHI has already moved to the business associate and governs what happens next.
The structure is visible in the remedy.
Under 164.504(e)(1)(ii), a covered entity falls out of compliance if it knew of a pattern of material breach by its business associate and failed to "cure the breach or end the violation" and, failing that, terminate the contract "if feasible."
That is a liability-allocation instrument working exactly as written. It is not a control on where the data lives. Our longer treatment of the contractual layer is in HIPAA-Compliant AI: Why a BAA Alone Is Not the Answer in 2026.
What does the HIPAA Security Rule actually require about where PHI is processed?
Nothing specific — and that is the point most procurement reviews miss. The Security Rule names no architecture, no region and no hosting model. It delegates the decision to the covered entity and then audits the reasoning.
45 CFR 164.306(b) is titled "Flexibility of approach." It permits regulated entities to "use any security measures" that reasonably implement the standards, then lists four factors they must weigh — including "the probability and criticality of potential risks to electronic protected health information."
That is a threat-ranking instruction written into the regulation. If the ranking omits the data path, the risk analysis required at 164.308(a)(1)(ii)(A) is incomplete by the rule's own standard, however complete the prompt-injection controls are.
Two more clauses sharpen it. Encryption of ePHI at rest (164.312(a)(2)(iv)) and encryption in transmission (164.312(e)(2)(ii)) are both marked Addressable, not Required — assess, implement if reasonable, or document why not and substitute an equivalent measure.
A rule that will not mandate encryption is plainly not going to mandate where your inference runs. The architecture is yours to choose, which means it is also yours to defend.
Who is liable when a healthcare AI vendor is breached, and what does the BAA give you then?
Liability is shared, and the notification duties run toward you rather than away.
Under 164.504(e)(2)(ii)(C) the business associate reports breaches of unsecured PHI to the covered entity; the covered entity then carries the Breach Notification Rule obligations toward patients, the Secretary and in large breaches the media.
In practical terms: the vendor's infrastructure fails, and the health system's name is on the OCR breach portal entry. The contract can allocate cost. It cannot allocate the incident.
The Department has acknowledged that satisfactory assurances on paper are thin.
In the January 2025 proposal it would require a regulated entity to obtain written verification, at least once every 12 months, that a business associate has deployed the technical safeguards at 164.312 — including a written analysis of the associate's relevant electronic information systems.
That proposal is not law. The comment period closed March 7, 2025, and as of October 2026 no final rule has been published; the current Security Rule remains the rule in force. Treat it as a signal about OCR's thinking, not a requirement — but note what the signal says: the Department itself does not consider a signature sufficient evidence of a control.
The financial exposure the Department cited while making that argument was an average of "almost $10.1 million per breach" for healthcare, and a survey finding that 92 percent of surveyed health care organizations had experienced a cyberattack in the prior year.
What changes when healthcare AI runs on-premise or air-gapped?
The data path changes, and with it the question an auditor can ask. When inference runs inside the hospital's own environment, PHI never crosses the boundary that already protects the EHR, so there is no third-party custody chain to govern.
Three deployment shapes do this, in ascending strictness. Managed VPC keeps PHI in the health system's own cloud tenancy. On-premise runs it on hospital hardware. Fully air-gapped removes outbound connectivity entirely — no API call to make, so no exfiltration path to monitor.
The clinical capability is the same work: documentation drafting, prior-authorization letters, coding support, patient-education material, message triage. What differs is whose audit log it lands in and whose retention policy governs it.
On ibl.ai you own all the code and the data — the full source under a perpetual license, running on your infrastructure, model-agnostic across any LLM you choose, with no per-seat pricing. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.
ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.
For regulated buyers evaluating where PHI will sit for the next decade, the ownership structure of the counterparty is part of the architecture question.
Related reading on the engineering side: Most Healthcare AI Pilots Never Reach Production — It Is an Architecture Problem and HIPAA-Compliant AI: Keeping PHI on Your Own Infrastructure.
Does per-seat cloud AI pricing make the healthcare architecture problem worse?
Yes, because per-seat pricing and third-party PHI custody are the same purchase.
Per-seat SaaS is the wrong shape for clinical AI before you reach the compliance question: the bill scales with headcount rather than with work done, so a health system pays for every badge whether that clinician drafts a letter or not.
The architecture consequence is the part usually left out of the spreadsheet. Every per-seat and every token-metered row puts PHI in someone else's cloud under a BAA. Only the self-hosted row does not.
Our full cost breakdown for a 5,000-clinician health system runs the prior-authorization workload across deployment shapes. Read the right-hand column:
| Deployment | Pricing shape | Monthly | Where PHI is processed |
|---|---|---|---|
| ChatGPT Enterprise | Per-seat, ~$60/user | $300,000 | Vendor cloud, under a BAA |
| Microsoft 365 Copilot | Per-seat, ~$30/user | $150,000 | Vendor cloud, under a BAA |
| Direct frontier API | Token-metered | ~$240–$500 | Vendor cloud, under a BAA |
| ibl.ai, self-hosted | Usage-based, no per-seat pricing | GPU + license | Inside your covered environment |
Figures are from our own 2026 cost analysis at 5,000 clinicians and 10,000 prior-authorization requests per month; the per-seat list prices are the vendors' published rates as cited there. The architecture column is the one the BAA cannot change.
What three questions should a health system ask before approving an AI tool?
Three, and all three are architecture questions a procurement reviewer can answer in writing before signature.
1. Where is PHI processed when a clinician uses this? If the answer names a third party's infrastructure, the health system has accepted a data path outside its perimeter. The BAA documents that acceptance; it does not reverse it.
2. Who chooses and changes the model? If the vendor can deprecate, update or restrict the model without the health system's involvement, the system has clinical dependency without operational control — and a validated workflow can change underneath it.
3. What happens the day you leave? If exit requires data migration, re-integration and re-validation, the lock-in compounds with every workload added. Owning the source code and the data makes exit an operational task rather than a renegotiation.
Those are the questions ibl.ai for healthcare is built to answer with a deployment diagram instead of a contract clause. The prompt-injection controls still matter — they just belong below the data path on the register, not above it.