A BAA is a promise, not a wall
When a clinician pastes a note into a public chatbot, protected health information leaves your network. A Business Associate Agreement can make that lawful, but it doesn't change where the data goes.
You are now trusting a vendor's controls, retention, and subprocessors with PHI you no longer hold.
That trust is the whole risk. HIPAA's Security Rule expects you to limit disclosure and control access. HITRUST attestation raises the bar further. A model you can't inspect, running on infrastructure you don't control, makes both harder to demonstrate to an auditor.
The popular tools — ChatGPT, Copilot, Claude — will sign a BAA for enterprise tiers. Useful, but the data still leaves your walls to be processed. For the most sensitive workflows, that's the line many compliance teams won't cross.
Private and on-premise means PHI never leaves
A private LLM runs inside infrastructure you control: your data center, or a cloud tenant under your governance.
Air-gapped goes further, with no path to the public internet at all. The note, the chart, the claim — all of it stays inside the boundary your security team already monitors.
This turns the compliance question inward. Instead of validating a vendor's promises, you apply your existing HIPAA controls — access management, audit logging, encryption, minimum necessary — to the AI the same way you do to your EHR.
Open models like Llama and Mistral now handle clinical summarization, drafting, and coding support at a quality that closed the old gap. Staying private no longer means accepting a weaker model.
Where it earns its place in the clinic
- Clinical documentation: drafting and summarizing notes from your own templates, with the text never leaving the environment.
- Patient education: plain-language explanations grounded in your approved materials.
- Medical coding: suggesting codes against the actual chart, with a human signing off.
- Prior authorization: assembling the supporting record so staff spend less time on paperwork.
Agents reach these through governed connectors to the systems you already run — Epic, Cerner/Oracle Health, athenahealth, Meditech — so there's one audited path, not another copy of PHI living somewhere new.
Owning it matters when the model changes
When a SaaS vendor updates the model behind its product, the behavior you validated changes too, often silently. In a regulated clinical setting, that's a governance gap: you're relying on a model you didn't review and can't freeze.
Owning the deployment closes it. You pick the model, pin the version, validate it, and update on your schedule. The audit trail is yours, and the capability doesn't reset when a vendor ships a release.
That's the basis for HIPAA-compliant AI for healthcare that you own: clinical, coding, and education agents on your servers, air-gapped if you need it, with PHI that never leaves your infrastructure.
A safe first step
Start with a workflow that has clear value and contained risk — internal clinical knowledge search or documentation drafting — and run it private against one department.
Prove the controls and the output quality on real charts, document it for your HIPAA program, then expand once the governance holds.