--- title: "When Compliance AI Hallucinates, Who Audits the Filing?" slug: "compliance-ai-hallucinations-audit-trail" author: "Jaione Amigot" date: "2026-08-15 10:00:00" category: "Premium" topics: "legal ai, compliance ai, ai governance, ai hallucinations, financial services, sec, finra, audit trail, self-hosted ai" summary: "A 125-year-old law firm was ordered to explain AI-hallucinated citations in a court brief. The same class of tool now drafts SEC and FINRA filings, where the reviewer is an examiner rather than a judge. The difference between a sanction and a clean examination is whether you can reconstruct what the model saw — which is an infrastructure property, not a model one." banner: "/images/blog/compliance-ai-hallucinations-audit-trail/ai-hallucinations-compliance-risk-cover.webp" thumbnail: "/images/blog/compliance-ai-hallucinations-audit-trail/ai-hallucinations-compliance-risk-cover.webp" linkedin: | A 125-year-old law firm was ordered to explain AI-hallucinated citations in a court brief. The instinct is to file that under "lawyers being careless with ChatGPT." That reading is comfortable and wrong, because the same class of tool now drafts SEC compliance reports and FINRA filings — where the reader is an examiner, not a judge, and the discovery window is years rather than weeks. A judge catches a fabricated citation because opposing counsel looks it up. Nobody looks up a footnote in a filing until an examination three years later asks how a number was derived. So the interesting question is not "did the model hallucinate." Models hallucinate; that is a known property, and every serious framework already assumes it. The question is what you can produce when someone asks how the output was generated. Answering it requires four things, and none of them are model capabilities: — the exact prompt and retrieved context, kept — the model version and parameters at that moment, not the current ones — the raw output before a human edited it — who reviewed it, and what they changed On a hosted API, most of that is a retention setting you do not control, on a system you cannot subpoena, for a model version that may have been silently replaced between the draft and the examination. On infrastructure you own, all four are just logs on your own disk. This is why "can you audit it" is an architecture question rather than a vendor-assurance question. SR 11-7 has expected documented model inventory, validation, and monitoring since 2011, and regulators have been clear that generative models fall inside that perimeter. You cannot validate what you cannot reconstruct. Compliance AI you cannot audit is a liability dressed up as efficiency. #iblai #LegalAI #Compliance #AIGovernance #FinancialServices #AgenticAI #EnterpriseAI --- ## The Short Answer **Compliance AI is auditable only when you can reconstruct the exact prompt, retrieved context, model version, raw output, and human review for any filing — which requires holding those logs yourself. On ibl.ai you own all the code and the data, so the entire inference path runs and is recorded inside your own perimeter, model-agnostic across any LLM and with no per-seat pricing.** A court ordering a 125-year-old firm to explain hallucinated citations is the visible version of this problem. The invisible version is a fabricated figure in a filing that nobody checks for three years. ## What actually went wrong in the law firm case? A century-old firm submitted a brief containing citations the underlying model invented, and the court ordered it to explain them. The failure was not that a model produced a plausible-looking case reference — models do that, and it is well documented. The failure was procedural: nothing in the workflow required the citation to be resolved against a real source before filing, and nothing preserved what the model was given at the moment it produced the text. Courts catch this quickly because litigation is adversarial. Opposing counsel reads every citation, and a fabricated one surfaces within weeks. That adversarial check is the only reason these cases become public at all. ## Why is a regulatory filing riskier than a court brief? Because nobody is reading it adversarially, and the clock runs much longer. SEC and FINRA filings are reviewed by examiners on a cycle, often years after submission. A hallucinated citation in a brief gets caught by an opponent with an incentive to find it. A fabricated basis for a number in a filing sits until an examination asks how the figure was derived. By then the model version that produced it may have been deprecated, the prompt template rewritten, and the retrieval corpus reindexed. The reviewer is asking a question about a system state that no longer exists. The exposure is not the error rate. It is the gap between when the error is made and when anyone asks about it. ## What does an auditable AI filing actually require? Four artifacts, captured at generation time. None of them are things a model provides:
Artifact Question it answers On a hosted API
Prompt + retrieved context What did the model actually see? Vendor retention setting
Model version + parameters Which system produced this? May be silently replaced
Raw output, pre-edit What did the human change? Usually not captured
Reviewer + revision trail Who signed off? Your system, if you built it
ibl.ai (self-hosted) All four Logs on your own disk
The pattern is that three of the four are decisions someone else makes for you unless the inference path is yours. ## Do existing model-risk rules already cover generative AI? Yes, and this is the part most teams underestimate. **SR 11-7**, the Federal Reserve and OCC model risk management guidance, has required documented model inventory, independent validation, and ongoing monitoring since 2011. Regulators have been explicit that AI models — including LLMs and agents — fall inside that framework. That has a hard consequence. Validation requires reproducing a model's behavior on known inputs. If you cannot reconstruct what the model saw, you cannot validate it, which means it should not have been in a regulated workflow to begin with. "We are still piloting" does not survive a second examination. The framework was already there; generative AI simply arrived inside it. [Financial services AI reference architecture](https://ibl.ai/blog/financial-services-ai-reference-architecture) walks the control layers this implies. ## Does privilege survive a third-party AI vendor? For legal work this is the sharper question. Sending client material to a third-party model provider introduces a party that was not in the engagement, and privilege analysis turns on who had access and under what terms. ABA Model Rule 1.6 obliges lawyers to make reasonable efforts to prevent unauthorized disclosure of client information. Whether a hosted API meets that standard depends on contract terms most firms never read closely — retention, subprocessing, training use, and jurisdiction. [ABA Model Rule 1.6-compliant AI](https://ibl.ai/blog/aba-model-rule-1-6-compliant-ai) covers the analysis. Self-hosting removes the question rather than answering it. If the material never leaves the firm's environment, there is no third party whose terms need construing. ## What does this change about how you deploy? Move the inference path inside the boundary you already defend, and the audit artifacts become a logging decision rather than a negotiation. ibl.ai is the agentic AI platform where **you own all the code and the data**. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with **no per-seat pricing** — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped. For firms comparing that against per-lawyer licensing, [AI cost math for law firms](https://ibl.ai/blog/ai-cost-math-for-law-firms-per-seat-vs-usage) shows the headcount-multiplied bill next to actual token cost. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. *Related: [Why Kenya Wrote Clearer AI Liability Law Than the US](/blog/kenya-ai-liability-law-full-chain) — the same evidence problem, arriving through liability rather than examination.* ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.