# How to Build an AI Governance Program > Source: https://ibl.ai/resources/guides/build-an-ai-governance-program > Last updated: 2026-08-19 *Inventory, risk tiering and controls that survive contact with fifty models — plus the evidence an auditor asks for and most programs cannot produce* Reading time: 15 min read | Difficulty: intermediate **On ibl.ai you own all the code and the data, run it model-agnostic across any LLM, and pay with no per-seat pricing — so you can deploy anywhere, from your own cloud to a fully air-gapped network.** ## How do you build an AI Governance Program? Most AI governance programs work at five models and break at fifty. The failure is structural: controls designed around manual review committees do not scale, so teams route around them and the program becomes a document rather than a control. The programs that hold share three properties. They know what exists, because there is a real inventory. They apply proportionate controls, because not every model warrants the same scrutiny. And they generate evidence automatically, because evidence assembled by hand at audit time is evidence that will not exist. This guide covers that sequence, and the technical prerequisite most governance frameworks omit: you cannot enforce what you do not control. ## Prerequisites - **Executive sponsorship with real authority:** Governance that cannot say no is advisory. The sponsor needs authority to block a deployment, which is what makes the rest of the program credible. - **An honest picture of current AI use:** Including shadow AI. A program built on the sanctioned inventory alone governs a fraction of the actual exposure. - **Agreement on which regulations apply:** Sector rules, the EU AI Act where relevant, and internal policy. Scope disagreements surface late and expensively if not settled at the start. - **A platform whose behaviour you can actually control:** Governance requires enforcing constraints and producing logs. If both live inside a vendor's platform, your program depends on their roadmap. ## Step 1: Build the inventory before writing any policy You cannot govern what you have not enumerated. Most organizations discover materially more AI in use than they expected, much of it embedded in vendor products nobody classified as AI. - [ ] Enumerate every model, agent and AI-enabled vendor feature - [ ] Record purpose, owner, data touched and systems reached - [ ] Include AI embedded in purchased software — This is the category most often missed entirely. - [ ] Survey for shadow AI without attributing blame — Punitive framing guarantees an incomplete inventory. ## Step 2: Tier by risk so controls are proportionate Uniform controls fail in both directions: too heavy for low-risk uses, so teams route around them; too light for high-risk uses, where the real exposure sits. - [ ] Define tiers by consequence of a wrong output, not by technology - [ ] Weight decisions affecting individuals' rights most heavily - [ ] Assign control requirements per tier - [ ] Make the lowest tier genuinely lightweight — If everything is high-risk, nothing is. ## Step 3: Define controls that a system can enforce A control that depends on a person remembering is a preference. Prefer controls the platform enforces: scoped permissions, routing rules, mandatory logging, human confirmation on consequential actions. - [ ] Scope agent permissions per task rather than per environment - [ ] Route regulated data to approved models by rule, not by instruction - [ ] Require human confirmation on irreversible actions - [ ] Make logging non-optional at the platform level **Tips:** - The strongest governance controls are indistinguishable from good security architecture. If your program and your security team are building different things, one of them is wrong. ## Step 4: Generate evidence automatically Audit evidence assembled by hand at audit time is evidence that will not exist. The program must produce its record as a by-product of normal operation. - [ ] Log the model and version behind every consequential output - [ ] Record the agent identity, delegating human and granted scope per action - [ ] Retain logs for at least the period your regulation requires — The EU AI Act sets a six-month floor for high-risk deployers. - [ ] Store evidence where you control retention, not in a vendor dashboard ## Step 5: Automate review so the program scales Manual review committees are the bottleneck that kills governance at scale. Reserve human review for the top risk tier and automate assessment for everything below it. - [ ] Template the assessment for low and medium tiers - [ ] Automate evidence collection into the assessment - [ ] Set a service level for review turnaround — Slow governance is the leading cause of shadow AI. - [ ] Re-review on a schedule and on material change, not once at launch ## Step 6: Close the loop with monitoring Approval at launch governs a system that no longer exists six months later. Models change, usage drifts, and new integrations appear without a fresh assessment. - [ ] Monitor for model version changes, including silent vendor updates - [ ] Track usage drift against the approved purpose - [ ] Re-run evaluation sets on a schedule - [ ] Feed incidents back into tiering and controls ## Common Mistakes ### Writing policy before building the inventory **Consequence:** The policy governs the AI you knew about, which is usually a fraction of what is running. **Prevention:** Enumerate first. The inventory almost always changes the policy you would have written. ### Applying uniform controls to every use **Consequence:** Low-risk uses route around a process that is too heavy, and high-risk uses get scrutiny calibrated for the average. **Prevention:** Tier by consequence and make the lowest tier genuinely lightweight, so the process is worth following. ### Treating governance as a launch gate **Consequence:** Systems are approved once and then drift — models update, usage changes, integrations appear — with no reassessment. **Prevention:** Schedule re-review and monitor for model version changes and usage drift as ongoing controls. ### Relying on evidence you do not hold **Consequence:** The audit trail lives in a vendor platform, so ending the contract or a dispute with the vendor removes your ability to answer a regulator. **Prevention:** Store audit evidence inside your own boundary in a schema you control and can export in full. ## FAQ **Q: Where should AI governance report?** Programs reporting into technology tend to focus on model performance and reliability; those reporting into legal or compliance emphasize regulatory adherence. The effective ones involve both, usually through a cross-functional committee with real authority to block. **Q: How do you govern AI embedded in purchased software?** It belongs in the inventory like anything else. Ask vendors which models they use, where processing happens, what logs you can obtain, and what notice you get before a model changes. Many cannot answer, which is itself a finding. **Q: What does the EU AI Act require of deployers?** Among other obligations, deployers of high-risk systems must retain automatically generated logs under their control for at least six months. Note that the Annex III high-risk deadline was moved to December 2027 by a provisional agreement in June 2026, so verify current timing. **Q: How do you stop governance from becoming a bottleneck?** Automate assessment for low and medium tiers, reserve human review for the top tier, and publish a turnaround service level. Slow governance does not prevent AI use; it relocates it somewhere you cannot see. **Q: Does self-hosting make governance easier or harder?** Easier on the controls that matter. You can enforce routing and permission rules in the platform itself, retain logs on your own terms, and pin model versions so past decisions remain reproducible. It adds operational responsibility in exchange. ## Can you do this on infrastructure you own? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.