--- title: "The 5-Layer Agent Stack: Most Vendors Ship Layer One" slug: "five-layer-agent-architecture-what-ships-is-layer-one" author: "ibl.ai Engineering" date: "2026-09-07 18:00:00" category: "Premium" topics: "AI agents, agent architecture, enterprise AI, governance, memory, orchestration, legal AI" summary: "A five-layer model of agent architecture — interface, orchestration, knowledge, memory, governance — is the most useful way we have found to audit an enterprise AI product. The uncomfortable part is that most enterprise AI products implement the first layer and describe the other four." banner: "" thumbnail: "" linkedin: | Here is the model we use to audit enterprise agent products: Interface → Orchestration → Knowledge → Memory → Governance. It is a useful lens. The uncomfortable part is what happens when you audit products against it. Most enterprise AI ships layer one and describes layers two through five. Layer 1 — Interface. The chat window. This is the part everyone builds, because it is the part that demos. Layer 2 — Orchestration. Multi-step planning, tool calls, retries, and knowing when to stop. Without it an "agent" is a chatbot that can call one function. Layer 3 — Knowledge. Live, role-scoped access to systems of record. Not a document upload — the actual CRM, the actual matter management system, at query time. Layer 4 — Memory. Context that persists across sessions and survives a model upgrade. If your institutional context resets when you change models, you did not have memory; you had a long prompt. Layer 5 — Governance. RBAC, audit trails, spend limits, data-handling rules — enforced in infrastructure where the model cannot route around them. Test any vendor with four questions: → Can it act across more than one system in a single task, and recover when a step fails? → Does it read from our systems of record live, or from a copy we uploaded? → If we switch the underlying model tomorrow, what does the agent forget? → Is a spend or access limit enforced server-side, or is it a line in a system prompt? A "no" on layer 5 is not a missing feature. It is the reason the deployment will not pass review in a regulated industry — and it is why so much legal, financial and clinical AI stalls after a strong pilot. With ibl.ai you own all the code and the data — self-hosted inside your own perimeter, model-agnostic across any LLM, usage-based with no per-seat pricing, deployable anywhere from your own cloud to a fully air-gapped network. Every layer is yours to inspect, which is the only way to verify any of the above. #iblai #AgenticAI #EnterpriseAI #AIGovernance #LegalTech #AIArchitecture --- ## The Short Answer **A five-layer model of agent architecture — interface, orchestration, knowledge, memory, governance — is the most useful way to audit an enterprise AI product, and most implement layer one while describing the rest. The layers that decide whether a deployment survives review are the last two, and both must be enforced in infrastructure rather than prompts. With ibl.ai you own all the code and the data, so every layer is inspectable rather than asserted.** The model is useful less as a design guide than as an audit instrument. Applied to a vendor demo, it tends to produce an uncomfortably short list. ## What are the five layers of a production agent? Each layer does work the layer above it cannot do. - **Interface.** How a person interacts with the agent — chat, embedded widget, API, or inside an existing application. This is the visible layer and the one that demos. - **Orchestration.** Multi-step planning: decomposing a request, calling tools in sequence, handling a failed step, deciding when the task is complete. Without it, an agent is a chat interface that can call one function. - **Knowledge.** Live, role-scoped access to systems of record — the CRM, the case or matter management system, the core platform — read at query time rather than uploaded in advance. - **Memory.** Context that persists across sessions: what this user is working on, what was decided before, what the institution knows. - **Governance.** Role-based access control, audit trails, spend limits and data-handling rules, enforced where the model cannot route around them. ## Why do so many enterprise AI products only implement the first layer? Because layer one is what a buyer sees in a demo, and layers three to five are what a buyer only discovers in production. A chat window over a capable model is genuinely impressive in a thirty-minute meeting. It answers questions, drafts documents, and reasons about uploaded material. Nothing in that meeting exercises whether the system can complete a task spanning three internal systems, or produce an audit trail an examiner will accept. The consequence is a familiar procurement failure: the evaluation measures the layer that is easiest to build, and the deployment fails on the layers that were never tested. **A vendor is not necessarily being deceptive — a demo simply cannot exercise governance.** That is the buyer's job, and it requires asking layer-specific questions. ## How do orchestration and memory differ from a long prompt? Both are frequently simulated by stuffing more text into the context window, and both break at the point that stops working. **Orchestration** is not a prompt instructing the model to "think step by step." It is a runtime that executes steps, observes results, retries failures, enforces limits on how long a task may run, and terminates cleanly. The difference shows up on the unhappy path: when a tool call fails, a prompt-based agent typically hallucinates a plausible result, while an orchestrated one retries or reports the failure. **Memory** is not conversation history pasted forward. The operative test is a model change: if switching the underlying model resets what the system knows about your institution, that was never memory — it was a long prompt. Real memory persists in your infrastructure, independent of which model reasons over it, which is also what makes model-agnostic architecture practical rather than theoretical. ## Why must governance be enforced in infrastructure rather than in prompts? Because a control the model can ignore cannot be demonstrated to a regulator, and demonstration is the entire point. Application-level governance means instructions: telling the model not to exceed a budget, not to access certain records, to log its reasoning. Every one of those depends on the model complying, and none produces evidence that it did. Infrastructure-level governance means the constraint sits beneath the model. A spend limit enforced **server-side** fails a request for every client regardless of what the model intends — which is why [spend limits belong at the workspace, agent and per-person scope with server-side enforcement](/updates/spend-limits) rather than in a system message. Role-based access tied to the institution's identity provider means an agent cannot read what the user cannot read. An audit trail written by the platform records what happened rather than what the model reported happening. The distinction is invisible until the first audit, and decisive at it. The scale of what layer five has to govern is not hypothetical. A January 2026 study by Liu et al. analyzed 31,132 agent skills from public marketplaces and found 26.1% carrying at least one vulnerability and 5.2% showing high-severity patterns strongly suggesting malicious intent — third-party code that layer five is the only thing standing between and your systems of record. This is precisely the gap that stalls AI in regulated industries — the same pattern behind [why only 15% of banking AI use cases reach production](/blog/why-only-15-percent-of-banking-ai-reaches-production). ## What should a buyer ask a vendor about each layer? Four questions, each targeting a layer that demos do not exercise. 1. **Orchestration:** Can it complete a task spanning more than one system, and what happens when a step fails partway through? 2. **Knowledge:** Does it read from our systems of record live and role-scoped, or from a copy we uploaded? What happens when the source record changes? 3. **Memory:** If we switch the underlying model next quarter, what does the agent forget? 4. **Governance:** Is a spend or access limit enforced server-side, or is it an instruction in a prompt? Show me the audit record for a single tool call. A vendor that answers the fourth question well will answer the others well, because governance is the layer that cannot be retrofitted. ## How does ibl.ai implement all five layers? By owning the whole stack, which is what makes every layer verifiable rather than asserted. With ibl.ai you own all the code and the data. The platform is deployed on your own infrastructure with full source code access, so the orchestration runtime, the retrieval path, the memory store and the governance enforcement are all yours to read. Agents read from systems of record in place under role-scoped permissions; memory persists in your infrastructure independent of the model; spend limits run at workspace, agent and per-person scope with server-side enforcement; and every interaction is audited. It is model-agnostic across any LLM, usage-based with no per-seat pricing, and deploys anywhere from your own cloud to on-premise, GovCloud, or a fully air-gapped network. The model is layer zero — deliberately the most replaceable part of the system. ibl.ai is family-owned and operated from New York, NY. *Related reading: [when agents exceed their scope](/blog/ai-agents-exceeding-scope-containment-architecture) — what happens when layer five is declared rather than enforced.* ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.