Chapter 18The Five Layers of a Production Agent

Chapter 17 described the harness from the inside. This chapter describes a production agent from the buyer's side, as five layers, because that framing makes it obvious what most products on the market are missing and what a CIO is actually paying for.

The five layers

Each layer does work the layer above it cannot.

Interface. How a person meets the agent: a chat window, a side panel, an embedded widget, an API, a place inside an application they already use. This is the visible layer, and it is the one that demos.

Orchestration. Multi-step planning: breaking a request into steps, calling tools in sequence, handling a step that fails, deciding when the task is done. Without it, an agent is a chat interface that can call one function.

Knowledge. Live, role-scoped access to the systems of record, read at the moment of the question rather than uploaded in advance. Part III of this book.

Memory. Context that persists across sessions: what this person is working on, what was decided before, what the institution knows. Chapter 17's three scopes.

Governance. Role-based access, audit trails, spend limits and data-handling rules, enforced where the model cannot route around them. The broker, the sandbox boundary, the approval modes, the routing policy.

A product with all five is a platform. A product with the first is a demo that costs per seat.

The layers also explain a pattern CIOs notice and rarely name: the agent that impressed everyone in the pilot and disappointed everyone in production. The pilot exercised the interface. Production exercised orchestration against real failures, knowledge against real permissions, memory across real weeks, and governance against real people trying to get around it. The pilot could not have revealed what was missing, because the missing layers are invisible until the work is real.

Why most products ship only the first

The economics explain it. The interface is cheap to build and easy to sell; a polished chat window with a strong model behind it impresses in a thirty-minute meeting. Orchestration is harder and shows its value only on real tasks. Knowledge requires connectors into the customer's systems, which means engineering per customer and a broker nobody wants to build. Memory requires a data model, storage the customer trusts, and privacy controls. Governance requires all of the above plus the discipline to enforce rules in infrastructure rather than in prompts.

A vendor optimizing for the first meeting ships layer one and describes the other four in a roadmap. A vendor optimizing for the fifth year ships all five and has a worse demo. A CIO who knows the layers can tell which they are talking to in the first ten minutes.

Orchestration and memory are not a longer prompt

Two of the layers are commonly faked, and it is worth knowing how.

Orchestration is faked with a long system prompt that tells the model to "break the task into steps and use the tools available." A model given that instruction will produce something that looks like a plan. It will not reliably recover from a failed tool call, will not know when it has finished, and will not stop when it should. Real orchestration lives in the harness: a loop with state, budgets, retries and a definition of done that the model participates in but does not own.

Memory is faked by stuffing prior conversations into the context window. That works for one session and collapses after a week: the context fills, the cost climbs, and nothing is organized by whose memory it is or who may see it. Real memory is a store with scopes, extraction rules, retrieval by relevance, and the switches that let a person and an administrator govern it.

The tell, in both cases, is to ask where the capability lives. If the answer is "in the prompt," it is layer one wearing a costume.

Governance belongs in infrastructure

The fifth layer is the one this book cares about most, and the one where the difference between prompt and infrastructure is a compliance finding.

A rule enforced in a prompt, "do not reveal other students' records," "always ask before sending," is a request to the model. Models comply with requests most of the time, and most of the time is not a standard any auditor accepts. A rule enforced in infrastructure holds regardless of what the model decides: the broker refuses the call the role does not have; the sandbox blocks the network route; the harness pauses for approval before the send because the action class requires it; the router confines restricted data to the local model.

The practical test is to try to break the rule from the chat window. Ask the agent, in a dozen creative ways, to do the thing the policy forbids. If the policy lives in a prompt, one of the dozen will work. If it lives in infrastructure, none will, and the audit log will show the twelve attempts.

The questions to ask about each layer

For a CIO in an evaluation, five questions, one per layer, separate platforms from demos.

Interface. On which surfaces does the agent appear, from one backend? Web only is a warning.

Orchestration. Show me a task that takes six tool calls, one of which fails. What happens, and where is the logic that handles it?

Knowledge. Which of my systems can the agent read today, through what, running where, as whom? Where are the credentials?

Memory. Show me what the agent remembers about a person, in which scopes, and show me that person turning it off. Where is the store?

Governance. Show me the policy table, the audit log of this demonstration, and an attempt to violate a rule from the chat window.

A platform built the way this book describes answers all five with a screen rather than a slide. Our own answers are, in brief: seven surfaces from one backend; a harness with budgets, retries and approvals; connectors on the Model Context Protocol through an institution-run broker; three memory scopes in the institution's database; and governance in the broker, the router, the sandbox and the approval card. The point of the chapter, though, is not our answers. It is that a CIO should insist on answers at every layer, because the layer that ships is never the one that costs the most to be without.