--- title: "AI Agent Observability Has No Standard to Buy Yet" slug: "agent-observability-fragmenting-stack-own-the-audit-trail" author: "ibl.ai Engineering" date: "2026-10-12 09:00:00" category: "Premium" topics: "AI agent observability, OpenTelemetry GenAI semantic conventions, agent audit trail, SEC Rule 17a-4, FINRA 4511, agent cost attribution, HIPAA agent runtime, KVM microVM sandbox, self-hosted AI agents, agent governance" summary: "OpenTelemetry deprecated every gen_ai.* attribute out of its main semantic-conventions repository on 12 June 2026 and moved them to a dedicated repository that still has zero tagged releases. Cross-layer agent tracing is therefore not a product anyone can sell you yet, while SEC Rule 17a-4's audit-trail alternative already requires reconstructing a record with attribution. This post gives the dated evidence and the five things agent observability has to capture." banner: "" thumbnail: "" linkedin: | The standard that would let you trace an AI agent decision across orchestration, memory, model and tools does not have a release yet. Three dates, all checkable: → 12 June 2026: OpenTelemetry semantic-conventions v1.42.0 deprecated every gen_ai.* attribute, metric, event and span in the main repository and moved them to a dedicated GenAI repository. → 5 May 2026: that dedicated repository was created. → 12 October 2026: it has zero tagged releases and zero published releases, and is still under active development. So when a vendor sells you "agent observability," they are selling a dialect of an unreleased spec. That is not a scandal. It is a young discipline moving fast. But it does mean cross-layer correlation is a property of the stack you operate, not a SKU you can buy. Meanwhile the regulator has already made up its mind. SEC Rule 17a-4's audit-trail alternative, compliance date 3 May 2023, requires a time-stamped audit trail capturing all modifications and deletions, the date and time of each action, "the identity of the individual creating, modifying, or deleting the record," and whatever else will permit re-creation of the original record. Read the word "individual" again. An agent is not one, and the requirement does not soften because the actor was a model. The circulating version of this story is that firms cannot say which tool failed or which step cost money. True, and too easy. The harder requirement is reconstruction with attribution, and it is already written down. Five things agent observability has to capture to be worth the name: decision provenance, cross-layer correlation, state reconstruction, cost attribution, and audit trails that map to your actual compliance framework rather than to a generic log format. One more that gets skipped. HIPAA conversations fixate on data at rest, and the agent runtime is where PHI is actually handled. Isolation at the infrastructure level beats a contractual promise, which is why our code-execution sandbox runs on KVM microVMs on your own servers with an egress firewall for every guest. With ibl.ai you own all the code and the data, self-hosted inside your own perimeter, model-agnostic across any LLM, usage-based with no per-seat pricing, deployable anywhere from your own cloud to a fully air-gapped network. #iblai #AgenticAI #AIGovernance #Observability #OpenTelemetry #EnterpriseAI --- ## The Short Answer **AI agent observability is the ability to reconstruct why an agent acted, not just that it did. No purchasable standard exists yet: OpenTelemetry deprecated every `gen_ai.*` convention out of its main repository on 12 June 2026, and the dedicated replacement repository still has zero tagged releases. Until it stabilizes, the durable audit trail is a self-hosted one, which is why on ibl.ai you own all the code and the data.** The usual framing is that the agent stack has outrun its governance. That is half right, and the missing half is more useful. Governance has not merely lagged. The specification that cross-vendor agent tracing would have to be built on is still unreleased, which means no vendor can sell you the thing, and the organizations treating it as a purchase are waiting for a delivery that is not scheduled. ## Why can't an enterprise trace an AI agent's decision across its own stack? Because the stack is assembled from independently governed parts, and no two of them agree on what to record. A production agent in late 2026 typically draws on an orchestration framework, a persistent memory layer, a code-execution sandbox, a set of tool or MCP connectors, and a model endpoint. Each arrives with its own permission model, its own audit surface, and its own failure domain. The scale of the choice is the problem. One community catalogue, [awesome-ai-agents-2026](https://github.com/caramaschiHG/awesome-ai-agents-2026), indexes **300+ AI agent resources across 20+ categories** and updates monthly. [e2b-dev/awesome-ai-agents](https://github.com/e2b-dev/awesome-ai-agents) carries **30,394 stars**. The memory tier alone has consolidated around a handful of serious options and a long tail of new ones. [Mem0](https://github.com/mem0ai/mem0) passed **67,000 GitHub stars** as of 12 October 2026, having been created in June 2023. This is healthy. Competition at the infrastructure layer is how the layer gets good. But it means that when an agent approves an expense, routes a ticket, or files a report, the record of why is scattered across four or five systems that were never designed to be read together. And the memory tier cannot always answer even on its own terms. On StateMemBench, released August 2026, leading agent memory systems returned the *current* state only 13 to 20 percent of the time, a failure we covered in [Agent Memory Fails at State Transitions, Not Retrieval](/blog/agent-memory-state-transitions-world-models-vs-rag). ## Is there an AI agent telemetry standard an enterprise can buy today? No. The standard exists as work in progress and has never been released, and the dates are checkable in a few minutes. OpenTelemetry's [semantic-conventions v1.42.0](https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.42.0), published **12 June 2026**, states it plainly in its own release notes: "All `gen_ai.*` attributes, metrics, events, and spans have been deprecated and moved to the dedicated OpenTelemetry GenAI Semantic Conventions Repository." That dedicated repository, [open-telemetry/semantic-conventions-genai](https://github.com/open-telemetry/semantic-conventions-genai), was created **5 May 2026**. As of 12 October 2026 it has **zero tagged releases and zero published releases**, carries 429 stars, and is under active development, with commits on its default branch as recently as 9 October 2026. The old location now redirects. The [GenAI conventions page](https://opentelemetry.io/docs/specs/semconv/gen-ai/) on the OpenTelemetry site reads: "This page has moved and is no longer maintained in this repository." Read the consequence carefully, because it cuts both ways. Vendor agent tracing is real, it works, and several implementations are good. What does not exist is a stable contract that makes two vendors' traces of the same agent run correlate. So cross-layer correlation is not a feature you procure. It is a property of a stack whose telemetry you emit, store and can reshape when the spec finally lands. This is the gap behind a result we examined earlier: Microsoft Foundry's agent tracing reached general availability at Ignite 2025, and what it narrows is *where* a 50-step run died, not why. See [Finding Where a 50-Step Agent Run Dies Is Not Solved](/blog/microsoft-agent-observability-debugging-multi-step-failures). ## What does the SEC actually require an AI agent audit trail to reconstruct? More than "which tool failed." It requires reconstruction of the original record, with attribution, for the full retention period. The common framing is that financial firms deploying agents cannot say which tool caused a failure, why the agent retried, or which step drove the cost. Those are fair questions and they understate the obligation. The SEC adopted amendments to Exchange Act Rule 17a-4 in October 2022, with a compliance date of **3 May 2023**, [published in the Federal Register](https://www.federalregister.gov/documents/2022/11/03/2022-22670/electronic-recordkeeping-requirements-for-broker-dealers-security-based-swap-dealers-and-major). They gave broker-dealers a second path alongside write-once storage: an **audit-trail alternative**, which requires a time-stamped audit trail capturing "all modifications to and deletions of a record," the date and time of each action, and "the identity of the individual creating, modifying, or deleting the record." It must also capture whatever else will "permit re-creation of the original record if it is modified or deleted." FINRA's own [chart of significant changes](https://www.finra.org/sites/default/files/2022-12/rule-17a-4-amendments.pdf) sets the two options side by side. Note the word *individual*. An agent is not one, and nothing in the rule softens because the acting entity was a model operating under a firm's credentials. [FINRA Rule 4511(c)](https://www.finra.org/rules-guidance/rulebooks/finra-rules/4511) closes the loop by requiring members to preserve records in a format and media that complies with Rule 17a-4. So the regulated-industry question is not whether agent observability is worth building. A regulator wrote the requirement down three years before the agents arrived. ## What does AI agent observability have to capture to be worth the name? Five things, and the fifth is the one most tooling skips.
Capability What it answers Why monitoring alone misses it
Decision provenance Which data was retrieved, which model processed it, under which prompt and tool permissions Monitoring records the response, not the inputs that produced it
Cross-layer correlation One trace spanning orchestration, memory, inference and tool execution Each layer emits its own dialect of an unreleased spec
State reconstruction The agent's memory contents, active context and permissions at a chosen moment Logs capture events; nothing marks a stored fact as superseded
Cost attribution Which business outcome a given token spend bought Aggregate usage dashboards sum tokens without attaching them to work
Governance mapping Records shaped to HIPAA, SOX, FERPA, NIST 800-53 or Rule 17a-4, not to a log format A generic trace is evidence of activity, not evidence of compliance
The HIPAA case shows why the fifth row is not paperwork. Most HIPAA review of an AI deployment concentrates on data at rest and on the business associate agreement. The agent runtime is where protected health information is actually handled: read into context, passed to a tool, written to a scratch file, sent to an endpoint. Isolation at the infrastructure level is a stronger control than a contractual promise, because it is observable and it fails closed. ## Why is vendor-supplied AI agent observability a structural conflict? Because it asks the party whose component may have caused the failure to also be the party that reports what failed. State this narrowly, since the overclaimed version is wrong. Model and framework vendors ship genuinely useful telemetry, and some of it is better than what a first attempt in-house would produce. The conflict is not competence. The conflict is custody. If your trace store, retention policy and export format all live inside a vendor's platform, then your ability to reconstruct an agent decision is a contractual right rather than an operational capability, and it ends when the contract does. That question gets answered at procurement, not at deployment. It is the same question the [Hugging Face agent intrusion](/blog/openai-eval-agents-hugging-face-intrusion-containment-you-own) raised about containment: who holds the control, and do you keep it afterwards. It is also the question a state regulator answered directly this month when it licensed an AI system to issue prescriptions and bought itself staged review and monthly reporting in exchange. We covered that agreement in [Utah Did Not Let AI Prescribe Without a Physician](/blog/utah-ai-prescribing-nolla-stage-gates-audit-trail). ## How does ibl.ai make AI agent decisions observable? By shipping the audit surfaces as platform features and handing you the deployment they run in. Every agent carries an [Audit panel](https://ibl.ai/docs/os/agent-settings/audit) recording who changed what and when, down to the specific configuration field, the user's email and the timestamp. It is permission-gated, so the change log is itself access-controlled. Run-level evidence sits in Analytics: [Transcripts](https://ibl.ai/docs/os/analytics/transcripts) for what was said and retrieved, [Costs](https://ibl.ai/docs/os/analytics/costs) for per-run spend, and [Data Reports](https://ibl.ai/docs/os/analytics/data-reports) for CSV and JSON export into whatever system of record your auditors already read. For the runtime itself, the ibl.ai [code-execution sandbox](https://ibl.ai/updates/platform-update-2026-10-09) runs on KVM microVMs on your own servers, with zero network by default, a built-in egress firewall for every guest, and session caps enforced globally and per host. The part that makes those features durable rather than rented is the deployment model. You receive the source under a perpetual license and run it inside your own perimeter, so the trace store is yours, the retention policy is yours, and reshaping the schema when the GenAI conventions finally tag a release is a change you make rather than a feature you request. --- *Sources: the deprecation and move of all `gen_ai.*` conventions from [OpenTelemetry semantic-conventions v1.42.0](https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.42.0), published 12 June 2026; the creation date, release and tag counts of [open-telemetry/semantic-conventions-genai](https://github.com/open-telemetry/semantic-conventions-genai) read from the GitHub API on 12 October 2026; the "no longer maintained" notice from [OpenTelemetry's GenAI conventions page](https://opentelemetry.io/docs/specs/semconv/gen-ai/); star counts and catalogue sizes from [mem0ai/mem0](https://github.com/mem0ai/mem0), [e2b-dev/awesome-ai-agents](https://github.com/e2b-dev/awesome-ai-agents) and [awesome-ai-agents-2026](https://github.com/caramaschiHG/awesome-ai-agents-2026), also read from the GitHub API on 12 October 2026; the Rule 17a-4 amendments, the audit-trail alternative and the 3 May 2023 compliance date from [the SEC's own summary of the amendments](https://www.sec.gov/investment/amendments-electronic-recordkeeping-requirements-broker-dealers), [the Federal Register notice](https://www.federalregister.gov/documents/2022/11/03/2022-22670/electronic-recordkeeping-requirements-for-broker-dealers-security-based-swap-dealers-and-major) and [FINRA's chart of significant changes](https://www.finra.org/sites/default/files/2022-12/rule-17a-4-amendments.pdf); the sandbox's microVM basis, egress firewall and session caps from [the 2026-10-09 platform update](https://ibl.ai/updates/platform-update-2026-10-09). One figure from the briefs was dropped: a developer's report of 43 billion tokens consumed with no breakdown of what they bought, which exists only as a social-media post and is not a citable source for a claim about agent cost attribution. A count of "10 memory frameworks released or updated in the last seven days" was also dropped, because no source establishes the window or the count.* ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.