--- title: "Finding Where a 50-Step Agent Run Dies Is Not Solved" slug: "microsoft-agent-observability-debugging-multi-step-failures" author: "ibl.ai Engineering" date: "2026-09-13 09:00:00" category: "Premium" topics: "agent observability, AI agent tracing, multi-step agents, LLMOps, Microsoft Foundry, tool calls, enterprise AI" summary: "Microsoft Foundry's agent tracing reached general availability at Ignite 2025, not this week, and the docs still mark workflow and external agents preview. It narrows where a 50-step run died, not why." banner: "" thumbnail: "" linkedin: | A claim going around: Microsoft just solved finding where a 50-step AI agent execution dies. Two corrections, both checkable. It is not new. Observability in Microsoft Foundry — tracing plus the evaluator suite — reached general availability at Ignite 2025, and the December 2025 roundup says so in those words. Build 2026, announced June 3, extended tracing and evaluations to LangChain, LangGraph, the OpenAI SDK and custom frameworks, most of it in public preview. And "solved" is not Microsoft's word. Their own documentation, updated 28 August 2026, notes that tracing is generally available for prompt and hosted agents while workflow and external agents remain in preview, and that the OpenTelemetry GenAI semantic conventions it depends on carry Development status and may change. That matters, because the thing being sold as solved is the easy half. → A trace tells you which step returned an error. It does not tell you that step 6 returned a plausible, well-formed, wrong answer that steps 7 through 50 reasoned over → The failure in a long run is usually a tool call, not the model — the model behaved correctly on bad input → So the record has to include the tool's arguments and result, the retrieved documents, and the model and provider that produced each turn → And it has to still be there next quarter: Azure stores traces in Application Insights, where retention is a billing decision, not a product guarantee With ibl.ai you own all the code and the data — self-hosted inside your own perimeter, model-agnostic across any LLM, usage-based with no per-seat pricing, deployable anywhere from your own cloud to a fully air-gapped network. #iblai #AgenticAI #EnterpriseAI #LLMOps #Observability #AIAgents --- ## The Short Answer **Microsoft did not just solve this. Foundry's agent tracing reached general availability at Ignite 2025 and records inputs, outputs, tool usage, retries, latency and cost per step, and Microsoft's own docs still mark workflow and external agents preview. Tracing narrows where a 50-step run died; it does not tell you why. With ibl.ai you own all the code and the data, so every turn's documents and tool calls stay yours.** The useful version of this story is not a launch. It is a specification of what an operator needs recorded before a long agent run is debuggable at all. ## What did Microsoft actually ship for agent observability, and when? Foundry observability, and not this week. Microsoft's own roundup of the period states that observability capabilities — tracing, quality, risk and safety evaluators, cloud evaluation runs, and the AI Red Teaming Agent — [became generally available at Ignite 2025](https://devblogs.microsoft.com/foundry/whats-new-in-microsoft-foundry-oct-nov-2025/), in a post published **18 December 2025**. The newer piece is cross-framework support. A Foundry post published **3 June 2026** extended [tracing and evaluations to LangChain, LangGraph, the OpenAI SDK, the Microsoft Agent Framework and custom frameworks](https://devblogs.microsoft.com/foundry/build-2026-from-observability-to-roi-for-ai-agents-on-any-framework/), with most of that surface — trace replay, intelligent sampling, multi-turn evaluation, traces-to-dataset — listed as public preview. Mechanically it is OpenTelemetry. Spans nest, so a top-level `invoke_agent` contains child spans for planning, tool execution and memory operations, with attributes carrying the tool arguments and the tool result. That is a real and well-built product. It is roughly ten months old, and describing it as this week's breakthrough is the tweet's framing, not Microsoft's. ## Does agent tracing solve finding where a 50-step run dies? No, and Microsoft does not claim it does. The [agent tracing documentation](https://learn.microsoft.com/en-us/azure/foundry/observability/concepts/trace-agent-concept), updated **28 August 2026**, says tracing is generally available for prompt and hosted agents while **workflow and external agents are in preview** — and the OpenTelemetry GenAI semantic conventions it standardizes on carry **Development status** and may change in future releases. The document is also honest about the problem it addresses. It frames tracing as answering "where did this response come from?" and "which step introduced an error or latency spike?" Those are locator questions. A 50-step run that throws on step 41 was never the hard case; the stack trace was already telling you that. The hard case is the run that completes, returns a confident answer, and is wrong — because a tool returned something plausible and malformed thirty steps earlier and nothing downstream objected. Tracing gives you the evidence to find that. It does not find it for you. ## Why is the failure in a long agent run usually a tool call, not the model? Because over 50 steps the model is the component being tested most often and changing least. A long run is mostly I/O. The agent queries a system of record, gets rows back, summarizes them, calls another tool with that summary, and repeats. Each hop is an opportunity for a payload to be empty, truncated, stale, or shaped differently than last week. When that happens the model usually behaves correctly. It reasons faithfully over what it was handed, which is the failure mode we described in [metacognitive failure and confidently wrong agents](/blog/metacognitive-failure-confidently-wrong-agents): fluent, internally consistent output built on a bad premise. So "the model hallucinated" is often a misdiagnosis. The model summarized an empty result set exactly as instructed. This is also why model-level metrics do not catch it. Token counts, latency and per-call error rates all look normal in a run that quietly went wrong at step 6. ## What does an operator actually need recorded for every step of an agent run? Five fields per turn, and the ability to query across them. - **Inputs and outputs** of each step, in invocation order, so the sequence can be replayed rather than reconstructed. - **The tool called, with its arguments and its result** — the argument tells you what the agent believed, the result tells you what it actually got. - **The model and provider** that produced the turn, because a regression that follows a provider change is not a prompt problem. - **The documents retrieved**, so a wrong answer can be traced to the passage that produced it rather than blamed on the model. - **The request context**, with credentials stripped, so the same run can be reproduced. This is the observability layer of an agent stack, and it is not a feature of the model. It sits alongside the other capabilities that decide whether an agent survives production review, which we set out in [the five-layer agent stack](/blog/five-layer-agent-architecture-what-ships-is-layer-one). ## Who owns the agent trace data, and how long does it survive? That is the question the launch coverage skips, and it is a procurement question rather than an engineering one. Foundry stores traces in Azure Monitor Application Insights. Per Microsoft's [retention documentation](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-retention-configure), Application Insights tables such as `AppTraces` and `AppDependencies` keep data for **90 days at no charge**, the Log Analytics workspace default for Analytics tables is **30 days**, analytics retention can be extended to **730 days at extra cost**, and total retention runs up to **12 years (4,383 days)**. So the answer to "can we still inspect the run that misfired last quarter?" is decided by a retention setting and a bill, and the tracing documentation notes that trace data volume and retention incur Application Insights costs. Traces also carry the most sensitive material your agent touches — user inputs, model outputs, tool arguments and tool results. Microsoft's own guidance is to treat them as production telemetry and apply the same access controls you apply to logs. Which makes where that telemetry lives an architecture decision, not an afterthought. ## How does ibl.ai make every agent turn inspectable? With ibl.ai you own all the code and the data. The [platform update of 11 September 2026](/updates/platform-update-2026-09-11) shipped exactly this shape of record. The conversation-detail endpoint now returns, on each AI turn, the `documents` retrieved, the `tool_calls` as `{name, input, output}`, the `metadata` naming the LLM model, provider and temperature, and the `request_context` with credentials stripped. The conversation list carries per-conversation rollups — `documents_count`, `tool_calls_count`, `used_documents` — so an operator can spot the runs worth opening without opening all of them. That week carried **62 production releases**. Because the platform is self-hosted with full source code, that telemetry lands in your infrastructure, under your retention policy, subject to your access controls. It is model-agnostic across any LLM, so the provider field is a fact you record rather than a vendor you are bound to, and it is usage-based with no per-seat pricing. It deploys anywhere — your own cloud, on-premise, GovCloud, or a fully air-gapped network. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY. *Related reading: [the five-layer agent stack](/blog/five-layer-agent-architecture-what-ships-is-layer-one) — where observability sits among the layers most vendors describe rather than implement; and [metacognitive failure in confidently wrong agents](/blog/metacognitive-failure-confidently-wrong-agents) — why a faithful model over a bad tool result looks like a hallucination.* *Sources: the Ignite 2025 general availability of Foundry observability from [Microsoft's October–November 2025 roundup](https://devblogs.microsoft.com/foundry/whats-new-in-microsoft-foundry-oct-nov-2025/); the cross-framework extension and its preview status from [the Build 2026 Foundry post of 3 June 2026](https://devblogs.microsoft.com/foundry/build-2026-from-observability-to-roi-for-ai-agents-on-any-framework/); the capture list, GA-versus-preview split and OpenTelemetry conventions from [the agent tracing overview](https://learn.microsoft.com/en-us/azure/foundry/observability/concepts/trace-agent-concept) and [the Foundry observability concept page](https://learn.microsoft.com/en-us/azure/foundry/concepts/observability); retention figures from [Azure Monitor data retention](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-retention-configure).* ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.