--- title: "Nemotron 3.5 Lightning and NeMo Switchyard: Why Agents Need an Open Routing Layer" slug: "nemotron-3-5-lightning-nemo-switchyard-open-routing-layer" author: "ibl.ai Engineering" date: "2026-08-11 14:00:00" category: "Premium" topics: "enterprise AI, model routing, NVIDIA, Nemotron, open-weight models, model agnostic, AI agents, AI cost" summary: "NVIDIA released Nemotron 3.5 Lightning (30B total, 3B active) and NeMo Switchyard, an open routing library. Together they make the model the cheapest part of an agent deployment — and move the value to the routing layer. Here is what enterprises should own, and the cost math for routing by task." banner: "" thumbnail: "" linkedin: | NVIDIA just released a 30B model that activates 3B parameters per call, and an open library that routes each step of an agent workflow to a different model. The second one is the bigger deal, and it is worth being precise about why. Most enterprise agents today run every step on one model. Classifying a support ticket and drafting a multi-step compliance analysis hit the same endpoint at the same price. That is not a model problem. It is an architecture problem, and it is where the money goes. Route by task and the arithmetic changes fast. In our own modeling of a 500,000-call-per-month agent workload, sending the ~70% of calls that are classification and extraction to a small open-weight model — and reserving frontier models for the reasoning steps — cuts inference spend by roughly 80%, with no accuracy loss on the simple steps, because those steps were never hard. So here is the part worth internalizing: when open weights make the model itself cheap, the model stops being the moat. What is left is the routing layer, the data integration, and the governance around them. Those are the pieces you want to own. Which is the distinction most "sovereign AI" conversations miss. Open weights are not the same thing as open infrastructure. Downloading a model you can run yourself, then wiring it into a managed platform you cannot see or leave, moves the lock-in one layer up. On ibl.ai you own all the code and the data — the routing layer included — so switching models is a configuration change, not a migration. #iblai #EnterpriseAI #AIAgents #NVIDIA #LLM --- ## The Short Answer **NVIDIA's Nemotron 3.5 Lightning and NeMo Switchyard shift enterprise AI value away from the model and onto the routing layer that decides which model handles each step — and that layer is the one you should own outright, because with ibl.ai you own all the code and the data, run any LLM model-agnostic and switch anytime, and pay by usage with no per-seat pricing.** Nemotron 3.5 Lightning activates 3 billion of its 30 billion parameters per call and runs on a single GPU. Switchyard routes each workflow step to the cheapest model that can do it. Both are open. What is not commodity is the orchestration around them — which is why renting that layer from a vendor recreates the lock-in the open weights just removed. ## What is NVIDIA Nemotron 3.5 Lightning? Nemotron 3.5 Lightning is an efficiency-optimized open-weight model built for high-volume agent workloads rather than benchmark leadership. It uses a mixture-of-experts architecture: 30 billion total parameters, of which roughly 3 billion are active on any given inference call. The practical consequences matter more than the architecture. It runs on a single GPU, with no multi-node cluster required. NVIDIA reports it running about 35% faster than Qwen3.6 35B and up to 4x faster than comparable open models on agentic coding tasks. It ships with open weights, open datasets, and open training recipes, distilled from the larger Nemotron 3 Ultra. The design premise is that most enterprise agent steps do not need frontier-scale reasoning. Routing a support ticket, checking a document against a policy, resetting a password — these need reliability and speed. Spending 400 billion parameters on them is a pricing decision, not a quality one. ## What does NeMo Switchyard do? NeMo Switchyard is an open-source routing library that sits between an agent and the model layer, selecting a model per workflow step instead of per deployment. For each step it weighs task complexity, latency budget, and cost, then dispatches to whichever model clears the bar most cheaply. This is the piece most enterprise agent deployments are missing. The common architecture wires an entire agent estate to one inference endpoint, which produces two failure modes at once: every trivial task is billed at frontier rates, and changing providers means rewiring every integration. Switchyard separates those decisions. Classification and extraction go to a small model like Nemotron 3.5 Lightning. Multi-step reasoning goes to a frontier model. Domain-specific analysis goes to a fine-tuned specialist. The agent's logic does not change when the routing table does. ## How much does routing by task actually save? Routing changes agent economics more than any model upgrade, because most agent traffic is simple. Below is our modeling of a 500,000-call-per-month enterprise agent workload — a realistic mid-size support and operations deployment — comparing a single-model deployment against task-based routing.
Architecture How calls are handled Monthly inference
Single frontier model All 500K calls at frontier rates ~$21,000
Routed, hosted models ~70% simple to a small model, ~30% to frontier ~$6,900
Routed, small model self-hosted Simple calls on your own GPU, frontier by API ~$4,200
The saving comes from traffic mix, not from accepting worse answers. The simple steps were never hard, so moving them to a 3-billion-active-parameter model costs nothing in quality. The third row adds the effect of owning the hardware for the high-volume tier: a single GPU absorbs the simple traffic at a fixed monthly cost regardless of how many calls hit it. Note what is absent from every row: a per-seat line. Per-seat AI pricing — ChatGPT Enterprise at roughly $60 per user per month, Microsoft Copilot at about $30, Glean at around $40 — scales with headcount rather than usage. At 5,000 employees that is $150,000 to $300,000 a month before anyone measures whether the tool was used. Routing optimizes a bill that per-seat pricing does not even expose. ## Why is open-weight not the same as sovereign AI? Open weights and open infrastructure are different claims, and conflating them is the most expensive mistake in this category. Downloading a model you can run yourself is necessary for sovereignty. It is not sufficient, because the model is only one layer of an agent deployment. Consider what remains proprietary in a typical "open model" deployment: the orchestration platform, the routing logic, the data connectors, the evaluation harness, the audit trail, and the admin controls. An organization can run Nemotron on its own GPUs and still be unable to leave the platform wrapped around it — the lock-in simply moved up a layer. Real sovereignty requires four things together: full source code ownership of the agent platform rather than the model alone, data integration that leaves institutional data in place, deployment flexibility spanning your cloud, on-premise, GovCloud, or air-gapped networks, and genuine model agnosticism. Miss any one and the other three are decorative. ## Which layers should an enterprise actually own? Own the layers that are expensive to rebuild and cheap for a vendor to hold hostage. With open weights driving model cost toward commodity, three layers carry the durable value, and all three are ones a managed platform keeps. The **routing layer** decides which model handles which task, at what cost, under what governance. The **data layer** connects agents to your SIS, CRM, HRIS, and ERP while leaving records in place. The **governance layer** carries audit trails, access controls, spend caps, and compliance monitoring. Everything else — the models, the cloud compute, the inference endpoints — is replaceable. That inversion is what NVIDIA's release makes concrete: when the model layer is commoditized, an organization spending its budget on model API costs while renting its orchestration is optimizing the one variable that no longer differentiates it. ## How do you tell whether you have a lock-in problem? Run three checks, in order, and the answer is usually clear within an hour. First, audit your model dependency. If switching LLM providers would take more than a configuration change, you have a lock-in problem regardless of how open your model's weights are. The test is not whether you *can* switch, but what it costs. Second, audit your routing. Look at what fraction of your agent calls are classification, extraction, or routing steps being served by a frontier model. In most deployments this is the single largest source of avoidable AI spend, and it is invisible on an invoice that reports only total tokens. Third, check who owns the orchestration. The routing, governance, and data-integration infrastructure should be yours — not a managed service whose terms, pricing, and model roadmap you do not control. ## Where does ibl.ai fit? **ibl.ai is the agentic AI platform where you own all the code and the data.** You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped. Applied to this release specifically: Nemotron 3.5 Lightning, GPT, Claude, Gemini, Llama, Command, or a model you fine-tuned yourself all run through the same routing layer, and that layer ships as source code you hold under a perpetual license. Changing which model serves which step is a configuration change, because the integrations point at your platform rather than at a vendor's API. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on. Related reading: [Open-Weight AI Models Just Reached Enterprise-Grade](https://ibl.ai/blog/open-weight-ai-enterprise-nemotron-3-ultra-2026) on the Nemotron 3 Ultra release, and [cost-optimizing model routing](https://ibl.ai/blog/iblai-openclaw-router-cost-optimizing-model-routing) on how routing is implemented in practice. For the platform itself, see [Agentic OS](https://ibl.ai/product/agentic-os). ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.