--- title: "NVIDIA's Open Routing Layer: Why the Model Stopped Being the Moat" slug: "nvidia-nemotron-lightning-switchyard-open-routing-layer" author: "ibl.ai Engineering" date: "2026-08-12 10:00:00" category: "Premium" topics: "NVIDIA Nemotron, NeMo Switchyard, model routing, enterprise AI, AI agents, open-weight models, model-agnostic architecture, AI inference cost" summary: "NVIDIA shipped an efficient open model and an open routing library on the same day. Together they commoditize the model layer and move the durable advantage to the routing layer — which is the one piece you should refuse to rent. What routing saves, what open weights do not buy you, and the three layers worth owning." banner: "" thumbnail: "" linkedin: | NVIDIA shipped two things on August 11 that matter more together than apart: → Nemotron 3.5 Lightning — 30B total parameters, only 3B active. Runs on a single GPU. 35% faster than Qwen3.6 35B. → NeMo Switchyard — an open routing library that picks the model per workflow step. The model is the headline. The router is the story. Most enterprise agents run every task through one frontier endpoint — the ticket triage, the password reset, the policy lookup, all of it. That is a 747 chartered to deliver a letter, repeated a million times a month. Routing simple steps to a 3B-active model and reserving frontier calls for the hard 10% is where the cost actually collapses. But here is the part teams get wrong: open weights are not sovereignty. Downloading a model gives you the model. It does not give you the routing layer, the data integration, or the governance — and those are what you end up renting. Three layers are worth owning: routing (which model, what cost, what policy), data (institutional data stays in place), governance (audit, access, spend caps). Everything else — models, GPUs, inference endpoints — is a commodity that gets cheaper each quarter. With ibl.ai you own all the code and the data, run it model-agnostic across any LLM, and deploy anywhere. Nemotron, Claude, GPT, Gemini, Llama — same routing layer, switch anytime. #iblai #AgenticAI #EnterpriseAI #NVIDIA #AI #LLM --- ## The Short Answer **NVIDIA's August 11, 2026 release of Nemotron 3.5 Lightning and NeMo Switchyard commoditizes the model layer and moves the durable advantage to the routing layer — the component deciding which model handles which step, at what cost, under what policy. Open weights are not sovereignty: with ibl.ai you own all the code and the data, run it model-agnostic across any LLM, and deploy anywhere.** Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model with only 3 billion active per call, running on a single GPU at 35% faster than Qwen3.6 35B. NeMo Switchyard is the open library that routes each workflow step to the right model. The pairing matters because most enterprise agent spend is frontier-model inference on tasks that never needed it. 1.6M+ users across 400+ organizations run the platform this way. ## What Is Nemotron 3.5 Lightning? Nemotron 3.5 Lightning is an efficiency-optimized open model built for high-volume agent workloads, not for benchmark leaderboards. Its architecture is the point: 30 billion total parameters with roughly 3 billion active on any given inference call, so about 90% of the weights stay dormant per request. That design has three consequences an infrastructure team cares about. It runs on a single GPU, removing multi-node cluster requirements. It benchmarks 35% faster than Qwen3.6 35B and up to 4× faster than comparable open models on agentic coding tasks. And it ships with open weights, open datasets, and open training recipes, so it can be customized rather than merely called. It is distilled from Nemotron 3 Ultra — frontier-class reasoning compressed into a runtime cheap enough to put in the hot path of an agent loop. ## Why Does a Routing Layer Matter More Than the Model? Because most enterprise agent workloads do not need frontier reasoning at every step, and today most of them get it anyway. A single-model deployment sends every task to the same endpoint: the ticket that needs classifying, the password reset, the policy lookup, and the genuinely hard synthesis problem all hit the same expensive inference call. NeMo Switchyard is an open-source routing library that selects a model per step in a multi-step workflow. That turns model choice from a procurement decision made once into a runtime decision made continuously — which is what makes the savings compound. The economics are easier to see than to argue about. The illustration below assumes 1,000,000 agent steps per month with a typical enterprise mix — roughly 90% routine classification, extraction, and lookup, and 10% genuinely hard reasoning — priced against public list rates:
Approach Routine 900K steps Hard 100K steps Relative monthly cost
Single frontier model Frontier API Frontier API 100% (baseline)
Routed, both hosted Small hosted model Frontier API ~25–35%
Routed, self-hosted small model Nemotron 3.5 Lightning on your GPU Frontier API ~10–15%
The exact figures move with your traffic mix and negotiated rates. The shape does not: the more of your volume that is routine, the more the single-model approach overpays, and the 90/10 split above is conservative for support, ITSM, and compliance workloads. ## Do Open Weights Give You Sovereignty? No — and this is the most common and most expensive misreading of a release like this one. Downloading open weights gives you the model. It does not give you the platform that calls the model, the integrations that feed it institutional data, or the governance that makes it auditable. Sovereignty needs four things, and open weights supply exactly one of them: full source-code ownership of the agent platform, data integration that leaves institutional data in place rather than extracting it, deployment flexibility across your cloud, on-premise, GovCloud, or air-gapped networks, and model agnosticism so you can route dynamically and switch anytime. An organization can run entirely open-weight models and still be locked in — if the orchestration layer calling those models is a vendor's managed service. The lock-in simply moved up the stack, where it is harder to see and more expensive to unwind. ## Which Layers Should an Enterprise Actually Own? Three, and the model is not among them. **The routing layer** decides which model handles which task, at what cost, under what policy — it is where model choice, spend, and governance are actually enforced. **The data layer** keeps institutional data in place, connected through MCP or an equivalent protocol rather than copied into a vendor index. **The governance layer** carries audit trails, access controls, spend caps, and compliance monitoring. Everything below those — the models, the GPUs, the inference endpoints — is a commodity that gets cheaper and better every quarter. NVIDIA just made that concretely true by giving away both an efficient model and the router for it. The pattern is visible in how sophisticated deployments are already built: Goldman Sachs runs hundreds of agents against a custom orchestration harness, where the harness is the durable asset and the model is swappable. Trajectory raised at a $300M valuation from Sequoia specifically to help enterprises customize open-source models — the market for customization infrastructure is growing faster than the market for the models themselves. ## How Should You Evaluate Your Own Stack? Start with a dependency test. If switching LLM providers would take more than a configuration change, you have a lock-in problem regardless of how open your model weights are. That single question separates model-agnostic architecture from the appearance of it. Then audit routing. Identify what share of your agent steps are routine classification, extraction, or lookup currently being served by a frontier model. In most deployments that share is the large majority, and it is the single most common source of avoidable AI spend. Finally, check who owns the orchestration. Routing, governance, and data integration should be infrastructure you hold, not a managed service you rent — because those are the layers that determine whether the next cheap, fast open model is an easy swap or a migration project. On ibl.ai you own all the code and the data, run it model-agnostic across any LLM, and deploy anywhere, with no per-seat pricing: cost tracks tokens actually consumed, not headcount. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on. ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.