Blog
LLM Infrastructure
Model selection, hosting, fine-tuning, cost optimization, and scaling LLM-powered systems in production.
773 articles in this category
Two Decision Models in Nine Days. One You Can Own.
TypeSafe shipped Jev on 15 September 2026 and Fastino shipped GLiNER2.5-Decide on 24 September. Two vendors, nine days, the same architectural claim: the routing and classification an agent does all day should not run on a frontier model. One of the two is Apache 2.0.
Letting a K-12 AI Agent Run Code Without Letting Data Out
An AI tutor that can actually run a student's Python is worth far more than one that can only talk about it — but only if the district can say where that code ran and what it could reach. Agent sandboxes answer that with a Linux VM that starts with no network at all.
Who Audits the AI Writing Into the Nurse's Flowsheet?
Oracle Health made its Clinical AI Agent for nurses available in the US on September 14, 2026, for structured documentation at the bedside, in fields that sit outside FDA device oversight.
An Agent That Moves Money Needs Row-Level Permissions
AWS open-sourced TOLAP on 22 September 2026 under Apache-2.0: row filtering and column masking enforced inside agent tools, across three SDKs and fourteen framework integrations.
Whose Agents Run Your Operations? The Managed Agent War
Five vendors now sell managed enterprise agent platforms, from AWS Bedrock AgentCore in October 2025 to Google's Gemini Enterprise Agent Platform in April 2026. All of them operate the agents for you, and that is the whole buying decision.
Most University AI Work Is Judgment, Not Generation
At published September 2026 prices, a million classification decisions cost $12,500 on GPT-6 Astra, $125 on GPT-6 Luna and $42 on TypeSafe's Jev — so most of the 99% saving is model choice, not a new model class.
Agent Infrastructure Went Open Source. The Record Didn't.
Chutes and researchers at Harvard and Chicago released 6,122,413,756 production LLM requests across 9,174 models — in twelve metadata fields that hold no prompts, no responses and no tool calls.
The AI Intel Report That Nearly Triggered a Ship Raid
CNN reported on 18 September 2026 that a chatbot-written intelligence report put armed personnel and aircraft in motion against a Chinese cargo ship. Four anonymous sources; the real cargo was never established.
Open Weights Tied Grok 4.7. What That Does to Procurement
Xiaomi's MIT-licensed MiMo-V2.6-Pro scored 46 on Artificial Analysis's Intelligence Index on 22 September 2026 — the same score as Grok 4.7, released a day earlier. DeepSeek V5, meanwhile, has not shipped.
Distributing Your Agent Platform Changes What You Own
Pine Labs and Google Cloud announced Gemini-powered merchant agents on 24 September 2026, serving over 1 million merchants on ₹17.15 trillion of FY26 transaction value, and distribution is what changes the ownership question.
Four Frontier Models in a Day: The Real Cost Is Migration
Four frontier models shipped on September 22, 2026, and seven Claude models were retired during 2026. AWS puts one production model migration at two days to two weeks — the recurring cost is migration, not licensing.
DoWI 8430.01 Bans External AI Hosting, Not Just Training
DoW Instruction 8430.01, signed August 31 and effective September 8, 2026, bars non-public department information from any generative AI service that does not reside on department systems and is not approved — with a six-question buyer checklist keyed to the instruction's own clauses.
Hospitals Buy the Charting Bot, Not the Early Warning
Documentation AI sells because its benefit is visible on day one; sepsis early warning saves more lives but the saved death is a counterfactual. Medicare now pays up to $61.84 per eligible case for it.
105 Forward-Deployed Roles, 5 Sales Roles: AI's Last Mile
On September 21, 2026 the job boards of OpenAI, Anthropic and Palantir list 105 forward-deployed engineering roles against 5 in sales development — Palantir posts 77 while carrying about 70 salespeople.
YC Open-Sourced QM: The Harness Became Infrastructure
Y Combinator open-sourced QM on 31 July 2026 under an MIT license, and says it runs the harness internally across four departments — not across its portfolio. The substance is scoping.
Agent Memory Fails at State Transitions, Not Retrieval
On StateMemBench, released August 2026, leading agent memory systems answered with the current state only 13–20% of the time. The failure is not retrieval quality — nothing marks a fact as superseded.
Clinical AI Trials Outlive the Models They Evaluate
Claude Opus 4.1 was retired on 5 August 2026, a year after release, while a 2013 PLOS Medicine study of 600 trials found a median of 21 months from trial completion to publication — and only 19.4% of completed AI imaging trials publish at all.
NYT v. OpenAI Discovery and the Law Firm AI Checklist
The unsealed filings reported on September 17, 2026 in NYT v. OpenAI are about training data, not client files. The user-log question was decided earlier on the same docket: 20 million ChatGPT logs, affirmed January 5, 2026.
94% of the Mid-Market Uses GenAI. 2% Have Scaled It
Kaufman Rossin found 94% of mid-market companies use generative AI tools and 2% have AI embedded across the business — and in the same sample only 33% run integration or API management.
LLMs Can't Invent: What That 2024 Paper Means for Buyers
"Theory Is All You Need" is not new and not a proof: Felin and Holweg published it in Strategy Science 9(4), 346–371, in 2024, arguing that LLMs are backward-looking and imitative rather than originating.
Backline Is a Control Plane, Not a Quantum Computer
Xanadu and AMD released Backline on September 10, 2026 under Apache 2.0. Its sub-3-microsecond figure is a 2.305 μs FPGA-to-CPU round trip over RoCE v2 — the GPU path is 4.5 μs, and the quantum side is simulated.
Beijing's AI Eye Clinic Hit 3.8% Clinician Adoption
A Nature Medicine Comment published 10 September 2026 reports that Beijing Tsinghua Changgung Hospital's AI-TEC agent clinic was used in 41 of 1,113 examinations — 3.8% — before workflow changes lifted it to 23%.
When Your Software Vendor Applies for a Bank Charter
Block applied on 8 September 2026 for an uninsured national trust bank that cannot take deposits or lend, and the OCC has taken 40 de novo charter applications in 18 months against 48 in the 14 years to 2024.
Generation Is Commoditized. Judgment Is the New Frontier
TypeSafe announced Jev on September 15, 2026 — a decision model priced at $0.042 per million input tokens with output unmetered. It is not the first model built to judge: CriticGPT and Prometheus 2 both shipped in 2024.
About LLM Infrastructure
Running large language models in production requires careful infrastructure planning—from model selection and hosting to fine-tuning, cost optimization, and GPU provisioning. Explore practical guides on building reliable, scalable LLM infrastructure that balances performance, cost, and latency for real-world applications.