The Short Answer
Extending a reasoning model's thinking time can reduce its accuracy, and the failure modes differ by model family. On ibl.ai you own all the code and the data, run it model-agnostic across any LLM, and pay with no per-seat pricing — so a legal team can route brief drafting to a deep-reasoning model and contract extraction to a cheap fast one — the correct response to this finding, and impossible on a platform locked to one vendor's models.
The finding comes from Inverse Scaling in Test-Time Compute, a 14-author study out of the Anthropic Fellows Program with collaborators at the University of Edinburgh, EPFL, UT Austin, Scale AI and Constellation.
One correction up front, because the result is widely recirculated with the wrong attribution: this is not Microsoft research. Microsoft is not among the affiliated institutions.
The finding is real and independently corroborated elsewhere in the literature, but it belongs to these authors.
What did the study actually find?
The researchers constructed evaluation tasks on which large reasoning models get worse as they are allowed to reason longer — an inverse relationship between test-time compute and accuracy.
The tasks span four categories: counting with distractors, regression with spurious features, deduction with constraint tracking, and advanced AI-risk evaluations.
They identify five failure modes, and the differences between model families are the useful part:
| Failure mode | Where it showed up |
|---|---|
| Distraction by irrelevant information | Claude models, increasingly with length |
| Overfitting to problem framing | OpenAI o-series (which resisted distractors) |
| Drift from sound priors to spurious correlations | Across families |
| Loss of focus on long deductive chains | All models tested |
| Amplification of concerning behaviors | Claude Sonnet 4, self-preservation expressions |
Note what this is and is not. These are constructed tasks chosen to expose the effect, not a claim that reasoning models are worse in general. Test-time compute scaling works. The finding is that it is not monotonic, and that "turn thinking up to maximum" is not a safe default.
Why does this matter more for legal work than for most domains?
Because legal workloads split unusually cleanly into two kinds of task, and firms routinely run both through the same expensive configuration.
Deductive work rewards deep reasoning. Drafting a brief, reconciling conflicting authority, reasoning about how a clause interacts with three others — these are genuinely multi-step. Constraint tracking is exactly the category where the study found all models struggle, which argues for the strongest model available and for human review, not for one or the other.
Extraction work does not. Pulling the renewal date, the governing-law clause and the notice period out of 4,000 contracts is retrieval with a schema. Extended reasoning on a retrieval task buys latency and cost, and — per the study's first and third failure modes — invites the model to reason its way past a correct answer it had already found.
Running both through a maximum-reasoning configuration is not conservative. It is more expensive and, on the extraction half, plausibly less accurate.
What does model-aware routing look like in practice?
Routing means the platform chooses the model per task, on criteria you set:
- Task class. Extraction, classification and summarization to a fast, cheap model with reasoning off or low. Drafting, analysis and multi-document reconciliation to a deep-reasoning model.
- Failure cost. A mis-extracted date is caught by validation against a schema. A flawed argument in a filed brief is not. Spend compute where errors are expensive and hard to detect.
- Volume. The extraction half of a legal workload is usually the high-volume half, so it dominates the bill. Moving it to a cheaper model is where the savings are.
- Verification, not trust. Structured outputs and deterministic validation on the extraction path mean a wrong answer is caught by the system rather than by a reader.
None of this is exotic. It is ordinary engineering, and the only thing that makes it hard is a platform that will not let you do it.
What blocks routing on most legal AI platforms?
Single-vendor lock-in, and per-seat pricing.
If your platform is built on one provider's models, "route this workload to a cheaper model" is not a setting — it is a migration.
And under per-seat licensing the economics point the wrong way: when you pay per lawyer per month regardless of usage, there is no budget line that improves when you route a workload to a cheaper model. The savings, if any, accrue to the vendor.
Per-seat legal AI runs roughly $300–500 per lawyer per month at the high end and $200–500 for adjacent research products.
That is a headcount-indexed cost for a workload whose actual shape is spiky, uneven, and dominated by bulk extraction that does not need a frontier reasoning model at all.
How does an owned, model-agnostic platform change this?
On ibl.ai you own all the code and the data.
The platform is model-agnostic across any LLM — Claude, GPT, Gemini, Llama, Qwen or your own fine-tune — so routing by task class is configuration rather than re-platforming, and adopting a better or cheaper model when one ships does not require anyone's permission.
Billing is usage-based with no per-seat pricing, which means the savings from routing land in your budget.
And because you deploy anywhere — your own cloud, on-premise, or a fully air-gapped network — privileged client material never has to leave a perimeter you control, which is the other reason legal teams end up self-hosting. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.
ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.
What should a legal team do with this finding?
Measure before you assume. The study's practical lesson is that reasoning depth is a parameter with an optimum, not a dial where higher is safer.
Split your workloads by task class and benchmark each against two or three models at different reasoning settings, scoring accuracy rather than plausibility. Put deterministic validation on everything extractive.
Then route — and re-check when models change, because these failure modes are model-family-specific and the next version may not share them.
Related reading: why model routing beats betting on a single frontier model and what an on-premise legal AI platform actually requires.