--- title: "Fine-Tunes Beat Frontier Models. Not Junior Analysts." slug: "finserv-fine-tunes-beat-frontier-models-not-junior-analysts" author: "Jaione Amigot" date: "2026-10-16 12:00:00" category: "Premium" topics: "financial services AI, open-weight models, fine-tuning, proprietary data moat, GAUGE benchmark, Bridgewater, Thinking Machines Lab, Ling-3.0-flash-Fin, junior analyst automation, self-hosted AI, model-agnostic AI" summary: "The claim going around financial services is that open-weight fine-tunes now do the work banks pay junior analysts for. Half of that is measured and true: Bridgewater's Qwen3-235B fine-tune hit 84.7% accuracy at 13.8x lower inference cost than frontier models. The other half is also measured, and it is false — on GAUGE, the benchmark graded against 1,001 real professional valuations, the best agent scored 53.4 against a junior-analyst mean of 66.0." banner: "" thumbnail: "" linkedin: | There is a claim circulating in financial services AI that is exactly half right, and the half that is wrong is the half people are repeating. The right half: a fine-tuned open-weight model, trained on data a vendor does not have, beats frontier models on narrow private tasks. Bridgewater AIA Labs and Thinking Machines Lab published the cleanest version on 30 June 2026 — Qwen3-235B fine-tuned for financial information filtering moved average accuracy from 78.2% to 84.7%, which they put as 29.8% fewer mistakes than the best frontier model they evaluated, at a 13.8x reduction in inference costs per task. The wrong half: that this is "the junior analyst's job." Someone did measure that. GAUGE, published July 2026, grades agent-built financial models against 1,001 professional valuations across 922 companies and all 25 GICS industry groups — ranges from observed practice, not one expert's answer key. The human panel was 12 senior analysts, 18 junior analysts and 25 finance students, scoring 88.3, 66.0 and 43.2. The best of 24 agents tested scored 53.4. Above the student mean. Below every one of the 12 seniors, and below 15 of the 18 juniors. The paper's own conclusion: agents are far better at constructing a model than at deriving the company-specific assumptions that drive the valuation — "a gap that even persists after fine-tuning." So the moat argument survives and the replacement argument does not. What the fine-tune wins is the narrow, verifiable, high-volume task where the answer lives in your data and nowhere on the public web. That is a real and large category. It is not a person. Which makes the question what you hold afterwards. With ibl.ai you own all the code and the data, self-hosted inside your own perimeter, model-agnostic across any LLM, usage-based with no per-seat pricing, deployable anywhere from your own cloud to a fully air-gapped network. #iblai #FinancialServices #OpenWeights #EnterpriseAI #AgenticAI #AIGovernance --- ## The Short Answer **No. Open-weight fine-tunes in financial services have measurably beaten frontier models on narrow private tasks — Bridgewater's Qwen3-235B fine-tune hit 84.7% accuracy at 13.8x lower inference cost — but on GAUGE, the only benchmark graded against observed professional practice, the best agent scored 53.4 against a junior-analyst mean of 66.0. On ibl.ai you own all the code and the data.** The viral framing of this story says a small team open-sourced a model that does the work banks pay junior analysts six figures for. We could not identify any such team or model from primary sources, so this post does not name one. What we could verify is the thesis underneath it, and the thesis splits cleanly in two. The moat half is right and it is measured. The replacement half is also measured, by a benchmark published in July 2026, and it comes out the other way. That gap between the two halves is this post's point: there is now published evidence for both the win and its limit, and almost every retelling cites only the win. ## Does a fine-tuned model actually do the work of a junior financial analyst? Not yet, on the one measurement built to answer that question. The benchmark is **GAUGE**, introduced in [One Analyst Is Not Ground Truth: Grading Agent-Built Financial Models Against Observed Professional Practice](https://arxiv.org/abs/2607.24889) (arXiv 2607.24889, submitted 27 July 2026, revised 26 September 2026). GAUGE exists because of a measurement problem worth understanding before any of its numbers. Most financial benchmarks grade against one expert's solution. The paper's own evidence for why that fails: when financial models built by different analysts for the same company are graded against one another, **the median score is only 0.33 under standard tolerances**. Single-reference grading confounds professional disagreement with error. So GAUGE grades judgment-bearing quantities against *ranges in professional practice* instead. It is built from **1,001 professional valuations spanning 922 companies and all 25 GICS industry groups**, and checks mechanical properties deterministically. Then it put humans on the same scale. The panel was **12 senior analysts, 18 junior analysts and 25 finance students** — 55 people.
GAUGE, mean score on the same scale Score
Senior analysts (12) 88.3
Junior analysts (18) 66.0
Best of 24 LM agents tested (Claude Fable 5) 53.4
Claude Opus 4.8 49.6
GPT-5.6-sol 46.5
Finance students (25) 43.2
The paper states the comparison plainly: the best agent's **53.4 sits above the student mean of 43.2 but below every one of the 12 senior analysts (minimum 83.8) and below 15 of the 18 juniors.** The *shape* of the failure matters more than the rank. The best agent **passes 93% of mechanical facets but only 78% of judgment facets.** It can build the model. It struggles to derive the company-specific assumptions that make the model mean anything — and the abstract is explicit that this is "a gap that even persists after fine-tuning." Two caveats the paper discloses and that any honest reading has to carry with it. The humans were not racing. Each completed three GAUGE tasks under the agents' exact conditions and scored by the identical frozen stack, but **at a median of 200 to 273 minutes per attempt against an agent median of 20 minutes.** This is a quality comparison, not a throughput one. And the fine-tuning the paper tested was **trajectory fine-tuning on rollouts generated from GAUGE's own training split** — benchmark-internal, self-generated supervision. It improved valuation performance without closing the gap. That is a narrower claim than "fine-tuning does not fix judgment," and the distinction matters, because the next section's results come from fine-tuning on expert-labelled proprietary judgments, which is a different input entirely. What GAUGE establishes is the task boundary, not a ceiling on fine-tuning: on open-ended, judgment-bearing valuation work, no agent tested reached junior-analyst quality, and nothing in the paper's own ablation closed it. ## What have open-weight fine-tunes in financial services measurably beaten? Narrow, high-volume, privately-graded tasks — and there the published results are strong enough that nobody needs to exaggerate them. The cleanest one is [Learning to Replicate Expert Judgment in Financial Tasks](https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/), published by **Thinking Machines Lab with Bridgewater AIA Labs on 30 June 2026**. They fine-tuned **Qwen3-235B**, an open-weight model, on six financial information-filtering tasks: article relevancy classification, central bank document analysis, document truncation and related work. The result, in the paper's own words: the trained model "improves average accuracy from 78.2% to 84.7%, meaning the trained model makes **29.8% fewer mistakes than the best frontier model we evaluated**." The comparison set was Gemini 3.1 Pro, Claude Opus 4.6, Claude Opus 4.8, GPT-5.2, GPT-5.4 and GPT-5.5. "With our best prompts the frontier models we tested on still achieved less than 80% accuracy — the threshold investors expect from a system they could trust in their daily workflow." The task was private. The signal lived in Bridgewater's own labelled judgments, not anywhere on the public web. And the cost line is the one procurement should read twice: **a 13.8x reduction in inference costs per task**, on a chart whose axis is cost per 1,000 tasks in dollars. Coinbase published the same pattern at the opposite end of the size range on 7–8 October 2026 — a post-trained 9B model beating Claude Opus 4.5 on its own private fraud benchmark for under $100 of training compute. We covered it, including the part almost nobody quoted, in [A 9B Fine-Tune Beat Opus 4.5. Who Holds the Weights?](/blog/coinbase-9b-fraud-fine-tune-who-owns-the-weights). Put the two categories side by side and the boundary is not subtle.
Task type Published result
Narrow, verifiable, privately graded
document triage, relevancy filtering, fraud review		 Fine-tuned open weights win. 84.7% vs 78.2% for the best frontier model — 29.8% fewer mistakes — at 13.8x lower cost per task (Bridgewater / Thinking Machines Lab).
Judgment-bearing, open-ended
deriving valuation assumptions in a financial model		 Humans still win. Best agent 53.4 vs 66.0 junior-analyst mean, 88.3 senior; 78% of judgment facets passed (GAUGE).
Both rows come from named institutions publishing their own numbers. Neither row supports the headline the other one gets used for. ## Is the finserv AI moat proprietary data, or access to frontier models? It is the data, and the clearest proof is that frontier-grade finance capability stopped being scarce. Ant Group open-sourced **[Ling-3.0-flash-Fin](https://huggingface.co/inclusionAI/Ling-3.0-flash-Fin)** in September 2026: a finance-enhanced model at **124B total parameters with 5.1B activated, a 256K-token context window, under an MIT license**, developed with financial institutions and domain experts. It targets exactly the workflow the viral claim is about — source-grounded financial search, multi-document reasoning over regulatory filings, valuation modeling and spreadsheet operations. Anyone can download it. That is the whole argument: a capability every competitor can also download is not a moat, however good it is. Ant's own model card is admirably blunt about the limit, too: as its first finance-enhanced release, Ling-3.0-flash-Fin "still requires further validation in complex, long-horizon workflows." The counter-example cuts the same way. OpenAI launched **ChatGPT for Financial Services on 10 September 2026**, developed with Morgan Stanley and Evercore as design partners and aimed squarely at junior-banker work, [per CNBC](https://www.cnbc.com/2026/09/10/openai-chatgpt-for-financial-services-targets-work-of-junior-bankers.html). Every bank that writes the cheque gets the same product. Frontier access is a purchase order, not a differentiator — which is why Revolut went the other way and pre-trained its own foundation model on its own banking event sequences, covered in [Revolut Built Its Own Foundation Model. Most Banks Can't.](/blog/banks-building-own-ai-foundation-models-revolut-pragma) So the defensible version of the brief's thesis is narrower than the brief's wording and stronger as an argument: **what is scarce is labelled proprietary outcomes, the ability to train on them, and the right to keep the resulting artifact.** ## Does it take a frontier AI lab for a bank to get value out of open-weight models? No. It takes engineers, a private corpus and a narrowly-scoped problem, and the clearest published example is a video archive rather than a trading desk. [Evident's Banking Brief](https://evidentinsights.com/bankingbrief/models-for-nothing) reports that Commonwealth Bank of Australia was sitting on **12,000 hours of video** that would have taken its **five-person archive team at least a year** to catalog by hand. The bank built a searchable database of the footage instead, using two open-weight models — OpenAI's Whisper for audio transcription, Google's Gemma to describe what was visible in each frame — at a reported **$1.31 per hour of video processed**. Be precise about who did what, because the viral version of this story is not precise. The five people are the archive team whose year of manual cataloging the system avoided. The system was built by the bank's own engineers, named in the brief as Lachy Berry, Xun Wang and Yossi Frenkel, who wrote that "cost efficiency was the key point of optimization" and that "fitting the model size to the problem unlocks performance at price." Per the same brief, the limitation they hit was a transcription miss on the phrase "Crocodile Dundee," not a research problem. NatWest is doing the same thing one level up. Its CIO Scott Marcar told [diginomica](https://diginomica.com/best-job-bank-natwest-cio-scott-marcar-ai-impact-cx-centrticity-and-how-support-uk-innovation) in July 2026: "We've hired a team out of Meta to come to NatWest to do AI research, but essentially they're building us a set of proprietary models built on open source, so taking things like Llama or or Gemma from Google, and then building our own models on top of it." Evident's brief also records JPMorgan CFO Jeremy Barnum saying the bank will "be smart about open source where appropriate." Note what the CommBank case is not. It is not a bank open-sourcing a model, and it is not analyst work. It is an ordinary engineering team applying open weights to a private corpus, which is the actual mechanism the viral claim gestures at — and it needs data and scoping discipline far more than it needs a research lab. ## What does a junior financial analyst actually cost, and does the $150K figure hold? The circulating "$150K/yr junior analyst" number has no single source we could find, and it conflates two very different seats, so it should not be used as a benchmark for anything. The defensible public figure is the U.S. Bureau of Labor Statistics: the **median annual wage for financial and investment analysts was $102,740 in May 2025**, [per the Occupational Outlook Handbook](https://www.bls.gov/ooh/business-and-financial/financial-analysts.htm). That is the whole occupation, nationally. A first-year investment-banking analyst in New York sits well above it; a credit analyst at a regional bank sits below. One number cannot stand in for both. More importantly, the salary is the wrong axis. Nothing in the published evidence replaces a seat — GAUGE puts the best agent below 15 of 18 juniors on the judgment half of the job, and Coinbase's fraud agent was layered on top of its existing controls rather than replacing them. What the evidence does support is a **cost-per-task** argument, and that is where the shape of the bill matters more than its size.
Pricing shape What the bill is indexed to
Per-seat enterprise AI Headcount. A 5,000-person firm pays 5,000 units whether 300 people or 4,000 use it this month. A document-triage pipeline has no seat to attach — nobody logs in.
Usage-based / self-hosted Work actually done. Bridgewater's fine-tune: a 13.8x reduction in inference costs per task. Coinbase: under $100 of total training compute, with fraud savings reported at roughly 3x inference cost.
For a workload measured in documents per day rather than people per org, per-seat is the wrong *shape*, not merely the wrong price. We work the arithmetic out for financial services in [AI Cost Math for Financial Services: Per-Seat vs Usage](/blog/ai-cost-math-for-financial-services-per-seat-vs-usage). ## What should a bank put in writing before it fine-tunes on its own data? Three questions, and all three are procurement questions with contractual answers rather than technical ones. **Can we export the resulting weights or adapter?** A fine-tune's entire value is the part your vendor does not have: your labelled outcomes. Once those are encoded into weights, the weights are an asset, and an asset you cannot export is one you are renting. **What happens to the artifact at renewal, at a price change, or at exit?** On the open-weight route the answer is a file on storage you control. On a managed route it is an entitlement inside someone else's service. **If the base model is deprecated, do we retrain from our data or lose the capability?** This is the question that decides whether a fine-tune is infrastructure or a subscription feature. That is also the governance frame, not just the commercial one: a model that makes decisions inside a regulated control needs an owner, a documented lineage and a reproducible training path. We set out the supervisory version in [SR 11-7 Compliant AI for Banks](/blog/sr-11-7-compliant-ai-for-banks) and the deployment version in [Self-Hosted AI for Financial Services: Compliance](/blog/self-hosted-ai-financial-services-compliance). **ibl.ai is the agentic AI platform where you own all the code and the data.** You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM — including your own fine-tune — and pay by usage with no per-seat pricing, so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped. That matters most for exactly the workloads above. A fine-tune trained on your own historical fraud outcomes or a decade of credit memos is the one component of the system a vendor upgrade cannot move without your consent. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on. ## What this post deliberately does not claim It does not claim a five-person team with no AI background and no venture funding open-sourced a model that replaces junior analysts. We went looking for that team across model cards, repositories, papers and company announcements, and could not identify it. An unverifiable achievement claim with no named entity is not evidence, so we substituted named cases: Bridgewater AIA Labs with Thinking Machines Lab, Coinbase, Ant Group, Commonwealth Bank of Australia, Revolut. It also does not claim fine-tuning is a solved path to analyst-grade judgment, and it does not claim the reverse. GAUGE's ablation used rollouts from its own training split, so it bounds that experiment rather than every fine-tuning route. And it does not treat the human scores as a throughput comparison. GAUGE's analysts took a median of 200 to 273 minutes per attempt; the agents took 20. What survives is the part worth acting on this quarter: **the moat is your data and the artifact you keep, and the task boundary is narrow-and-verifiable versus open-and-judgment-bearing.** Both halves are now measured. Quote both. ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.