--- title: "GPT-6 Astra, ARC-AGI-3, and the Harness Footnote" slug: "gpt-6-astra-arc-agi-3-model-agnostic-architecture" author: "ibl.ai Engineering" date: "2026-09-07 10:00:00" category: "Premium" topics: "model-agnostic, vendor lock-in, enterprise AI, GPT-6 Astra, ARC-AGI-3, benchmarks, AI architecture" summary: "GPT-6 Astra's headline 98.6% on ARC-AGI-3 came from a harness OpenAI built for it. On the standard harness — the one ARC Prize calls apples-to-apples — it scored 62.7%. Both numbers are real, and the gap between them is an argument for model-agnostic architecture." banner: "" thumbnail: "" linkedin: | GPT-6 Astra scored 98.6% on ARC-AGI-3. That number is real. It is also not the one to compare against other models. ARC Prize published three figures for the same model: → 62.7% on the Standard harness (Semi-Private, $26,098) → 99.9% with a Provider Adapter at high reasoning ($18,817) → 98.6% with a Provider Adapter at max reasoning ($17,332) The Standard harness makes a model carry forward its own notes. The Provider Adapter preserves opaque reasoning state between requests and compacts long conversations, so the model reuses prior work. ARC Prize says plainly that these are different evaluation conditions, and that the Standard harness is the "apples-to-apples comparison across providers." So the like-for-like line on the leaderboard is 62.7% for Astra against 30.2% for Claude Opus 5 — still more than double, and still an enormous jump from GPT-5.6 Sol's 7.8% six months ago. Astra is a real leap. It is just not a 3x leap over the field, and the widely-quoted number is the one its own provider's harness produced. Here is the part enterprises should sit with: if the comparison surface itself is shaped by the provider, you cannot fully outsource your model choice to a leaderboard. You have to be able to run the candidates against your own work. That is only possible if swapping models is cheap. If your authentication, data pipelines, guardrails and compliance documentation are welded to one vendor's API, every re-evaluation is a rebuild — so you don't re-evaluate, and the lock-in becomes self-enforcing. Three frontier generations have shipped in six months. Astra is priced at $10 per million input tokens and $50 per million output. It is also the first OpenAI model classified Critical for cybersecurity under their Preparedness Framework. With ibl.ai you own all the code and the data — self-hosted inside your own perimeter, model-agnostic across any LLM, usage-based with no per-seat pricing, deployable anywhere from your own cloud to a fully air-gapped network. Swap the model, keep the stack. #iblai #AgenticAI #EnterpriseAI #ModelAgnostic #LLM #AIArchitecture #VendorLockIn --- ## The Short Answer **GPT-6 Astra's headline 98.6% on ARC-AGI-3 came from a harness OpenAI built for it; on the standard harness ARC Prize calls "apples-to-apples," it scored 62.7% against Claude Opus 5's 30.2%. Both numbers are real. When the comparison surface is vendor-shaped, the only durable answer is architecture that treats models as swappable — with ibl.ai you own all the code and the data, model-agnostic across any LLM.** Astra is a genuine leap. The point of this post is not that the number is fake — it is that there are three numbers, they differ by 36 points, and which one you quote depends on whose harness ran the test. ## What did GPT-6 Astra actually score on ARC-AGI-3? Three different figures, all published by [ARC Prize](https://arcprize.org/blog/astra) for the same model on the Semi-Private set:
Harness Reasoning effort Score Cost of run
Standard (cross-provider) — 62.7% $26,098
Provider Adapter High 99.9% $18,817
Provider Adapter Max 98.6% $17,332
The 98.6% that circulated is the third row. Note the cost column, because it inverts the usual intuition: the adapter run scored far higher *and* cost roughly $8,000 less, and ARC Prize records the adapter runs as approximately **3.66x faster** by aggregate elapsed time. Efficiency and score moved together, which is what makes the harness difference substantive rather than cosmetic. ## What is the difference between the standard harness and a provider adapter? It is a difference in what the model is allowed to carry between steps. The **standard harness** requires a model to carry forward whatever notes it chooses to keep as it moves through the environment, under a neutral interface that every provider meets on the same terms. The **provider adapter** preserves opaque reasoning state between requests and uses compaction for longer conversations, so the model can reuse prior work rather than reconstructing it. That is not cheating — it is the model using infrastructure its provider built for it, and it reflects how the model would actually be deployed. ARC Prize is explicit that these are **different evaluation conditions**, and that the standard harness is the one providing "an apples-to-apples comparison across providers." Both numbers describe something true. They just answer different questions: *how good is this model in a neutral rig* versus *how good is this model with its vendor's own scaffolding*. ## How does Astra compare to other frontier models on the same benchmark? On the standard harness, which is the like-for-like column:
Model ARC-AGI-3
GPT-6 Astra62.7%
Claude Opus 530.2%
GPT-5.6 Sol7.8%
Claude Opus 4.81.5%
GPT-5.6 Terra0.8%
Gemini 3.1 Pro0.4%
Two things are true at once, and enterprise readers need both. Astra genuinely leads: 62.7% is more than double the next model, and the jump from GPT-5.6 Sol's 7.8% is enormous for six months. Anyone dismissing this as benchmark theater is not reading the standard-harness column either. And the headline overstated the margin. Comparing Astra's adapter-harness 98.6% to Opus 5's 30.2% puts two different evaluation conditions in the same sentence. The real like-for-like gap is large; it is not near-saturation against a field stuck in the twenties. ## Why does a benchmark footnote matter for enterprise architecture? Because it means you cannot fully outsource model selection to a leaderboard. If the scaffolding a vendor supplies can move its own score by 36 points, then published comparisons are partly a measurement of vendor tooling, not only of model capability. That is not an accusation — ARC Prize published all three numbers precisely so the distinction is visible. But it does change what a procurement team can conclude from a headline. The only reliable way to know which model is best for your workload is to run the candidates against your own data, your own prompts and your own evaluation set. That is a straightforward exercise — *if* switching models is cheap. It usually is not. Most enterprise AI deployments weld authentication, data pipelines, prompt engineering, safety guardrails and compliance documentation to one provider's API. Under that architecture every re-evaluation is a rebuild, so the re-evaluation never happens, and the lock-in becomes self-enforcing: you stay because leaving is expensive, not because the model is still the best one. We wrote about this pattern when [GPT-5.6 shipped and model routing became the enterprise question](/blog/gpt-5-6-model-routing-enterprise-ai-agnostic); Astra is the same lesson at a larger magnitude. ## What does GPT-6 Astra cost, and what else shipped with it? Astra was released on **September 3, 2026**, priced at **$10 per million input tokens and $50 per million output tokens**, with cached input at $1 and cache writes at $12.50. It also arrived with a classification worth noting: it is the **first OpenAI model to cross the "Critical" threshold for cybersecurity** under the company's Preparedness Framework, meaning it can identify and develop working exploits against hardened systems without step-by-step human direction. Exploit-creation capability is gated behind a limited-access program. In OpenAI's own cyber-safety testing the model refused 91.5% of requests that should not receive cyber assistance, against 59% for GPT-5.6 Sol, as [reported by CSO Online](https://www.csoonline.com/article/4218679/openai-launches-gpt-6-astra-its-first-model-to-cross-a-critical-cybersecurity-threshold.html). For a regulated institution, that is a second argument for the same architecture. A model whose capability profile now includes autonomous exploit development is a model your security and compliance teams will want to reason about specifically — including the ability to route certain workloads away from it, or to keep sensitive work on a self-hosted open-weight model entirely. Routing is only an option if the platform is model-agnostic to begin with. ## What does model-agnostic architecture actually require? Four things, and none of them is the model itself. - **A unified API layer** that abstracts provider-specific integration, so swapping a model does not change application code. - **A context layer** that connects agents to institutional systems — SIS, HRIS, CRM, ERP, EHR — independently of which model reasons over them. - **Memory and evaluation that persist across model changes**, so an upgrade does not reset your accumulated context or invalidate your benchmark set. - **Routing**, so sensitive queries can go to a self-hosted open-weight model while general queries go wherever is cheapest that week. With ibl.ai, you own all the code and the data — the platform is deployed on your own infrastructure with full source code access, model-agnostic across any LLM, usage-based with no per-seat pricing, and deployable anywhere from your own cloud to on-premise, GovCloud or a fully air-gapped network. The practical test is simple. When the next frontier model ships — and on the current cadence that is weeks, not quarters — can you evaluate it against your own workload and switch if it wins, without a migration project? If the answer is no, you are not choosing your model. Your first integration decision is still choosing it for you. ibl.ai is family-owned and operated from New York, NY. For institutions weighing who will still be standing behind a deployment several model generations from now, that is not a small consideration. *Related reading: [why vendor lock-in is the real risk in model-agnostic AI](/blog/model-agnostic-ai-the-real-risk-is-vendor-lock-in), and [what published inference benchmarks reveal about the cost floor](/blog/inference-cost-floor-jalapeno-published-benchmarks).* *Sources: ARC-AGI-3 scores, harness definitions and run costs from [ARC Prize's Astra report](https://arcprize.org/blog/astra); pricing and release details from [OpenAI](https://openai.com/index/gpt-6-astra/); cyber-safety classification via [CSO Online](https://www.csoonline.com/article/4218679/openai-launches-gpt-6-astra-its-first-model-to-cross-a-critical-cybersecurity-threshold.html) and [The New Stack's analysis of the harness caveat](https://thenewstack.io/astra-arc-agi-benchmark/).* ## Why does owning the AI stack matter? **ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.** - **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours. - **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform. - **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ. - **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University. ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.