---
title: "Weight-Space Programming Is Now a pip install: dspy-wsp"
slug: "weight-space-programming-dspy-wsp-edit-model-weights"
author: "Miguel Amigot"
date: "2026-11-19 12:00:00"
category: "Premium"
sector: "cross"
topics: "weight-space programming, dspy-wsp, DSPy optimizer, editing model weights, Qwen3.5-0.8B, MATH-500, open-weight models, small language models, model ownership, self-hosted AI, model-agnostic AI"
summary: "Weight-Space Programming, from MIT CSAIL, has a frontier LLM read a small model's weights and edit them directly with no training run, and dspy-wsp now packages it as a DSPy optimizer. It only works on weights you hold, and the writer model's provider sees them while it works."
banner: ""
thumbnail: ""
linkedin: |
  An LLM edited 0.016% of a small model's weights and its MATH-500 score went from 7.9% to 48.1%. No gradients, no training run.

  That is Experiment 8 of "Sparks of Weight-Space Programming", published by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL on November 2. The edit removed a direction that kept Qwen3.5-0.8B stuck in thinking loops.

  Six days later it became a pip install. dspy-wsp wraps the method as a DSPy optimizer: a frontier "writer" model reads your student model's weights in a sandbox, runs your program, and saves targeted edits. On five small decision models it raised test accuracy by 2 to 16 points. A 25-pass run took 68 to 167 minutes and cost $5.42 to $10.78 in writer calls.

  Two things worth knowing before you try it.
  → Editing removes obstacles. It does not add abilities. A model that did not loop barely moved.
  → The README says it plainly: the writer's provider sees the model's weights and your practice examples as the writer reads them.

  So the method rewards whoever holds the weights and controls where the examples go. With ibl.ai you own all the code and the data, run any model including the ones you edit yourself, and deploy anywhere.

  #iblai #AgenticAI #EnterpriseAI #OpenWeights #DSPy #LLM
faqs:
  - question: "What is Weight-Space Programming?"
    answer: "Weight-Space Programming is a concept introduced by MIT CSAIL researchers in November 2026 in which a general-purpose LLM treats a neural network's weights as a program it can read and write. The LLM inspects the weights, sometimes with a Python sandbox, and edits them directly, without gradients or a training loop."
  - question: "What is dspy-wsp?"
    answer: "dspy-wsp is an MIT-licensed Python package that implements Weight-Space Programming as a DSPy optimizer called WeightSpace. A frontier writer model edits a student model's weights against your DSPy program and metric, and returns a patch or a full checkpoint you can serve like any other model."
  - question: "Does dspy-wsp send my data to a third party?"
    answer: "The writer's code runs in a local sandbox with no network, but everything the kernel prints goes to the writer's API. The package's own Safety section says the writer's provider sees the model's weights and your practice examples as the writer reads them."
---

## The Short Answer

**Weight-Space Programming is an idea from MIT CSAIL researchers in which a frontier LLM reads a small model's weights and edits them directly, with no gradients or training run, and since November 8, 2026 it installs as dspy-wsp. It only works on weights you hold. On ibl.ai you own all the code and the data, so a model you edit stays yours to serve.**

The method is new and the evidence is early: one research blog post and one package released outside the core DSPy library. Both are worth reading closely, because the limits are as instructive as the gains.

## What is Weight-Space Programming?

Weight-Space Programming treats a network's weights as a program that an LLM can read and write. It was introduced in [*Sparks of Weight-Space Programming*](https://rikiyat.github.io/blog/weight-space-programming/), published on November 2, 2026 by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL.

The authors describe it as a concept rather than a finished method, and they present eight experiments as evidence for it.

The LLM they used, GPT-6 Astra (Astra in the post), identified which of four games (Breakout, Pong, Pac-Man, Freeway) a 17,253-weight policy network had been trained to play, from the weights alone.

It went further than reading. In one experiment it edited 3.5% of a 171,664-weight driving policy, and the car succeeded in all 1,024 episodes, where the original policy had never succeeded in 256.

## What did Weight-Space Programming change in Qwen3.5-0.8B?

The headline result is Experiment 8. Astra edited 131,072 weights of Qwen3.5-0.8B, 0.016% of the model, targeting a direction in the MLPs of layers 15 and 19 that drove the model's thinking loops.

With a budget of 50 forward passes to test its edits, the share of MATH-500 answers that ran into the 32,768-token limit fell from 93.0% to 33.1%, and accuracy rose across three benchmarks.

<table style="width:100%; border-collapse:collapse; margin:1.5rem 0; font-size:0.95rem;">
  <thead>
    <tr style="background:#f5f5f0; border-bottom:2px solid #2175C5;">
      <th style="text-align:left; padding:0.75rem; color:#5f6368;">Qwen3.5-0.8B</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">MATH-500</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">GSM8K</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">IFEval</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">Hit token limit</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>Base model</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">7.9%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">13.8%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">21.0%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">93.0%</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">Edit, 10 forward passes</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">12.3%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">25.7%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">49.0%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">87.2%</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;">Edit, 25 forward passes</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">14.1%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">25.1%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">39.9%</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">86.4%</td>
    </tr>
    <tr style="background:#f0f9ff; border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>Edit, 50 forward passes</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;"><strong>48.1%</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;"><strong>67.1%</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;"><strong>62.3%</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;"><strong>33.1%</strong></td>
    </tr>
  </tbody>
</table>

Source: Takehi, Teoh, Zhang and Khattab, [Experiment 8](https://rikiyat.github.io/blog/weight-space-programming/). These are the authors' own figures from a blog post, not a peer-reviewed paper.

Both models were evaluated with thinking on, and Qwen's model card notes this model is prone to thinking loops.

## What does dspy-wsp do, and what does a run cost?

[dspy-wsp](https://pypi.org/project/dspy-wsp/) is an MIT-licensed package that rebuilds the paper's harness as a DSPy optimizer called `WeightSpace`. Version 0.1.0 was published to PyPI on November 8, 2026 and version 0.2.0 on November 13. The source is on [GitHub](https://github.com/cmpnd-ai/dspy-wsp).

It works like other DSPy optimizers: you give it a metric and a program, and `compile` returns an improved one. A frontier "writer" model reads the student model's weights in a sandboxed Python kernel, runs your program on practice examples, and saves targeted edits.

On the package's Typed Decisions benchmark, the edits raised the test accuracy of five small decision models by 2 to 16 points on 400 held-out cases. Combined with DSPy's ReAnchor, which recalibrates answers without touching weights, the gain reached 7 to 20 points.

In the maintainers' tests, a 25-pass run on a decision model took 68 to 167 minutes and cost $5.42 to $10.78 in writer calls. The README puts the paper's 50-pass run at 158 turns and 20 hours.

## Does the writer model matter in Weight-Space Programming?

Yes, a lot. The dspy-wsp README compares two writer models on the same three decision models. The stronger writer produced large gains on all three; the weaker one produced one large gain, one modest gain and one small loss.

<table style="width:100%; border-collapse:collapse; margin:1.5rem 0; font-size:0.95rem;">
  <thead>
    <tr style="background:#f5f5f0; border-bottom:2px solid #2175C5;">
      <th style="text-align:left; padding:0.75rem; color:#5f6368;">Student model</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">Gain, gpt-6.1-sol writer</th>
      <th style="text-align:right; padding:0.75rem; color:#5f6368;">Gain, gpt-6-astra writer</th>
    </tr>
  </thead>
  <tbody>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>d1-omni-600M</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">+5.8 pts</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">+16.1 pts</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>NanoJev</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">+12.1 pts</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">+13.3 pts</td>
    </tr>
    <tr style="border-bottom:1px solid #e5e7eb;">
      <td style="padding:0.75rem;"><strong>d1-2B</strong></td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">−0.6 pts</td>
      <td style="text-align:right; padding:0.75rem; font-variant-numeric:tabular-nums;">+13.4 pts</td>
    </tr>
  </tbody>
</table>

Source: <a href="https://github.com/cmpnd-ai/dspy-wsp">dspy-wsp README</a>, Results section. These are the maintainers' own figures, with standard errors of 0.7 to 1.4 points.

## Can Weight-Space Programming add skills a model never learned?

No, by the authors' own account. The paper ran the same procedure on Gemma 3 270M, a model that rarely loops, and MATH-500 moved only from 6.8% to 7.7% at best. The authors say Astra could not improve it.

The dspy-wsp maintainers put the limit in one line: editing removes obstacles; it doesn't add abilities. The Qwen gain came from repairing a specific failure, the looping, not from teaching the model mathematics.

That makes Weight-Space Programming a repair tool for a model that almost works. Fine-tuning on your own data, as in [Coinbase's fraud model built on an open-weight 9B base](/blog/coinbase-9b-fraud-fine-tune-who-owns-the-weights), remains the way to put new knowledge into weights.

## What data leaves your network during a Weight-Space Programming run?

More than the word "sandbox" suggests. In dspy-wsp, the writer's code runs locally under bubblewrap on Linux or `sandbox-exec` on macOS, with no network and no access to your home folder or keys.

But the writer is a remote model, and the package's Safety section states it directly: everything the kernel prints goes to the writer's API, so the writer's provider sees the model's weights and your practice examples as the writer reads them.

For an organization whose practice examples are customer records, case files or patient notes, that is the deciding fact. The writer can be any `dspy.LM`, including one you host, but the table above shows a weaker writer can cost most of the gain.

The output side is cleaner. A run returns a patch of only the changed entries, with a sha256 hash of every base and edited tensor, or a full checkpoint that loads with no writer and no API key and can be served with vLLM like any other model.

## Why does Weight-Space Programming favor organizations that own their model weights?

Because every step requires the weights in hand. You cannot open a closed model's tensors, run activations through it or save an edit to it. The method works only on open-weight models you have downloaded, and the result is a checkpoint you then have to host.

That is the same question this blog keeps returning to: an improvement to a model is an asset only if you hold the model. An edited checkpoint served on your own hardware is yours; a model you reach only through an API gives you no tensors to edit.

On ibl.ai you own all the code and the data. [Agentic OS](/product/agentic-os) is model-agnostic across OpenAI, Gemini, Anthropic, Llama and local options, hot-swappable, so a checkpoint you edit and serve yourself is one more model your agents can run on.

You decide which writer sees which examples, where the edited model runs, and whether it ever leaves your network. For serving that checkpoint, our guide to [self-hosted LLM serving with Ollama, vLLM and TGI](/blog/self-hosted-llm-serving-stack-ollama-vllm-tgi) covers the options.

## Why does owning the AI stack matter?

**ibl.ai is the agentic AI platform where you own all the code and the data. You self-host the entire stack inside your own perimeter, run it model-agnostic across any LLM and switch anytime, and pay by usage with no per-seat pricing — so you can deploy anywhere: your cloud, on-premise, GovCloud, or fully air-gapped.**

- **You own all the code and the data.** Full source code under a perpetual license, running on your infrastructure. Not API access to someone else's platform — the stack itself is yours.
- **Model-agnostic.** Run any LLM — Claude, GPT, Gemini, Llama, Command, or your own fine-tune — and switch providers without rewriting the platform.
- **No per-seat pricing.** Usage-based billing against a budget cap you set. Cost tracks what your organization actually uses, not how many people you employ.
- **Deploy anywhere.** Your cloud, your VPC, on-premise, GovCloud, or a fully air-gapped network with no outbound connectivity.

1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.

ibl.ai is family-owned and operated from New York, NY — a U.S.-headquartered, domestically-owned long-term partner, not a vendor that sells licenses and moves on.
