MIT researchers show GPT-6 Astra can rewrite other models' weights

The researchers call the approach "Weight-Space Programming": a language model acts like a mechanic turning individual knobs instead of paying for a full retraining run. Depending on the experiment, Astra worked directly with lists of numbers or used programming tools and feedback from test runs. This is a research blog, not a shipped product.

What Astra did to Qwen

The most practical test used Qwen3.5-0.8B. Astra was asked to improve its math skills using a coding environment and 50 test iterations with practice problems, then evaluated on problems it had never seen. It found a fault Qwen's own documentation warns about: thinking loops, where the model repeats the same text instead of reaching an answer. Astra changed 0.016 percent of all weights. MATH-500 accuracy rose from 7.9 to 48.1 percent and instruction following from 21 to 62 percent.

The loops were not fully eliminated. A third of responses still hit the maximum text length, down from 93 percent before. With Gemma 3 270M, a smaller model that rarely fell into such loops, Astra produced no measurable improvement. The researchers suspect it removed a blockage suppressing capabilities Qwen already had; skills a model fundamentally lacks could not be added by rewriting individual weights in this experiment.

Reading weights, and writing them

Fed the raw weights of trained networks, Astra correctly identified which of four computer games each network was built to play. It recognised the basic shapes an image generator produced, though it confused filled shapes with outlines, and matched the training language of small language models without training data and without running them.

Given two fine-tuned language models, it could not tell from the weights alone which had been trained to always agree with users. Only when allowed to observe what the models paid attention to during text processing did it identify the sycophantic model across all three pairs.

Astra also wrote weights from scratch. Given game rules and source code, it produced models for three simple games. In Breakout its model averaged 517 points against 25 for a baseline trained over 100 million rounds. It fell behind the longest-trained versions in the other two games. On 9x9 Go it wrote the weights of a network with about 1.5 million parameters; given 50 test iterations, it beat a comparably sized model trained on roughly 20 million games.

Old hat, or a new handle on model behaviour

AI pioneer Jürgen Schmidhuber called the approach "old hat" on X, pointing to his own work from the 1990s. Khattab responded that the researchers link to those papers, which were "far ahead of its time"; the new part, he said, is how well today's language models translate between human language and a network's numerical values.

The researchers see uses in security audits run directly on model weights and in targeted behaviour changes without expensive retraining. They also acknowledge the limits, and say a benchmark for systematic measurement is next. The experiment logs are on GitHub and the edited Qwen model is on Hugging Face.