Open 27B Model Rewrites Its Own Weights for 6.6-Point Gain
TL;DR
- An open 27B language model rewrote its own weights using only forward passes and became 6.6 points more accurate on problems it never saw, with no gradients and no training.
- Across eight consecutive self-edit sessions the model tested each edit on practice problems and kept only what worked, and accuracy rose each time.
- The ability emerged only at scale: in the same model family, smaller models either made no edit or broke themselves, and only the largest improved and repaired a damaged copy.
An open 27B language model rewrote its own weights without any gradient updates and became 6.6 points more accurate on problems it had never seen. That is the central finding of a new arXiv preprint led by Jiajun Fan and colleagues, titled "Language Models Can Rewrite Their Own Weights" and revised on November 18.
The authors describe what they call "self-rewriting": the model is handed its own weights, its own graded answers, and the ability to run itself, and asked to improve. "The model is its own optimizer," they write. "Its own edit makes it 6.6 points more accurate on problems it never saw." The gains compound across repeated attempts because the model tests each edit on practice problems and submits only what works, and "over eight consecutive sessions its accuracy only rises."
A second experiment stress-tests the claim. The model is given a broken copy of itself that "solves almost nothing" and asked to fix it without retraining and without ever reading the original weights. The paper reports the model brings the copy back to "over 80% of the original accuracy on image tasks it never saw."
Scale matters. "The smaller models make no edit or break themselves, and only the largest improves and repairs itself," the paper says of the one model family it tested. It does not name that family, does not name the benchmarks behind the 6.6-point figure, and does not publish the base accuracy the gain is measured from.
The authors frame the result as a conceptual move, not a product: "The wall between training and inference is a limit of our tools, not a property of intelligence."
Originally reported by paper
Read the original article →Original headline: Open 27B Model Rewrites Its Own Weights, Gains 6.6pp Accuracy — No Gradients