Nesformer compiles an NES CPU into Qwen3.5 model weights

Nesformer, a project published on GitHub, encodes a working NES console inside the weights of a stock Qwen3.5 model and plays Super Mario Bros. by predicting the CPU's execution log one token at a time. The NES 6502 CPU, 2 KB of work RAM and a cartridge's program ROM all live inside a standard `Qwen3_5ForCausalLM` checkpoint. "Nobody trained these weights; a compiler computed every one," the README states.

The whole thing hinges on one sentence: "the model's Gated DeltaNet state is the machine's memory." Each DeltaNet head keeps a 128 × 128 state matrix that the delta rule updates at every token; with a one-hot key, an update overwrites exactly one row, so every row becomes a memory cell for the cartridge, RAM and registers. The checkpoint stacks 7 Gated DeltaNet layers plus one all-zero attention layer the Hugging Face cache requires. Its total size is 466.6M parameters in bf16, but only 9.3M of them are nonzero, so the compressed weights zip to 3.3 MB.

The README reports several checks against a conventional 6502 emulator. In a 600-frame run of the test ROM Thwaite, the model emitted 1,916,527 tokens over 10.0 seconds of NES time, and "every frame also matches a conventional 6502 emulator's output exactly." A 420-frame run of Doug Fraker's Full Game produced 2,645,885 tokens with the same identical-frame result. The nestest conformance run ended in "the expected HALT" after 4,586 tokens.

It is also slow. The fastest engine, a numba-compiled forward pass that skips the all-zero blocks, "ran at about 1,650 tokens/s on one CPU core," while the stock transformers path manages about 10. "A real NES would need about 700,000 tokens per second," the project notes, putting nesformer somewhere between 150 and 290 times slower than the console it emulates. Only NROM cartridges (iNES mapper 0) are supported, so Super Mario Bros., Donkey Kong and Excitebike work; anything that bank-switches does not.

Nesformer builds on Qwenputer by Divyansh Agrawal, which earlier compiled CHIP-8 and then a RISC-V core into Qwen3.5-4B. The checkpoint itself is a plain Hugging Face folder: `Qwen3_5ForCausalLM.from_pretrained` loads it with no custom code.