TinyWorldformer runs world model and actor on Pi Pico 2 W

The whole thing fits on a Raspberry Pi Pico 2 W with 512 KiB of SRAM. SheepSeb's TinyWorldformer pairs a looped transformer that predicts the next game frame with an MLP actor that picks the next move, and runs both at once on the microcontroller.

"Using 2 small neural networks to generate and play the game at the same time on a Raspberry Pi Pico 2 W," the README states. The author adds: "I got interested in looped transformers, where one transformer block is applied several times with the same weights instead of stacking different layers."

Five games ship: Breakout, Doom-lite, Flappy Bird, Pong, and Pac-Man lite. World model sizes run from 26k parameters for Breakout up to 131k for Pac-Man; the actor sits between 9k and 16k. Compute per frame ranges from 2.0M to 8.2M multiply-adds. Flash footprint stays between 361 KB and roughly 600 KB, SRAM between 227 KB and 460 KB. Weights are int8 with one scale per output row, trained with QAT and a straight-through estimator; softmax runs on a 1024-entry exp table producing uint8 probabilities. The actor is trained with Stream AC(λ), without a replay buffer or target network.

At two loop iterations, exact-frame match hits 100% on Flappy Bird, 99.99% on Breakout, 99.45% on Pong, 86.4% on Pac-Man lite, and 83.6% on Doom-lite. The developer flags why Doom lags: "Doom-lite's 3D view has 768 cells and a single shaded wall pixel off counts the whole frame as wrong, so its number looks worse than it plays; the game state is almost always right."

It sits alongside a run of int8-on-small-silicon work we've been tracking in edge AI this week, including the related Doomformer project that compiles integer neural networks into playable DOOM maps.