Recurrent-depth LM hits 50B-equivalent compute at 3.5B params

A 3.5-billion-parameter language model can improve on reasoning benchmarks up to a computation load equivalent to a 50-billion-parameter model, its authors report, by iterating a recurrent block at inference time instead of writing out longer chains of thought.

The paper on arXiv, posted in February by Jonas Geiping, Sean McLeish, Tom Goldstein and colleagues, describes what they call a recurrent-depth approach to test-time compute. 'Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time,' the abstract states. 'This stands in contrast to mainstream reasoning models that scale up compute by producing more tokens.'

The pitch against chain-of-thought is direct: the method 'does not require any specialized training data, can work with small context windows, and can capture types of reasoning that are not easily represented in words.' The proof-of-concept was trained on 800 billion tokens, and the model plus code and data recipes are posted publicly on Hugging Face and GitHub. Two researchers we track had already circulated the link.

The abstract advertises gains 'sometimes dramatically' on reasoning benchmarks but names none of them, quotes no per-benchmark scores, and gives no wall-clock or memory figure for how the deeper unroll compares to just decoding more tokens.