This work builds on LCKV (final-layer caches + KV matching), CLA (cross-layer reuse), and Huginn (recurrent depth), but we test cache-history mismatch and repair in a tied core using LSTM cells aclanthology.org/2024.acl-lo... arxiv.org/abs/2405.12981 arxiv.org/abs/2502.05171
AI Weekly's analysis
→
- A 3.5-billion-parameter model iterates a recurrent block at inference to reach reasoning gains equivalent to a 50-billion-parameter compute load.
- The approach reasons in latent space rather than writing out chain-of-thought tokens, and needs no specialized reasoning training data.
- The proof-of-concept was trained on 800 billion tokens; weights are on Hugging Face and code and data recipes are on GitHub.
Read full analysis →
