Ant International's PAR Hits 16× Extrapolation on Swap-S5 Task

Researchers at Ant International and the National University of Singapore report that a retrofitted Qwen2.5-0.5B trained with their new Progress-Aligned Recurrence scheme holds 95.5% full-state accuracy on the swap-S_5 benchmark at 16× the training length, against 55.7% for the prior RLT recipe and 0.8% for explicit chain-of-thought fine-tuning. The paper, PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking, was posted October 20, 2026.

The setup is a looped language model. Following McLeish et al., the authors split the 24 layers of Qwen2.5-0.5B into a prelude (layers 1-8), a weight-tied core (layers 9-16), and a coda (layers 17-24), with the core applied repeatedly. The twist is in supervision. "At a supervised iteration k, the prediction is trained to match the state after the first k operations," the authors write, so on a rule-table chain iteration k predicts the value reached after k lookups. Each token's recurrent state is then rescaled to its first-iteration RMS norm to stop drift when the loop runs hundreds of times at test.

Depth extrapolation holds across scales. On larger Qwen2.5 backbones at 1.5B, 3B, and 7B, the paper reports that "the best PAR seed maintains 100% accuracy through d=256, whereas fixed-depth baselines remain near chance (≤26.5%)." A learned halting read-out, which fires when a completion token clears a "6 of 8" confirmation window, reaches 92.9% at 16× without being told the problem length. A separate variant called prefix self-distillation learns the depth-length correspondence from final answers alone and reaches 85% at d=256.

The authors flag where this does not help. Graph reachability "is solved by expanding a frontier in parallel rather than by a fixed sequence of steps, so it has no canonical intermediate state for iteration k to hold; all recipes stay below 3.5× on it." On Qwen3-4B at the 16× horizon, PAR reaches 53% mean accuracy with increased seed variance, well above fixed-depth collapse but short of the near-ceiling the 0.5B retrofit posts. State tracking is a busy corner of the research feed; we have logged 38 prior stories tagged research in the past 90 days on our research tracker.