Mind Lab's LongStraw runs RL post-training beyond 2M tokens

A twenty-author paper out of Mind Lab, posted to Hugging Face on July 16, is worth reading past the leaderboard-style numbers, because the interesting claim is about GPU budgets, not model quality. The group calls their system LongStraw, and the pitch is that reinforcement-learning post-training has been stuck near 256K tokens even as inference contexts have marched toward a million and beyond. LongStraw's job is to close that gap without asking anyone to buy more GPUs.

The mechanics are memory bookkeeping. Under Group Relative Policy Optimization, the shared prompt gets evaluated once without autograd, only the model-specific state later tokens actually need is retained, and short response branches are replayed one at a time instead of held simultaneously. The result the paper reports is grouped scoring plus response backward at 2.1 million token positions on 8 H20 GPUs using Qwen3.6-27B, with a stress test pushed to 4.46 million positions, and increasing the group size costs about 0.21 GB of peak allocated memory. A 32-H20 run validates a 2.1M-token prompt across all 78 layers of GLM-5.2. Code is at github.com/MindLab-Research/longstraw.

The reason this matters for anyone running a post-training team is straightforward. If grouped RL rollouts can be scheduled at inference-scale context length on the boxes you already own, million-token RL fine-tuning stops being a hyperscaler-only sport. That is the kind of shift that changes which labs get to try the technique at all.

The honest caveat is one the authors themselves flag. The paper 'establishes execution capacity rather than complete training correctness', which is a careful way of saying the runs fit in memory and finish, not that they demonstrably produce a better policy than a shorter-context baseline would. What the write-up does not give you is wall-clock throughput at these scales, whether the sequential-replay trick degrades gradient quality relative to a full-group backward, or whether the technique carries over to dense transformer architectures rather than the hybrid-attention and mixture-of-experts models tested here. Independent replication has not landed yet.

The forward look is straightforward. If the open-sourced code holds up under third-party runs on Qwen and GLM checkpoints, the long-context RL playbook gets cheaper for everyone downstream, and the complaint that RL context is a generation behind inference context stops being the default excuse.