LongStraw runs GRPO post-training at 2.1M tokens on 8 GPUs

Modern AI systems have a lopsided quality most users never see. Inference can hold a million tokens of context, but reinforcement learning post-training on those same models has been stuck at roughly 256K tokens or below. A new paper from Changhai Zhou and collaborators, posted to arXiv, introduces LongStraw, an execution stack designed to close that gap.

The core claim is a systems one. LongStraw runs Group Relative Policy Optimization, or GRPO, at 2.1 million positions for groups of two and eight on eight H20 GPUs. Stress testing pushes it to 4.46 million positions. The authors also validate end-to-end execution at 2.1M tokens on a 32-GPU H20 configuration. The trick, in the paper's telling, is to evaluate shared prompts without computing gradients, keep only essential model-specific state, and replay response branches sequentially so the active training graph stays small.

The team tests on two frontier open models: Qwen3.6-27B, described as a hybrid recurrent and full-attention design, and GLM-5.2, a compressed-attention mixture-of-experts with 78 layers. Both are the kind of long-context models that inference stacks now serve, but that RL post-training has struggled to keep up with.

Why this matters if you are not a systems researcher: long-horizon agentic training is where a lot of frontier work is heading, and the training-side bottleneck has been one of the more expensive constraints. Pushing GRPO past the million-token mark on a modest H20 cluster puts long-context RL closer to reach for teams that do not own hyperscaler-tier fleets.

The honest caveat is that this is a systems paper, not a downstream benchmark study. It shows the machinery runs at these token lengths on these specific models; it does not show that the resulting agents are better at anything. The reporting also does not give you wall-clock training cost, comparisons to prior long-context RL approaches, or evidence that other teams have reproduced the numbers on non-H20 hardware. The upside, if the recipe holds up outside the paper, is that smaller labs and open-model teams get a plausible path to training long-horizon agents at contexts that until now needed the largest clusters.