Preprint maps LRM training on an L0–L4 human-supervision ladder

A five-level ladder from L0 to L4 that tries to name which parts of a large reasoning model's training loop still depend on a human is the framing of a new preprint on arxiv titled 'Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence.'

The authors split the problem into two connected axes. 'The reward axis traces the development from per-instance human judgments to reusable verifiers and rewards that operate even without human feedback,' the abstract states. 'The experience axis examines how learning can progress from human-curated tasks and environments toward self-generated curricula, constructed environments, and autonomous co-evolution.' Reinforcement learning with verifiable rewards, the paper notes, has already worked in mathematics and code 'where outcomes can be checked automatically'; extending the approach to open-ended and agentic tasks is where 'direct human supervision cannot keep pace with the scale and complexity of model-generated experience.'

The paper also names the failure modes that come with pulling humans out of the loop: reward hacking, feedback drift, curriculum collapse, and environment errors. It proposes assessing any such system along three dimensions: policy capability, feedback fidelity, and experience quality.

The abstract names the ladder but does not publish what each rung actually contains.