Study: Max LLM reasoning tier costs up to 42x on easy tasks

On easy tasks that every reasoning effort level solves, the paper finds that the highest setting can cost up to 42 times more than the lowest.

That number sits inside a new arxiv preprint, "What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents," by Kaiser Sun, Yongchan Kwon, Zain Hasan, Shang Zhu, Kaitlyn Zhou and Federico Bianchi. The core move is a reframing: "Reasoning effort is therefore a credence good: buyers observe the outcome but cannot assess the marginal value of the computation they paid for," the authors argue. The user picks before knowing whether more thinking helps, and a successful run still never tells them whether the premium was necessary.

The authors measured counterfactual effects on proprietary and open-weight models across four benchmarks, spanning "single-call reasoning and long-horizon agentic tasks". Higher effort raised accuracy in aggregate but "frequently fails on tasks that lower effort solved". The behavior change at more effort is specific: agents "verify their own work more heavily and more often search for solutions than work through problems directly, both of which lengthen trajectories without necessarily altering the outcome".

The paper's recommended fix is cheap and mechanical: "starting at the lowest effort and escalating only after a verified failure" matches or improves on fixed top effort across every model and benchmark they tested, "at lower cost for most proprietary models". The hedge "most" is the one the abstract itself keeps.

The abstract publishes no per-model, per-benchmark numbers. The 42x is a worst-case ratio on easy tasks; the model names stay inside the paper proper.