Probes Reveal LLM Forecasters Pre-Commit Before Reasoning

The uncomfortable finding in a new arxiv paper from Raphaël Sarfati and colleagues is that when a language model fine-tuned to forecast future events writes out its chain of thought, the reasoning is often narration rather than deliberation. Probes trained on intermediate activations recover both the answer and the confidence level from a single pass before the reasoning has run, and route questions well enough to conserve 30 to 47 percent of generated tokens with no loss of accuracy.

That matters because a lot of the industry has been treating visible reasoning traces as the honest signal. Regulators want them, evaluators read them, and enterprise buyers point to them as the reason to trust a probabilistic answer over a black box. The authors tested three models, Eternis-Forecaster 8B alongside GLM-4.7-Flash and GLM-4.5-Air, and found that probes on the models' internal states were substantially better calibrated than the models' own generated predictions. In an evidence-ablation experiment, removing influential sources often changed the forecast while, in the paper's phrasing, the reasoning trace remained untouched. That is a fairly direct way of saying the trace was not doing the work the trace claimed to be doing.

There is also a lie-detection angle worth pulling out. Internal activation patterns predicted the direction of the forecast change in 84 percent of cases, far better than what you could read off the stated reasoning. If that holds, activation probing becomes a concrete auditing tool rather than a research curiosity, and the cost-saving story, routing easier questions on the pre-commitment distribution, is a very practical secondary benefit for anyone paying for reasoning tokens at scale.

The honest caveats are the ones the paper's setup itself implies. The result is on forecaster-tuned open models, not the frontier closed systems most teams actually deploy, and probing requires white-box access to intermediate activations that API customers simply do not have. What the reporting does not give you is whether the same pre-commitment pattern shows up on math, code, or multi-step agentic reasoning, or how large the calibration gain looks in absolute rather than relative terms.

Even with those caveats, the direction is the part worth watching. If the model already knows the answer before it starts talking, the interesting question is what the reasoning trace is actually for, and which parts of the current auditing stack were only ever reading the narration.