ICML 2026 paper argues LLMs must optimize for self-consistency

A new position paper accepted to ICML 2026 puts a name on something that has been quietly circling the alignment discussion for a while. The claim, from Itamar Pres, Belinda Z. Li, Jacob Andreas and six co-authors, is that the way language models are currently trained and evaluated, one input, one output, judged in isolation, is why so many familiar failure modes keep resurfacing. The paper proposes self-consistency as the missing objective.

The authors' list of persistent failures reads familiar: models overcondition on user framing, what they call sycophancy; they exhibit incomplete logical generalization; and they produce confident but incorrect responses. Their point is that none of these are visible if you only look at one response to one prompt. You have to compare a model's answers across related inputs to see the crack.

The other half of the argument is a synthesis move. Techniques already in the literature, with adversarial robustness and factual coherence named as examples, can be understood as special cases of a common consistency optimization procedure, addressable with a standard set of optimization tools. That reframing is the paper's real bid, not a new benchmark or a new algorithm, but a lens under which several corners of LM research collapse into one problem.

The honest caveat is that this is a position paper, not an empirical result. There are no benchmark numbers, no head-to-head against a baseline. What the paper does not give you is a recipe for how expensive consistency optimization is at scale, or how it trades against helpfulness when a user's framing is the very signal a chat model is supposed to follow.

Where it might matter is downstream. Evaluation teams, red-teamers and post-training groups now have a shared vocabulary for arguing that a sycophantic model and an inconsistent reasoner are failing at the same underlying thing.