Paper flags a 'curse of ambiguity' in language models

A new arXiv paper by Nicolas Zucchet, Hyun Dong Lee and Scott Linderman argues that language models — and, more broadly, any neural network producing discrete probability distributions — suffer from what the authors call a 'curse of ambiguity': the more ambiguous a next-token distribution is, the harder it is for the model to learn it accurately.

The tradeoffs are architectural. 'More ambiguous distributions require more capacity to be stored, larger embeddings to be represented, more steps to be fitted, and amplify token-sampling noise,' the paper reports. The authors trace the effect through a theoretical analysis, then validate it on synthetic tasks with controlled ground truth before saying they 'observe the same signatures in language models trained on real data.'

The framing matters because, as the abstract puts it, 'large language models increasingly rely on sampling as a driver of their own improvement.' If the sampled distribution is systematically less faithful in exactly the ambiguous regions, self-improvement loops inherit that bias. The paper positions its result as offering 'a practical framework for when to trust their output distribution.' The abstract publishes no per-model numbers and does not name which real-data models carry the signatures. Two experts we track have shared the paper.