Nokia Bell Labs Paper Finds Constrained Decoding's Reasoning Hit Comes From Prompting, Not Masking

Ngoc Trinh Hung Nguyen (Télécom Paris) and Armen Aghasaryan (Nokia Bell Labs) separate constrained decoding into format, validity, and reasoning constraints across four model families and 11 open-source LLMs. Causal ablations argue the long-held 'decoding masks hurt reasoning' story is wrong: apparent drops come from non-equivalent prompting and output formats, not decoder-level masking. Validity constraints improve reliability by eliminating invalid answers, and reasoning constraints lift accuracy when they enforce beneficial trajectories.