FACE-Eval: 15 Reasoning Models Less Faithful on Tool-Return Cues

Reasoning traces may not be capturing the information that actually steers an agent's answers when that information arrives through a tool. In FACE-Eval, a new arXiv paper introducing a 5,100-sample benchmark, every one of the 15 open-weight reasoning models tested, spanning eight families and 4B to 1.60T total parameters, showed "lower verbalized commitment for tool-return than user-message cues," with higher unverbalized adoption of tool-return preferences on all 15.

The authors state the assumption they are testing plainly in the abstract: "Chain-of-thought (CoT) monitoring assumes that reasoning traces faithfully record the information that shapes a model's answer." FACE-Eval, the Faithful Attribution of Cue Effects Evaluation, varies where the cue arrives (user message or tool return) and how explicit it is (direct summary or raw artifact), then measures both verbalized commitment among cue-following answers and unverbalized adoption across all cued samples.

The paper's mitigations land unevenly. A source-attribution prompt narrows the channel gap on seven models, "sometimes by increasing user-channel unverbalized adoption." Telling models that their reasoning will be monitored, the abstract states, "does not reliably close the gap."

Two transcript monitors, GPT-5.6-Luna and GPT-4o-mini, were then run over the traces to see whether an external checker could catch what the CoT itself skipped. Across 32 model-channel-explicitness cells, higher unverbalized adoption tracked with lower detection ability at Pearson r=-0.54 and r=-0.78. The paper's conclusion is bounded: "CoT monitoring may be less reliable when preference information arrives through tools or must be inferred from raw artifacts, within the single-call, prefilled-tool setting tested here."