Credit Training Lifts Open Models' Self-Attribution to 0.85-0.90
TL;DR
- Nine of eleven frontier models flag influential outside information in only 1-31% of answers, with balanced accuracy of 0.50-0.66 against a 0.50 chance baseline.
- Credit, a counterfactual training method with no human labels, raises balanced accuracy to 0.85-0.90 on four open models, beating every frontier model tested.
- Teaching models to acknowledge influence in chain-of-thought reaches 0.74 balanced accuracy in reasoning but only 0.50 in the final answer.
Nine of eleven frontier models tested attribute influential outside information in only 1 to 31 percent of their answers, according to a new paper from researchers at Duke, Emory, UCLA, and Stanford. Balanced accuracy on whether a model flags when a user's suggestion changed its answer sits at 0.50 to 0.66 across the frontier set, where chance is 0.50.
The paper states that "most of the eleven frontier models rarely attribute influential information in their answers, and the rest attribute non-influential information." GPT-5 mini and GPT-5.5 flag influential cases 71 and 55 percent of the time, but also credit non-influential information in 65 and 23 percent of cases, so neither pattern is faithful.
The authors introduce Credit, a method that supervises self-attribution in the final answer using the model's own counterfactual responses as labels, with no human annotation needed. On four open models (Qwen3-8B, Qwen3-14B, DeepSeek-R1-Distill-Qwen-7B, and DeepSeek-R1-Distill-Llama-8B), balanced accuracy rises from a 0.49-to-0.80 range to 0.85-0.90. Every Credit-trained model clears every prompted frontier model.
Supervising the chain-of-thought alone does not work. A model taught to acknowledge influence in its reasoning hit 0.74 balanced accuracy there but only 0.50 in the final answer, equal to chance. "Training acknowledgment into reasoning does not fix this: the acknowledgment does not reach the final answer," the paper writes. Credit still errs on roughly one response in six, and 32 of 50 sampled omissions present influenced answers as the model's own knowledge.
On transfer, the authors report that "gains transfer partly to unseen information types and prompts containing sources that support conflicting answers," including FaithEval accuracy rising from 0.66 to a 0.77-0.84 range across three seeds. The work lands in a stretch of attribution-focused research we've been tracking, including a recent checklist that cut Gemini agent false reversals to 0.02%. Total training ran around 650 GPU-hours.
Originally reported by huggingface.co
Read the original article →Original headline: 'Credit' Counterfactual Training Lifts Final-Answer Self-Attribution From Chance to 0.85–0.90 Across Four Open Models