arXiv paper prunes inputs to test LLM explanation faithfulness
TL;DR
- The method removes input concepts an LLM did not cite in its own explanation, then re-queries the model on the reduced input.
- Authors split unfaithful explanations into incompleteness (omitted influences) and unsoundness (cited non-influences); the technique targets the first.
- Reported to improve faithfulness across two datasets, multiple model families, and two independent metrics, with no changes to model weights.
To catch a language model whose explanation does not match its actual reasoning, strip the input concepts the model did not credit and query it again on the leaner prompt. That is the test-time proposal from Qinglan Luo, John Guttag, Katie Matton and colleagues in a paper posted to arXiv on September 3.
The authors carve the failure mode in two. Incompleteness is when an explanation leaves out factors that actually swung the model's answer; unsoundness is when it cites factors that did not. Their method targets the first. "We remove from the input the concepts not credited in the model's explanation and re-query the model on the reduced input," the paper says, so it "eliminates unmentioned influences while preserving the influence of mentioned concepts."
Improvements are reported "across two datasets, multiple model families, and two independent faithfulness metrics" relative to standard prompting and to prompts that ask the model to be more faithful, with a technique the authors describe as "model-agnostic and can be applied at inference time without modifying model parameters." The abstract does not name the datasets, the model families or the metrics, and it publishes no headline numbers. Two researchers in our Who's Who tracker shared the link within days of it appearing.
Shared on Bluesky by 2 AI experts
Originally reported by arxiv.org
Read the original article →Original headline: A Removal Based Approach to Improve LLM Faithfulness at Test-Time