JHU/NYU 'Accurate but Not Humble' Shows High-Scoring LLM Agents Hide Unresolved Conflicts

Johns Hopkins and NYU researchers introduce an Identify-Solve-Escalate framework for evaluating 'epistemic humility' in LLM agents when retrieved evidence contradicts parametric knowledge. Across four agents in controlled and natural knowledge-conflict settings, higher task accuracy did not translate to humility: agents often detect conflicts early then fail to maintain or communicate uncertainty in incorrect final answers, with model-level interventions improving EH but trading off accuracy.