Jerry Spanakis 🟥

If an LLM agent notices conflicts, will it tell you? Our #EMNLP2026 Findings paper finds that higher accuracy doesn't always mean greater epistemic humility: agents can spot conflicts early yet leave uncertainty out of wrong answers. 📄 arxiv.org/abs/2610.12360 🧵👇 #NLProc #LLM

Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict arxiv.org

AI Weekly's analysis →

• A new paper proposes 'epistemic humility' as an evaluation axis separate from task-success benchmarks for LLM agents.
• Across four agents tested, higher task accuracy did not correspond to greater willingness to flag uncertainty in final answers.
• Interventions that improved epistemic humility often reduced task accuracy, the authors report.

Read full analysis →

View on Bluesky →

If an LLM agent notices conflicts, will it tell you? Our #EMNLP2026 Findings paper finds that higher accuracy doesn't always mean greater epistemic humility: agents can spot conflicts early yet leave uncertainty out of wrong answers. 📄 arxiv.org/abs/2610.12360 🧵👇 #NLProc #LLM

Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict arxiv.org

AI Weekly's analysis →

• A new EMNLP 2026 paper argues LLM agents can be accurate without being epistemically humble when retrieved evidence contradicts their prior knowledge.
• The authors propose a three-part evaluation — Identify, Solve, Escalate (ISE) — applied to four agents across controlled and naturally occurring knowledge conflicts.
• Agents often flag a conflict in early execution steps but fail to maintain or resolve it later, and humility interventions tend to cost accuracy.

Read full analysis →

View on Bluesky →

📊 How do people in NLP build their careers, and what shapes those decisions? We’re running a survey on career paths in NLP and we'd love to hear from you! The survey is anonymous and takes only 10-5 minutes: 👉 acpaths.ims.uni-stuttgart.de With @esradonmez.bsky.social and @morl…

NLP Career Paths and Challenges — Survey acpaths.ims.uni-stuttgart.de View on Bluesky →

🏆 Join the ChildSafeAds shared task on detecting and classifying commercial content in child-facing YouTube videos, @nllpworkshop.bsky.social @emnlpmeeting.bsky.social Expression of interest via this form: forms.office.com/e/q0JSkSuGzD Expand for more info > (1/5)

Microsoft Forms forms.office.com View on Bluesky →