Kaiser Sun

Why they matter

Directory member with public evidence across NLP & language.

AI signals
4
past 30d
Sources
3
distinct domains
Discussões
0
past 30d
Latest signal
21d ago
View every signal from Kaiser Sun →
Ph.D. student at @jhuclsp, human LM that hallucinates. Formerly @MetaAI, @uwnlp, and @AWS they/them🏳️‍🌈 #NLProc #NLP Crossposting on X.

Articles & links

If an LLM agent notices conflicts, will it tell you? Our #EMNLP2026 Findings paper finds that higher accuracy doesn't always mean greater epistemic humility: agents can spot conflicts early yet leave uncertainty out of wrong answers. 📄 arxiv.org/abs/2610.12360 🧵👇 #NLProc #LLM

Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict arxiv.org
AI Weekly's analysis →
  • A new paper proposes 'epistemic humility' as an evaluation axis separate from task-success benchmarks for LLM agents.
  • Across four agents tested, higher task accuracy did not correspond to greater willingness to flag uncertainty in final answers.
  • Interventions that improved epistemic humility often reduced task accuracy, the authors report.
Read full analysis →
View on Bluesky · ♥ 7 ↻ 2 ↩ 1 · 2 from the directory shared this · 36d ago

If an LLM agent notices conflicts, will it tell you? Our #EMNLP2026 Findings paper finds that higher accuracy doesn't always mean greater epistemic humility: agents can spot conflicts early yet leave uncertainty out of wrong answers. 📄 arxiv.org/abs/2610.12360 🧵👇 #NLProc #LLM

Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict arxiv.org
AI Weekly's analysis →
  • A new EMNLP 2026 paper argues LLM agents can be accurate without being epistemically humble when retrieved evidence contradicts their prior knowledge.
  • The authors propose a three-part evaluation — Identify, Solve, Escalate (ISE) — applied to four agents across controlled and naturally occurring knowledge conflicts.
  • Agents often flag a conflict in early execution steps but fail to maintain or resolve it later, and humility interventions tend to cost accuracy.
Read full analysis →
View on Bluesky · ♥ 7 ↻ 2 ↩ 1 · 2 from the directory shared this · 36d ago

Effort should be chosen per task, and preferably, per agentic step! Paper: arxiv.org/abs/2610.34491 HuggingFace: huggingface.co/datasets/Ka... Code: github.com/KaiserWhoLe...

What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents arxiv.org
AI Weekly's analysis →
  • On tasks every effort tier solves, the max setting costs up to 42x more than the lowest, with no accuracy guarantee.
  • The paper reframes reasoning effort as a credence good: buyers see the outcome but cannot judge whether the extra compute paid off.
  • A cheap-first, escalate-on-verified-failure policy matched or beat fixed top effort across every tested model and benchmark.
Read full analysis →
View on Bluesky · ♥ 0 ↻ 0 ↩ 1 · 21d ago

Recent commentary

Your agent solved the task at xhigh. Would low have solved it too? You can’t tell without rerunning it, and you still pay for every token. We tested models x reasoning effort level x benchmarks. On tasks low effort can already solve, top effort costs up to 42× more #LLM #AI 🧵👇

View on Bluesky · ♥ 0 ↻ 0 ↩ 1 · 21d ago

In Kaiser Sun's orbit

Center = Kaiser Sun. Left = members they follow (green edges). Right = members who follow them (blue edges). Top = mutual follows (orange edges, slightly larger). Drag any node to reposition; click to open that profile.

Are you Kaiser Sun? Show it.

Add the Who’s Who of AI badge to your site or bio. It links back to this profile.

Listed in AI Weekly's Who's Who of AI

Markdown: [![Listed in AI Weekly's Who's Who of AI](https://aiweekly.co/modules/custom/aiweekly_whoswho/images/whoswho-badge.svg)](https://aiweekly.co/whos-who/person/kaiserwholearns-bsky-social)