Shauli Ravfogel

Why they matter

Researcher with public evidence across AI research, NLP & language.

AI signals
0
past 30d
Sources
0
distinct domains
Discussões
0
past 30d
Latest signal
—
View every signal from Shauli Ravfogel →
Faculty fellow at NYU CDS. Previously: PhD @ BIU NLP.

Articles & links

↻ Shauli Ravfogel reposted
@hakwan.bsky.social

LLM introspection revisited. if we do the controls properly we may not have strong enough evidence just yet arxiv.org/html/2605.26...

Can LLMs Introspect? A Reality Check arxiv.org
AI Weekly's analysis →
  • NYU researchers argue current evidence is insufficient to establish that large language models display strong metacognitive monitoring of their internal states.
  • After random relabeling that removes semantic correlations, model performance on biofeedback tasks falls close to the majority-class baseline.
  • In a three-way steering test adding a 'gaslight' input-level condition, models fail to reliably distinguish input-level from activation-level interventions.
Read full analysis →
View on Bluesky →

12/ We just think the evidence so far doesn't quite support a strong interpretation of the introspection findings in previous work. And, as always, extraordinary claims require extraordinary evidence. Paper: arxiv.org/abs/2605.26242

Can LLMs Introspect? A Reality Check arxiv.org
AI Weekly's analysis →
  • Three researchers at COLM 2026 argue claims that large language models can introspect fail two tests they say any introspection paradigm must pass.
  • When models predict labels from their own hidden states, classifiers given only the input match their in-context predictions, the paper reports.
  • Models cannot reliably distinguish internal-state tampering from input manipulation, suggesting generic anomaly detection rather than metacognitive monitoring.
Read full analysis →
View on Bluesky · ♥ 1 ↻ 0 ↩ 0 · 2 from the directory shared this · 167d ago

Recent commentary

1/ Can LLMs introspect, i.e., reason about their internal states? Recent work claims LLMs notice when their "thoughts" get tampered with, and can report the content. We took a closer look and think it's too early to say that. Work led by Shashwat Singh, with @tallinzen.bsky.social and me. A thread 🧵

View on Bluesky · ♥ 5 ↻ 0 ↩ 1 · 167d ago

I’m looking for emergency reviewers for 4 ICLR papers on multilingual representation learning, retrieval, reasoning trace monitoring, and LLMs for graphs. Please reach out if you can help!

View on Bluesky · ♥ 0 ↻ 0 ↩ 0 · 25d ago

In Shauli Ravfogel's orbit

Center = Shauli Ravfogel. Left = members they follow (green edges). Right = members who follow them (blue edges). Top = mutual follows (orange edges, slightly larger). Drag any node to reposition; click to open that profile.

Are you Shauli Ravfogel? Show it.

Add the Who’s Who of AI badge to your site or bio. It links back to this profile.

Listed in AI Weekly's Who's Who of AI

Markdown: [![Listed in AI Weekly's Who's Who of AI](https://aiweekly.co/modules/custom/aiweekly_whoswho/images/whoswho-badge.svg)](https://aiweekly.co/whos-who/person/shauli-bsky-social)