Naomi Saphra

Our ICML HiLD workshop paper shows that the reason why bigger models learn more complex tasks is because they are able to saturate the gradients for easier tasks; different tasks are competing for the same parameters and gradient mass.

Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention arxiv.org

Our new paper sets the stage for the biggest practical use case of model interpretability: stress testing and dataset development. All you need is interpretable linear features and simple geometry.

Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry arxiv.org

I'm so happy when other people write papers on nondeterministic factors in training. embrace the chaos

Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns arxiv.org

If you're at COLM next week, check out our interpretability work, led by Isabella Gidi, on how grammatical inflection transfers across languages! Poster on Tuesday afternoon.

Shared Circuits for Shared Grammar: Tracing Subject-Verb Agreement Across Languages arxiv.org

We don’t always know what problems are hard for LLMs. So devs evaluate on tasks HUMANS find hard or on broad benchmarks. What if we could instead anticipate which scenarios a model will fail on—all without evaluating specific input examples? 🧵NEW PAPER by @jenniferlumeng.bsky.social

if you think students will be routinely collaborating with LLMs after graduation, #1 priority is to study rhetoric because you need to recognize BS and critique the rigor of an argument that *looks* good. the elite users are going to be like, philosophy, classics, and english majors.

if you are a PhD student in AI, remember it is in your interests to distract your advisor from how much money they could be making in industry. should be a daily priority.

Yes, some people are worse at writing than an LLM. Those people are also usually incoherent thinkers. If they learned to write better, they would get better at thinking.

Are students embarrassed by AI cheating? Like, there has always been rare but unstigmatized dishonesty (memorizing a frat’s archive of finals questions) and common but stigmatized (saying you’ve started when you definitely have not started it). AI cheating should be the most stigmatized. Is it?

ok the thing about erdos is he obviously loved collaborating with humans. he could have done a lot on his own, but math was how he chose to connect. I'm not sure he would have been very into chatbots?

All scientific field distinctions are ultimately socially defined categories. So the Transformers paper was an “NLP paper” not because it was about language modeling and MT, but because Vaswani went on to do grammatical inference algorithms for mRNA. That is real “NLP researcher“ behavior.

ACL needs to adopt the expectation from ML conferences that workshops be exciting. If you come from NLP you know ACL workshops are mostly terminal venues for abandoned/unambitious work, but it's clear that the correct approach is to host cutting-edge WiP.

How do you make LLMs actually good at explaining new math? It's like reading a badly written reference for people who already know the subject. When I ask a question, it never matches my level. If I try to rephrase to test my understanding, it just sycophantically agrees.

my new literary award cannot be won by a commercial frontier LLM because I will require that 10% of each submission is smut