Leshem (Legend) Choshen @EMNLP

Huge thanks to my collaborators @RheaSukthanker, @CameronPashmina, and @Emy_Aze. Paper: https://arxiv.org/abs/2608.28444

Sliding-window beats linear attention arxiv.org

https://arxiv.org/abs/2608.17981 Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training? — via @rosinality https://x.com/rosinality/status/2089988698246942874

Full-bandwidth transformer arxiv.org

Many other details and experiments are in the paper: https://arxiv.org/pdf/2608.09703 We released our checkpoints here: https://huggingface.co/nthngdy/matryoshka-3B Many thanks to my co-author and advisor @yoavartzi for the precious guidance, and to @NVIDIAAI and to Alps for t…

arxiv.org

Many other details and experiments are in the paper: https://arxiv.org/pdf/2608.09703 We released our checkpoints here: https://huggingface.co/nthngdy/matryoshka-3B Many thanks to my co-author and advisor @yoavartzi for the precious guidance, and to @NVIDIAAI and to Alps for t…

nthngdy/matryoshka-3B · Hugging Face huggingface.co

https://arxiv.org/abs/2608.17981 Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training? — via @rosinality https://x.com/rosinality/status/2089988698246942874

Recirculation arxiv.org

Effective language identification based on a tokenizer UnigramLM tokenizer already gives probabilities, testing those to identify a language is fast and effective. Whiceh leads me to wonder, can we identify language during training and affect behavior? arxiv.org/abs/2602.17655…

What Language is This? Ask Your Tokenizer arxiv.org

arXiv: https://arxiv.org/abs/2608.25832 alphaXiv: https://alphaxiv.org/abs/2608.25832 HF Paper: https://huggingface.co/papers/2608.25832 Code: https://github.com/TextArena/TextArena

Skill Issue: Are Skills Language-Invariant in LLMs? arxiv.org

Paper : arxiv.org/abs/2607.18235 Github : github.com/akshat57/har...

Automated Discovery Has No Universally Superior Harness arxiv.org

Do you think Anthropic's PR were worried they miss out the positive hype of openAI breaking laws and attacking harmfully two companies. So they ran to look for places to say they also do it? (and pretend remorseful) "Everything you can do I can do" better? apnews.com/article/a…

Anthropic says its AI models hacked 3 organizations during testing apnews.com

arxiv.org/abs/2408.10646 arxiv.org/abs/2502.21228 And so much more (this nuance is somehow not being recognized and following the literature is so hard this days this well established branch on lack of sharing is hardly discussed outside the branch)

Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs arxiv.org

Will the same model become a different player depending on the interface language? We report our findings in our paper "Skill Issue: Are Skills Language-Invariant in LLMs?". And with it, TextArena is now multilingual 🌍 with 65 games in 193 languages. 🧵

Sota on 89 languages speech models. There's plenty of speech data it appears, so the simplest fine-tuning plus tokenization on public data just improves everything substantially. And it's not even with any tricks or all the data... #conll #acl @catherinearnett.bsky.social @alexwarstadt.bsky.social

Livetweet @mcxfrank.bsky.social talk: If we care about intelligence and cognition, AI has recently allowed a change; we now have two ways to study them. AI is of course allowing us a lot that we wouldn't dare on our Children (brain surgery, never tell a child about cats...) #acl #conll

At last, a way to unlock mulitlingual knowledge sharing in LLMs! 🌍 ​By pretraining an English/Arabic model and swapping Arabic for a word-wise, 1-to-1 translation mapping to English, we saw a massive boost in cross-lingual knowledge transfer 🚀

Counterfactual futures are the holy Grail of, well of a lot, right? It's predicting the future plus making A/B tests. Humanities come with this question now. Can we now use AI to reimagine history and see how it would unfold differently?

AI language acquisition. Geniuses debated how humans learn language, what their stimulus is, and what kind of rules do they apply. Why is the way AI learns language, less legitimate field of study? I want the theory of AI language acquisition

“Why Pretraining Fails to Share Cross-Lingual Knowledge” LLM's knowledge learned in one language Sometimes barely transfers to another. 🔁 x-post from @askalphaxiv

If you reimagined how AI science should be done in Academia, what would you change? I keep posing this question to people I meet lately (and I am going to act on the answers), and thought, there are so many people I don't physically meet...