Glenn Matlin, Chandreyi Chakraborty, Saehee Eom, Mika Okamoto, Rayan Castilla, Louis Jaburi, Alvin Deng, Taywon Min, Lucia Quirke, Stella Biderman, Mark Riedl Where Does Social Reasoning Come From? Capability Provenance in Language Models https://arxiv.org/abs/2606.19625
Where Does Social Reasoning Come From? Capability Provenance in Language Models arxiv.org
AI Weekly's analysis
→
- OLMo3-7B's social and STEM reasoning trace to 'qualitatively distinct corpus regions' of its Dolma3 pretraining mix, per gradient-based training-data attribution.
- The social-STEM split is sharper at the reasoning level (SocialIQA, ARC-Challenge) than at the knowledge level (MMLU Social Sciences, MMLU STEM).
- Unlearning high-attribution topics like Literature degraded SocialIQA more than within-topic random baselines, giving what the authors call 'partial causal validation.'
Read full analysis →
View on Bluesky ·
♥ 0
↻ 0
↩ 0
·
3 from the directory shared this ·
120d ago
DiffusionGemma Team, Adrien Ali Ta\"iga, James Assiene, Daniele Calandriello, Rahma Chaabouni, Jo\~ao Gante, Tamara von Glehn, Nate Keating, Chris Knutsen, Martin Kukla, Tianlin Liu, Ivan Lobov, Ofir Nabati, ... DiffusionGemma Technical Report https://arxiv.org/abs/2608.00146
DiffusionGemma Technical Report arxiv.org
AI Weekly's analysis
→
- DiffusionGemma reportedly generates around 1,500 output tokens per second on a single NVIDIA H100 by refining blocks of 256 tokens in parallel.
- The model is a fine-tune of the mixture-of-experts Gemma 4 base, which has 3.8B activated and 25.2B total parameters.
- Its two-stage training pipeline of supervised denoising plus RL with sampler distillation used less than 10% of the original AR model's token budget.
Read full analysis →
Adnan El Assadi, Niklas Muennighoff, Jinhyuk Lee The Embedder's Dilemma: LLMs Are Better, but at What Cost? https://arxiv.org/abs/2608.12875
The Embedder's Dilemma: LLMs Are Better, but at What Cost? arxiv.org
AI Weekly's analysis
→
Read full analysis →
Fanfei Li, Jana Zeller, Manuel Prada-Corral, Thadd\"aus Wiedemer, Prasanna Mayilvahanan, Ryan Cotterell, Wieland Brendel LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure https://arxiv.org/abs/2608.13545
LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure arxiv.org
AI Weekly's analysis
→
- LITTLECURRICULUM is an 88-billion-token pretraining corpus curated to U.S. elementary school material through Grade 5.
- A 5-billion-parameter model trained from scratch on that corpus, LITTLELEARNER, has knowledge boundaries mapped to interpretable curriculum guidelines.
- Post-training and in-context learning helped the model use existing knowledge but did not raise out-of-scope capabilities.
Read full analysis →
Jenna Russell, Ben Glickenhaus, Katherine Thai, John Wieting, Mohit Iyyer, Max Spero, Bradley Emi How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text https://arxiv.org/abs/2609.40295
How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text arxiv.org
AI Weekly's analysis
→
- Pangram labeled 27.5% of June 2026 web tokens as AI-generated, rising to 31.1% by August, after FineWeb quality filtering.
- For high-human-budget runs AI tokens raise loss almost immediately; data-starved runs see an early benefit that saturates and reverses.
- The authors' new scaling law predicts effects on models up to 3.6x larger with 41% lower error than existing laws across all AI ratios.
Read full analysis →
Mandana Samiei, Eunice Yiu, Anthony GX-Chen, Dongyan Lin, Jocelyn Shen, Blake A. Richards, Alison Gopnik, Doina Precup Human Adults and LLMs as Scientists: Who Benefits from Active Exploration? https://arxiv.org/abs/2606.06464
Human Adults and LLMs as Scientists: Who Benefits from Active Exploration? arxiv.org
AI Weekly's analysis
→
Read full analysis →
View on Bluesky ·
♥ 0
↻ 0
↩ 0
·
3 from the directory shared this ·
134d ago
Vil\'em Zouhar, Niyati Bafna, Mukund Choudhary, Maike Z\"ufle, Sara Rajaee, Pinzhen Chen, Jannis Vamvas, Sara Papi, Ona de Gibert, Bhavitvya Malik, Eliya Habba, Orfeas Menis Mastromichalakis, Patr\'icia Schmidtov\'a, ... Last Translation Benchmark https://arxiv.org/abs/2609.04173
Last Translation Benchmark arxiv.org
AI Weekly's analysis
→
- Last Translation Benchmark ships peer-reviewed hard examples, each paired with handcrafted verification rules that flag concrete failure cases instead of assigning a metric score.
- LTBv1 covers submissions accepted before September 1st 2026, spans texts, images, audio and videos, and is designed to accept ongoing contributions.
- The authors argue MT is stuck: standard benchmarks are saturating, automatic metrics are gameable, and human evaluation lacks reproducibility and scalability.
Read full analysis →
Zeyu Zhang, Ziliang Guo, Yihang Sun, Xichong Zhang, Xixuan Hao, Zehao Lin, Yang Zhang, Xiaoyan Zhao, Tong Shen, Bo Tang, Zhi-Qin John Xu, Junchi Yan, Haofen Wang, Xu Chen, Feiyu Xiong, Zhiyu Li, Tat-Seng Chua Metis: Memory Foundation Model https://arxiv.org/abs/2607.26760
Metis: Memory Foundation Model arxiv.org
AI Weekly's analysis
→
- Metis introduces a persistent, dynamically evolving memory state built into the model backbone rather than as an external retrieval module.
- The authors train Metis at 4B, 9B, and 27B parameters on a Qwen3.5 backbone using 8 H100 GPUs and around 406M synthesized tokens.
- Metis-27B reports 73.77% average on the paper's constructed test set versus 1.69% for the baseline Qwen3.5-27B without context.
Read full analysis →
Fitsum Reda, John Kamalu, Roger Waleffe, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro Nemotron-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context https://arxiv.org/abs/2606.26493
Nemotron-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context arxiv.org
AI Weekly's analysis
→
- NVIDIA's Nemotron-TwoTower splits an LM into a frozen autoregressive context tower and a trainable diffusion denoiser with bidirectional block attention.
- The system is built on Nemotron-3-Nano-30B-A3B, a 30B hybrid Mamba-Transformer MoE backbone, and trained on roughly 2.1 trillion tokens.
- The authors report retaining 98.7% of the autoregressive baseline's quality while delivering 2.42x higher wall-clock generation throughput.
Read full analysis →
View on Bluesky ·
♥ 0
↻ 0
↩ 0
·
3 from the directory shared this ·
113d ago
