My book, Reinforcement Learning from Human Feedback is done! This is the book I wish I had when learning to fine-tune, align, & now post-train models since ChatGPT. The resource has been built by me finding time to study and document the fundamentals on nights and weekends since 2024.
Thinking Machines just released with a ~1T param, 41B active, apache-2 model Benchmarks are a clear step up from Nemotron Ultra (55B active), new best American model, and omni input. A bit behind GLM 5.2 on agentic benches, and Kimi K 2.6 on multi modal Super exciting!!
Major restructuring at Gemini (Jef Dean out, Hassabis no longer CEO). This story will be studied forever as the incumbent with all the advantages not being able to get going. P.s. OpenAI accomplished their original goal.
I think what is pretty clear is that the Chinese labs are far more capital efficient. In a world where scaling labs are intelligence is proportional to effective capital (buys compute, data, & talent) that may be the greatest strength your AI industry could ever have.
Anthropic's political pressure on distillation is regulatory capture and most of the employees are blind to it under their veil of safety. Or their paycheck helped them buy into safety, is only human nature, I don't even fault them that much.
Being out of SF has lowered my information proximity but with the big upside of giving me space to cultivate my own beliefs and values around ai. We need more people zagging in AI, the monoculture just helps the incumbents win at this point.
The pace of progress on models from so many organizations at once is genuinely incredible. Building LLMs isn't driven by rare secrets, but consistent effort, mass capital, and effective organization design. It is great that know-how of such a powerful technology is diffused.
Kimi K3 with more likes than downloads on HuggingFace is definitely showing us a glimpse of the future on open models. It's way less about individual access, and more of a distributed platform layer for companies.
Personal milestone: 1000 true fans of my newsletter Interconnects! Hitting a very long term goal feels great. I’m happy to get to be an independent voice in AI. Cultivating a paid base helps me commit to that longer term, and scale Interconnects’ impact.
Making talks with AI agents is awesome. I just told Fable to make a slide with real data on the KL distance from one of our reference Olmo 2 models and it made this with the wandb api (I edited text slightly).
