My book, Reinforcement Learning from Human Feedback is done! This is the book I wish I had when learning to fine-tune, align, & now post-train models since ChatGPT. The resource has been built by me finding time to study and document the fundamentals on nights and weekends since 2024.
In light of kind of insane AI safety discussions recently: 1. AI progress is very fast 2. we should be careful about how we roll out the tech 3. the world is not actively ending
Thinking Machines just released with a ~1T param, 41B active, apache-2 model Benchmarks are a clear step up from Nemotron Ultra (55B active), new best American model, and omni input. A bit behind GLM 5.2 on agentic benches, and Kimi K 2.6 on multi modal Super exciting!!
With the latest external hack of OpenAI via Claude, closed models continue to be the tip of the iceberg on AI risks, not open models. They have been 1) much easier to get started with, 2) more capable & 3) shipped with leaky safeguards. Finetuning open models to specific attacks is harder.
Major restructuring at Gemini (Jef Dean out, Hassabis no longer CEO). This story will be studied forever as the incumbent with all the advantages not being able to get going. P.s. OpenAI accomplished their original goal.
I think what is pretty clear is that the Chinese labs are far more capital efficient. In a world where scaling labs are intelligence is proportional to effective capital (buys compute, data, & talent) that may be the greatest strength your AI industry could ever have.
Anthropic's political pressure on distillation is regulatory capture and most of the employees are blind to it under their veil of safety. Or their paycheck helped them buy into safety, is only human nature, I don't even fault them that much.
Being out of SF has lowered my information proximity but with the big upside of giving me space to cultivate my own beliefs and values around ai. We need more people zagging in AI, the monoculture just helps the incumbents win at this point.
The pace of progress on models from so many organizations at once is genuinely incredible. Building LLMs isn't driven by rare secrets, but consistent effort, mass capital, and effective organization design. It is great that know-how of such a powerful technology is diffused.
Kimi K3 with more likes than downloads on HuggingFace is definitely showing us a glimpse of the future on open models. It's way less about individual access, and more of a distributed platform layer for companies.
