Nathan Lambert ships RLHF post-training textbook via Manning

Nathan Lambert's long-in-the-works textbook on RLHF is finally out, and the useful thing about the launch is how he's carved up the subject. In a post on Interconnects, Lambert says *Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs* is shipping via Manning, with roughly 25% of the page count spent on RL algorithms themselves and the rest on the surrounding craft of turning a base model into something useful.

The five things he flags as most useful read as a snapshot of where the field's practical knowledge actually lives. Algorithm intuition comes first, and he is explicit that the interesting arc runs from the policy-gradient theorem through PPO to newer variants like GSPO and CISPO, on the grounds that knowing the family tree is how you judge whether "a new algorithm is fake or has potential." Second is the systems view: Lambert calls modern RL "a systems problem balancing a few problems," naming off-policy data, training-inference mismatch, and throughput as the tradeoffs to hold in mind. Third is a walk through three eras of post-training from roughly 2018 through 2023. Fourth is a chapter on distillation, which he frames as usefully "boring" in the middle of the ongoing AI policy debate about how outputs from one model get used to train another. Fifth is a second-half tour of over-optimization, regularization, evaluation, and character training.

Alongside the book, Lambert is shipping a 12-hour accompanying course with slides and YouTube video, plus a discount code PBLambert for 50% off through August 19. The launch post says the book assumes a bachelor's degree in CS level of knowledge, and that UK Amazon availability lands in October.

A printed textbook is a snapshot of a fast-moving area, and the launch post does not address whether Manning will ship revised editions as new post-training algorithms appear, which is the obvious question for anyone thinking about it as a long-lived reference rather than a one-time read.

For engineering managers onboarding a new post-training hire, or for instructors putting together a graduate seminar on alignment, a consolidated text that treats distillation and character training as normal chapters rather than blog posts is the sort of resource this corner of the field has been missing.