Alexandre Torres Leguet ships Little Book of RL with PyTorch code

A new short-form RL primer landed on GitHub this month, and the pitch is refreshingly narrow: one small book, from the basics up through PPO, with the code for every algorithm sitting right next to the theory. The Little Book of Reinforcement Learning, by Alexandre Torres Leguet, is the sort of thing that keeps showing up in my browser tabs the week before someone on the team actually has to train a policy.

The design choice worth flagging is that this is not just a PDF. The repo bundles PyTorch-based implementations of the algorithms the book covers, from Monte Carlo up to PPO, in an `algos/` folder. There is also a `supplementary/` folder with what the author describes as detailed explanations and rigorous proofs for the dynamic programming algorithms, a document he says was originally written in 2021 and folded back in here.

Why this matters when there is already a canon of RL textbooks and courses out there: the fastest way back into RL for an engineer who did it once and forgot most of it is short and code-first, not comprehensive. If your team is being pulled toward RLHF, agent training loops, or reward-model work, a small primer paired with runnable implementations is a lower-friction on-ramp than a heavyweight textbook. V1 shipped in June 2026 under a non-commercial Creative Commons license, and a printed copy is available via Amazon for anyone who prefers paper.

The honest caveat is that the README does not spell out the chapter list, the page count, or whether modern RL-for-LLMs approaches like RLHF get any airtime at all. From basics to PPO is a classical trajectory, so treat it as a foundations refresher rather than a guide to whatever your frontier lab's alignment team is doing today. Still, small, opinionated, code-anchored books are the format that actually gets read, and the upside is that a solo author has done the compression work so a reader does not have to.