Nathan Lambert's RLHF Course Reaches Tool Use and Agents

A free lecture on how language models actually acquire tool use is not the flashiest release of the week, but for anyone trying to ship an agent it may be the more useful one. Nathan Lambert's RLHF & Post-Training Course has reached Lecture 11, covering tool use, function calling and the path to agents, and mapping to Chapter 13 of the accompanying book.

The framing in the companion chapter is pleasingly unglamorous. Tool use is defined mechanically: the model emits a structured request, an orchestrator executes the tool, results are appended to the context, and the model continues generating. Function calling narrows that to arguments that must conform to a declared schema, usually JSON Schema. Code execution is a special case where the tool is a code interpreter and results come back as tool output. MCP, the Model Context Protocol, is described as an open standard using JSON-RPC 2.0 with three core primitives (resources, prompts and tools) so any harness can discover and call any provider's capabilities.

The section the lecture title actually flags is how these behaviors get trained. Lambert's chapter argues multi-step tool reasoning looks more like classic reinforcement learning than the per-sample RLHF loop, because the agent interacts with an environment and its tools over a full trajectory before any reward is assigned. A prompt is sampled, the model alternates actions with observations, and the completed trajectory is graded to produce a single reward that drives the policy update. That is a different shape of training data and infrastructure from preference tuning, and it hints at why teams shipping serious agents are quietly rebuilding their post-training stacks around long-horizon environments rather than static human ratings.

The honest caveat is that this is a lecture and a book chapter, not a benchmark result or a released model, so nothing here is a claim about who is winning at agents. What the material does not give you is the proprietary detail of how frontier labs actually structure their tool-use RL environments, or how reward hacking and trajectory-level over-optimization are being handled in production. For teams building on top of MCP or writing their own harnesses, the practical value is having one grounded reference to point engineers at before they invent their own vocabulary.