Distill's 'Zoom In' argues neural nets are readable circuits

There is a foundational essay in machine learning interpretability that keeps getting cited as the field's origin document, and it is worth reading directly rather than through the summaries that have accreted around it. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov and Shan Carter published "Zoom In: An Introduction to Circuits" in Distill in 2020, and it lays out a specific bet about how to make sense of what a neural network is actually doing.

The bet has three parts. Features, meaning the network learns meaningful, understandable pieces (edge or curve detectors early, floppy ear detectors and wheel detectors deeper in). Circuits, defined by the authors as a computational subgraph of the network consisting of a set of features and the weighted edges between them. And universality, the claim that analogous features and circuits form across models and tasks, the way analogous organs recur across species.

What makes the essay unusual is that it does not just assert this and move on. The authors point at specific neurons inside InceptionV1 as worked examples. They present curve detectors organized by orientation, supported by maximizing activations, by drawing curves by hand, and by rotating those curves and watching one orientation's neuron drop out. They say plainly that you can read a curve detection algorithm off the weights. They walk through a dog head detector built from smaller curve and snout detectors wired by both excitatory and inhibitory connections, and they name the phenomenon where a single neuron in InceptionV1 responds to cat faces, fronts of cars, and cat legs at once. Their term for it is polysemanticity, and their proposed reason is superposition: the model uses fewer neurons for more concepts as long as those concepts do not tend to co-occur.

The frame the piece pushes is that interpretability should behave like a natural science. In the authors' own words, circuits sidestep the usual challenges by focusing on tiny subgraphs of a neural network for which rigorous empirical investigation is tractable, and they are falsifiable in the sense that understanding a circuit should let you predict what changes when you edit the weights. That is the shift being sold, from waiting for a full theory of deep learning to cataloguing motifs the way biologists catalogued cells.

The honest caveats are the ones the essay itself flags. The three claims are labelled speculative, the worked examples sit inside a convolutional vision model rather than the transformer language models that dominate current attention, and the universality claim is a hope with early evidence rather than a proof. What the piece does not give you is any guarantee that the same weight-reading approach scales to models with billions of parameters, or to the messier features inside modern LLMs. Even so, the shape of the argument, that useful interpretability comes from stacking small verifiable circuit explanations rather than from one grand theory, is the thing that later mechanistic interpretability work has largely inherited.