Biderman Paper: AI Science Must Study Training Dynamics

A new position paper argues the field has been looking at models from the wrong end. In Don't Just 'Fix it in Post': A Science of AI Must Study Training Dynamics, Stella Biderman, Mohammad Aflah Khan, Niloofar Mireshghallah, Catherine Arnett, Fazl Barez and Naomi Saphra put the complaint in a single sentence.

"Models are not static objects: they are snapshots of time-evolving processes shaped by data, objectives, architectures, and optimization dynamics," the authors write. "Yet much of AI research treats models as fixed artifacts, analyzing behaviors after training rather than asking why they emerge."

What they want is a science that can do three things in sequence: "predicting outcomes from early training signals, intervening when trajectories go wrong, and ultimately designing training procedures that more reliably produce desired properties." Scaling laws, they note, have already made prediction routine for loss. The open task is extending that to capabilities, biases, robustness and safety-relevant behaviors.

The abstract names four areas where it says progress is already visible: mechanistic interpretability, fairness, memorization and simplicity bias. It does not list which specific results it is counting. Two researchers in our Who's Who tracker shared the preprint shortly after it appeared.