Found first: a primary source the press has not covered yet.
Guide Labs submitted "Scaling Inherently Interpretable Language Models" to arXiv on August 6, 2026, presenting Steerling-8B as empirical evidence that interpretability and capability do not trade off when interpretability is built into training. The paper tests this relationship across three orders of magnitude of compute, in both autoregressive and diffusion language models. Across every scale tested, the two properties improved together.
What the source says
The paper comes from the Guide Labs team, with Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen, Isaac Plant, Muawiz Chaudhary, Nathaniel Monson, Saqib Azim, Zhichen Guo, and Julius Adebayo listed as authors. Steerling-8B is a causal diffusion language model. Its architecture decomposes model representations into human-understandable concepts connected to outputs through a linear path, making each concept's contribution to each generated token exactly computable. The model is competitive with models trained on up to 16x more compute. Across the compute range tested, representations become more disentangled and more aligned with human-understandable concepts as scale increases, in both autoregressive and diffusion LMs.
Why it matters
The prevailing assumption in interpretability research is that making a model interpretable costs performance. Guide Labs' results run counter to this across the tested compute range. The design enables inference-time concept steering and per-token attribution to source training data without retraining the model. Both capabilities currently require separate post-hoc systems with no guarantee of faithfulness to the model's actual computation. The paper's evidence is that they follow directly from treating interpretability as a training constraint, and that the interpretability strengthens rather than erodes as scale increases.