AutoTraceGT recovers 73–91% of agent failure taxonomies

A group of researchers has automated grounded theory — the qualitative coding methodology from the social sciences — and pointed it at agent trajectories, producing behavioral taxonomies that recover 73 to 91 percent of the failure modes previously catalogued by human annotators. The system, called AutoTraceGT, is described in a paper accepted to Findings of EMNLP 2026 and posted on arXiv.

The pipeline runs the three canonical phases of grounded theory — open, axial and theoretical coding — iteratively over trajectory data until it hits what the authors call "theoretical saturation." The paper reports that the resulting codebooks "recover 73-91 percent of the failure modes" in existing human-annotated taxonomies while also flagging patterns those taxonomies had missed. The authors also test the codebooks as feature representations for a downstream task and report they "outperform zero-shot and few-shot LLM baselines on downstream failure prediction."

The experiments span six trajectory corpora, and the authors say results align with previously documented expert interpretations on the same data.

What the abstract does not break out is per-corpus performance inside that 73–91 band, the compute cost of iterating to saturation, or how the additional novel patterns were adjudicated as genuine failures. The 33-page camera-ready is under a CC BY 4.0 license.