ByteDance and USTC paper targets agent 'topological collapse'
TL;DR
- The paper diagnoses standard full-trajectory imitation learning as causing 'severe topological collapse' that indiscriminately penalizes valid alternative reasoning paths in agents.
- DART-SD identifies a Critical Topological Breakpoint during rollouts and applies training loss only to recovery steps, protecting the valid reasoning prefix.
- On multi-turn tool-calling benchmarks, the ByteDance and USTC team reports the method significantly outperforms traditional full-trajectory baselines.
The most striking claim in a new ByteDance and USTC preprint is a diagnosis, not a benchmark: standard full-trajectory imitation learning, the workhorse used to fine-tune multi-turn tool-calling agents, is wrecking the very thing it is meant to train.
The paper's framing: for tasks with multiple order-independent sub-goals, "the optimal solution space forms a vast combinatorial diamond lattice." Forcing that lattice into single trajectories "causes a severe topological collapse, indiscriminately penalizing valid alternative explorations and severely degrading policy diversity."
The proposed fix, DART-SD, tries to stop treating "different valid path" as "wrong." It models the agent's execution as an Interaction-State Transition Graph, identifies where the student first diverges from a recoverable teacher path (the Critical Topological Breakpoint), and computes the training loss "exclusively on the generated recovery steps while strictly protecting the valid reasoning prefix from destructive gradient updates."
On "complex multi-turn tool-calling benchmarks," the paper reports DART-SD "significantly outperforms traditional full-trajectory baselines." The abstract publishes no per-benchmark numbers, and this is a single preprint from one lab.
Shared on Bluesky by 1 AI expert
Originally reported by paper
Read the original article →Original headline: ByteDance's DART-SD Shows Diamond-Topology Training Prevents Multi-Turn Agent Policy Collapse