OpenHands-Trained Coding Agents Degrade on Other Scaffolds

A short arXiv paper posted on August 6, 2026 makes a claim that should quietly worry anyone shipping open-weight coding agents. The paper, DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds by Kishanthan Thangarajah, Boyuan Chen, and Ahmed E. Hassan, argues that "the open ecosystem has converged on a single training environment," and that models tuned that way stop working when you swap that environment out.

That environment is OpenHands. According to the abstract, "trajectory datasets used to fine-tune open models are collected almost exclusively under OpenHands," and models fine-tuned on that data "score well under OpenHands but degrade substantially when deployed under any non-training scaffold." The authors frame the culprit as "planning structure," which they split into explicit planning (visible plan-then-act patterns) and implicit planning (habits baked into the trajectory itself). Both, they argue, are scaffold-specific, so the fine-tuned model has learned to be good at OpenHands more than it has learned to be good at coding in general.

Their proposed fix is DCAS, "a backend-substitution interception layer that routes API traffic between any CLI scaffold and any backend model." It lets researchers collect trajectories in a planning-aware way and evaluate the same weights across scaffolds without touching the scaffolds themselves. The reported result is that a model fine-tuned on a small set of those planning-aware trajectories "gains consistently across non-training scaffolds."

The abstract is thin on the usual receipts. It does not name the specific models tested, the scaffolds compared, the benchmarks used, or the size of the "substantially" it reports, and the retrieved arXiv record shows no venue attached. So the strong reading, that many published open coding-agent numbers are partly a measurement of OpenHands rather than of the model, is a hypothesis the paper advances rather than a settled result.

If the effect holds up, the interesting shift is competitive. Teams shipping coding agents that need to run inside whatever CLI a customer already uses, and benchmark maintainers willing to publish scaffold-agnostic evaluations, gain a real edge over anyone whose leaderboard score is really an OpenHands score.