DataFlow-Harness hits 93.3% pass rate, 72.5% cost cut vs Claude

The interesting thing in this paper is not the headline pass rate but why the authors think it exists. Their framing is that when you ask a general-purpose code agent like Vanilla Claude Code to build a data pipeline, you get a throwaway script rather than the persistent, editable DAG the platform actually needs to keep running. They call the disconnect the NL2Pipeline gap.

Their fix is to stop treating pipeline construction as free-form code generation. In the arxiv paper, Runming He and coauthors describe DataFlow-Harness as three coordinated pieces: DataFlow-Skills carrying procedural guidance, a Model Context Protocol layer exposing live operator registries to the agent, and a DataFlow-WebUI that keeps conversational authoring and visual DAG editing in sync. The agent is pushed to build platform-native DAGs through typed, incremental mutations rather than free-form scripts.

The numbers reported on the 12-task benchmark are the reason to look. A 93.3% end-to-end pass rate is high in absolute terms, but the cost picture is the more telling part: 72.5% cheaper than Vanilla Claude Code, 49.9% lower latency, and within 0.9 percentage points of a Context-Aware Claude baseline while spending 42.8% less. The claim, read carefully, is that grounding the agent into live platform state does most of the work that context stuffing was doing, at a fraction of the token bill.

The honest caveat is scale. Twelve tasks is a small benchmark and every comparison point is a variant of Claude Code, not the wider field of agentic pipeline tools. What the paper does not give you is the task list, a head-to-head against orchestration-native LLM platforms, or a read on how DataFlow-Skills generalize outside the DataFlow platform they were built for. Take the specifics as reported, not settled.

If the pattern holds beyond twelve tasks, the takeaway for anyone building enterprise data agents is that the grounding — a real operator registry the model can query through MCP, not a prompt dump — is where the cost and reliability savings come from.