GPT-6 Astra leads ProgramDistill app-cloning bench at 49.2%

Given a working web app and a partially built copy, the strongest frontier coding agent tested finished 49.2% of the cumulative workflows on the full-application task. That was GPT-6 Astra. Claude Opus 5 came in at 28.8%.

The benchmark, ProgramDistill, reframes what the agent is asked to do. "Coding agents are typically evaluated with desired behavior specified through issues or instructions," the authors write. "In practical web development, however, agents may need to infer behavior from working software and implement it in an incomplete application."

To build it, the team factored 26 applications into features of different granularities and ran a pipeline called mine-craft-patch, which "discovers 1,975 replay-verified behaviors across 26 applications and constructs 4,063 tasks without human intervention." Nine frontier coding agents were evaluated in all.

Difficulty scales with how much of the app the agent has to put back. In partial-application reconstruction, success falls from 100% to 64.0% and from 96% to 32% as restoration depth increases from 1 to 8. The abstract does not name which two models produced those two curves.