Berkeley and NVIDIA's GaP Auto-Builds Robot Task Graphs

A robotics paper from UC Berkeley and NVIDIA, with collaborators at Carnegie Mellon and Bosch, has been circulating on Hugging Face's daily papers list, and the framing is worth pulling out because it points at a different bet than the usual end-to-end vision-language-action push. The system, GaP (Graph-as-Policy), replaces the single-big-model approach with a multi-agent LLM harness that composes the task as a directed computation graph, drawn from a 51-skill Modular Open Robot Skill Library the authors call MORSL.

The category they are targeting matters. They call it Variational Automation, which sits between fixed-cell industrial automation (identical parts, identical motion) and generalist robotics (any task, any environment). It is the space where the workcell is known but the objects move around, get swapped, or arrive in different poses, think grocery packing, cable insertion into a socket bank, or making popcorn on a portable stove. GaP handles this by iteratively rehearsing candidate graphs in NVIDIA Isaac Lab, analyzing failures with an LLM, and rewriting the graph.

The numbers, as reported by the authors, are the part that will get attention. On physical trials, GaP is reported at 25/25 (100%) on fulfilling grocery orders, 28/30 (93%) on packing grocery items, and 18/20 (90%) on the popcorn sequence, all on a Franka arm. Cable insertion on a UR5 came in at 121/130 (93%) across a six-socket bank. Across the simulated benchmarks in the paper, GaP holds success rates in the 0.93 to 0.99 range where baseline VLA models like π₀.₅ and MolmoAct2 collapsed into the 0.10 to 0.43 range under positional variation.

The caveat the paper itself makes is the honest one to lead with. Cycle times of 30 to 100+ seconds per instance are still roughly an order of magnitude slower than the roughly 7 seconds the authors cite as the industrial standard, and the benchmarks are quasi-static pick-and-place. Deformables, high-speed dynamics, and heavy force-feedback tasks are largely out of scope. What the reporting doesn't give you is a compute-cost breakdown for the self-learning loop or how the pipeline degrades when the underlying VLM or grasp model is swapped.

The forward-looking part: if graph-of-skills orchestration can be composed by an LLM from a shared open library, integrators building per-workcell automation get a cheaper path to a reliable policy than fine-tuning a VLA per task. That is a direction worth watching, even before the throughput closes on factory standards.