HexaFuture EmbodiedLab: Claude Opus 5.5 Hits 43.3% on 30 Labs

On 30 textbook lab experiments, the best frontier coding agent reaches a 43.3% success rate when it is given only camera images, the robot state and joint control. That is Claude Opus 5.5 running in Claude Code. GPT-6 Astra, in Codex, reaches 40.0%. DeepSeek-V4.1-Flash-1017 and Kimi K3 reach 10.0%; GLM-5.3-Flash reaches 0%.

The benchmark, EmbodiedLab, comes from HexaFuture Inc., Tsinghua University, the National University of Singapore and Peking University. The authors call the setting 'acting to know': the agent receives an objective and the apparatus, 'chooses its own procedure, and operates a physics-based simulated laboratory, reading the instruments through its cameras.' Hidden parameters are resampled per instance, so the answer cannot be memorized, but it can be scored exactly against a tolerance.

The apparatus set is deliberately pedestrian. Thirty experiments span mechanics, electricity, optics, thermal physics and chemistry on 14 benches, all simulated in Isaac Sim and operated by a Franka Panda arm with a two-finger gripper and three cameras, two fixed and one on the wrist. Tolerances run from 0.2% to 5%. 'Every experiment is verified to be solvable,' the paper reports, by a reference solution that reads the same instruments as the agent, without a language model and without access to the hidden parameters.

Where the strongest agents fail is the part that cuts against the usual manipulation story. 'For the strongest agents the stages with the lowest pass rates are knowing the result and restoring the bench, while the other agents lose most of their runs before their readings suffice,' the paper writes. GPT-6.1 Sol leaves the bench unrestored after an accurate result in 23% of its runs. For Claude Opus 5.5, observing errors (one misread digit on a dial decides the run) account for 23% of its failed runs, its second-largest cause. Of the 30 experiments, 27 are solved by at least one agent; none of the six clears the black-box resistor, the focal length of a lens, or the two-lens relay.

More help only buys so much. Adding privileged segmentation with true depth and collision-free motion planning (level T1) raises success by between 1.7 and 10.0 points. A library of manipulation skills distilled from the GPT and Claude runs (T2) raises the weaker agents by 10 to 27 points, enough to put DeepSeek-V4.1-Flash-1017 at GPT-6 Astra's T0 level. 'Yet no agent that receives them reaches the strongest agent at T1.'