paper web signal

Alibaba's MAI-Gym Hits 78.4% MobileWorld, Open-Sources RL Stack

TL;DR

  • MAI-Gym-9B scores 78.4% on MobileWorld and 76.2% on OSWorld-Verified, with the platform scaling to 20,000 parallel environment instances.
  • RL inside a mocked train-ticketing app lifted real-phone task success from 28.8% to 48.1%, a rare published sim-to-real datapoint for GUI agents.
  • Alibaba's Tongyi Lab released the framework, synthesis skills, mocked apps, 2,000 verified tasks, and the model checkpoint.

Alibaba's Tongyi Lab released MAI-Gym, an open platform that unifies GUI agent evaluation, data collection, and online reinforcement learning training across mobile, computer, and web environments. The 9-billion-parameter MAI-Gym-9B scores 78.4% on MobileWorld and 76.2% on OSWorld-Verified; the paper says the platform "supports ten established benchmarks, and scales to 20,000 parallel environment instances."

Sim-to-real carries the sharper figure. The authors write that "RL in a mocked train-ticketing app transfers to its real-world counterpart, improving task success on a physical phone from 28.8% to 48.1%," measured on an actual device rather than inside the simulator.

The team released the framework, synthesis skills, mocked applications, 2,000 verified tasks, and the model checkpoint. The abstract does not disclose compute cost, name the ten supported benchmarks, or say whether the physical-phone transfer holds up on apps outside the train-ticketing domain.