8B RL agent on live iOS ties Gemini-3.1-Pro, beats 3.5-Flash

An 8B reinforcement-learning agent trained directly on live iOS devices ties with Gemini-3.1-Pro on real iOS applications and outperforms Gemini-3.5-Flash, according to a new arXiv preprint by Peitong Duan, Khashayar Gatmiry, Jeffrey Nichols, Samy Bengio and Emmanuel Abbe. The same agents post "nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments."

The framework splits the agent into two specialized components, "a Planning agent and a UI Grounding model," and uses "an LLM-as-a-Judge to provide comprehensive reward signals, combining dense, step-level feedback to evaluate intermediate progress with a terminal reward for overall task completion." Training runs on Proximal Policy Optimization with step-level advantage estimation, plus a new method the authors call GiGPPO, a "neighbor-conditioned value estimation method that improves critic stability by leveraging reward signals from visually similar states across concurrent trajectories."

To make online training on real phones tractable, the team built "an efficient infrastructure that orchestrates hundreds of concurrent virtual iOS devices." Evaluations run on UINavBench, iOSWorld and AndroidWorld, and the authors report that "our online RL trained agents substantially outperform both SFT and terminal-only RL baselines while accelerating training convergence."

The abstract reports relative gains and head-to-head frontier comparisons. It does not publish per-benchmark absolute scores, name the specific SFT baseline, or quantify the compute behind spinning up hundreds of iOS VMs in parallel.