Alibaba Qwen-UI-Agent posts 92.2% on real-device mobile bench

The interesting thing in Alibaba's Qwen-UI-Agent technical report is not the leaderboard number, it is where the number came from. The team's headline claim is 92.2% on something called MobileWorld-Real, a benchmark of 409 tasks across 104 apps that runs on more than 100 physical phones rather than on a simulator. Most GUI-agent scores you see are from Android emulators or web sandboxes, which is a very different problem to expired logins, real CAPTCHAs, and pop-ups that actually appear.

According to the report, the same model lands 97.5% on AndroidDaily, 79.5% on OSWorld-Verified for computer use, and 73.6% on WebArena for browser tasks. On MobileWorld-Real the team reports it ahead of Claude Opus 4.8 by 7.5 points, GPT-5.6 Sol by 6.8, and Gemini 3.1 Pro by 6.0. It ships in three sizes: a 27B dense model, a 35B-A3B mixture-of-experts that activates only 3B parameters per token, and a 4B.

Two design choices are worth flagging. The action space is unified so the model can issue GUI clicks and bash CLI commands in the same trajectory, and the team says more than 40% of computer-use actions are emitted as batched actions to cut unnecessary inference steps. Training used online RL across up to 10,000 concurrent environments on trajectories that can exceed 100 turns, plus an agent-driven data flywheel that constructs tasks and diagnoses failures.

The honest caveat is that this is a technical report from the team that built both the model and the new benchmark, scored by their own AutoJudge grader that they claim hits 92.8% agreement with expert annotations. MobileWorld-Real is a Chinese-ecosystem-first benchmark and independent replication on the frontier baselines named here does not exist yet. What the report does not give you is cost per task, release license, or how the proactive features that let the agent act on incoming notifications are gated against misuse.

If the numbers hold up under outside eyes, the meaningful part for small teams is the 4B variant. A mobile agent that runs on ordinary hardware and can also drive a shell is a very different tool from a paid cloud API. It is cheaper to deploy, easier to keep on-device, and the kind of thing that changes what a single developer can ship.