Skip to main content

Alibaba's Qwen-UI-Agent Outshines GPT and Claude in Mobile GUI Benchmarks

Alibaba has officially thrown its hat into the GUI agent ring with the release of Qwen-UI-Agent on August 20. This new foundation model isn't just another player—it's setting the pace. Covering mobile, desktop, web, and deep search environments, Qwen-UI-Agent has already matched or surpassed several industry heavyweights on key benchmarks.

Mobile Mastery

On the mobile front, Qwen-UI-Agent is a force to be reckoned with. It scored a stellar 82.1% on the MobileWorld benchmark, leaving GPT-5.6Sol and Claude Opus4.8 trailing by 12 and 14.6 percentage points, respectively. But it doesn't stop there. On the real-world MobileWorld-Real benchmark, it hit 92.2%, outperforming Gemini3.1Pro, Claude Opus4.8, and GPT-5.6Sol. And on AndroidDaily, it nearly aced the test with a 97.5% score—just a hair's breadth from perfection.

Desktop and Web Prowess

Desktop and browser tasks are no slouch either. Qwen-UI-Agent achieved 79.5% on OSWorld-Verified, surpassing GPT-5.5 and Gemini3.1Pro. On WebArena, it took the top spot among all compared models with a 73.6% score. Even in GUI grounding tests like ScreenSpot-Pro, it scored 81.5%, setting new records on four other evaluation benchmarks.

Image

From Simulation to Reality

One of the biggest hurdles in GUI agents is bridging the gap between simulated environments and real-world usage. Qwen-UI-Agent tackles this head-on by building a real mobile environment with over 100 actual smartphones and more than 150 applications for task creation, trajectory collection, model training, and evaluation. It also introduced its own real-device benchmark, MobileWorld-Real, which includes over 400 tasks across more than 100 apps. Beyond conventional click operations, the model supports direct command-line inputs and can output multiple actions in a single decision—about 40% of desktop task actions are batch-processed this way.

Image

Safety and Long-Horizon Tasks

Security is woven into every step of Qwen-UI-Agent's task execution. If it encounters illegal or high-risk requests, it outright refuses and terminates the task. For sensitive scenarios like payments, data deletion, or privacy authorizations, it pauses at critical junctures and notifies the user. The model also supports online reinforcement learning on ultra-long trajectories exceeding 100 steps, with about 10,000 concurrent environments rolling out simultaneously. This continuous training loop helps it tackle increasingly complex, long-term tasks.

Key Points

  • Benchmark Dominance: Qwen-UI-Agent leads on MobileWorld (82.1%), MobileWorld-Real (92.2%), AndroidDaily (97.5%), OSWorld-Verified (79.5%), and WebArena (73.6%).
  • Real-Device Training: Built with 100+ real smartphones and 150+ apps, plus a new MobileWorld-Real benchmark with 400+ tasks.
  • Efficient Actions: Supports batch output of multiple actions, with ~40% of desktop actions processed in batches.
  • Security First: Refuses illegal requests and pauses for sensitive operations like payments or data deletion.
  • Long-Horizon Learning: Online RL on 100+ step trajectories with 10,000 concurrent environments.