Xiaomi-GUI-0 Technical Report

• Real-device GUI agents face distribution shifts from dynamic apps, account states, abnormal dialogs, and long-tailed user queries that static benchmarks and simulations do not capture.The paper argues that real devices, applications, requests, and failure states should be central to training and evaluation because errors can persistently divert execution from the task objective.Source: §1 · p. 3
• A real-device-dominant infrastructure supports multi-source data collection across physical devices, authentic app and account states, abnormal states, and long-tail intents.Physical devices are the primary execution environment, with sandboxes providing auxiliary support; training data also targets reflection and memory.Source: §1 · pp. 3–4
• An error-driven data flywheel repairs failures through annotated erroneous steps and rationales, plus teacher-scored takeovers that generate recovery trajectories.The resulting data include error explanations, reflective rationales, and recovery processes to strengthen error recognition and self-correction.Source: §1 · p. 4
• A three-stage pipeline combines SFT, step-level RL, and agentic RL to develop execution, fine-grained correction, and long-horizon planning and recovery.Agentic RL uses real or near-real mobile environments for continuous interaction.Source: §1 · p. 4
• RealMobile evaluates end-to-end task completion, exception recognition, and robustness on physical devices and live apps, using fine-grained sub-goals and cross-application scenarios.The public evaluation uses the simulated task-execution benchmark AndroidWorld [26].Source: §1 · p. 4
• 72.0% success rate on RealMobile and 78.9% on AndroidWorld, with improved success and robustness on real devices.These are the reported results for Xiaomi-GUI-0.Source: §1 · p. 4

2 Infrastructure: Real Devices and Sandboxes

• Hundreds of physical phones and dozens of physical tablets form the primary execution substrate, complemented by hundreds of sandbox instances for scalable, reproducible collection.Figure 1 depicts the hybrid infrastructure, with physical devices as the primary environment and sandboxes as auxiliary support.Source: Figure 1 · p. 5

2.1 Design Motivation

• Physical devices capture authentic accounts, live networks, native behaviors, diverse layouts, and abnormal states that emulators often cannot reproduce reliably.Examples of difficult-to-reproduce states include captchas, payment authentication, expired logins, risk-control interception, and manual verification.Source: §2.1 · p. 5
• Tasks go to sandboxes when apps remain stable under virtualization and to physical devices when they depend on real devices, networks, or native client states.This hybrid routing supports scalable, stable, reproducible collection while keeping the core interaction distribution aligned with real applications.Source: §2.1 · p. 5

2.2 System Architecture

• The infrastructure separates resource management, task-to-resource scheduling, and execution and trajectory collection into three layers.The resource layer spans nearly ten mainstream device brands; phones cover 100 high-traffic commercial apps, while tablets and cockpits together cover 20 representative optimized apps. Sandboxes support ablations, grounding subtasks, and reproducible experiments.Source: §2.2 · pp. 5–6
• Device-Pull scheduling assigns queued tasks when an idle device’s current readiness profile matches their requirements, avoiding assignments to devices that become ineligible.Profiles include device type, resolution, installed apps, account availability, and risk-control level; workers refresh them through health reports.Source: §2.2 · p. 6
• The execution layer runs an observe–decide–act loop, supports browser-based recovery of abnormal states on physical devices, and archives trajectories for training and evaluation replay.Archived records include task, device, app and account state, observations, actions, execution status, timestamps, and exception type, supporting failure attribution and data-flywheel feedback.Source: §2.2 · p. 6

3 Training Data Collection and Construction

• The three data tiers and error-driven flywheel address execution precision, abnormal-state handling, long-tail generalization, correction signals, and planning, reflection, and memory.The tiers comprise high-frequency task data, high-generalization data, and agent-capability enhancement data; the flywheel targets corrections to the model’s own error distribution.Source: §3 · pp. 6–7

3.1 High-Frequency Task Data

• For each in-scope function, real-user query variants are paired with multiple functionally equivalent trajectories spanning different entry points and intermediate pages, teaching equivalent execution strategies across diverse UI configurations.The dataset also includes approximately 5,000 samples spanning 14 abnormal-state types; it teaches the model to return control for captchas and close or bypass safely skippable pages such as advertisements and non-critical pop-ups.Source: §3.1 · p. 7

3.2 High-Generalization Data

• High-generalization data scales long-tail intent coverage through automated query synthesis, device rollouts, and trajectory- and step-level cleaning.A manually annotated function tree underlies the largely automated stages; rollout episodes are filtered to retain complete, high-quality successes and remove repeated or stagnant labels.Source: §3.2 · p. 8, Figure 2 · p. 8, §3.2.3 · p. 10

3.2.1 Function Tree Construction

• Each application’s five-level function tree records entry paths, example queries, and function pages, with human checks for completeness and structure.Annotators traverse from top-level entries to concrete function points, marking reused branches rather than duplicating them; tree coverage is used to assess synthesized-data coverage.Source: §3.2.1 · p. 8

3.2.2 Query Synthesis

• Behavior buckets synthesize natural, intent-driven queries, while function-tree paths preserve fine-grained coverage; query types span single- and cross-application tasks.Single-app queries include function-point, complex, and summary tasks; cross-app queries include relay, contrast, and parallel relations. Candidates are judged for realism, completability, and complexity, then polished and back-tagged.Source: §3.2.2 · p. 9

3.2.3 Trajectory Synthesis and Cleaning

• Trajectory cleaning keeps only episodes whose ordered subtasks are all judged complete, and removes repeated actions from labels while retaining them as history for recovery.The subtask-level VLM judge agreed with human spot-checks 94% of the time on 320 trajectories; corrective steps after stalls remain labels conditioned on the failed attempts.Source: §3.2.3 · p. 10

3.3 Agent-Capability Enhancement Data

• Structured chain-of-thought annotations supervise planning, reflection, and memory with a consistent five-tag schema aligned between training and inference.The schema separates observation, optional reflection, planning or replanning, decision, and memory; synthesis constraints require decisions to follow from observations and plans and match the original action label.Source: §3.3 · pp. 10–11

3.4 Error-Driven Data Flywheel

• The error-driven flywheel turns model failures into first-error corrections and teacher-guided recovery examples, then adapts supervision to the model’s evolving errors.Annotators correct the first key error while retaining the erroneous action as history; a teacher also takes over for bounded steps after sustained low scores to return execution to a workable path.Source: §3.4 · pp. 12–13

4.1 Training Recipe

• SFT, Step RL, and Agentic RL progressively initialize the policy, correct response-level behavior, and optimize complete trajectories over long horizons.The curriculum orders stages by supervision density, avoiding sparse trajectory rewards before the policy can produce structurally valid responses.Source: §4 · p. 13, §4.1 · p. 13

4.2 Supervised Fine-Tuning

• SFT trains on four data sources and supervises each trajectory step to produce structured reasoning and an action from the task, recent history, and screen observation.The sources are high-frequency task, high-generalization, agent-capability enhancement, and error-flywheel corrected-action data; the stage establishes a stable policy before reinforcement learning.Source: §4.2 · p. 14

4.3 Step-Level Reinforcement Learning

• Step RL uses GSPO [52] to optimize complete agent responses, with a cascade reward that checks response requirements hierarchically and exits at the first failure.Rule-based checks reject malformed responses before LLM-as-judge checks; rewards are −1.0 for parse or execution failures, −0.5 for other failures, and 1.0 when all levels pass.Source: §4.3 · pp. 14–15, Table 1 · p. 15
• Step RL targets local errors including malformed actions, incorrect parameters, incomplete reasoning, missing memory updates, and reasoning-action inconsistency.Its sequence-level importance ratio and group-relative advantages match the optimization unit to the structured response rather than individual tokens.Source: §4.3 · p. 14

4.4 Agentic Reinforcement Learning

• Agentic RL optimizes whole task trajectories to improve long-horizon state tracking, recovery, cross-application consistency, and task completion.Before rollout, task sampling uses smoothed success-rate estimates and a softmax score; α and β shift sampling from easier tasks toward harder ones during training.Source: §4.4 · pp. 15–16
• Agentic RL decouples inference, environment execution, and data transfer through asynchronous trajectory state machines so slow device interactions do not stall training.Inference uses SGLang [53] with session-affinity routing, while TransferQueue [12] separates control metadata from large payloads stored and accessed by reference.Source: §4.4 · pp. 15–16, Figure 6 · p. 16
• Agentic RL batches training by turn while retaining trajectory IDs and turn indices, then regroups turns so each inherits an advantage derived from its trajectory return.This reuses the GSPO objective while keeping training sequences short through fixed-window histories.Source: §4.4 · p. 16

5 The RealMobile Benchmark

• RealMobile evaluates GUI agents on physical devices and live applications, scoring each task through fine-grained sub-goals and requiring state retention across applications in most tasks.Unlike emulator- or mock-based benchmarks, it is built from real user traffic and hand-crafted for reproducible evaluation.Source: §5 · p. 17

5.1 Benchmark Construction

• The five-stage construction pipeline designs tasks, collects real-device trajectories, specifies sub-goals and vetoes, encodes executable rules, and iteratively validates them against expert judgments.Additional successful and failed trajectories are sampled with multiple agents.Source: §5.1 · p. 17

5.2.1 Application Selection

• RealMobile selects 14 widely used applications across functional categories, installs them on physical Android devices with logged-in test accounts, and stops payment tasks before transaction confirmation.It retains applications requiring authentication or sensitive operations, while preserving the reasoning chain for payment tasks.Source: §5.2.1 · p. 17

5.2.2 Task Construction

• The 100 tasks span Foundation, Safety & Reflection, Memory & Knowledge, and Complex Reasoning & Planning, testing interface operations, boundary judgment, information retention, and long-horizon planning.Complex tasks include multi-source comparisons, numerical reasoning, and open-ended planning across applications.Source: §5.2.2 · pp. 17–19

5.3.1 Fine-Grained Scoring

• RealMobile decomposes tasks into typically 3–6 verifiable sub-goals, awards equal credit for each completed goal, and assigns zero when a veto condition is triggered.Conditional branches allow full credit for valid paths that depend on account or environment state.Source: §5.3.1 · pp. 19–20

5.3.2 Executable Verification Rules

• A dual verification framework combines XML structure matching with logical semantic rules to verify sub-goals and semantically equivalent actions, including sequential and cross-step consistency constraints.OCR supplements XML matching when on-screen text is missing from the UI hierarchy.Source: §5.3.2 · pp. 20–21

5.3.3 Auto-Eval Pipeline

• The auto-evaluation pipeline applies OCR, checks veto conditions, evaluates sub-goals with XPath queries and code functions, and scores trajectories by the fraction of completed sub-goals.A triggered veto terminates evaluation with a score of zero.Source: §5.3.3 · p. 22

6.1 Experimental Setup

• Training uses approximately 1.2 million SFT GUI steps, 4.4 million grounding samples, 0.4 million Step RL GUI steps, and several thousand Agentic RL tasks.SFT trajectories come from real devices and Android emulators; Agentic RL trajectories are generated online through environment interaction. Grounding samples train the model to parse visible screen elements and predict their coordinates, functions, text, and other attributes.Source: §6.1 · p. 22

6.2 Main Results

• 72.0% on RealMobile and 78.9% on AndroidWorld are Xiaomi-GUI-0’s mean success rates across four runs.On AndroidWorld, it exceeds UI-Venus-1.5-30B-A3B at 77.6%; on RealMobile, it outperforms MAI-UI-8B at 33%. It also exceeds Gemini 3.1 Flash at 58%, Claude Opus 4.7 at 60%, and Claude Opus 4.6 at 33%, while approaching Gemini 3.1 Pro at 85% and Seed 2.0 Pro at 80%.Source: §6.2 · pp. 22–23, Table 4 · p. 23
• Foundation success reaches 100.0%, while Safety & Reflection is Xiaomi-GUI-0’s weakest domain at 43.8%.The Foundation result matches the strongest proprietary models; Safety & Reflection is the weakest domain for every evaluated model, and Xiaomi-GUI-0’s 43.8% is highest among open-source models (Gemini 3.1 Pro reaches 62.5%).Source: §6.2 · p. 23, Table 5 · p. 23
• Memory & Knowledge success is 66.7%, while Complex Reasoning & Planning success is 80.5%.Xiaomi-GUI-0 leads open-source models in Memory & Knowledge but trails Gemini 3.1 Pro at 93.9% and Seed 2.0 Pro at 90.9%; its planning score approaches Gemini 3.1 Pro at 82.9%, exceeds Seed 2.0 Pro at 73.2%, and tops the strongest open-source baseline at 31.7%.Source: Table 5 · p. 23, §6.2 · p. 24

From Framework Agents to Native GUI Agents

• Xiaomi-GUI-0 follows the native-agent paradigm but targets real-mobile constraints rather than static or emulator-dominated benchmarks.Framework agents such as Mobile-Agent [38] [39] and PC-Agent [18] orchestrate specialized modules around general-purpose VLMs; native agents integrate perception, grounding, planning, and execution into one model, as in UI-TARS [37], AutoGLM [19], and other recent systems. Framework performance depends on the underlying VLM, and module depth and interaction rounds increase latency.Source: From Framework Agents to Native GUI… · p. 24

Real-Device and Virtualized GUI Benchmarks

• Real-device benchmarks evaluate agents on physical devices and live applications, while virtualized benchmarks offer greater reproducibility but differ from real-app states and behaviors.AndroidWorld [26] uses programmatic checks in emulator or sandbox tasks; MobileGym [43] and SimuWoB [17] simulate commercial apps. MobileBench-OL [45] and Knowu-bench [7] evaluate real-device behavior. Xiaomi-GUI-0’s benchmark retains rule-based programmatic evaluation and separates navigation, grounding, planning, long-horizon execution, memory, reflection, and safety constraints.Source: Real-Device and Virtualized GUI… · p. 24

RL-based GUI Agent Training

• Reinforcement learning has become a central direction for improving GUI agents’ long-horizon execution, error recovery, and environmental adaptability.Early work explored curriculum training and R1-style RL [20] [21] [32] [11], while DigiRL [4] showed online RL can outperform supervised fine-tuning for device control. Further directions include self-supervised RL [9], preference learning [13], trajectory-efficient optimization [8], process-level rewards [46], error detection and backtracking [44], large-scale rollouts [37], model fusion [35], and asynchronous online RL [42].Source: RL-based GUI Agent Training · pp. 24–25

8 Conclusion

• 72.0% success on RealMobile and 78.9% on AndroidWorld, with improved success and robustness on real devices.Source: §8 · p. 25
• Xiaomi-GUI-0 is trained and evaluated in a closed loop centered on real-device execution to address the gap between benchmark scores and real-world usability.Source: §8 · p. 25
• Its real-device-dominant hybrid infrastructure makes physical devices primary and sandboxes complementary, aligning data collection, training, and evaluation with deployment conditions.Source: §8 · p. 25
• An error-driven data flywheel turns failure patterns from real rollouts into supervision for reflection and recovery, rather than merely scaling successful trajectories.Source: §8 · p. 25
• Progressive training moves from supervised fine-tuning through step-level to agentic reinforcement learning, developing interface operation, long-horizon execution, and error recovery.Source: §8 · p. 25
• The report argues that real execution is useful as a continual source of supervision, not only as final evaluation, and offers a practical path toward deployable mobile GUI agents.Source: §8 · p. 25

• 100 high-frequency commercial mobile applications target everyday mobile usage.Source: A Data Collection Applications · p. 30
• 20 tablet and cockpit applications target large-screen interaction patterns.Source: A Data Collection Applications · p. 30
• 20 AndroidWorld applications provide a stable benchmark environment.Source: A Data Collection Applications · p. 30

B Action Space

• The action space covers touch, text input, system navigation, user communication, and task termination; each step emits one action with JSON arguments and relative coordinates in [0, 1]².Coordinates use (0, 0) for the top-left corner and (1, 1) for the bottom-right.Source: B Action Space · p. 31, Table 7 · p. 31

B.1 Anomaly Semantics

• More than fifty online abnormal-state causes are consolidated into fourteen handoff semantics, each a valid Fail.type value for stopping, pausing, or returning control to the user.When an abnormal state blocks reliable automation, the model should hand control back rather than continue issuing UI actions.Source: §B.1 · p. 31, Table 8 · p. 32
• Table 8 defines the fourteen anomaly semantics used by the Fail.type field.The taxonomy provides the handoff categories for abnormal states.Source: §B.1 · p. 31, Table 8 · p. 32

C Function Tree Example

• Figure 9’s Bilibili function tree lists sibling functions at each level while expanding one representative branch to the next.The simplified example illustrates the functions available for sampling within the tree.Source: C Function Tree Example · p. 33, Figure 9 · p. 33
• The function tree represents both which functions can be sampled at each node and the access path from an application entry to a concrete function point.Source: C Function Tree Example · p. 33, Figure 9 · p. 33

D Behavior-Bucket Examples

• Behavior buckets capture coherent user-request motivations, unlike the function tree’s record of exposed functions and how users reach them.Each bucket corresponds to a stable usage motivation and contains several concrete behavioral phrases.Source: D Behavior-Bucket Examples · p. 33
• Query synthesis samples a behavior bucket, then binds concrete entities, filters, pages, or function-tree paths into an executable natural-language instruction.Source: D Behavior-Bucket Examples · p. 33

Local life and exploration; E-commerce and market shopping

• Xiaohongshu tasks cover nearby-content and destination discovery alongside Market browsing, seasonal food selection, image-based product search, and cart or wishlist management.Examples include filtering local content by city or distance and finding nearby activities, while shopping spans product categories and image recognition.Source: §D.1 · p. 34

Live broadcast interaction; Order and asset management

• Tasks include switching among themed livestream channels, joining Market night-event lotteries, tracking order statuses, handling after-sales messages, and managing wallet assets and coupons.Order examples include pending-payment, pending-shipment, pending-receipt, and pending-review states.Source: §D.1 · p. 34

Content creation and publishing

• Creation tasks range from selecting or taking media and writing captions to publishing posts and using creator-center activities, templates, AI effects, or filters.The example also includes discovering inspiration and templates for creation.Source: §D.1 · p. 34

Social connection and groups

• Social tasks include forming interest-based group chats, adding friends through several channels, and checking likes, saves, followers, comments, and private messages.Group chats can have entry rules, and friend-adding includes scan, contacts, or third-party social channels.Source: §D.1 · p. 34

Creator growth and learning

• Creator-growth tasks cover analytics, interaction metrics, monetization income and benefits, plus learning platform rules, reviewing violations, and using creator-academy tutorials.Source: §D.1 · p. 34

Profile and preference settings

• Profile tasks include editing personal details, adjusting display and functional preferences, and configuring interaction permissions, blocked users, protection mode, and personalized content.Source: §D.1 · p. 34

History and space management

• History and storage tasks involve reviewing note, livestream, and Market histories, then clearing cache, downloads, or chat files to free device storage.Source: §D.1 · p. 35

D.2 Cross-Application Example: Weibo and Bilibili

• Cross-application buckets require relay, contrast, or parallel relations rather than concatenating independent tasks, using Weibo’s real-time social signals and Bilibili’s long-form video content.The example pair spans trends, discussion, images, and fan activity on Weibo and analysis, tutorials, replays, and fan creations on Bilibili.Source: §D.2 · p. 35
• For idol support, the agent can check in, post, or boost rankings on Weibo, then watch a Bilibili music video or performance and send supportive bullet comments.The example frames this as parallel fan-support execution across related but independent subtasks.Source: §D.2 · p. 36
• Leisure examples pair Weibo humor or local-destination rankings with Bilibili lifestyle videos or travel guides, while an event example relays convention updates into immersive vlog recaps.The examples span parallel text-image and video consumption as well as relay from fragmented offline-event posts to a full-scene recap.Source: §D.2 · p. 36

E System Prompt

• At each step, Xiaomi-GUI-0 receives the user instruction, interaction history, device and foreground app, and screenshot, then emits reasoning, an action description, and exactly one JSON tool call.The required output uses the ordered XML-like spans <think>, <action>, and <tool_call>, with no extra text.Source: E System Prompt · p. 37
• The system prompt offers GUI actions including tap, long press, swipe, type, search, app open, back, home, and wait, plus request, failure, completion, and answer tools.Failure types include login, permission, network, payment-authentication, and verification conditions; Complete confirms non-Q&A tasks, while Speak presents Q&A answers.Source: E System Prompt · p. 37
• Operational constraints require normalized three-decimal coordinates, dismissal of unrelated pop-ups, and loop correction after three unchanged steps or repeated actions.If self-correction fails, the agent should call Fail.Source: E System Prompt · p. 37
• The reasoning framework calls for screen observation, optional reflection on deviations, a 2–4-step plan with progress markers, an exact action decision, and persistent memory.A failed prior plan triggers Replan.Source: E System Prompt · p. 37

F Case Study

• The agent turns a user instruction into intermediate sub-goals and GUI actions, completing a task on a physical device through its unified action space.The trajectory runs from intent understanding to task completion.Source: F Case Study · p. 38, Figure 10 · p. 39
• When the observed screen deviates from expectations, the agent records the discrepancy, revises its plan, and selects a corrective action instead of continuing its original trajectory.The case illustrates reflection enabling recovery from intermediate uncertainty during interaction.Source: F Case Study · p. 38, Figure 11 · p. 40