GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL

Motivation

• Native GUI agents remain less effective at long-horizon decision making because existing data and generic post-training recipes provide weak action-aligned reasoning and mishandle reasoning–grounding trade-offs.Partial verifiability further introduces biased gradients, destabilizes training, and weakens offline metrics as predictors of online success.Source: §1 · pp. 1–2

Conservative RL

• Conservative RL uses GRPO with moderate KL regularization and success-adaptive negative-gradient scaling to stabilize learning under partially verifiable rewards.These constraints address ambiguous failures when multiple actions are valid but offline supervision verifies only one demonstrated action, improving offline–online alignment.Source: §1 · pp. 2–3

Action-aware SFT

• Action-aware SFT mixes reasoning-then-action with direct-action supervision and reweights tokens to prioritize action and grounding, reducing CoT-related grounding degradation.The approach targets the tension in which CoT tokens dominate SFT loss while executable actions depend directly on grounding and action tokens.Source: §1 · pp. 2–3

Data construction

• GUI-Libra releases a curated 81K GUI reasoning dataset built through scalable construction and filtering to improve alignment between reasoning traces and executable actions.The pipeline uses existing open-source resources rather than costly online environment interaction.Source: §1 · pp. 2–3

Results

• +15.6% and +12.2% on AndroidWorld, and +4.0% and +8.7% on Online-Mind2Web, are the gains of GUI-Libra-4B and GUI-Libra-8B over their base models, respectively.The GUI-Libra series also consistently improves offline step-wise accuracy and online completion across AndroidWorld, WebArena-Lite-v2, and Online-Mind2Web.Source: §1 · p. 3

2.1 Datasets for Training GUI Agents

• Grounding datasets provide large-scale annotated screenshots and UI-element supervision for visual perception and screen parsing, including SeeClick (Cheng et al., 2024b), UGround (Gou et al., 2025), GUIAct, ScaleCUA (Liu et al., 2025c), and GUI-360 (Mu et al., 2025).Related visual-supervision resources include Deka et al. (2017), Li et al. (2020b,a), Bai et al. (2021), Wu et al. (2023), Yang et al. (2025a), Zheng et al. (2025b), Wu et al. (2025b), Nayak et al. (2025), and Luo et al. (2025).Source: §2.1 · pp. 3–4
• Context-aware and trajectory datasets model evolving UI states, while AndroidControl (Li et al., 2024) and JEDI add low-level action descriptions for fine-grained GUI policies.Examples include AITW (Rawles et al., 2023), MM-Mind2Web (Zheng et al., 2024; Deng et al., 2023), AMEX (Chai et al., 2025), GUI Odyssey (Lu et al., 2024), and Aria-UI (Yang et al., 2024c).Source: §2.1 · p. 4
• Reasoning-annotated datasets inject observation interpretation and planning, but short, noisy traces limit reliable long-horizon reasoning, recovery, and strategy adaptation, while AgentNet (Wang et al., 2025d) remains primarily desktop-focused.AITZ (Zhang et al., 2024), AgentTreck (Xu et al., 2025a), OS-Genesis (Sun et al., 2024), Aguvis (Xu et al., 2025c), GUI-Net-1M (Zhang et al., 2025a), and WebSTAR (He et al., 2025) provide rationales; AgentNet adds reflective thoughts, but high-quality mobile and web reasoning data remains scarce.Source: §2.1 · p. 4

2.2 VLM Post-training for GUI Agents

• GUI post-training spans SFT, explicit-attention grounding, RL, and SFT+RL hybrids, while unified native GUI models combine grounding, planning, and navigation through SFT-only or RL-based training.Remaining limitations include free-form reasoning hurting grounding accuracy (Zhou et al., 2025b; Yang et al., 2025b; Tang et al., 2025; Lu et al., 2025; Chen et al., 2025b), costly online RL (Wang et al., 2025a; Zhou et al., 2025a; Bai et al., 2026), and ambiguous step-wise RLVR rewards under partial verifiability (Hong et al., 2025).Source: §2.2 · p. 4

VLM-based GUI agents

• A VLM GUI agent receives an instruction and screenshot-based partial observations, conditions on interaction history, and selects executable actions under sparse success-only rewards.The environment is a goal-conditioned POMDP with latent states, action transitions, and termination on success or a maximum horizon.Source: §3 · p. 5

High-level vs. low-level GUI tasks

• Low-level GUI tasks require one atomic interaction, whereas high-level tasks require multi-step trajectories; grounding is low-level coordinate prediction, while this paper focuses on high-level navigation.Examples include typing a URL or clicking a button for low-level tasks, versus purchasing a textbook across multiple screens for high-level tasks.Source: §3 · p. 5

Post-training for GUI Models

• SFT trains GUI models on expert trajectories to generate a reasoning trace followed by an executable action, while RLVR samples candidate actions and applies group-relative policy optimization using rule-based rewards.RLVR incorporates action type, text value, and coordinate matching, but step-wise rewards can be ambiguous because multiple actions may be valid in the same state; explicit KL regularization is controlled by β and is often set to zero in prior work.Source: §3 · p. 5

4 Reasoning Data Curation for GUI Agents

• GUI-Libra introduces an automated pipeline that constructs and filters the 81K-step GUI-Libra-81K reasoning dataset.The pipeline addresses scarce high-quality, action-aligned reasoning data for GUI agents.Source: §4 · p. 5

4.1.2 Unified Structured Format

• Each sample combines screenshots, instructions, interaction history, a CoT trace, and a JSON action with one of 13 common control types.Actions may include targets, descriptions, text values, or coordinates, while descriptions are appended to later interaction history.Source: Figure 2 · p. 6, §4.1.2 · p. 7

4.1.1 Data Sources

• The pipeline aggregates diverse web and mobile trajectories, adds GUIAct’s Chinese subset, and cleans traces to 19K trajectories containing 170K steps.Cleaning removes incomplete trajectories, traces shorter than 3 or longer than 50 steps, and compound actions outside the action space.Source: §4.1.1 · pp. 6–7

4.1.3 Action-aligned Reasoning Augmentation

• Reasoning augmentation uses GUI-specific prompting and model selection, while treating annotated actions as references and allowing justified alternative actions.The generator produces reasoning, descriptions, action types, targets, and values, after which a dedicated filtering stage addresses annotation and reasoning-action mismatches.Source: §4.1.3 · p. 7

4.1.4 Data Filtering for SFT

• SFT filtering combines stochastic action re-prediction with bounding-box verification to remove uncertain coordinates and retain reliable grounding supervision.Steps are discarded below 0.3 re-prediction accuracy, and retained coordinates must fall inside predicted target boxes.Source: §4.1.4 · p. 8
• Filtering yields 81K SFT steps from 9K trajectories, with 14.3% web data and roughly 60% Click actions, motivating RL for rare actions.Mobile datasets provide most SFT data, while LongPress and Select are rare.Source: §4.1.4 · p. 8

4.1.5 Data Filtering for RL

• The RL subset contains 40K steps after downsampling early steps and mobile trajectories to reduce step-index and domain imbalance.The resulting RL data is substantially more balanced than the SFT dataset.Source: §4.1.5 · p. 8

5.1 SFT with Long CoT Hurts GUI Grounding

• Longer responses correlate with worse ScreenSpot-v2 grounding, while long-CoT SFT substantially broadens response lengths and causes severe performance drops.Grounding-only SFT modestly improves performance, SFT without CoT slightly degrades it, and long-CoT SFT causes a substantial drop.Source: §5.1 · p. 9, Figure 4 · p. 9

5.2 Action-Aware Supervised Fine-Tuning

• ASFT mixes reasoning-then-action and direct-action supervision, strengthening action-centric learning while reducing reliance on verbose reasoning during inference.It further reweights action and grounding tokens through αa and αg, recovering standard, CoT-free, and grounding-only SFT as special cases.Source: §5.2 · pp. 9–10

5.3 Reinforcement Learning from Partial Verifiable Rewards

• Offline step-wise matching can poorly predict online task success because multi-step errors cause occupancy shift and single-action verification misses valid alternatives.Theorem 5.1 identifies occupancy mismatch C(π) and off-demo validity mass η̄π as the two governing factors; predictability improves when C(π) is bounded and η̄π is small or stable.Source: §5.3 · pp. 10–11

5.3.1 Why Standard RLVR is Easier to Predict?

• Standard RLVR is easier to predict when it is single-step and fully verifiable, whereas multi-step GUI navigation combines error accumulation with partial verifiability.For H = 1 and ηπ ≡0, one-step matching directly reflects correctness and offline-to-online alignment is substantially tighter.Source: §5.3.1 · p. 11

5.3.2 KL Regularization Improves Predictability

• KL regularization constrains policy drift, simultaneously limiting occupancy mismatch and probability movement away from demonstrated actions so offline matching remains a more stable online-success proxy.This trust-region effect directly targets both quantities identified by Theorem 5.1.Source: §5.3.2 · pp. 11–12

5.4 Success-adaptive Negative Gradient Scaling

• SNGS downweights ambiguous negative policy-gradient updates while preserving reliable positive feedback, reducing over-penalization of valid but uncredited alternatives.Its state-conditioned scaling uses the GRPO group success rate; with κ > 0, scaling increases as the policy concentrates on the demonstrated action and approaches standard GRPO.Source: §5.4 · p. 12

5.5 Reward Function Implementation

• GUI-Libra’s reward combines format validity with action correctness, where correctness factors action type, value similarity, and point grounding.The format weight is wfmt = 0.1, so the step-wise reward mainly emphasizes action correctness while retaining ambiguity for valid but uncredited alternatives.Source: §5.5 · p. 13

5.6 Overall Training Framework for GUI-Libra

• The complete framework applies ASFT in SFT and conservative GRPO in RL, using KL regularization and SNGS to make offline improvements more behaviorally meaningful.This enables scalable optimization from large static datasets without costly online interaction during training.Source: Figure 5 · p. 13, §5.6 · p. 14

6.1 Experimental Setups

• Experiments use refined offline datasets with Pass@1/Pass@4 step accuracy and online benchmarks reporting task success under fixed step budgets, alongside native, proprietary, and modular-agent baselines under a unified protocol.The evaluation includes AndroidControl-v2, MM-Mind2Web-v2, AndroidWorld, WebArena-Lite-v2, and live-site Online-Mind2Web; training uses GUI-Libra-81K for SFT and a downsampled 40K subset for RL.Source: §6.1 · pp. 14, 16

6.2.1 Offline Benchmarks

• On offline benchmarks, GUI-Libra achieves the best Pass@1 results and consistently outperforms corresponding baselines and similarly sized models, including stronger parameter-matched backbones.GUI-Libra-3B improves Pass@1 over Qwen2.5-VL-3B by +20.9 and +14.8 on AndroidControl-v2 high- and low-level tasks, and by +19.3 on average MM-Mind2Web-v2; GUI-Libra-4/8B exceed Qwen3-VL-4/8B by 13.0/9.5 and 8.8/6.7 points on the reported benchmarks.Source: Table 3 · p. 15, Table 4 · p. 15, §6.2.1 · p. 17

6.2.2 Online Benchmarks

• GUI-Libra improves AndroidWorld success from 3.5 to 25.2 (+21.7) at 3B and reaches 42.6 at 8B, while generalizing strongly on locally deployed WebArena-Lite-v2 with only 15K web-related samples.On WebArena-Lite-v2, GUI-Libra-7B improves from 4.9 to 22.6 and GUI-Libra-8B from 15.3 to 26.6, outperforming ScaleCUA-32B, UI-TARS-72B, and Aguvis-72B.Source: §6.2.2 · pp. 17–18

6.3 Action-aware SFT and RL Mitigate Grounding Performance Degradation

• Action-aware SFT mitigates grounding degradation from long CoT outputs, while subsequent RL largely eliminates the remaining reasoning-versus-grounding gap without direct grounding supervision.At 7B, mixed-data SFT raises reasoning-mode grounding accuracy from 79.0% to 81.4%, and action-aware weighting raises it to 83.4%; GUI-Libra-7B reaches 89.3% versus 88.5% in no-reasoning mode.Source: §6.3 · pp. 19–20, Figure 8 · p. 20, Table 8 · p. 20

6.4 On the Effectiveness of KL Regularization for RL

• KL regularization stabilizes RL and substantially improves the predictability of online success from offline evaluation by preventing entropy collapse and reward-hacking-like degradation.KL-regularized models achieve Pearson/Spearman correlations of 0.89 (p < 10^-4) and 0.83 (p = 2×10^-4), versus 0.63 (p = 0.015) and 0.53 (p = 0.053) without KL.Source: Figure 9 · p. 20, §6.4 · pp. 20–21, Figure 10 · p. 21

6.5 Ablations

• Ablations show that cleaner filtered data, mixed supervision and action-aware weighting, and reinforcement learning with moderate KL regularization (KL = 0.001) improve navigation performance and generalization.Filtering yields +4.5 Pass@1 and +6.3 Pass@4 in SFT, and +0.5 Pass@1 and +3.7 Pass@4 in RL; ASFT raises AndroidWorld success from 3.5 to 13.0, while RL raises it to 25.2.Source: Figure 11 · p. 21, §6.5 · p. 22
• Reasoning is especially important for online generalization: removing CoT during training lowers GUI-Libra’s AndroidWorld success from 25.2 to 12.2, while removing it only at inference also degrades RL-trained models.ASFT can sometimes benefit from direct-action inference on in-distribution offline tasks, but GUI-Libra generally relies on the coupled reasoning-and-action traces.Source: §6.5 · pp. 22–23

6.6 RL with Mixed Navigation and Grounding Data

• Mixing direct grounding data improves localization but generally reduces navigation performance, revealing competing optimization pressures between spatial alignment and reasoning-intensive decision making.Grounding accuracy improves by 2–7 points on ScreenSpot-v2 and ScreenSpot-Pro, while MM-Mind2Web-v2 and AndroidControl-v2 performance generally declines.Source: Table 11 · p. 23, §6.6 · pp. 23–24

• The work introduces GUI-Libra as a unified framework built around the curated GUI-Libra-81K dataset.Source: §7 · p. 24
• GUI-Libra achieves strong offline and online results across diverse mobile and web benchmarks without expensive online interaction during training.The framework also offers favorable data and parameter efficiency.Source: §7 · p. 24
• Action-aware SFT preserves grounding under long reasoning traces, while conservative RL improves decision making from partially verifiable feedback by controlling policy drift.Source: §7 · p. 24
• GUI-Libra makes offline evaluation more reliable and more predictive of online task success, supporting deployment of interactive GUI agents in real-world settings.Source: §7 · p. 24

Limitations

• The framework trains on a relatively limited amount of existing open-source data and does not study fully online, interactive training.Scaling to broader, more diverse trajectories and systematically extending the framework to fully online training are left for future work.Source: Limitations · p. 24
• Fully online RL remains a challenging extension because it can be expensive, slow, and infrastructure-intensive.The paper notes that robust infrastructure and careful system design are typically required.Source: Limitations · p. 24

A Additional Related Works

• RLVR replaces potentially misaligned learned reward models with ground-truth verifiers, such as exact answer matching or code execution.This line of work addresses reward hacking and misalignment concerns in RLHF (Ouyang et al., 2022; Wang et al., 2023; Yang et al., 2024b; Gao et al., 2023; Yang et al., 2024a).Source: A Additional Related Works · pp. 31–32
• GRPO, DAPO, Dr. GRPO, and GSPO improve verifiable-reward optimization through policy optimization, clipping, dynamic sampling, and sequence-level importance sampling (Shao et al., 2024; Yu et al., 2025; Liu et al., 2025d; Zheng et al., 2025a).These methods target training effectiveness, bias mitigation, or stability, including for mixture-of-experts models.Source: A Additional Related Works · p. 32
• Directly applying RLVR recipes to step-wise agent training can produce suboptimal policies because of distribution shift and ambiguous intermediate rewards.This limitation distinguishes step-wise agent training from settings with more directly verifiable outcomes.Source: A Additional Related Works · p. 32
• VLM-agent post-training commonly uses SFT followed by RL: curated demonstrations establish grounding, reasoning, and action skills, while interaction improves long-horizon decision-making.SFT can compound errors on out-of-distribution states because its coverage and diversity are limited, whereas RL supports exploration, error recovery, and self-correction (Chen et al., 2025a; Liu et al., 2026; Deng et al., 2025; Bai et al., 2024a; Qi et al., 2025; Putta et al., 2024; Feng et al., 2025; Wang et al., 2025c).Source: A Additional Related Works · p. 32

B Implementation Details

• RL initializes from SFT or ASFT checkpoints and applies GRPO with online rollouts, 300 iterations, global batch size 128, and rollout group size n = 8.KL regularization is 0.001 by default and 0.005 for GUI-Libra-7B, while SNGS uses model-specific parameters.Source: B Implementation Details · pp. 32–33
• The unified action space represents each GUI step as action type, target, value, and optional 2D coordinate, supporting web, mobile, device controls, and termination.Qwen2.5-VL uses absolute pixel coordinates, while Qwen3-VL uses normalized coordinates in [0, 1000].Source: B Implementation Details · p. 32, Table 12 · p. 33
• Inference uses vLLM with temperature 0.0, top-p 1.0, and up to 1024 completion tokens, while available actions can be adjusted to deployment environments.When OpenAPP is unavailable on mobile, the model can instead scroll to access the app drawer.Source: B Implementation Details · p. 33
• Mixed direct-action models can receive an explicit prompt requiring valid JSON action outputs with action descriptions, types, values, and point_2d coordinates.The format uses [-100, -100] when coordinates are not applicable.Source: B Implementation Details · p. 33
• SFT fully tunes Qwen2.5-VL and Qwen3-VL models from 3B–8B with learning rate 1 × 10−5 and effective batch size 256.Training uses two epochs on GUI-Libra-81K or one epoch when mixing reasoning and direct-action data; ASFT defaults to αa = 2 and αg = 4, except GUI-Libra-4B.Source: B Implementation Details · p. 32, Table 13 · p. 34

C Benchmark Details

• The benchmark suite evaluates GUI navigation through step-wise success or task-level completion and separately measures grounding after long chain-of-thought generation.Grounding benchmarks assess whether predicted clicks correctly identify instructed interface elements.Source: C Benchmark Details · p. 34

C.1 Grounding Benchmarks

• ScreenSpot-V2 contains 1,269 corrected-label tasks, whereas ScreenSpot-Pro contains 1,555 higher-resolution tasks that demand more precise grounding.Both use screenshot instructions and bounding-box targets, with success determined by whether the predicted click falls inside the box; Pro reaches 5120×2880 resolution.Source: §C.1 · p. 34

Multimodal-Mind2Web-v2

• MM-Mind2Web-v2 evaluates offline web navigation across 100+ websites using cached multimodal demonstrations and natural-language action-history descriptions.Its Cross-Task, Cross-Website, and Cross-Domain subsets contain 1,328, 1,019, and 1,002 samples, respectively.Source: §C.2 · p. 34
• Step success on MM-Mind2Web-v2 requires both correct element grounding and exact operation execution, including serialized action-string F1.The operation string uses the format “ActionType Value,” with values such as typed text or application/website identifiers.Source: §C.2 · p. 34

AndroidControl-v2

• AndroidControl-v2 filters noisy demonstrations from 500 UGround examples, yielding 398 samples evaluated under high- and low-level instructions.A step succeeds only when action type, textual value when applicable, and target coordinates are all correct.Source: §C.2 · p. 35
• AndroidControl-v2 maps predicted coordinates to nearest accessibility-tree elements, avoiding fixed distance thresholds that are not comparable across differently sized UI elements.This follows UGround’s evaluation strategy rather than protocols used by OS-Atlas and GUI-R1.Source: §C.2 · p. 35

C.3 Online GUI Navigation Benchmarks

• AndroidWorld evaluation covers 115 usable tasks across 20 apps with a 20-step horizon and requires both an explicit Terminate action and a satisfied completion state.Task #82 could not be initialized, so it was excluded; the agent removes step-wise reflection and summary modules to test native model capability.Source: §C.3 · pp. 35–36
• WebArena-Lite-v2 evaluation uses 154 tasks, four deterministic runs, improved action parsing and execution, and GPT-5-based fuzzy evaluation.The setup handles response actions, automated termination, multiline answers, blank-content clearing, and reports averages because of high result variance.Source: §C.3 · p. 36
• Online-Mind2Web evaluates 300 tasks across 136 live websites with a 30-step limit and WebJudge-based binary completion judgments.The judge combines task descriptions, textual actions, completion key points, and selected key screenshots, using either o4-mini or WebJudge-7B.Source: §C.3 · p. 36

D Additional Results

• Additional analyses cover single-step grounding, reasoning-augmentation generators, and complementary offline metrics to clarify behavior under partial verifiability.The grounding study contrasts multi-step navigation, while the reasoning and offline-metric analyses isolate contributors to training and evaluation outcomes.Source: D Additional Results · p. 36

D.1 Grounding as a Single-step Verifiable Setting

• Grounding RL improves steadily before plateauing, with strong ScreenSpot-v2/Pro correlation and Pearson ≈0.98 without KL regularization.Unlike multi-step navigation, single-step grounding shows no offline–online gap or major reward-hacking-style drops, indicating stable predictability even without KL.Source: §D.1 · p. 36, Figure 12 · p. 37

D.2 Comparing Different Models for Reasoning Augmentation

• GPT-4.1 reasoning augmentation outperforms GPT-4o by +3.7 and o4-mini by +2.1 on the MM-Mind2Web Cross-Website subset.The comparison uses identical 30K AGUVIS samples, prompts, Qwen2.5-VL-3B-Instruct fine-tuning, and UGround-7B-v1 coordinate prediction.Source: Figure 13 · p. 37, §D.2 · p. 37

D.3 Comparing with Uniform Negative Gradient Scaling Strategy

• Adaptive negative-gradient scaling outperforms uniform λg = 0.75 and λg = 0.9 across MM-Mind2Web-v2, AC-v2, and AndroidWorld.Uniform scaling cannot reflect heterogeneous state difficulty and ambiguity, whereas adaptive scaling supports more flexible optimization and decision making.Source: Table 15 · p. 38, §D.3 · p. 38

D.4 Additional Metrics on Offline Benchmarks

• The additional metrics decompose step accuracy into grounding and action prediction, using grounding and action-type accuracy on AndroidControl-v2 and grounding accuracy and operation F1 on MM-Mind2Web-v2.Operation F1 follows the protocol in Appendix C.2.Source: §D.4 · p. 38
• 76.3 and 95.5 Pass@1 grounding accuracy make GUI-Libra-8B best on AndroidControl-v2 high-level and low-level tasks, surpassing GPT-5 + UGround-7B-v1 and larger open-weight models.Action-type accuracy follows the same overall trend but shows smaller baseline gaps, indicating that grounding contributes more strongly to the gains.Source: §D.4 · p. 38, Table 16 · p. 39
• 57.8 average Pass@1 grounding accuracy makes GUI-Libra-8B best across all three MM-Mind2Web-v2 subsets, ahead of Qwen3-VL-32B, Qwen2.5-VL-72B, and GPT-5 with UGround.At Pass@4, Qwen2.5-VL-72B is 1.4 points higher than GUI-Libra-8B; the paper attributes this gap mainly to the training-set imbalance of roughly 85% mobile and 15% web data.Source: §D.4 · p. 38, Table 18 · p. 41
• 85.1 Best@1 operation F1 makes GUI-Libra-7B the strongest model on that metric, while GUI-Libra-3B remains competitive with 32B–72B baselines.Best@N reports the maximum F1 over N ∈{1, 4} samples because F1 is continuous; larger models are strong at Best@4.Source: §D.4 · p. 38
• GUI-Libra’s improvements extend beyond step accuracy to its underlying grounding and action components, providing a more complete account of the sources of improvement.The AndroidControl-v2 results specifically suggest that grounding, especially for high-level goal-directed tasks, is more challenging than action-type prediction.Source: §D.4 · p. 38

E Proofs for Theoretical Analysis

• The section gives detailed proofs for the theoretical results stated in Sec. 5.3.The supplied table references include AndroidControl-v2 grounding accuracy, AndroidControl-v2 type accuracy, and Multimodal-Mind2Web-v2 grounding accuracy.Source: E Proofs for Theoretical Analysis · p. 38, Table 16 · p. 39, Table 17 · p. 40

E.0.1 When does offline step-wise matching predict online success?

• Offline matching becomes predictive of online success when occupancy mismatch C(π) is bounded and off-demo validity mass η̄π is small or stable across policies.The offline-to-online relationship follows an affine lower bound under these conditions.Source: §E.0.1 · pp. 41, 43
• Offline matching can fail under distribution shift, error accumulation, or unstable off-demo validity mass because it credits only the demonstrated action while online success accepts any valid action.Large C(π) makes offline scores uninformative about online-dominant states, while varying η̄π makes score changes non-identifiable as correctness improvements.Source: §E.0.1 · p. 43
• A higher offline score can coincide with lower online success: π1 has Moff(π1)=0.2 and J(π1)=0.9, whereas π2 has Moff(π2)=0.4 and J(π2)=0.5.The change reflects probability reallocation between uncredited valid and invalid actions, which offline matching cannot distinguish.Source: §E.0.1 · p. 43
• Offline overfitting to the demonstrated action can reduce behavioral diversity, weaken recovery behaviors, amplify state-distribution shift, and decrease long-horizon online success.Thus Moff(π) may improve while J(π) decreases, especially when early mistakes move the agent into poorly covered states.Source: §E.0.1 · p. 43

E.0.2 KL regularization improves predictability

• KL regularization limits occupancy mismatch C(π) by constraining policy divergence from a reference policy and thereby bounding induced state-visitation drift.This follows from a per-state KL constraint, Pinsker’s inequality, and propagation of action-distribution differences through the dynamics.Source: §E.0.2 · p. 44
• KL regularization also limits off-demo validity mass by preventing excessive probability movement away from the demonstrated action when the reference policy is demo-concentrated.Together, these controls keep optimization in a regime where improvements in Moff(π) are more likely to reflect improvements in J(π).Source: §E.0.2 · pp. 45–46

F.1 Prompt for Reasoning Augmentation

• The reasoning-augmentation prompts combine screenshots, instructions, interaction history, and reference actions to generate initial reasoning traces for the dataset.The prompts were inspired by AGUVIS (Xu et al., 2025c) and refined iteratively during data generation.Source: §F.1 · p. 46
• The templates instruct agents to inspect the screenshot, reason toward the task goal, explain why an action is appropriate, and avoid hindsight-based reliance on reference highlights.They also require first-person reasoning and prescribe click-then-type descriptions for combined interactions.Source: §F.1 · pp. 46–47
• Both web and mobile prompts require exactly one <think></think> plus <answer></answer> pair containing step-by-step reasoning and a structured action response.The reasoning covers screenshot observations, instruction and history context, and the next action decision.Source: §F.1 · pp. 46, 48
• The web action schema records an action description, action type, target element, and value, with fields specialized for clicks, writing, scrolling, keyboard input, termination, and answers.Targets include element identity, type, text, and disambiguating location hints, while values encode exact text, options, keys, or directions.Source: §F.1 · p. 47
• The mobile schema extends the action vocabulary with LongPress, Swipe, NavigateHome, NavigateBack, Wait, OpenAPP, and Answer, while defining corresponding target and value rules.For example, mobile responses can specify wait times, app names, swipe directions, or answer text.Source: §F.1 · p. 48

F.2 SFT Data Example

• The SFT data example represents GUI actions with action type, target, value, and optional 2D coordinates, covering answer, click, write, long-press, scroll, swipe, wait, navigation, app opening, and termination.The example reasons from a Blossom screenshot to click the search bar labeled “Type a plant name” before entering “ihi.”Source: §F.2 · pp. 50–51

F.3 RL Data Example

• The RL data example requires valid JSON action output with screen coordinates and supplies normalized ground-truth action metadata for reward computation.Its ZArchiver example labels the target as the “Extract...” option, with gt_action “Click” and gt_point_2d [0.194, 0.843].Source: §F.3 · p. 53

AndroidWorld case study

• GUI-Libra-7B completes AndroidWorld Task 18 by deleting School Supplies, Religious, and Flight Tickets, whereas Qwen2.5-VL-7B-Instruct fails after deleting only one item.The comparison is shown in Figures 14 and 15.Source: G Long-Horizon Trajectory Case Studies · p. 53, Figure 14 · p. 54, Figure 15 · p. 55
• GUI-Libra sustains multi-step progress by alternating iterative reasoning with grounded actions, unlike the base model’s difficulty discovering and executing successive deletions.This trajectory illustrates the advantage of GUI-Libra for long-horizon decision making.Source: G Long-Horizon Trajectory Case Studies · p. 53

WebArena-Lite-v2 case study

• GUI-Libra-4B successfully completes a long-horizon WebArena-Lite-v2 task, illustrating generalization to multi-step web interactions.The web navigation case study is presented in Figure 17.Source: G Long-Horizon Trajectory Case Studies · p. 53, Figure 16 · p. 56, Figure 17 · p. 57