PaperlayerView exact revision on arXiv

Source-linked AI summary

Task-Adaptive Rubrics for GUI Reward Modeling

Tao Xiong, Xavier Hu, Wenkai Wang, Qinzhuo Wu, Changqiao Wu, Pengzhi Gao, Wei Liu, Jian Luan, Shengyu Zhang

arXiv:2608.24174v1cs.AI

TL;DR

Existing GUI reward verifiers often use generic or implicit criteria that do not adapt sufficiently to each instruction’s concrete requirements. ADAPTRUBRIC constructs task-adaptive criteria through category-level retrieval and instance-level cue generation, improving offline reward evaluation and online task success. Its evaluation is bounded by the tested environments and a fixed offline rubric bank.

  • Problem

    Existing verifiers may use generic rubrics or implicit reasoning that overlook concrete instruction details or enforce unstated requirements.

  • Method

    ADAPTRUBRIC retrieves a reusable category-level rubric, generates compact instance-level cues, and fuses them into one criterion for VLM verification.

  • Results

    ADAPTRUBRIC achieves 86.6 F1 offline, improving F1 by 3.6 points over the baseline average under a matched image budget, and yields a 4.23-point absolute online task-success gain.

  • Takeaways & Limitations

    Across offline and online experiments, explicitly constructing task-adaptive criteria supports reliable GUI reward modeling across GUI interaction settings.

  • Takeaways & Limitations

    Evaluation does not exhaust all applications, interface designs, or instruction styles, and the offline rubric bank remains fixed during evaluation.

Abstract

from arXiv · show

Recent studies on GUI agents have increasingly focused on outcome reward modeling, which assigns outcome rewards by judging whether an executed trajectory satisfies the success criteria implied by the user instruction. Existing GUI reward verifiers, however, often under-specify how these criteria should be constructed for each task instance. Whether using generic rubric structures or implicit model reasoning, their judging criteria are not sufficiently task-adaptive: they can transfer checks across tasks, overlook concrete constraints in the current instruction, or become overly strict by enforcing unstated requirements. To address this limitation, we propose AdaptRubric, a Coarse-to-Fine Rubrics Framework that constructs task-adaptive judging criteria through a category-level coarse stage and an instance-level fine stage. AdaptRubric performs category-level coarse rubric retrieval by routing the instruction to a GUI task family and retrieving reusable task-family criteria, then conducts instance-level fine rubric generation to surface compact cues for concrete values, scopes, and constraints in the current instruction. Across offline reward evaluation and online reinforcement learning optimization, AdaptRubric consistently outperforms prior reward agents, improving F1 by 3.6 points over the baseline average under a matched image budget and yielding a 4.23-point task-success gain.

1 Introduction

Existing GUI reward verifiers often use generic or implicit judging criteria that miss instruction-specific requirements. ADAPTRUBRIC addresses this with coarse-to-fine task-adaptive rubrics and improves offline and online evaluation results.

  • Outcome reward models must identify success criteria implied by the instruction, including goals, constraints, and needed visual evidence.
  • Static rubrics remain generic, while implicit reasoning may overlook instruction details or enforce unstated requirements.
  • ADAPTRUBRIC routes instructions to GUI task categories, retrieves reusable rubrics, and generates compact instance-level cues for concrete requirements.
  • 86.7% accuracy and 86.6 F1 were achieved offline, with F1 improving by 3.6 points over the baseline average under a matched image budget.
  • A 4.23-point absolute gain in online task success rate was obtained using ADAPTRUBRIC as the reward verifier.
  • ADAPTRUBRIC combines category-level and instance-level rubrics into one task-adaptive criterion for VLM-based reward verification.

2 Related Work

GUI agents have progressed from structured interface representations toward screenshot-based multimodal interaction. Outcome reward modeling now commonly uses programmatic evaluators or model-based judges, each with different scalability and precision trade-offs.

  • Early GUI agents commonly relied on DOM/HTML trees and accessibility metadata as structured interface representations.
  • Later GUI agents increasingly use screenshot-based observations, often augmented with Set-of-Mark-style representations.
  • Programmatic or rule-based evaluators are precise for deterministic checks but costly to design and difficult to scale to open-ended GUI tasks.
  • Model-based evaluators estimate task success from screenshots, action histories, final states, or UI metadata.
  • LLM-as-a-judge methods feed trajectories and instructions into a model to obtain binary or graded rewards.

3 Preliminary

A GUI trajectory is a sequence of screenshots and actions generated from an instruction until termination. An outcome reward model maps the instruction and trajectory to a binary success judgment, using an explicit criterion for task-adaptive verification.

  • A GUI agent receives an initial screenshot, executes actions, and observes successive visual states until termination.
  • The completed trajectory is represented as τ = (s1, a1, s2, a2, . . . , sT, aT, sT+1).
  • Each st is a GUI screenshot, each at is an executed action, and sT+1 is the terminal screenshot after the final action.
  • An outcome reward model maps the instruction and trajectory to a binary success judgment.
  • Task-adaptive verification derives an explicit criterion R for the current instruction before assigning reward.
  • The verifier uses a selected trajectory context τ′ ⊆ τ together with R to judge whether the trajectory satisfies the specified success criteria.

4 Method

ADAPTRUBRIC builds task-adaptive judging criteria through category-level rubric retrieval and instance-level fine-rubric generation. It fuses these criteria with selected trajectory screenshots before unchanged verifier architecture produces a binary reward.

  • Category-Level Coarse Rubric Retrieval: ADAPTRUBRIC constructs a category-level rubric bank from successful and failed development trajectories, grouping verification dimensions into eight GUI task categories.The categories include info_query, create_modify, delete_cleanup, communication, transfer, state_navigation, composite_workflow, and general.
  • Category-Level Coarse Rubric Retrieval: At inference time, a task router predicts the instruction’s category and retrieves its corresponding reusable rubric, falling back to the general entry when necessary.The coarse rubric specifies verification steps, common pitfalls, special rules, and output-format requirements.
  • Instance-Level Fine Rubric Generation: The fine stage generates a compact list of additional checks grounded in explicit phrases, values, or constraints from the current instruction.The generator receives the instruction, predicted category, and retrieved coarse rubric; it may abstain when no reliable instance-level item is needed.
  • Criterion Fusion and Reward Verification: Criterion fusion preserves the coarse rubric as the main body and appends the fine rubric as a separated task-specific block, or retains only the coarse rubric after abstention.This preserves category-level verification boundaries while adding instruction-specific checks when available.
  • Criterion Fusion and Reward Verification: Before verification, the system selects screenshot-bearing trajectory steps, including initial and final states and frames around high-signal actions.The instruction, selected context, and fused criterion are passed to a verifier that outputs a parsable binary reward, while the verifier architecture remains unchanged.

5 Experiment

Experiments evaluate ADAPTRUBRIC across offline reward discrimination, online reinforcement learning, and reward-guided test-time scaling under controlled comparisons. Results show stronger trajectory judgment and downstream task selection, while ablations and a failure case support the complementary value of coarse and fine rubrics.

  • Offline Evaluation: OGRBench contains 1,409 trajectories from five environments, with 700 positive and 709 negative trajectories, and evaluates Accuracy, Precision, Recall, and F1.All offline verifiers use a matched ten-screenshot budget, with Qwen, Qwen3.5, Qwen3.6, and Gemini judge backbones evaluated.
  • Offline Evaluation: 86.7% accuracy and 86.6 F1 are ADAPTRUBRIC’s best average OGRBench results, improving F1 by 3.6 points over the two-baseline average.Recall rises from the baseline average of 80.2% to 86.5%, while precision remains comparable at 86.8%.
  • Offline Evaluation: ADAPTRUBRIC achieves the best accuracy on all tested backbones and the best F1 on five of six Qwen-family backbones under the matched ten-screenshot budget.The expanded comparison includes DigiRL, DistRL, AndroidGen, and WebRL in addition to ZeroGUI and OS-Themis.
  • Online Reinforcement Learning: 23.93% task success is achieved with ADAPTRUBRIC in online reinforcement learning, a 4.23-point absolute gain over the 19.70% no-external-reward result.The training protocol keeps the policy, environment, and optimization procedure fixed while replacing only the reward agent.
  • Reward-Guided Test-Time Scaling: On the 113-task AndroidWorld pool, ADAPTRUBRIC improves EarlyStop@7 by +11.88 points and BestOfN@8 by +13.28 points over Random, with 88.14 accuracy and 88.41 F1.Its false-positive rate is 11.17, close to OS-Themis; EarlyStop reaches 64.60% task success at N=7, while BestOfN reaches 65.49% at N=8.
  • Ablation and Analysis: Removing either the Coarse or Fine rubric lowers F1 by 2.0 points, while removing both lowers F1 by 5.9 points on Qwen3.5-122B-A10B.A representative failure case shows baselines accepting text placed below existing note content despite the instruction requiring placement at the top.
  • Efficiency: ADAPTRUBRIC achieves 84.0 Acc. and 84.2 F1 with fewer calls, tokens, and runtime than OS-Themis and ZeroGUI, while DigiRL is cheaper but 3.4 F1 points lower.The comparison identifies ADAPTRUBRIC as having the strongest quality–cost trade-off for repeated reward verification.

6 Conclusion

ADAPTRUBRIC is a coarse-to-fine framework for constructing task-adaptive GUI outcome-reward criteria. Its category-level rubric anchors verification to task-family criteria, while its instance-level rubric specifies requirements for the current instruction.

  • ADAPTRUBRIC combines category-level and instance-level rubrics into a task-adaptive criterion for GUI outcome reward modeling.
  • The Coarse rubric anchors verification to task-family success criteria, while the Fine rubric specifies criteria needed for the current instruction.
  • ADAPTRUBRIC consistently demonstrates the value of explicitly constructing task-adaptive success criteria across offline and online experiments.

Limitations

The evaluation covers mobile, desktop, and web environments but does not exhaust all applications, interface designs, or instruction styles. Its coarse rubric bank is built offline and fixed during evaluation, limiting coverage of newly emerging applications and domain-specific workflows.

  • The evaluation does not cover all possible applications, interface designs, or user instruction styles.
  • The coarse rubric bank is constructed offline and kept fixed throughout evaluation for reproducibility and controlled comparison.
  • A fixed offline rubric bank may not capture every newly emerging application or domain-specific workflow.

C Online RL Training Details

Online reinforcement learning experiments use ClawGUI-RL with GRPO on MobileWorld. Compared reward agents share the same policy backbone and training protocol, with only the reward verifier changed.

  • Online reinforcement learning experiments use ClawGUI-RL with GRPO on MobileWorld.
  • All compared reward agents use the same policy backbone and training protocol, changing only the reward verifier.
  • Table 7 summarizes the key hyperparameters for online reinforcement learning training.

D Image Budget Analysis

Trajectory context and image budget affect GUI reward verification performance. Increasing the image budget improves ZeroGUI mainly by raising recall, motivating matched-budget comparisons.

  • The main experiments use at most ten trajectory screenshots, compared with ZeroGUI’s original final-two-screenshot setting.
  • Increasing ZeroGUI’s image budget mainly improves recall.
  • Increased trajectory context substantially improves several baselines, motivating the matched-budget comparison in Figure 3.

E Additional Baselines under Matched Image Budget

This section details the expanded matched-budget comparison, using controlled trajectory-context and judge-backbone settings to evaluate additional baselines. Table 11 reports the full results, including ZeroGUI and OS-Themis for reference.

  • Every baseline receives ADAPTRUBRIC’s ten-screenshot trajectory budget and the same Qwen-family judge backbones for controlled evaluation.
  • ADAPTRUBRIC maintains the strongest mean overall F1 across the expanded baseline set.ZeroGUI and OS-Themis are included for reference.

F Reward-Guided Test-Time Scaling Details

The reward-guided scaling evaluation uses a shared AndroidWorld trajectory pool and controlled candidate ordering to compare verifier-based selection. The protocol defines EarlyStop@N, BestOfN@N, Random@N, and Oracle@N over common candidate prefixes.

  • Reward-guided trajectory selection is evaluated on 113 AndroidWorld tasks using complete rollout traces and ground-truth labels.A pre-collected pool simulates online repeated attempts so verifiers compare the same candidate trajectories.
  • The heterogeneous pool is necessary because mixed tasks are the only cases where verifier-guided selection can change final task success.The 8B-only pool has 18.6% mixed tasks, while the 235B-only pool has 13.3%.
  • Per-task shuffling prevents source-model ordering from dominating EarlyStop@N and hides source-model metadata from reward methods.A fixed seed-42 shuffle places a 235B trajectory in each position with approximately 20% probability.
  • EarlyStop@N scans candidates until the first predicted success, whereas BestOfN@N scores the entire prefix before selecting a predicted-success trajectory.Random@N samples uniformly from the prefix, and Oracle@N provides the oracle reference.
  • Matched-budget comparisons align baselines with ADAPTRUBRIC using at most ten trajectory screenshots.Table 10 describes the context-budget alignment.

G Detailed Case Study

A representative AndroidWorld case shows how generic verifiers can reward an incomplete edit when they check content presence but miss required ordering. ADAPTRUBRIC combines category-level structure with instance-specific cues to judge the final state correctly.

  • Case study: The case asks the agent to place “Hello, World!” at the top of note_SiFbv.txt, but the original sentence remains above it, making the ground-truth label failure.
  • Baseline errors: ZeroGUI and OS-Themis assign false-positive success rewards because they verify text presence without correctly checking the note’s final ordering.
  • AdaptRubric judgment: ADAPTRUBRIC routes the task to create_modify and adds the instruction-specific top-placement cue to the final-state check.
  • Ablation: The coarse-only verifier misses exact placement, while the fine-only verifier misreads the final order despite receiving the positional cue.
  • Ablation: Only the fused coarse-to-fine criterion combines structured final-state verification with the positional constraint and produces the correct failure judgment.
  • Rubric bank: The rubric bank stores task-family sections for verification steps, common pitfalls, special rules, and output format, plus named rubric items for structured ablations.
  • Rubric families: The create_modify rubric checks explicitly stated properties in the end state and warns against inventing filename, folder, toast, or UI-path requirements.
  • Rubric families: Other rubric families address deletion, communication, transfer, and state navigation through checks for target scope, confirmation, fidelity, persistence, and final-screen match.
Loading 2608.24174v1…