PaperlayerView exact revision on arXiv

Source-linked AI summary

SmartSnap: Proactive Evidence Seeking for Self-Verifying Agents

Shaofei Cai, Yulei Qin, Haojia Lin, Zihan Xu, Gang Li, Yuchen Shi, Zongyi Li, Yong Mao, Siqi Cai, Xiaoyu Tan, Yitao Liang, Ke Li, Xing Sun

arXiv:2512.22322v2cs.CLcs.AIcs.CVcs.LGcs.MA

TL;DR

Open-ended GUI tasks are difficult to verify scalably when verifiers must review full, noisy trajectories. SmartSnap trains agents to complete tasks and proactively curate evidence for verification; the reported gains span model families and scales. The paper reports performance gains over 16% across investigated models.

  • Problem

    Open-ended GUI tasks lack success criteria amenable to simple manual rules, while post-hoc VLM review of full trajectories is costly and risks hallucinations and false positives.

    Source: §1 · p. 2

  • Method

    SmartSnap trains Self-Verifying Agents to execute tasks and curate a minimal evidence set, guided by the 3C Principles and verifier feedback on evidence quality.

    Source: §1 · pp. 2–3

  • Results

    All investigated models achieve performance gains over 16%, and the Qwen3-8B and Qwen3-32B agents reach on-par performance with DeepSeek-V3.1 and Qwen3-235B-A22B, respectively.

    Source: §4.3 · p. 8

  • Takeaways & Limitations

    The authors posit that requiring agents to find and present evidence develops task understanding and evidence curation as complementary capabilities.

    Source: §1 · p. 3

    Experiments cover only one benchmark, with extension to more heterogeneous environments left for future work.

    Source: §5 · p. 16

Abstract

from arXiv · show

Agentic reinforcement learning (RL) holds great promise for the development of autonomous agents under complex GUI tasks, but its scalability remains severely hampered by the verification of task completion. Existing task verification is treated as a passive, post-hoc process: a verifier (i.e., rule-based scoring script, reward or critic model, and LLM-as-a-Judge) analyzes the agent's entire interaction trajectory to determine if the agent succeeds. Such processing of verbose context that contains irrelevant, noisy history poses challenges to the verification protocols and therefore leads to prohibitive cost and low reliability. To overcome this bottleneck, we propose SmartSnap, a paradigm shift from this passive, post-hoc verification to proactive, in-situ self-verification by the agent itself. We introduce the Self-Verifying Agent, a new type of agent designed with dual missions: to not only complete a task but also to prove its accomplishment with curated snapshot evidences. Guided by our proposed 3C Principles (Completeness, Conciseness, and Creativity), the agent leverages its accessibility to the online environment to perform self-verification on a minimal, decisive set of snapshots. Such evidences are provided as the sole materials for a general LLM-as-a-Judge verifier to determine their validity and relevance. Experiments on mobile tasks across model families and scales demonstrate that our SmartSnap paradigm allows training LLM-driven agents in a scalable manner, bringing performance gains up to 26.08% and 16.66% respectively to 8B and 30B models. The synergizing between solution finding and evidence seeking facilitates the cultivation of efficient, self-verifying agents with competitive performance against DeepSeek V3.1 and Qwen3-235B-A22B. Code is available at: https://github.com/TencentYoutuResearch/SmartSnap

1 Introduction

SmartSnap makes GUI agents responsible for both completing tasks and proactively proving completion with curated evidence, rather than leaving verification to post-hoc review. Its 3C-guided evidence curation and verifier feedback accompany reported gains from fine-tuning and reinforcement learning.

1 Introduction

  • SmartSnap brings significant success-rate gains over vanilla prompting through fine-tuning and reinforcement learning, with agents competitive with larger LLMs.The figure reports AndroidLab results across model families and scales.Source: Figure 1 · p. 1, §1 · p. 3
  • Post-hoc VLM verification reviews noisy, full interaction trajectories, incurring high API costs and latency while increasing hallucination and false-positive risks.Open-ended task success criteria are difficult to specify with simple manual rules, which are labor-intensive and unscalable.Source: §1 · p. 2
  • SmartSnap introduces Self-Verifying Agents that execute tasks and curate concise, decisive evidence, including through additional evidence-oriented actions in the live environment.The 3C Principles are Completeness, Conciseness, and Creativity.Source: §1 · pp. 2–3
  • SmartSnap shifts verification responsibility from external handcrafted verifiers to the agent, reducing the verifier’s cognitive load under GUI tasks.The stated contribution is to enhance verification efficiency, scalability, and robustness.Source: §1 · p. 3
  • The 3C Principles formalize evidence curation, while verifier feedback evaluates evidence quality and shapes rewards to guide curation skill.The verifier supplies structured, dense feedback on evidence quality as well as task completion.Source: §1 · p. 3

2 Related Works and Preliminaries

Prior GUI-agent work spans supervised learning and reinforcement learning, with approaches seeking to reduce costly data collection and task-specific reward design. The section also introduces GRPO, which updates the policy using rewards relative to a sampled group of trajectories.

2.1 Agents based on Large Language Models

  • GUI-agent training includes demonstration-based SFT and distillation, online filtering of successful trajectories, and RL methods using handcrafted rewards, outcome models, or VLM review.The cited discussion notes that high-quality trajectory collection remains costly and full-trajectory VLM review is expensive and demanding of long-range reasoning.Source: §2.1 · pp. 3–4

2.2 Group Relative Policy Optimization

  • GRPO updates the policy using advantages derived from sampled trajectory rewards relative to the group, without a separate critic network.The advantage uses group reward statistics; the displayed expression gives the group-relative reward term.Source: §2.2 · p. 4

3 Method

SmartSnap models task execution and evidence curation as one agent policy, which selects proof from interaction history and can take additional actions to strengthen it. Its verification framework scores evidence and task completion, while shaping learning toward concise, well-formatted submissions.

3.1 Overall Pipeline and Problem Formulation

  • An augmented MDP lets one policy interact with the environment and submit a final message with curated evidence to a verifier.The verifier scores the instruction, message, evidence, and evaluation guidance.Source: §3.1 · p. 4

3.2 Grounding Evidence in Tool Interactions

  • An exhibit is an action paired with its immediate observation; the agent selects indices for a minimal, decisive evidence set from its interaction history.This replaces subjective self-declarations with direct action–observation evidence.Source: §3.2 · p. 5

3.3 Foundational Principles of Evidence Curation

  • Completeness includes pivotal interactions, while conciseness reduces redundancy and verifier burden; both principles guide the agent through system-prompt instructions.Completeness targets fewer false negatives from missing proof, and conciseness targets fewer false positives and less context rot.Source: §3.3 · p. 5
  • Creativity allows evidence-oriented actions after task completion when history lacks proof, such as inspecting a browser toolbar for a newly installed extension.The paper also illustrates proactive snapshots listing a target transaction as evidence.Source: §3.3 · p. 5, Figure 3 · p. 6
  • The evidence-curation principles distinguish task-oriented actions from additional actions taken solely to produce stronger proof.For example, checking the browser toolbar can verify that an extension loaded when the confirmation dialog disappearing cannot.Source: §3.3 · p. 5

3.4 Evidence Verification and Reward Shaping

  • The verifier rewards relevant evidence, including evidence of failure, but grants completion only for unequivocal, traceable proof and adds format and evidence-size regularizers to the summed reward.The size penalty favors succinct evidence once completeness is met.Source: §3.4 · pp. 6–7
  • The authors optimize the policy with GRPO under the VeRL implementation.Source: §3.4 · p. 7

4 Experiment

Experiments evaluate SmartSnap on AndroidLab and report task-success gains across model families and sizes. The analyses also track evidence curation, interaction efficiency, and performance on difficult app domains.

4.1 AndroidLab

  • AndroidLab evaluation uses 138 tasks across nine apps, while training and cold-start SFT/RL use the same 726 tasks; observations are compressed XML trees.The experiments omit AndroidLab’s built-in rule-based evaluator and add a submit tool for evidence snapshots.Source: §4.1 · p. 7, Table 1 · p. 7

4.2 Implementation Details

  • The study tests four LLM backbones, uses DeepSeek-R1 for evidence-based rewards, and evaluates task completion with GLM4 as an LLM judge.Training uses VeRL for SFT and RL; validation scoring analyzes interaction trajectories.Source: §4.2 · p. 8

4.3 Main Results

  • Over 16% performance gains occur across model families and sizes; SmartSnap Qwen3-8B and Qwen3-32B match DeepSeek-V3.1 and Qwen3-235B-A22B, respectively.The comparison reports task success rate on AndroidLab.Source: §4.3 · p. 8, Table 2 · p. 9
  • Self-Verifying SFT achieves much higher task success than vanilla Android Instruct fine-tuning on the same 726 tasks, without task-distribution expansion.The authors attribute an advantage to evidence curation supporting problem decomposition; vanilla fine-tuning places models near 22% success rate.Source: §4.3 · p. 10
  • SFT and RL improve success on most app categories except Maps.me, while gains track training-task distributions and Maps.me remains nearly unchanged across models.The authors identify task balance and within-app diversity as training-data priorities and describe a knowledge gap for difficult Maps.me tasks.Source: Table 3 · p. 10, §4.3 · p. 10

4.4 Qualitative Analysis

  • During RL, average reward rises as within-group variance falls, which the authors interpret as gradual overfitting to the 726 training tasks.The average number of submitted evidences converges toward 1.5, and interaction turns decrease across model families and scales.Source: §4.4 · pp. 11, 13
  • Validation accuracy improves on most app domains but fluctuates near zero on Calendar, Maps.me, and Zoom, where scarce test samples and weak tactics are cited as factors.The authors identify a potential mobile-domain knowledge gap in these scenarios.Source: §4.4 · pp. 11, 13
  • Case studies show agents learning to use Bluecoins’ floating add button with intermediate evidence snapshots and Settings’ search tool with a final dark-theme snapshot.Before RL, the agents repeatedly mis-tap or enter the wrong settings page; afterward, they select evidence showing task-relevant states.Source: §4.4 · p. 13, Figure 6 · p. 14, Figure 7 · p. 15

5 Conslusion

SmartSnap shifts verification to the agent, which curates minimal, decisive evidence; experiments on AndroidLab report lower LLM-verification costs and higher overall success. The conclusion also identifies limitations in model preparation and benchmark scope.

5 Conslusion

  • On AndroidLab, SmartSnap reduces costly LLM-based verification while enhancing overall task success.The conclusion describes the approach as proactive self-verification guided by Completeness, Conciseness, and Creativity.Source: §5 · p. 16
  • The study omits comprehensive CPT and SFT for preparing an RL-ready starting point; some domains may therefore lack domain-specific knowledge that requires additional training.The authors identify downstream task-corpus collection as a future need for cultivating both task-solving and evidence-creation skills.Source: §5 · p. 16
  • Experiments cover only one benchmark because RL with sandbox environments is resource-intensive; the authors aim to extend evaluation to heterogeneous environments.They name WebShop and WebArena as examples of future benchmarks.Source: §5 · p. 16

Contributions

The supplied section lists the paper’s authors, their affiliations, and an equal-contribution note.

Contributions

  • The author roster names Shaofei Cai, Yulei Qin, Haojia Lin, Zihan Xu, Gang Li, Yuchen Shi, Zongyi Li, Yong Mao, Siqi Cai, Xiaoyu Tan, Yitao Liang, and Ke Li.Shaofei Cai and Yulei Qin are marked as equal contributors.Source: Contributions · p. 17
  • The listed affiliations are Tencent Youtu Lab and the Institute for Artificial Intelligence at Peking University.Source: Contributions · p. 17
  • Shaofei Cai’s work is marked as having been conducted during an internship at Tencent.Source: Contributions · p. 17

A Prompts

The appendix provides prompts for the 3C principles, task instructions, Android interaction tools, and LLM-as-a-Judge evidence verification.

A Prompts

Loading 2512.22322v2…