Source-linked AI summary
HiconAgent: History Context-aware Policy Optimization for GUI Agents
Xurui Zhou, Gongwei Chen, Yuquan Xie, Zaijing Li, Kaiwen Zhou, Shuai Wang, Shuo Yang, Zhuotao Tian, Rui Shao
TL;DR
GUI agents need useful historical context, but full visual histories increase computational cost and may include irrelevant information. HiconAgent trains history-aware policies with variable-length sampling and action-anchored history compression. It reports improved benchmark performance alongside lower computational cost, including gains over a larger agent on GUI-Odyssey.
Problem
GUI agents must use historical context for navigation, but omitting past observations can discard useful visual cues while retaining full history raises computational overhead.
Source: §1 · pp. 1–2Method
HCPO combines variable-length history sampling with dual-branch policy updates that compress history observations while retaining action tokens as anchors.
Source: §1 · p. 2, §5.2 · p. 6Results
HiconAgent-3B reports +11.32% step successful rate on GUI-Odyssey over GUI-R1-7B, alongside 2.47× computational speedup and 60% FLOPs reduction.
Source: §1 · p. 2Takeaways & Limitations
The authors present HiconAgent as a practical path toward lightweight, high-performance GUI agents.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Graphical User Interface (GUI) agents require effective use of historical context to perform sequential navigation tasks. While incorporating past actions and observations can improve decision making, naive use of full history leads to excessive computational overhead and distraction from irrelevant information. To address this, we introduce HiconAgent, a GUI agent trained with History Context-aware Policy Optimization (HCPO) for efficient and effective utilization of historical information. HCPO optimizes history usage in both sampling and policy updates through two complementary components: (1) Dynamic Context Sampling (DCS) presents the agent with variable length histories during sampling, enabling adaptive use of the most relevant context; (2) Anchor-guided History Compression (AHC) refines the policy update phase with a dual branch strategy where the compressed branch removes history observations while keeping history actions as information flow anchors. The compressed and uncompressed branches are coupled through a history-enhanced alignment loss to enforce consistent history usage while maintaining efficiency. Experiments on mainstream GUI navigation benchmarks demonstrate strong performance. Despite being smaller, HiconAgent-3B outperforms GUI-R1-7B by +8.46 percent grounding accuracy and +11.32 percent step success rate on GUI-Odyssey, while achieving comparable results on AndroidControl and AITW with up to 2.47x computational speedup and 60 percent FLOPs reduction.
1. Introduction
GUI reinforcement-learning agents face a trade-off between useful visual history and the computational cost of retaining full histories. HCPO addresses this by varying history during sampling and compressing observations during policy updates; HiconAgent-3B reports gains over larger agents with lower computation.
1. Introduction
- Prior GUI RL methods often omit history observations to reduce cost, but those screenshots can help resolve ambiguity, ground similar elements, and maintain temporal consistency.Including complete history instead raises computational overhead from attention's quadratic complexity and high-resolution visual tokens.Source: §1 · pp. 1–2
- HCPO improves both sampling and policy updates through Dynamic Context Sampling (DCS) and Anchor-guided History Compression (AHC).DCS uses varied history lengths; AHC compresses observations while retaining history actions as anchors and aligning compressed and uncompressed branches.Source: §1 · p. 2
- The framework targets effective history use while reducing redundant context and its computational burden.AHC drops history observations after early fusion, retains action tokens, and aligns the compressed branch with the uncompressed branch using a history-enhanced loss.Source: §1 · p. 2
- +11.32% step successful rate on GUI-Odyssey, with 2.47× computational speedup and 60% FLOPs reduction, is reported for HiconAgent-3B.The paper also reports consistent outperformance of existing reinforcement-learning agents across three GUI navigation benchmarks.Source: §1 · p. 2
2. Related Work
Related work describes both the value and cost of historical information in GUI agents and the use of rule-based reinforcement learning. It identifies effective history use as an underexplored concern in GUI RL.
2.1. GUI Agents and History Utilization
- Full visual history can improve GUI agents but costs more computation, especially for long-horizon or high-resolution tasks; prior work also finds benefits from past actions [7] [29].The paper frames selective or compressed historical context as a response to this efficiency trade-off.Source: §2.1 · p. 2
2.2. Rule-based Reinforcement Learning
3. Preliminaries
The preliminaries frame GUI interaction as sequential action selection conditioned on an instruction, current screenshot, and a history window. They also introduce the group-based response setup used for GRPO training.
3. Preliminaries
- Figure 2 indicates that the reward-maximizing history length differs across samples and that more history is not always optimal.The figure evaluates different history lengths τ and selects the one with the highest mean reward for each sample.Source: Figure 2 · p. 3
- A GUI agent completes a high-level task through a sequence of low-level actions conditioned on the instruction, current screenshot, and preceding history.The interaction is formulated as a Markov Decision Process, with τ denoting the number of previous steps included in the history window.Source: §3 · pp. 2–3
- For each input q, GRPO generates a group of candidate responses, each containing a thought process and corresponding action.The objective uses a clipping parameter ϵ, hyperparameter β, and normalized group-reward advantages representing relative response quality.Source: §3 · p. 3
4. Rethinking History Usage: Limitations of Fixed Context and the Anchoring Role of Actions
The section finds that useful history length varies by sample and that action tokens anchor the flow of visual history through the model. These findings motivate adaptive context lengths and compressing visual history only after early fusion.
4.1. Analyzing the Impact of History Length
- Different samples can favor different history lengths, and shorter histories can outperform longer ones when added context introduces irrelevant information.The rollout analysis evaluates τ ∈ {0, 1, 2} and identifies the per-sample history length with the highest mean reward.Source: §4.1 · p. 3, §4 · p. 3
4.2. History Actions as Information Flow Anchors
- In shallow layers (k<12), dropping history actions causes a larger performance decline than dropping history images, indicating that actions anchor useful visual cues.The analysis reports that later layers cannot directly extract effective cues from retained visual history without those action anchors.Source: §4.2 · p. 4
- At deeper drop depths, the action-drop and image-drop curves approach one another; for k ≥24, all three curves converge to no-compression accuracy.This pattern is presented as evidence that most historical information has already been integrated by then.Source: §4.2 · p. 4
- History images contribute through intermediate-depth interactions with action tokens, which deliver extracted visual information to subsequent tokens.The analysis cautions that history actions alone provide limited benefit without their paired screenshots.Source: §4.2 · p. 4
- The proposed compression rule prunes history images after early fusion while retaining actions, reducing sequence length and computation cost.Training strengthens cross-modal interaction in the first k layers so relevant visual context transfers into the retained action tokens before compression.Source: §4.2 · p. 4
- HCPO combines variable history lengths during sampling with dual-branch updates that share sampled responses and group-wise advantages.The compressed branch is trained with policy gradients and aligned to the uncompressed branch using a history-enhanced alignment loss.Source: Figure 4 · p. 5
5. HiconAgent
HCPO improves GUI agents’ use of historical context during sampling and policy updates. Its components adapt history lengths, compress redundant visual history while retaining actions, and align compressed and uncompressed branches.
5. HiconAgent
- HCPO improves history utilization in GUI agents by strengthening both sampling and policy updates through Dynamic Context Sampling and Anchor-guided History Compression.Source: §5 · p. 5
5.1. Dynamic Context Sampling
- Dynamic Context Sampling exposes the policy to varied history lengths and shifts sampling toward larger contexts as training progresses.The sampled responses are evaluated under a unified full-history context during optimization.Source: §5.1 · p. 5
5.2. Anchor-guided History Compression
- Anchor-guided History Compression removes history vision tokens while retaining past actions, then aligns compressed-branch outputs with detached full-history guidance.The compressed branch uses the uncompressed branch’s responses and advantages.Source: §5.2 · p. 6
5.3. Reward Design
- Reward design combines format compliance, exact action-type matching, and action-value feedback based on the action’s value format.Text values use F1 with a threshold above 0.5, discrete values require exact matches, and coordinate actions receive a continuous reward of 1 − d.Source: §5.3 · p. 6
6. Experiments
Experiments evaluate HiconAgent on three GUI navigation benchmarks and compare it with supervised and reinforcement-learning baselines. Results show gains on GUI-Odyssey and OOD evaluation, while ablations examine history sampling, compression, and computational cost.
6.1. Implementation Details
- Evaluation uses test splits in an OOD setting on AndroidControl-High, AITW, and GUI-Odyssey, reporting Type, Grounding, and SR.HiconAgent-3B is based on Qwen2.5-VL-3B; experiments use a history window of up to two past interaction steps.Source: §6.1 · pp. 6–7
6.2. Experiment Results
- +5.85% SR over GUI-R1-3B and +0.73% SR over GUI-R1-7B: HiconAgent-3B improves AndroidControl results under the same training setup.Source: §6.2 · p. 7
- +8.46% grounding accuracy and +11.32% step successful rate: HiconAgent-3B outperforms GUI-R1-7B on GUI-Odyssey despite having less than half its parameters.The table compares performance on AndroidControl-High and GUI-Odyssey.Source: Table 1 · p. 7, §6.2 · p. 7
- 51.47% average step successful rate: HiconAgent achieves the highest average across three OOD benchmarks using 3K unfiltered training samples.The comparison includes models trained with larger filtered data volumes.Source: §6.2 · pp. 7–8, Table 2 · p. 7
6.3. Ablation Study
- Exponential-biased DCS sampling gives the best reported performance-cost trade-off; uniform sampling degrades short-history response quality as training proceeds.Uniform sampling updates parameters using τ = 2 contexts, while including all τ values in optimization partially recovers performance but adds training overhead.Source: §6.3 · p. 8, Table 3 · p. 8, Figure 5 · p. 8
- The AHC ablation performs best when DCS is enabled, after adding an uncompressed branch and KL alignment to the compressed branch.Training only the compressed branch yields the weakest performance; the table evaluates SR with compression enabled.Source: §6.3 · p. 8, Table 4 · p. 8
- 59.54% FLOPs reduction and 2.47× computational speedup: k = 6 balances computation with competitive performance.Earlier dropping reduces FLOPs more but degrades performance across metrics.Source: §6.3 · p. 8, Table 5 · p. 8
- HCPO improves most action categories, especially finished, scroll, and type actions; examples show correct Delhi entry and Red Chief selection where the base model fails.Source: §6.3 · pp. 8–9, Figure 7 · p. 9, Figure 6 · p. 9
7. Conclusion
The conclusion presents HiconAgent as a history-aware GUI agent whose HCPO training combines DCS and AHC, drawing on findings about history length and action anchors.
7. Conclusion
- Pairing DCS and AHC lets HiconAgent outperform larger models with fewer FLOPs.The paper presents HiconAgent as a practical path toward lightweight, high-performance GUI agents.Source: §7 · p. 9
- Different decision steps prefer different history lengths, according to the paper’s empirical investigations.The conclusion reports this finding as motivation for adaptive history use.Source: §7 · p. 9
- Historical actions serve as information-flow anchors in the paper’s account of GUI-agent history use.Source: §7 · p. 9
A. Visualization of SSR curve during training
The training-accuracy results favor DCS over fixed-length histories, while its sampling schedule shifts from near-uniform exploration toward longer histories as training progresses.
A. Visualization of SSR curve during training
- DCS achieves higher final execution accuracy and faster convergence than the AHC-only baseline.The training curves show improvement in both training dynamics and final accuracy.Source: A. Visualization of SSR curve during… · p. 13
- DCS sampling shifts from near-uniform exploration early in training toward an exponential bias favoring larger history lengths.As λ(u) grows, the distribution increasingly favors larger τ_i values and approaches full-context history.Source: B. Visualization of sampling… · p. 13, Figure 9 · p. 13
- The schedule increases λ from 0 to λmax during the first αT steps; implementation sets λmax = 2 and α = 1.Source: B. Visualization of sampling… · p. 13
C. GUI datasets
The paper describes datasets used for GUI-agent training and evaluation, including their scale and interaction scope.
C. GUI datasets
- Training data come from 3,000 randomly sampled AMEX steps, with the original action distribution preserved and no additional filtering or cleaning.AMEX contains more than 104K high-resolution screenshots from 110 Android apps, with multi-level annotations.Source: C. GUI datasets · p. 13
- AndroidControl contains 15,283 demonstrations across 14,548 unique tasks and 833 Android apps, with high- and low-level instructions.Source: C. GUI datasets · p. 13
- GUI-Odyssey covers cross-app mobile workflows, while AITW comprises 715,000 Android-control episodes with more than 30,000 unique instructions.The paper reports GUI-Odyssey as 7,735 episodes across 201 apps and about 1.4K app combinations; AITW spans eight device types and four Android versions.Source: C. GUI datasets · pp. 13–14
D. Effect of history observation and compression
The section reports gains from history-aware compression, outlines the HCPO update procedure, and gives the evaluation prompt format.
D. Effect of history observation and compression
- HiconAgent-3B improves SR by +5.06 % over the compressed 2AO baseline and exceeds the uncompressed 2AO baseline in SR and grounding accuracy.Adding visual history to the 2A model yields +8.96% SR, while inference-only compression of 2AO reduces SR by -4.95%.Source: Table 6 · p. 14, D. Effect of history observation and… · p. 14
E. Algorithm details
- HCPO reuses sampled responses for uncompressed and compressed policy passes, dropping history vision tokens after layer k while retaining action tokens and aligning branches with a token-level consistency term.The uncompressed branch supplies detached teacher logits for the consistency term; the resulting objective combines both policy losses and the KL loss.Source: E. Algorithm details · pp. 14–15
F. Prompts for training and evaluation
- Evaluation prompts instruct agents to reason from the task, current image, and history, then emit reasoning in <think> tags and an action in <action> tags.The action space varies by evaluation prompt.Source: F. Prompts for training and evaluation · p. 15
G. Model Behavior Across Different History Lengths
The case study contrasts how the base model and HiconAgent-3B respond to longer histories: extra observations distract the base model at τ = 2, while the DCS-trained model remains correct.
G. Model Behavior Across Different History Lengths
- At τ = 2, distracting or misleading observations lead the base model to attend to the wrong UI element and produce an incorrect action, despite correct behavior at τ = 0 or 1.The case study compares the base model and HiconAgent-3B across τ ∈ {0, 1, 2}.Source: G. Model Behavior Across Different… · p. 16
- HiconAgent-3B still produces the correct action at τ = 2 after training with Dynamic Context Sampling.DCS exposes the agent to diverse, progressively biased history lengths during optimization.Source: G. Model Behavior Across Different… · p. 16
- The case study supports the conclusion that naively increasing history is suboptimal, while HCPO provides robustness across variable context windows and lets the agent benefit from longer history when needed.The authors present this as qualitative evidence supporting their quantitative results.Source: G. Model Behavior Across Different… · p. 16
H. Visualization of successful trajectories
The paper presents representative successful GUI-navigation trajectories and identifies a separate case study of model behavior across history lengths.
H. Visualization of successful trajectories
- Figure 11a and Figure 11b visualize representative successful trajectories from evaluation benchmarks.The trajectories illustrate the proposed HCPO framework's role in robust decision-making.Source: H. Visualization of successful… · p. 16
- Figure 10 presents a case study of model behavior under different history-length contexts.Source: Figure 10 · p. 17
- Figure 11 is described as case studies of the model on downstream GUI navigation tasks.The caption does not specify the trajectory encoding or report a comparison outcome.Source: Figure 11 · p. 17