GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness

1 INTRODUCTION

• High-resolution screenshot histories make GUI-agent inference costly, and storing all KV states can exceed 80GB for a five-screenshot UI-TARS-1.5-7B inference.Existing compression methods had not been established for GUI tasks despite their spatial and temporal redundancy.Source: §1 · pp. 1–2
• GUI screenshots show uniformly extreme attention sparsity across layers (> 0.99), motivating equal KV-cache budgets per layer instead of layer-varying schedules.The paper reports that varying schedules can amplify small sparsity differences and misallocate cache.Source: §1 · p. 2
• GUI-KV combines hidden-state L2-norm saliency guidance with temporal scoring that uses QR decomposition to identify redundant visual information in prior frames.The two components determine spatio-temporal token saliency.Source: §1 · p. 2
• Across six GUI benchmarks, GUI-KV often recovers near-full-cache accuracy at 10–20% budgets and matches or outperforms the best competing compression baselines.With five screenshots at a 40% budget, it reduces FLOPs per decoded token by 38.9% while improving step accuracy by +4.1%.Source: §1 · p. 2
• With five screenshots at a 40% budget, GUI-KV reduces FLOPs per decoded token by 38.9% while improving step accuracy by +4.1%.Prefilling overhead increases by less than 0.29%.Source: §1 · p. 2

2.1 GUI AGENT PROBLEM DEFINITION

• A GUI-agent task is modeled as a POMDP: the agent acts from partial observations and a goal, the environment transitions, and the agent receives rewards.The formulation uses states, actions, transitions, rewards, and observations.Source: §2.1 · p. 2

2.2 KV CACHE IN VISION-LANGUAGE MODELS

• During prefill, visual and language inputs are encoded and attention key-value states are cached; autoregressive decoding appends new states, so long histories can cause memory bottlenecks.Compression selects a subset of the full cache through layer-budget allocation and token-selection policies.Source: §2.2 · p. 3
• For a cache budget γ, token scoring ranks input tokens by importance, and the selection rule retains the top-scoring indices up to the budget.The scoring uses attention received from the last ω input tokens.Source: §2.2 · p. 3

2.3 HIGH SPATIAL REDUNDANCY IN GUI

• ScreenSpotV2 GUI attention sparsity averages above 0.99 across layers and remains nearly flat, versus natural-image averages of 0.92 and minima around 0.88.GUI sparsity has lowest values around 0.98, while natural-image sparsity decreases in deeper layers.Source: §2.3 · p. 4
• On ScreenSpotV2, SnapKV’s uniform allocation significantly outperforms PyramidKV and VL-Cache, with the gap widening at 5–20% cache budgets.The comparison supports uniform layer budgets for GUI settings with consistently high attention sparsity.Source: §2.3 · p. 4, Figure 2 · p. 4
• The temporal-scoring illustration projects prior-frame keys onto the current screenshot’s QR subspace; larger orthogonal residuals indicate less-redundant tokens.The example identifies k3 and k4 as less redundant than k1 and k2 because their residuals are larger.Source: §2.3 · p. 5, Figure 3 · p. 5

3 GUI-KV

• GUI-KV combines spatial saliency guidance with temporal redundancy scoring to compress GUI-agent KV caches.The method is designed to retain important visual tokens while removing redundant history.Source: §3 · p. 5, §3.3 · p. 6

3.1 SPATIAL SALIENCY GUIDANCE

• Residual-stream hidden-state norms supplement attention scores to identify visual tokens with strong content and contextual relevance.The method standardizes and softmaxes token norms, then adds the resulting saliency weights to attention scores; the additive form supports numerical stability and easier calibration.Source: §3.1 · p. 5

3.2 TEMPORAL REDUNDANCY SCORING

• Temporal redundancy scoring compares earlier-frame keys with a low-dimensional subspace of the latest image’s keys, favoring tokens that add new information.Keys whose information lies largely in the latest image’s subspace are treated as redundant.Source: §3.2 · p. 6

3.3 SPATIAL AND TEMPORAL COMBINATION

• The combined scoring retains visual tokens that are both important and temporally unique, filtering earlier-frame tokens that duplicate the current frame.The method scores previous-frame tokens for redundancy and combines that signal with spatial saliency before selecting tokens.Source: §3.3 · p. 6

4.1 EXPERIMENTAL SETTINGS

• The evaluation covers visual grounding, offline action prediction, and online task completion using two GUI agent models, benchmark-specific metrics, and three cache-compression baselines.The models are UI-TARS-1.5-7B and OpenCUA-7B; baselines are SnapKV, PyramidKV, and VL-Cache.Source: §4.1 · pp. 6–8

4.2 ACCURACY EVALUATION

• GUI-KV consistently matches or surpasses the best competing compression schemes across six benchmarks, often recovering near-full-cache accuracy with 10–20% budgets.On four of six benchmarks, its average absolute gains over the best baseline range from +0.7 to +3.6.Source: Table 1 · p. 7, §4.2 · p. 8
• UI-TARS-1.5-7B retains more full-cache performance under compression than OpenCUA-7B, which is more sensitive to KV eviction and typically needs more budget.On AndroidControl, GUI-KV step accuracy is 41.2% at 10% and 46.0% at 20% for UI-TARS-1.5-7B, versus 23.8% and 23.3% for OpenCUA-7B.Source: §4.2 · p. 8

4.3 EFFICIENCY ANALYSIS

• At γ = 40% with 5 screenshots, GUI-KV reduces MFLOPs per decoded token by 38.9% and increases step accuracy by 4.1% over the full-cache baseline.Prefill overhead is below 0.29% across settings, while decoding compute drops substantially.Source: §4.3 · pp. 8–9, Table 2 · p. 8

4.4 ABLATION STUDIES

• Combining spatial saliency and temporal redundancy scoring performs best across screenshot counts; spatial-only guidance helps more at ≤7 screenshots, while temporal-only excels as screenshots increase.At 20% cache budget and 7 screenshots, GUI-KV matches the full-cache counterpart; performance peaks at 7 screenshots for all methods.Source: Figure 4 · p. 8, §4.4 · p. 9

• Visual-token reduction includes architectural compression and adaptive pruning based on token importance, with some approaches extending these ideas to GUI agents.The passage describes pooling projectors, attention-based importance scores, and pruning across model layers.Source: §5 · p. 9
• Post-training KV-cache compression spans token eviction or merging, static layer-wise reduction, and quantization, with methods trading retained context, layer allocation, or precision.The passage also notes that many methods focus on text, while multimodal approaches may use fixed allocation or dynamic layer allocation.Source: §5 · p. 9
• Existing approaches leave multimodal challenges and GUI-specific spatial and temporal redundancy insufficiently addressed.The related-work discussion notes that most methods focus on text-based compression and describes multimodal methods with fixed or adaptive allocation.Source: §5 · p. 9

• GUI-KV combines residual-stream saliency for important visual tokens with temporal scoring that preserves information not captured by the most recent frame.The approach is presented as exploiting GUI interactions’ spatio-temporal structure.Source: §6 · p. 9
• Across six visual-grounding, offline action-prediction, and online-evaluation benchmarks, GUI-KV recovers near-full-cache accuracy at modest budgets and consistently outperforms strong baselines.On UI-TARS-1.5-7B, 10–20% of cache typically closes most of the visual-grounding performance gap.Source: §6 · p. 9
• GUI-KV substantially reduces decoding compute with negligible prefill overhead, supporting GUI-agent deployment under tight memory constraints.Source: §6 · p. 9

• The study uses existing public GUI-agent benchmarks under their licenses and reports no new human-subject data collection, user studies, crowdsourcing, or added personal data.Source: §7 · p. 10
• Evaluation uses existing UI-TARS-1.5-7B and OpenCUA-7B models without additional training or fine-tuning on user data; online tasks follow the official protocol and are limited to benign workflows.Source: §7 · p. 10
• The authors caution that GUI automation could be misused without safeguards and state that large language models were used only for writing polish, not experiments or data or label generation.Source: §7 · p. 10

8 REPRODUCIBILITY STATEMENT

• The paper specifies models, datasets, metrics, and implementation details in the main text and appendix to support independent verification.It also identifies where the method, experimental settings, hyperparameters, ablations, and profiling results are documented.Source: §8 · p. 10
• The evaluated datasets are public and cited in the paper.The implementation is intended to be released as open-source code upon publication.Source: §8 · p. 10
• Appendix materials include additional ablations and profiling, including prefill-time overhead and sensitivity to the rank parameter.Hyperparameters such as rank r, temperature τ, and trade-off coefficients are also reported there.Source: §8 · p. 10

A DETAILS OF GUI-KV

• GUI-KV combines spatial saliency guidance with temporal redundancy scoring to retain the most informative KV-cache tokens.Its three stages compute spatial scores, identify redundant past tokens by subspace projection, and select the final retained set.Source: A DETAILS OF GUI-KV · p. 14
• For visual tokens, spatial saliency combines attention-based scoring with the L2 norm of residual-stream hidden states.The algorithm defines the hidden-state norm as the visual-token saliency signal.Source: A DETAILS OF GUI-KV · p. 14
• Temporal scoring uses the current frame’s key subspace to assess redundancy among past visual tokens.The algorithm forms a projector from a rank-r decomposition and uses residual norms to score past tokens.Source: A DETAILS OF GUI-KV · p. 14
• The algorithm partitions context tokens into current visual, past visual, and text tokens, then computes retained indices separately for each attention head.The token budget determines how many tokens are retained.Source: A DETAILS OF GUI-KV · p. 14
• The algorithm applies a temporal filter to past visual tokens, while retaining spatial scores for current visual and text tokens before selecting tokens to meet the budget.The algorithm’s final selection uses the number of tokens to keep based on the budget.Source: A DETAILS OF GUI-KV · p. 14

B.1 COMPRESSION OVERHEAD

• On UI-TARS-1.5-7B on AgentNetBench, profiling during prefill shows QR decomposition, top-k selection (0.007 GFLOPs; 0.15%), and memory gathering (0.057 GFLOPs; 1.22%) together contribute under 2% overhead.Attention scoring dominates the cost.Source: §B.1 · p. 15

B.2 THE IMPACT OF r IN TEMPORAL REDUNDANCY SCORING

• Across tested rank settings, results do not differ significantly, and r = 32 gives the best overall performance on AgentNetBench.All tested settings outperform the other baselines in Table 1.Source: §B.2 · p. 15

B.3 EFFECTIVENESS OF OTHER SPATIAL SALIENCY GUIDANCE

• Alternative saliency signals include image-patch histogram entropy, Sobel-filter output, and center-surround color contrast.Center-surround contrast compares a patch’s average color with that of a larger surrounding region in CIELAB space.Source: §B.3 · p. 15
• At 10–20% budget, Center-Surround Contrast is most effective, Pixel Histogram Entropy performs best at 1%, and residual-stream L2 performs best overall.The comparison uses UI-TARS-1.5-7B on ScreenSpot-V2 under different budgets.Source: §B.3 · p. 15, Table 5 · p. 16

C HYPER-PARAMETERS SELECTION

• The experiments use QR rank r = 32, temperature τ = 3.5, saliency-to-attention trade-off α = 2, and ω = 8 observation tokens.The rank sets the compact current-frame projector; ω defines the observation tokens used to form scoring queries.Source: C HYPER-PARAMETERS SELECTION · p. 16