PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents

Yuhao Wang, Mu Qiao, Xindong Zhang, Yunzhi Zhuge, Lei Zhang, Huchuan Lu

arXiv:2609.10297v1cs.CV

TL;DR

GUI agents must prune reusable visual evidence before future interaction targets are known, without losing coverage under tight budgets. TRACE addresses this with trajectory-robust ordering and coverage repair, and ranks first on four of six benchmarks while retaining 52.9% of dense performance under a tight budget.

  • Problem

    Reusable-cache pruning is an irreversible admission decision that must preserve future utility and spatial coverage despite unknown later GUI targets.

  • Method

    TRACE combines layout-derived interaction priors, instruction relevance, feature novelty, native-token coverage repair, and monotone KV contraction into one nested evidence order.

  • Results

    TRACE ranks first on four of six benchmarks and retains 52.9% of dense performance on average under the tight budget, versus 43.5% for PruneSID.

  • Takeaways & Limitations

    TRACE enables visual evidence to remain reusable across shrinking budgets while preserving coverage without re-encoding historical frames.

  • Takeaways & Limitations

    Under aggressive context reduction, failures mainly reflect insufficient detail for precise localization, especially for thin or low-contrast widgets.

Abstract

from arXiv · show

GUI agents accumulate high-resolution screenshots as the trajectory unfolds, increasing inference latency and memory usage. Training-free visual token pruning can reduce this cost, but cache reuse introduces a fundamental constraint. Once tokens are discarded, the corresponding visual evidence cannot be recovered without re-encoding. Pruning therefore becomes an \textit{irreversible admission decision} that must remain useful for unknown future targets while preserving coverage of operable regions under tight budgets. To address these challenges, we propose \textbf{\method{}}, a training-free framework for \emph{\textbf{T}rajectory-\textbf{r}obust \textbf{A}dmission and \textbf{C}overage-aware \textbf{E}vidence ordering}. Specifically, we combine a query-independent layout-derived interaction prior with instruction relevance and feature novelty to rank visual evidence according to both potential future utility and diversity. Then, we reserve part of the budget for native visual tokens distributed across the screen, repairing missing spatial coverage without breaking the ordering. Together, these mechanisms produce a nested token order, allowing retained visual evidence to shrink monotonically across budgets while remaining reusable throughout the trajectory. Finally, our monotone KV contraction incrementally contracts retired frames into compact session state, avoiding repeated visual encoding or pruning. Extensive experiments across six GUI benchmarks and diverse models verify the effectiveness of our proposed \method{} under tight budgets. The source code will be released.

1 Introduction

TRACE frames GUI visual pruning as an irreversible, write-time commitment under trajectory uncertainty, where evidence must remain reusable across unknown future targets and preserve spatial coverage. It addresses this with a training-free framework combining layout-aware admission, nested evidence ordering, coverage repair, and monotone state contraction.

  • Motivation: GUI agents face growing visual computation and memory costs because each trajectory step processes high-resolution screenshots alongside accumulated interaction context.The introduction identifies visual computation as increasingly dominant as trajectories unfold.
  • Motivation: Existing training-free pruning methods reduce redundancy but can recompute selection when queries change, conflicting with reusable cache-based GUI serving.The cited methods use attention, feature diversity, instruction relevance, or synthetic merging, whereas lifecycle-aware pruning requires commitment at write time.
  • Formulation: TRACE formulates lifecycle-aware visual pruning as irreversible admission with nested keep sets across budgets, enabling reusable visual state without repeated commitment.The formulation explicitly treats pruning as write-time visual evidence commitment under trajectory uncertainty.
  • Challenges: Write-time admission must commit evidence before later targets are known, while tight budgets require preserving operable regions and spatial coverage.Feature diversity is query-independent but does not distinguish operable regions from background; interface layout provides operable-region information before demand is observed.
  • TRACE: TRACE introduces four components—Layout-derived Interaction Prior (LIP), Nested Evidence Ordering (NEO), Native-token Coverage Repair (NCR), and Monotone KV Contraction (MKC)—for efficient GUI agents.LIP uses layout detections to favor operable regions before later targets appear, while the framework commits evidence once and contracts it monotonically thereafter.

2 Related Work

Prior GUI-agent efficiency methods comprise training-based designs that learn cheaper perception and training-free visual token pruning that removes or aggregates redundant tokens. Representative approaches use architectural redesign, history-policy retraining, attention statistics, feature dispersion, or instruction relevance and diversity.

  • Training-based Designs: GUI-agent methods address the computation and memory costs of accumulating high-resolution screenshots through training-based and training-free designs.The related work frames these costs as arising from agents grounding actions in an accumulating screenshot stream.
  • Training-based Designs: Training-based designs learn cheaper perception by retraining the architecture or history policy, including CogAgent’s low-resolution backbone with a high-resolution cross-attention module.
  • Pruning-based Methods: Visual token pruning accelerates MLLM inference by discarding native tokens or aggregating them into synthetic tokens.
  • Pruning-based Methods: Pruning-based methods rank tokens using language-model attention statistics, feature dispersion, or joint diversity and instruction relevance, as in FastV, DivPrune, and CDPruner.

3 Method

TRACE formulates reusable visual admission for multi-step GUI agents as a monotone lifecycle: tokens are selected before prefill, retained sets shrink after retirement, and original rows remain available for cache reuse. It combines layout-aware interaction priors, nested evidence ordering, native-token coverage repair, and monotone KV contraction to preserve future utility and spatial coverage under changing budgets.

  • Reusable Visual Admission: TRACE constrains reusable admission to pre-prefill information, monotone retirement, and original visual tokens, enabling cache reuse while leaving future utility unknown.Current-frame retained positions can be reduced to a smaller history subset without synthetic merges that cannot later be dropped as cache rows.
  • Layout-derived Interaction Prior: Layout-derived Interaction Prior maps detected interface elements such as buttons and text fields into interaction density, distinguishing operable regions from background.The prior addresses instruction relevance’s inability to anticipate later demands and uniform geometry’s failure to separate operable regions from background [Zhang et al., 2025b].
  • Nested Evidence Ordering: Nested Evidence Ordering fuses interaction prior, instruction relevance, and feature novelty into a single nested order whose prefixes support every later budget.Prior-weighted normalized features emphasize layout-dense tokens while retaining eligibility through a unit baseline; greedy residual ordering balances novelty against relevance.
  • Native-token Coverage Repair: Native-token Coverage Repair restores spatial coverage with native tokens after biased ordering, preventing tight prefixes from leaving entire interface regions uncovered.The method selects coverage tokens differently for the current frame and history, with the current frame receiving the larger repair budget.
  • Monotone KV Contraction: Monotone KV Contraction contracts each retired frame into the next prefill, advancing the reusable visual state without a second visual encoding.The lifecycle is organized into three serving phases illustrated for a past frame and the current frame.

4 Experiments

TRACE outperforms existing methods across six GUI benchmarks, retaining 78.7% and 61.1% of dense performance at mild and tight budgets while maintaining stronger results as budgets tighten. Ablations show that layout priors, evidence ordering, and coverage repair jointly drive performance, though window truncation still leaves latency and memory costs high.

  • Single-step Evaluation: TRACE achieves the best existing-method performance, retaining 78.7% and 61.1% of dense GUI-Owl-1.5-8B performance at mild and tight budgets, respectively.At the mild budget, TRACE leads PruneSID by 14.94% on SS-v2 and VisPruner by 8.12% on MMBench-GUI; at r = 5%, its SS-v2 margin over PruneSID grows to 20.99%.
  • Multi-step Evaluation: On multi-step benchmarks, TRACE’s margin over the best existing method grows from 0.54%–0.74% at mild budgets to 1.30%–3.53% at tight budgets.On Mind2Web, TRACE reaches 34.20% versus VisPruner’s 30.67%.
  • Analysis of Key Modules: Combining the layout prior with NEO reaches 56.45% on SS-v2 at r=10%, 19.97% higher than either stage alone, while NCR raises it to 73.90%.Adding NCR also lifts tight Mind2Web from 27.79% to 34.20%, showing complementary contributions from localization, ordering, and repair.
  • Analysis of Key Modules: NEO’s diversity term spreads retained tokens across screenshots instead of concentrating them on repeated glyphs, while NCR rescues targets on 44 frames and loses them on only 5.Without diversity, the order spends its budget on repeated glyphs; coverage repair reduces uncovered targets on most frames at the same budget.
  • Serving Efficiency Analysis: Window truncation remains inefficient: three recent frames still consume 90% of the dense budget, and one frame costs 1.66× TRACE without improving accuracy.This leaves latency and memory usage high despite the absence of an accuracy gain.

5 Conclusion … A.1 Lifecycle Contract and Method Space

The paper frames reusable GUI-agent visual pruning as an irreversible write-time commitment requiring nested admission orders and spatial coverage, and introduces TRACE as a training-free framework. The appendix documents the lifecycle contract, evaluation protocol, implementation choices, and additional evidence, including comparisons of prior methods against lifecycle constraints and setting-specific axes.

  • 5 Conclusion: TRACE treats cache reuse as an irreversible write-time pruning commitment, requiring nested orders that remain useful under unknown future grounding demands while preserving spatial coverage.The conclusion motivates TRACE around reusable visual state and tight-budget coverage.
  • 5 Conclusion: TRACE is presented as a training-free framework for efficient GUI agents built around the lifecycle requirements identified in the conclusion.The supplied conclusion passage introduces TRACE after stating the lifecycle problem and its constraints.
  • APPENDIX: The appendix covers lifecycle behavior, implementation, evaluation protocol, configuration choices, benchmark breakdowns, ablations, serving efficiency, backbone transfer, and evidence limitations.These topics are organized across Sections A–C of the appendix.
  • A Method, Protocol, and Configuration: Section A specifies the lifecycle contract, method implementation, comparison protocol, and configuration choices used for the paper’s evaluation.The appendix overview identifies these as the contents of Section A.
  • A.1 Lifecycle Contract and Method Space: Table 8 evaluates nine existing methods against three lifecycle constraints for deletion-only serving, scoring each method from its official selection rule.Its marks indicate compatibility with the specified serving path rather than a universal judgment; nesting may require modification for some quota, merge, or de-duplication rules.
  • A.1 Lifecycle Contract and Method Space: Table 9 compares representative prior families across six axes: multi-step visual-state reuse, GUI-screen design, pre-language-model decisions, native token positions, nested two-budget ordering, and historical-frame re-encoding.The comparison notes that TopV’s no-re-encoding property applies within a single generation rather than across frames.

A.2 Method Specification

The method specifies an inference pipeline that orders visual evidence using interaction priors, instruction relevance, novelty, and coverage, then preserves nested native-token subsets for budget changes. It also contracts retired frames through native history representatives and cache replay while retaining positional and contextual consistency.

  • Layout-derived Interaction Prior: The interaction prior uses detector-derived layout information to identify potentially operable regions that generic detectors may miss on dense GUI screens.The specification uses the icon_detect branch of OmniParser-v2 (Lu et al., 2024) and combines four normalized attributes in the evidence score.
  • Nested Evidence Ordering: NEO combines instruction relevance, prior-weighted novelty, and residual coverage into a single nested sequence reusable across visual-token budgets.Greedy matching pursuit forms the ordering, after which current-frame stride representatives and protected history prefixes are inserted without breaking nesting.
  • Nested Evidence Ordering: At r=10%, doubling the log aj coefficient reduces ScreenSpot-v2 accuracy by 10.53%, while max-over-query-rows and instruction-token queries are retained in the adopted scoring form.The ablation compares substitutions to the adopted NEO form and reports the largest stated degradation for doubling the relevance coefficient.
  • Native-token Coverage Repair: Native-token coverage repair keeps repaired rows in the cache, preserves Shhist_t ⊆ Scur_t, and makes retirement deletion-only without synthetic features or reconstructed KV rows.History medoids are selected from contiguous, nearly equal-sized complement regions by dynamic programming that minimizes the maximum within-region squared distance to the region mean.
  • Monotone KV Contraction: Monotone cache contraction reuses replayed kh visual rows, keeps original positional indices after row deletion, and addresses downstream mixed-context states through context-consistent replay.The serving-cost formulation counts visual KV rows as O(kc + Hkh), while the replay mechanism targets text and action states computed before retiring visual rows were removed.

A.3 Evaluation Protocol and Baseline Reproduction · A.4 Configuration Validation · A.4.1 Fixed-Configuration Check

The evaluation reproduces nine training-free pruning baselines under matched budgets and validates TRACE’s configuration choices across benchmarks and token budgets. A global configuration remains close to selected per-benchmark-budget settings while the selected configuration improves the median result and most combinations.

  • A.3 Evaluation Protocol and Baseline Reproduction: The protocol evaluates nine leading training-free methods against random and uniform pruning under each method’s official selection settings and target budget.The baselines cover diversity, relevance, saliency, merging, posterior attention, sensitivity, and semantic grouping.
  • A.3 Evaluation Protocol and Baseline Reproduction: Among tuned baselines, VisionTrim is strongest at a DVTS share of 0.9, reaching 61.24%.PruneSID and ZOO-Prune vary across their tested threshold or direction-count settings, while VisionTrim attains the highest reported tuned-baseline score.
  • A.4 Configuration Validation: TRACE selects the prior cap from {1, 2, 4, 6} for each benchmark and budget, or applies the effective-support rule in Eq. 9.The configuration-validation section tests whether these choices remain useful when tasks or budgets change.
  • A.4 Configuration Validation: Effective support equals the number of equally weighted active tokens, ranges from approximately 1 under single-token concentration to N under uniform mass, and is scale-independent.Squared masses make the measure increase with concentration while cancelling overall mass scale.
  • A.4 Configuration Validation: The applied prior strength is min(ᾱ, α⋆), with no prior for frames lacking detector support; current and history passes use separate doses when history tokens are fewer.Main-table runs use one dataset-and-budget configuration, with ρcur for current frames and ρhist only when kh < kc.
  • A.4.1 Fixed-Configuration Check: The selected configuration improves the median result by +1.0% and improves 14 of 16 benchmark-budget combinations, while remaining within a 1.9% gap from the global configuration.Table 12 compares Fixed (α=2 with default current dose) against Selected (adopted cap, calibration mode, and doses) across 16 combinations.

A.4.2 Benchmark and Budget Dependence · A.4.3 Prior and Coverage Interaction · A.4.4 Budget and Dose Sensitivity

The appendix examines how task family, budget allocation, prior strength, and coverage dose affect evidence selection. It finds that current-frame evidence is more valuable than history, while strong priors and coverage repair interact with available budget.

  • A.4.2 Benchmark and Budget Dependence: Task families differ in their preferred prior strength, spanning candidate-based selection, coordinate grounding, and structured action prediction.Mind2Web Task uses candidate-based selection; MMBench-GUI L2, ScreenSpot-v2, and ScreenSpot-Pro use coordinate grounding; OmniGUI and the truncated final benchmark entry use structured action prediction.
  • A.4.2 Benchmark and Budget Dependence: Prior-strength sensitivity is evaluated with task-family-specific candidate grids, using ᾱ ∈ {1, 2, 4, 6} where available and ᾱ ∈ {1, 2, 4} for structured action prediction.The adopted settings are indicated in bold in Table 13 for GUI-Owl-1.5-8B.
  • A.4.3 Prior and Coverage Interaction: Strong prior caps are usually preferred, but the best prior–coverage configuration depends on the available budget.At tight budgets, the prior may already cover the interactive surface, making extra repair compete for the same tokens; at milder budgets, a strong cap with a larger current dose is preferred.
  • A.4.3 Prior and Coverage Interaction: Coverage repair and prior selection address complementary needs: the prior identifies likely interactive regions, while coverage preserves spatial support when the budget allows.The supplied sensitivity analysis covers both prior strength and current dose.
  • A.4.4 Budget and Dose Sensitivity: On OmniGUI, halving the current budget costs 5.29%, whereas halving the history budget costs 0.74%, showing that current-frame evidence carries most useful information.The adopted split therefore assigns a larger budget to current evidence and a smaller budget to history evidence.
  • A.4.4 Budget and Dose Sensitivity: After fixing the current–history split, the dose allocates part of each budget to coverage tokens rather than relying only on greedy selection.This allocation follows the finding that current evidence is more valuable while still reserving budget for coverage.

A.4.5 Calibration and Robustness

TRACE remains usable under incomplete detections and untuned instruction-relevance scaling on ScreenSpot-v2 at r = 10%, with gradual degradation rather than abrupt failure. The default temperature τ = 1 is retained without tuning across experiments.

  • Detector degradation: Accuracy falls gradually from 73.90% to 68.55%, 64.15%, 55.82%, and 53.85% as 25%, 50%, 75%, and 100% of detections are randomly removed.With every detection removed, the prior becomes flat and selection uses only the remaining signal.
  • Calibration and Robustness: TRACE remains robust to imperfect inputs, including incomplete detections and the default instruction-likelihood temperature τ = 1.Both robustness tests use ScreenSpot-v2 at r = 10%, and the default τ = 1 is used in every experiment without tuning.
  • Instruction-likelihood temperature: Using τ = 0.5 loses 10.5%, while flattening relevance to τ = 4 loses 7.1%, motivating the fixed default τ = 1.The lower temperature concentrates relevance on too few tokens, whereas the higher temperature dilutes the instruction signal.

B Additional Experimental Evidence · B.1 Lifecycle Behavior

Lifecycle experiments show that TRACE preserves later-usable evidence through nested retirement, avoids re-admission and re-encoding, and retains more future-target coverage than a no-prior keep under a 10% budget. Instrumented traces and cache visualizations support these conclusions across OmniGUI and ScreenSpot-v2.

  • B Additional Experimental Evidence: The broader evidence block evaluates lifecycle retention alongside benchmark gains, selector ablations, and serving measurements that separate cache reuse from selection cost.These four evidence chains organize the additional experiments beyond lifecycle traces.
  • B Additional Experimental Evidence: Nested retirement preserves a committed prefix, whereas fresh 65-token reselection would add and drop 12 tokens and move 23.5% of the surviving set over 1000 steps.This comparison uses GUI-Owl-1.5-8B and contrasts nested retirement with unconstrained reselection.
  • B.1 Lifecycle Behavior: The lifecycle study tests whether first-admitted keeps contain later targets and whether history keeps remain prefixes of current keeps, avoiding re-encoding.It measures later-target retention, compares nested retirement with independent reselection, and inspects cache states.
  • B.1 Lifecycle Behavior: At r = 10%, TRACE grounds 82.5% of ScreenSpot-v2 frames versus 40.0% without the prior, while an OmniGUI episode retires from about 50% to a 10% prefix with zero nesting violations.The committed keep remains nested throughout the episode, and no token is re-admitted after its frame is written.
  • B.1 Lifecycle Behavior: On ScreenSpot-v2, both methods hit the photos-app target, but only TRACE with the prior hits the AirDrop-setting target by retaining the settings row.The comparison is conducted at r = 10% against the no-prior keep.
  • B.1 Lifecycle Behavior: The cache-state visualization shows each frame’s input, its admitted cache, and the contracted history for an OmniGUI Alipay episode at (c, h) = (50%, 10%).The diagonal represents admission, while the same frame to the right represents contracted history.
  • B.1 Lifecycle Behavior: TRACE’s committed cache retains displayed evidence usable for later actions after retirement, while the nested prefix is not rebuilt.The lifecycle visualizations are presented as evidence for robustness across future-target reuse.

B.2 Benchmark Breakdowns

This subsection breaks down benchmark performance by target category, action metric, budget, and model scale. TRACE retains strong tight-budget grounding and multi-step transfer, while category-level results reveal uneven advantages.

  • Benchmark scope: The subsection organizes benchmark averages by accuracy and reports Avg. (%) as retained performance relative to the dense model across target categories, action metrics, budgets, and model scale.The breakdown explicitly separates target categories from action metrics, examines budget curves, and repeats comparisons at another 2B scale.
  • Single-step Breakdown: At r = 10%, TRACE reaches 73.90%, 37.63%, and 50.47% on ScreenSpot-v2, ScreenSpot-Pro, and MMBench-GUI, retaining 64.5% of dense performance on average.These are single-step grounding results with GUI-Owl-1.5-8B; Table 15 reports the per-metric breakdown and retained performance relative to the full-token upper bound.
  • Multi-step transfer: Under mild and tight budgets, TRACE exceeds uniform on element accuracy and joint Step SR, while operation F1 remains close between methods.The supplied comparisons are 39.41% versus 26.32% and 34.20% versus 20.61% at the tight budget, while operation F1 is 83.90% versus 83.49%.
  • Multi-step transfer: TRACE’s multi-step results are 36.08%, 37.92%, and 56.53% Step SR on OmniGUI, Mind2Web, and AndroidControl, respectively.The passage identifies these as multi-step transfer results from Table 19, but the supplied excerpt ends before additional comparisons.
  • Single-step Breakdown: At r = 5% on ScreenSpot-Pro, TRACE leads FastV on text targets at 6.86% versus 5.53% but trails on icon targets at 6.79% versus 8.28%.TRACE’s overall lead is only 0.25 points, with 6.83% versus 6.58%, showing that an aggregate ranking does not imply category-wise leadership or high absolute accuracy.

B.3 Ablations and Visual Evidence

Ablations show that spatial repair, instruction relevance, diversity, and the layout prior each materially affect TRACE’s tight-budget performance. Visual evidence confirms improved target coverage and spatial structure, while exposing failures on very small controls.

  • Coverage Recall: TRACE reports 90.06% coverage recall on the 120-frame dump, measuring whether detector-proposed elements retain at least one token.This diagnostic measures coverage of detector proposals rather than every true interface element.
  • Ablations: Adding spatial repair raises tight-budget accuracy from 38.36% to 55.19% on ScreenSpot-v2 and from 28.15% to 37.63% on ScreenSpot-Pro.Recoveries outnumber losses across examined frames, although repair does not benefit every image.
  • Ablations: Removing instruction relevance costs 15.57 points on tight ScreenSpot-v2 and 14.17 points on ScreenSpot-Pro, while removing diversity costs 5.35 and 4.74 points, respectively.Both terms contribute, with the instruction term producing the larger losses.
  • Layout Prior Comparison: The layout prior improves most baselines, yet the best compatible result trails TRACE by 13.37 points at the mild budget and 20.83 points at the tight budget.Different baseline responses show that the detector prior is not a universal replacement for each selector’s scoring rule.
  • Keep Maps: At the tightest budget, TRACE retains three of four marked targets, whereas instruction-conditioned scoring retains one on the same screen.The gallery also shows a concrete MISS for a tiny notification control that is insufficiently represented in retained tokens.

B.4 Serving Efficiency · C Transfer and Scope

TRACE improves serving efficiency by combining low-cost selection with reusable visual state, and transfers to UI-TARS across single-step and multi-step GUI benchmarks without retuning. Its gains are strongest under tighter budgets, though performance varies by benchmark and baseline.

  • B.4 Serving Efficiency: Serving gains depend on both selection cost and reusable visual state: when selection overhead exceeds the prefill benefit, end-to-end serving cost increases.The comparison separates selector cost from lifecycle reuse and shows that pruning alone does not explain the serving benefit.
  • B.4 Serving Efficiency: 60.20%: AndroidControl mild-budget Step SR rises from 59.84% through 59.91% to 60.20% as current-frame dose increases, while tight-budget scores reach 55.45% at dose 0.3 and 56.58% at dose 0.5.The nested history rung is separately enabled by the star and is not another point on the current-dose-only curve.
  • B.4 Serving Efficiency: 5.1×: the dense curve reaches 5.1× TRACE’s TTFT at step 5, while TRACE’s latency rises overall but avoids the sharper accumulation caused by repeated encoding.The within-episode measurement uses AndroidControl Step SR for dose response and OmniGUI for TTFT.
  • C Transfer and Scope: At matched retention on the nested MKC path, TRACE transfers its serving framework to existing selectors for a paradigm-transfer comparison on GUI-Owl-1.5-8B.The appendix places DivPrune, VisPruner, and ZOO-Prune on the same nested MKC path to match their serving state.
  • C Transfer and Scope: TRACE leads ScreenSpot-Pro at 25% and 5% retention with 16.76% and 3.04% average accuracy, and leads ScreenSpot-v2 at 25% and 10%.The ranking is not uniform: DivPrune leads MMBench-GUI, while PruneSID leads several higher-retention settings; TRACE’s advantage is clearest at tighter budgets.
  • C Transfer and Scope: TRACE gains 1.05 and 1.18 points over the strongest baseline on OmniGUI and 2.80 and 4.92 points on pooled Mind2Web across two budgets.The transfer uses UI-TARS’s native action grammar and MKC path across 1,521 of 2,572 target-location OmniGUI steps, while reusing cached rows instead of re-encoding prior frames.

C.1 Limitations and Failure Modes

TRACE is evaluated on shorter trajectories than its supported history horizon, while its remaining errors are concentrated in precise localization under aggressive context reduction rather than failure to reach target regions.

  • Trajectory Horizon: TRACE supports histories of five or six frames, exceeding the two- or three-frame horizons retained by most existing GUI benchmarks.As H grows, compact historical evidence reuse becomes increasingly beneficial, with serving cost expressed as O(kc + Hkh).
  • Failure Modes: 93.0% of remaining failures retain a token overlapping the target, while removing the target’s spatial evidence explains only 0.8% of the gap to dense.These results indicate that the selector usually reaches the target neighborhood, but may discard detail needed for precise localization.
  • Failure Modes: The dominant failure mode is insufficient localization precision under aggressive context reduction, not failure to reach the target region.Errors mainly occur when the target neighborhood is retained without enough detail for precise localization.
Loading 2609.10297v1…