Source-linked AI summary
Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents
Beining Wu, Zihao Ding, Jun Huang
TL;DR
Studies disagree about whether self-generated GUI experience should be put in model weights or retrieved into the prompt, in part because trajectories bundle different kinds of items. This paper routes components using pre-training properties and finds that the preferred destination differs by component, while rule-based routing improves on single-destination choices.
Problem
Studies comparing fine-tuning and retrieval disagree, while whole-trajectory comparisons change the experience items and do not compare destinations per component.
Source: §1 · p. 2Method
Results
+3.5 points on average: rule-based routing beats each backbone’s better single destination, while locators and lessons favor weights and procedures and state facts favor context.
Source: §1 · pp. 2–3, §4.1 · p. 7Takeaways & Limitations
The value of a destination varies with the component and with training or producer–consumer differences, including lower note readout after the same component is written into weights.
Source: §1 · p. 3
Abstract
from arXiv · showhide
Self-improving GUI agents keep the trajectories they produce and return them to the agent, by fine-tuning or by retrieval into the prompt, and studies that compare the two destinations disagree. We attribute this to the unit of experience: a trajectory bundles items with different properties, so a conclusion about the bundle depends on its mix. To address this, (i) we introduce component routing, which splits the experience into locators, procedures, state facts and lessons and sends each component to the context or to the weights, compared on the same items across three backbone families, two environments and three seeds. One pool has two destinations: locators and lessons win in the weights, procedures and state facts in the context. (ii) We fit a rule in two properties measured before any training, recurrence and state-conditionality; it recovers the destination of a held-out backbone family in 24 of 24 cells, two interventions move a component toward the boundary, and routing by the rule beats every whole-trajectory baseline and, by +3.5 points on average, the better single destination of each backbone. (iii) We identify how training and producer-consumer differences change the value of the two destinations: note readout decreases after the same component is written into the weights, most for the items that recur most, context gains increase with the information gap, and weights gains decrease with the policy gap. Code and data will be released.
1 Introduction
The paper argues that conflicting findings about where self-generated experience belongs arise because trajectories bundle items with different properties. It introduces component routing and finds that the best destination differs by component, with a pre-training rule that improves on single-destination choices.
1 Introduction
- A trajectory bundles items with different properties, so conclusions about the whole bundle depend on its component mix.The motivating example contrasts a locator that fine-tuning learns with a preview-screen procedure it loses.Source: §1 · p. 2
- Existing comparisons change the experience unit or pipeline, while this study compares destinations on the same items and held-out instances.Earlier studies disagree about whether fine-tuning or prompt-side reuse performs better, and none asks the question per component.Source: §1 · p. 2
- Locators and lessons favor weights, while procedures and state facts favor context, showing that one pool can have different winning destinations by component.The reported route gains are +4.9 points for locators and +1.5 for lessons, versus −4.2 for procedures and −4.1 for state facts.Source: §1 · pp. 2–3
- +3.5 points on average: rule-based routing beats each backbone’s better single destination across three backbones and two environments.The rule recovers the winning destination in 24 of 24 held-out cells; locators and lessons favor weights, while procedures and state facts favor context.Source: §1 · pp. 2–3
2 Related Work
Prior work studies experience reuse through whole trajectories, skills, or other fixed representations and destinations. This paper distinguishes itself by splitting one experience pool into components and routing them using measured properties.
Two destinations for experience
- Earlier work compares fine-tuning and retrieval with whole trajectories as the unit, while component routing holds items fixed and compares destinations per component.Prior conclusions differ across tool tasks, web tasks, and skill representations.Source: §2 · p. 3
Experience forms in GUI agents
- GUI experience methods vary what they keep, including skills, hindsight relabeling, and memories, but each fixes one form and one destination.Prior diagnoses also find that trajectory fine-tuning can learn interface shortcuts and that a form helping grounding can hurt action prediction.Source: §2 · p. 3
Producer, consumer and fragility
- Studies of producer–consumer differences and self-improvement fragility report net effects; this paper separates information and policy gaps on one pool.Reported sources of fragility include seeds, task order, backbone, and repeated internalization.Source: §2 · p. 3
3 Experimental Setup
The setup represents a trajectory pool as four keyed experience components and compares context retrieval with fine-tuning on the same items. It measures recurrence and state-conditionality before training and evaluates the routes on paired held-out tasks.
Problem formulation and components
- Each trajectory pool is extracted into locators, procedures, state facts, and lessons, with keyed items that support context retrieval or weights training.Locators encode screen positions; procedures encode action-target sequences; state facts encode preconditions; lessons pair failed and successful actions.Source: §3 · p. 4
Route gain
- The route gain is a paired held-out comparison of weights versus context, with positive values indicating that the component is better off in the weights.Source: §3 · p. 4
Recurrence and state-conditionality
- Recurrence measures how often an item’s key returns across unrelated tasks, while state-conditionality measures dependence on matching the source state.Both properties are measured on the pool before any route is trained.Source: §3 · p. 5
Two destinations
- The context route retrieves same-application items as prompt notes; the weights route fine-tunes on the same items, and a coexistence arm uses both.Weights training uses LoRA, with direct preference optimization for lessons and supervised fine-tuning for other components.Source: §3 · pp. 4–5
Evaluation setup
- Evaluation uses MobileGym and AndroidWorld held-out tasks, paired instances, three seeds, and deterministic environment checkers.The setup compares three 7–8B backbone families across the two environments.Source: §3 · p. 6
4 Observations
Across components, the better destination depends on the component and producer–consumer match; writing items into the weights also makes their notes less useful, especially for recurrent items.
4.1 Components Split by Destination
- Locators gain +4.9 points in weights, while procedures and state facts favor context by −4.2 and −4.1; lessons lean weights by +1.5 across three backbone families and two environments.Both destinations help every component; adding notes to the tuned model yields +0.1 points for locators and +3.5 for state facts. Components also occupy distinct regions in recurrence and state-conditionality before training.Source: §4.1 · pp. 6–7
4.2 Producer and Consumer
- Context procedures help across producers, while weights gains depend on producer–consumer alignment: the 8B consumer gains +4.8 points from 32B procedures but loses −7.5 from another family.The Figure 5 comparison includes solid context gains, dashed weights gains, and filled markers for training on the consumer’s own procedures only.Source: Figure 5 · p. 7, §4.2 · p. 7
4.3 Readout after Training
- Training on a component makes its notes redundant, and note readout declines more as component recurrence rises.Figure 6 measures action changes after reading notes before and after the same items enter the weights, and plots the after-to-before ratio against recurrence.Source: §4.3 · p. 8, Figure 6 · p. 8
5 The Routing Rule
The section fits a routing rule using recurrence and state-conditionality, tests interventions on those properties, examines readout under version conflict, and compares routing with alternative destinations and whole-trajectory methods.
5.1 A Rule in Two Properties
- A linear rule scores recurrence against state-conditionality and predicts whether each component’s route gain favors weights or context, holding out one backbone family at a time.A held-out cell goes to weights when its score is positive, otherwise to context.Source: §5.1 · p. 8
5.2 Interventions
- Changing either property moved outcomes as predicted: eightfold procedure copying shifted route gain from −4.2 to −0.8, while removing state condition cost state facts 3.4 points.Reading state from visible screen elements cost 0.7 points for procedures and state facts; higher learning rates and shuffled items did not reproduce the reported gains.Source: §5.2 · pp. 9–10
5.3 Readout and Conflict
- Under version conflict, high-recurrence cases followed the note less often than low-recurrence cases (43% vs 52%) and took neither action more often (36% vs 14%).The experiment trains weights on version 1 and serves notes from version 2 of the applications.Source: §5.3 · p. 10
5.4 Routing against the Alternatives
- Routing by the held-out rule beats every whole-trajectory baseline and the better single destination on every MobileGym backbone, by +4.4 points on average there.On AndroidWorld, routing leads the better single destination by +5.0 with Qwen3-VL-8B and +3.2 with ScaleCUA-7B, and matches GUI-Owl-8B; the best whole-trajectory baseline trails routing by 9.3 points.Source: §5.4 · pp. 10–11, Table 2 · p. 11
6 Producer and Consumer Shifts
The section examines how producer–consumer differences affect the value of context and weights routes. It finds that information gaps favor context, policy gaps reduce weights gains, and notes can retain value across consumer upgrades.
6.1 Three Gaps
- +0.2 to +11.3 points: context gains rise with the information gap, while weights gains fall from +4.0 to −5.7 points as the policy gap grows.The trends come from 720 producer–consumer cells; the information and policy gaps vary almost independently.Source: §6.1 · pp. 11–12, Figure 9 · p. 12
6.2 Producer Shifts
- A foreign producer with a policy gap can supply useful notes but harmful weights samples; retraining it on the consumer’s successes narrows that gap and improves the samples’ value.For GUI-Owl-8B, retraining reduced the policy gap from 0.46 to 0.17 while the information gap stayed at 0.07.Source: §6.2 · p. 12
6.3 Consumer Shifts
- After a consumer upgrade, notes from the earlier producer still help, while the weights route is rebuilt from the upgraded consumer’s own successes.The 8B producer’s notes gave the 32B consumer +3.2 points, while the same samples in its weights gave −1.4.Source: §6.3 · p. 13
7 Conclusion
The conclusion presents component routing as a way to assign GUI-agent experience to context or weights using recurrence and state-conditionality. The appendices describe the item representations, extraction, and fixed experimental design behind that approach.
7 Conclusion
- Locators and lessons win in the weights, while procedures and state facts win in context; routing by recurrence and state-conditionality beats whole-trajectory baselines and, on average, each backbone’s better single destination.The rule recovers the destination on a held-out family and for both halves of a component split at its boundary.Source: §7 · p. 13
A.1 Component Cards
- Each item has a key, payload, and state condition, and is rendered both as a context note and as a training sample for the weights route.For locators, for example, the note lists subgoals and is served when the task and screen match; its training sample maps a screenshot and history to an action.Source: A Components and Items · p. 20, §A.1 · pp. 20–21
- The cards distinguish component content: locators capture recurring target runs, while state facts encode action preconditions and lessons pair a failed action with one that later succeeded.State-fact notes state a precondition; lesson notes name the failed and successful actions.Source: §A.1 · pp. 20–21
A.2 Extraction
- Two rule-based extractors emit component items from accessibility states and actions; pools use the lenient extractor, and item-level recurrence and state-conditionality are summarized in Table A.1.The strict extractor keeps element names verbatim; the lenient one normalizes them before comparing keys.Source: §A.2 · p. 21, Table A.1 · p. 22
B Arms and Protocol
- The appendix records design constants fixed before runs, with measured outcomes reported in the main text and Appendix E.Source: B Arms and Protocol · p. 23
B.1 Environments and Splits
This appendix specifies the environments, pools, experimental arms, interventions, and interaction budget used in evaluation. It describes shared held-out instances and how context and weights routes use the same extracted items.
B.1 Environments and Splits
- MobileGym and AndroidWorld use held-out evaluation splits, while their experience pools are drawn from training or remaining tasks; the producer–consumer batch groups held-out templates into 16 task families.Each arm uses the same stored instances; MobileGym reports the mixed seen/unseen 32-template group separately from main-text numbers.Source: §B.1 · p. 23
B.2 Arms
- Single-component arms route one component through context or weights, while all_C, all_W, routed, anti_routed, and routed_self compare combined or rule-assigned destinations.The routed assignment is written before adapter training; routed_self is an in-sample reference used for GUI-Owl-8B only.Source: §B.2 · p. 24
Interventions:
- Interventions vary training dose, retrieval state matching, learning rate, recurrence splits, or controls such as shuffled payloads and random cards.The copy intervention increases the weights-route training set while leaving the notes unchanged.Source: Interventions · pp. 24–25
B.3 Interaction Budget
- Arms generally run one episode per held-out instance with the same step limit; context notes and adapter training add different budget costs, and paired differences use matched instances and seeds.The interaction budget defines when two actions count as the same for readout, policy-gap, and conflict measurements.Source: §B.1 · pp. 23–24, §B.3 · p. 25
C Training Setup
The weights route uses LoRA adapters with a fixed configuration and converts each component’s items into training samples. Aggregate adapters combine component samples and restrict the training stages to the components assigned to weights.
C Training Setup
- Weights-route adapters use LoRA with rank 16, α 32, three epochs, and learning rate 10−5; the vision tower stays frozen and the base prompt is used without notes.Source: C Training Setup · p. 26
C.1 Weights Route
- Component items become task-specific samples: locator clicks, procedure steps, state-fact predicates with actions, or lesson preference pairs trained by preference optimization.Aggregate and routed adapters train on the locator, procedure, and state-fact samples before a lesson preference stage, restricted to the components assigned to weights.Source: §C.1 · p. 26
C.2 Context Route
Context keeps state-matched notes in the prompt, while the accompanying rule-fit procedure uses pool properties to assign experience destinations before evaluation.
C.2 Context Route
- The context route ranks same-application items by source-state similarity and appends the five best notes, exact matches first, while leaving the base prompt otherwise unchanged.Procedures are retrieved first by task text and then by step-level items; aggregate context arms serve at most two notes per component and eight in total.Source: §C.2 · p. 26
- Table C.1 reports training samples and development token accuracy for each adapter, averaged over its seeds.Adapters use LoRA rank 16, α 32, three epochs, learning rate 10−5, and a frozen vision tower, with stated exceptions for learning-rate rows and preference optimization.Source: Table C.1 · p. 27
D The Rule Fit
- Appendix D specifies the rule fit's notation, implemented objective, execution routine, and per-cell results behind Figure 7.Source: D The Rule Fit · p. 28
D.1 Notation and Objective
- A held-out-family component goes to the weights when its fitted score is positive and to the context otherwise; uncertain cells count half in the reported score.Source: §D.1 · p. 28
D.2 The Routine
- The routine measures the pool, fits the rule with one backbone family held out, routes that family's components by the fitted sign, then evaluates on held-out instances.Source: §D.2 · p. 28
D.3 Held-out Table
The held-out folds recover the observed destination signs across all cells, and recurrence-based splits of lessons and state facts fall on opposite sides of the rule boundary.
D.3 Held-out Table
- The held-out routine measures pool properties before training, routes each component by the fitted sign, and pairs evaluation episodes with baseline arms on the same instances and seeds.Source: §D.3 · p. 29
- Splitting lessons and state facts by recurrence yields opposite-side halves whose route gains differ by +4.3 and +4.9 points, respectively.The halves differ in recurrence but hardly in state-conditionality, and their destinations were fixed before runs without fitting on the halves.Source: §D.3 · pp. 28, 30, Table D.4 · p. 32
- 24 of 24 cells match the observed route-gain sign across folds, with 19 intervals clear of zero and S = 21.5/24.The full fit agrees with all twenty-four cells in sample, and every fold assigns locators and lessons to weights and procedures and state facts to context.Source: §D.3 · p. 28, Table D.2 · p. 31
E Full Results
The full-results appendix reports route gains and intervention checks, alongside AndroidWorld contrasts, producer-consumer comparisons, readout rates, and sensitivity settings.
E Full Results
- The full-results tables report paired route differences over three-seed means, and Figure E.1 plots per-cell gains plus gains on seen and unseen applications.Source: E Full Results · p. 33, Figure E.1 · p. 33
E.1 All Intervention Arms
- 0.7 points are lost for both procedures and state facts when notes use screen-element signatures instead of environment state; tenfold learning rates raise their weights-route gains by +1.0 points each.With the higher rate, route gains remain −3.3 and −3.2 points, respectively; threefold rates yield −3.4 points for both.Source: §E.1 · p. 33
E.2 AndroidWorld Contrasts
- AndroidWorld contrasts compare routing with each single-destination arm and the reverse assignment across 232 instances and three seeds per family.Source: §E.2 · p. 33, Table E.2 · p. 35
E.3 Producers and Consumers
- The producer-consumer table compares procedures-route gains from producer notes, producer-trained weights, mixed samples, and consumer-specific practice.Each adapter in this batch trains on 5,600 samples, or 1,900 for 32B consumers.Source: Table E.3 · p. 36, §E.3 · p. 36
E.4 Readout
- Readout rates measure how often removing a note changes the action before and after the same component is written into the weights, including recurrence thirds.Source: Table E.4 · p. 36, §E.4 · p. 36
F Sensitivity
- Sensitivity tests vary matching thresholds, note counts, adapter rank, pool size, library cap, and routing-rule input on GUI-Owl-8B, seed 0.Each variant changes one default-arm setting; the table reports its gain and difference from the default.Source: F Sensitivity · p. 37
- A further sensitivity variant removes the per-component note quota and uses online state-conditionality rather than κ0 for routing.Source: F Sensitivity · p. 37
G Cases
- The illustrative cases expand Figure 8 episodes with each action marked as coming from weights or context.Source: G Cases · p. 39
G.1 Case 1: One Task, Two Sources
In one case, combining a locator from weights with state facts and a procedure from context completes the task; in another, a note overrides a conflicting learned route.
G.1 Case 1: One Task, Two Sources
- The locator from weights and state fact and procedure from context complete the task, while the all-weights agent reaches the preview but leaves without adding Tokyo.The mixed-source episode saves Tokyo; the all-weights outcome leaves Tokyo unsaved.Source: §G.1 · p. 39
G.2 Case 2: A Note and a Learned Route Disagree
- When a note conflicts with a learned route, the agent follows the note and selects Birthday; without the note, the weights route selects generic Event.The note-guided episode completes the task with a Birthday entry, while the weights-only episode saves the wrong entry type.Source: §G.2 · p. 40
H Compute and Tokens
The appendix specifies resource accounting and the prompt, turn, and retrieved-note formats used across backbone families.
H Compute and Tokens
- Compute and token accounting covers adapter training samples and GPU-hours, plus per-episode prompt tokens and their ratio to the base agent.Training and pool-sampling compute are booked at specified rates; the table reports means over seeds.Source: Table H.1 · p. 41, H Compute and Tokens · p. 41
I.1 Agent Prompts
- The agent prompts specify each model’s action space and required response structure, including tool-call formats for GUI-Owl and tagged output for ScaleCUA.Qwen3-VL instead returns one JSON action after its THINK block.Source: §I.1 · pp. 42, 44, 45
I Prompts
- The prompts include backbone-specific system instructions and action formats; the Qwen3-VL rendering uses a THINK block followed by one JSON action.The MobileGym applications, prompts, notes block, and notes are in Chinese, with an English rendering supplied.Source: I Prompts · p. 42, §I.1 · p. 45
I.2 User Turn
- User turns carry the current screenshot and task/history context; context-route arms append retrieved notes, while weights-route and base arms receive the same messages without that block.The readout probe also runs steps with a notes block once without it; the action produced with the block is executed.Source: §I.2 · p. 45
I.3 Notes Block
- The notes block has a fixed header and one line per retrieved item, with templates for locators, procedures, state facts, and lessons.Its header identifies the notes as prior same-application experience and says the current screenshot and task take precedence.Source: §I.3 · pp. 45–46