SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents

1 Introduction

• Prior evaluations either lack interactive executable-skill transfer, use fixed website snapshots, or vary task content, interface state, and runtime conditions together.Existing robustness benchmarks primarily assess general agent robustness rather than whether reusable skills remain executable when task semantics are preserved.Source: §1 · p. 1
• SkillShift-Bench preserves task intent, critical entities, functionality, and success criteria while targeting skill invocation, grounding, or runtime feasibility.The three controlled mismatch modes are Contextual, Perceptual, and Execution Shift, respectively.Source: §1 · p. 1
• Controlled same-site evolution produces consistent transfer degradation, with Execution Shift exerting the strongest pressure.The authors attribute the pattern to local executability mismatch: task-aligned skills may remain available while the next operation is infeasible in the current page state.Source: §1 · pp. 1–2
• The paper formulates shifted skill execution as nonstationary local executability and analyzes SWOP with a variation-dependent dynamic-regret guarantee.It also proposes MPCR, a fixed-library executor-side reranker that uses feasibility-aware action selection at test time.Source: §1 · p. 2

2.1 Skill learning and reusable web procedures

• Reusable procedures can reduce repeated planning and improve web-agent efficiency, while this work asks whether they remain usable as the same website changes.The changes of interest affect context, interface evidence, or runtime conditions needed for execution.Source: §2.1 · p. 2

2.2 Web agent benchmarks and environment realism

• Web-agent benchmarks trade off task coverage, reproducibility, and realism, while SkillShift-Bench focuses on aligned transfer with preserved task semantics and varying local execution conditions.Offline and self-hosted benchmarks offer broad coverage or reproducible snapshots; remote and live benchmarks improve realism but may vary multiple factors together.Source: §2.2 · p. 2

2.3 Environment variation and diagnostic robustness

• Environment-variation benchmarks motivate non-static evaluation, but this work targets whether previously useful procedures remain invokable, groundable, and executable under aligned task semantics.The focus is learned-skill transfer rather than general agent robustness alone.Source: §2.3 · p. 3

2.4 Breakage and repair of recorded web procedures

• Web-test repair addresses locator, structure, or wait breakage, whereas this paper adapts the executor because skills are selected at runtime and Execution Shift changes runtime state.A still-valid procedure may be invoked where it no longer applies; stored-script repair cannot anticipate runtime-state interruptions.Source: §2.4 · p. 3

2.5 Test-time executor-side adaptation for shifted skill execution

• The paper connects nonstationary decision making, GUI grounding, reranking, and reliability analysis through fixed-library selection using current-page feasibility evidence.The executor chooses among prior-guided skill and primitive candidates rather than learning new skills.Source: §2.5 · p. 3

3.1 Controlled Transfer Setting

• Aligned task families preserve goal, entities, required functionality, and outcome criteria while changing website conditions rather than redefining the task.Each family pairs a source context, where a skill is learned or selected, with an evolved target context.Source: §3.1 · pp. 3–4

3.2 Skill Environment Mismatch Modes

• Contextual Shift targets skill invocation, Perceptual Shift changes grounding evidence, and Execution Shift tests runtime feasibility through modal blockers.The perturbations preserve task-critical objects and functionality; Execution Shift tests recovery from interruptions while continuing the original task flow.Source: §3.2 · p. 4

3.3 Benchmark Instantiation

• The benchmark uses GitLab, Magento, and WordPress and constructs validated task families with aligned entities, automatic state scorers, and executable solutions across conditions.The construction process yields 355 natural-language tasks; evaluation uses six conditions spanning clean source and target contexts plus perceptual or execution shifts.Source: §3.3 · pp. 4–5

3.4 Diagnostic Metrics

• Success Rate measures task correctness, Average Steps interaction cost, Average Skill Calls library reliance, and Skill Failure Rate failures during attempted skill execution.Skill Failure Rate combines retrieval-level and executability-level failures and is unstable when skills are rarely invoked, so it is supplementary to Average Skill Calls in low-invocation regimes.Source: §3.4 · p. 5

4.1 SWOP as Prior-Guided Adaptation

• SWOP models a skill-guided executor under nonstationary execution conditions, anchored to a reference prior and updated with recent evidence about action reliability.Its information-state MDP sequence represents environment evolution through changing rewards and transitions.Source: §4.1 · p. 5
• With fixed problem-dependent factors suppressed, SWOP has sublinear dynamic regret when the variation budget grows sublinearly and the stated problem dimensions remain fixed.The guarantee assumes bounded rewards, fixed episode-invariant action domains, a full-support prior, stated sampling and concentration conditions, and an oracle-tuned fixed window.Source: §4.1 · p. 6

4.2 From SWOP to Practical Executor Design

• The SWOP design guide adapts from recent execution evidence, retains uncertainty-aware alternatives, and regularizes selection toward learned skill preferences.This translates into test-time selection among primitive and skill-action candidates based on current-page feasibility evidence.Source: §4.2 · p. 6

4.3 MPCR as Feasibility-Aware Candidate Reranking

• MPCR proposes primitive and skill-action candidates, checks current-page feasibility, scores executability, and reranks the shortlist.It selects only among proposed candidates, carries execution feedback into the next decision, and keeps the learned skill library fixed.Source: §4.3 · p. 6

5.1 Experimental Setup

• The evaluation compares AWM, ASI, SkillWeaver, and WALT across six source and transfer conditions, with a frozen ASI library shared by ASI and MPCR for RQ2 and RQ3.The benchmark reports success rate, average steps, average skill calls, and skill failure rate; the fixed-library comparison isolates test-time selection from additional skill learning.Source: §5.1 · p. 7

5.2 RQ1: How Do Existing Learned-Skill Methods Fail under Controlled Environment Evolution?

• 40.97% to 18.69%: pooled success falls from Base to Execution, while average steps rise from 8.49 to 11.99 and skill failure rate from 9.83% to 27.60%.Execution Shift exerts the strongest pressure; the diagnostics also reveal different failure channels, including longer trajectories and more skill calls for SkillWeaver.Source: §5.2 · p. 7, Figure 3 · p. 7

5.3 RQ2: Are the Observed Adaptation Dynamics Consistent with the SWOP Theoretical Account?

• 11.0, 20.0, and 32.0: MPCR's cumulative task-failure advantage over ASI increases from Base to Perceptual to Execution conditions.The observed pattern is consistent with SWOP's account: current feasibility evidence can be more useful when grounding changes or runtime blockers appear.Source: §5.3 · pp. 7–8, Figure 4 · p. 8

5.4 RQ3: Can Test-Time Executor-Side Adaptation Improve Transfer under a Fixed Skill Library?

• 37.90% to 41.26%: MPCR raises average success over ASI by 8.87% relative while sharing the same fixed skill library.Across all six environments, MPCR changes test-time selection rather than acquiring additional skills; average skill calls rise from 0.06 to 0.15.Source: §5.4 · pp. 8–9, Table 2 · p. 9

• SkillShift-Bench preserves task semantics while varying contextual, perceptual, and execution conditions that affect whether skills can be invoked, grounded, or executed.The benchmark is designed to expose skill-environment mismatch under same-site evolution.Source: §6 · p. 9
• SWOP formalizes skill-environment mismatch, and MPCR improves fixed-library transfer through feasibility-aware reranking.The paper concludes that robust web agents should complement reusable skills with test-time adaptation to current feasibility evidence under evolving environments.Source: §6 · p. 9

• The study focuses on self-hosted same-site evolution, leaving broader website coverage to future work.It also mainly uses relatively capable LLM-based web agents; smaller or resource-constrained models remain for future study.Source: Limitations · p. 9
• The benchmark focuses on English desktop web interactions, with multilingual, mobile, and accessibility-oriented settings identified as extensions.Source: Limitations · p. 10

• Evaluation uses seeded task data, synthetic or benchmark-specific accounts, and automatic grading, without interacting with live third-party services, real user data, or private accounts.Released artifacts are limited to permitted benchmark-specific materials for reproducible robustness analysis under controlled-use settings.Source: Ethics Statement · p. 10

A Comparison with Existing Web Agent Benchmarks

• Unlike general robustness benchmarks, SkillShift-Bench measures whether skills acquired in a source environment remain executable on the same task after changes to invocation context, grounding evidence, or runtime preconditions.The source is where skills are acquired, selected, or induced; the target tests their transfer while preserving task intent, entities, functionality, and grading semantics.Source: A Comparison with Existing Web Agent… · p. 14

B Benchmark Construction Details

• The benchmark construction pairs aligned task families across source and evolved target contexts, with environment selection and shifted variants designed for controlled evaluation.The construction covers environment and version selection, task-family validation, and implementation of shifts.Source: B Benchmark Construction Details · p. 14

B.1 Environment Selection and Version Pairing

• Three self-hosted systems—GitLab, Magento, and WordPress—provide contrasting interface evolution while preserving task-level semantics and data alignment.GitLab and Magento use version pairs; WordPress changes themes while holding seeded content fixed. Version pairs require substantial interface change and feasible data migration.Source: §B.1 · pp. 15–17

B.2 Shift Perturbation Implementation

• Runtime modules implement Perceptual Shift and Execution Shift, while version or theme pairings implement Contextual Shift.The interventions preserve task semantics while changing grounding evidence or action feasibility.Source: §B.2 · p. 18

B.2.1 Perceptual Shift Implementation

• Perceptual Shift deterministically alters visual presentation and DOM-surface cues without changing task data or functionality.Fixed seeds make repeated runs consistent, and transformations also apply to dynamically inserted elements throughout an episode.Source: §B.2.1 · p. 18

B.2.2 Execution Shift Implementation

• Execution Shift inserts a modal blocker that intercepts otherwise valid actions, requiring agents to detect and dismiss it before resuming the task.The controller confines interruptions to the task execution window and records recovery after dismissal.Source: §B.2.2 · p. 18

B.2.3 Injection Regimes for Execution Shift

• Main-paper Execution Shift results use the once schedule, so one successful recovery leaves the page unobstructed for the rest of the episode.The authors describe this conservative schedule as a lower bound on runtime-interruption pressure; multiple and random schedules are also available.Source: §B.2.3 · pp. 18–19, Table 4 · p. 19

B.3 Task Validation Protocol

• Before inclusion, task families are manually checked for valid instructions, correct scoring, available functionality, and a solution path under all six evaluation conditions.The protocol establishes solvability, not a human performance reference; the benchmark reports no human baseline.Source: §B.3 · p. 19

B.4 Shift-Induced Failure Modes

• Perceptual and Execution Shift failures roughly double trajectory length and almost always exhaust the step budget, while Contextual Shift causes smaller increases and truncates fewer than half of cases.The first pattern indicates unrecoverable looping; the contextual pattern indicates early termination on a wrong route or stale answer.Source: Table 5 · p. 19, §B.4 · pp. 19–20
• Contextual Shift breaks skill invocation: the agent fires a reporting step before re-grounding on a relocated order page and returns a stale figure in one action.The skill remains task-relevant, but the learned procedure no longer locates the target page state.Source: §B.4 · p. 20
• Perceptual Shift breaks skill grounding when altered DOM structure and locator signals turn a selection control into a table header, causing repeated type errors and a budget-exhausting loop.The perturbation preserves task-critical objects and functionality but violates the skill’s assumed mapping between control type and node.Source: §B.4 · p. 20
• Execution Shift breaks feasibility when a modal overlay intercepts correctly targeted interactions, and recognizing the obstruction does not prevent a ten-step loop without an answer.Repeated click and navigation timeouts are followed by the search-box identifier becoming unavailable after the overlay reshuffles the document.Source: §B.4 · p. 20
• Skill Failure Rate combines incorrect results after skill execution with correct steps that cannot be realized, so it is supplementary and unstable when skills are rarely invoked.The analysis distinguishes wrong-route or early-termination behavior, un-actionable-locator loops, and interception-induced stalls; ASC is used to report skill avoidance when the SFR denominator is near zero.Source: §B.4 · p. 20

C.1 Nonstationary Model and Objective

• SWOP models episodic environment change with a fixed feasible action domain, representing local-executability changes through nonstationary rewards and transitions, and optimizes reward against deviation from a behavioral prior.Its dynamic comparator is each episode’s own optimal KL-regularized policy, and its regret separates estimation error from environmental variation.Source: §C.1.1 · p. 21, §C.1.2 · p. 22, §C.1.5 · p. 23

C.1.6 Sliding-Window Empirical Model and Bonus

• SWOP uses recent episodes to estimate rewards and transitions, adds an upper-confidence bonus, and performs optimistic KL-regularized soft planning before executing the resulting policy.Discarding older data tracks environmental change and reduces drift-related bias.Source: §C.1.6 · pp. 23–24

C.2 Main Theoretical Guarantees

• The appendix states a high-probability fixed-window dynamic-regret bound and an oracle-tuned fixed-window upper envelope; under fixed problem complexity, sublinear regret follows when VK = o(K).The oracle-tuned window is selected before deployment as a deterministic function of the variation budget.Source: §C.1.5 · p. 23, §C.2.1 · p. 26, §C.2.2 · p. 26

C.2.2 Oracle-Tuned Window Corollary

• With fixed problem parameters, the tuned-window bound gives sublinear dynamic regret whenever VK = o(K), while growing parameters have corresponding sufficient conditions.The supplied passage does not state the explicit conditions for growing parameters.Source: §C.2.2 · p. 27

C.2.3 Proof Roadmap and Dependency Graph

• The proof roadmap combines near-optimism, certificate decomposition, summed-bonus control, martingale control, and window-drift counting into estimation, drift, and martingale terms.Source: §C.2.3 · p. 27

C.3.1 Soft Bellman Recursion

• KL regularization yields a closed-form softmax greedy policy and a unique Bellman maximizer, avoiding hard-argmax discontinuity and tie-breaking issues.Source: §C.3.1 · p. 28, §C.3.2 · p. 28

C.3.3 Log-Sum-Exp and Bellman Non-Expansiveness

• The supporting Bellman facts establish log-sum-exp non-expansiveness and monotonicity, translation equivariance, and non-expansiveness of the soft Bellman operator.Source: §C.3.3 · pp. 29–30

C.3.4 Boundedness of Soft Value Functions

• Optimal soft values lie in [0, H − h + 1], but arbitrary policy values need not share this envelope because KL penalties can make regularized rewards negative.The proof separately uses a conservative envelope for its executed policy, derived from clipped optimistic quantities.Source: §C.3.4 · pp. 30–31

C.3.5 Bellman Stability under Model Perturbation

• Bellman backup and cross-episode stability results yield a cumulative value-drift bound, but these are contextual; the main proof instead uses conditional-mean drift and window counting.Source: §C.3.5 · pp. 31–33

C.4.1 Sampling Model and Filtration

• The sampling framework requires predictable environments and plans based on completed history, with conditional reward means and transition laws but no required reward-transition independence.Source: §C.4.1 · p. 34

C.4.3 Predictable-Skipping Concentration

• The appendix supplies Azuma–Hoeffding control and predictable-skipping concentration, requiring hit indicators to be known before outcomes and forbidding conditioning on the full hit-location vector.Source: §C.4.2 · p. 35, §C.4.3 · p. 35
• Selected observations form a bounded martingale-difference sequence, enabling simultaneous concentration over selected-sample partial sums.When no observations are selected, no empirical average is formed; the cold-start bonus clipping mechanism handles that case.Source: §C.4.3 · pp. 36–37

C.4.4 Reward and Transition Concentration

• Reward and transition estimates concentrate uniformly around their conditional means, and Lemma C.15 combines these bounds into a total concentration event.The concentration statements apply to positive empirical counts; zero-count cases are covered by the clipped cold-start bonus.Source: §C.4.4 · pp. 38, 39, 41

C.5.1 Current True Model

• The stability and regret analysis separates nonstationarity, represented through window-average conditional models, from statistical concentration, represented through empirical models.This separation aligns the optimism and certificate arguments with the sliding-window estimator.Source: §C.5.1 · p. 41

C.5.2 Primitive Drift and Window Cumulative Drift

• The analysis defines a sliding window and its cumulative drift, using the cumulative quantity throughout the proof because it has a clean summability relation.For episode 1, the window is empty and the cumulative drift is zero.Source: §C.5.2 · p. 42

C.5.3 Window Reference Model

• The proof relies on the conditional-mean reference model, the concentration event, and window drift; it does not require an additional window-average model.Source: §C.5.3 · p. 42

C.5.4 Conditional-Mean Model

• Concentration of the empirical reward model is established around the conditional-mean model, which is then used together with drift in the proof.Source: §C.5.4 · p. 42

C.5.5 Empirical Model and Optimistic Planning Objects

• Subsequent optimism, certificate, summed-bonus, and dynamic-regret results use the same window-truncated bonus interface.The bonus convention uses (+1) to remove cold-start singularity and ∧W to match the sliding window's effective sample size.Source: §C.5.5 · pp. 43–44
• The empirical planning objects are horizon-clipped, and reward and transition deviations are absorbed into a unified bonus on the concentration event.The stated value ranges are 0 ≤ eQ_k^h(ξ,ω) ≤ H − h + 1 and 0 ≤ eV_k^h(ξ) ≤ H − h + 1.Source: §C.5.5 · p. 44

C.5.6 Model Difference Bounds

• Theorem C.1’s model-comparison argument uses only the conditional-mean drift bound, with the positive-count regime handled by Lemma C.18.The zero-count regime is covered instead by optimistic clipping and the maximal bonus.Source: §C.5.6 · pp. 45–46

C.5.7 Near-Optimism under Sliding-Window Estimation

• The analysis establishes single-step optimistic domination, including zero-count cases, and near-optimism under window drift.The single-step result handles both positive- and zero-count regimes; the window-drift proposition extends the guarantee across stages.Source: §C.5.7 · pp. 46–47

C.5.8 Certificate Decomposition

• A pathwise bridge identity yields a unified remainder, which is then bounded on the concentration event to obtain an episode-wise certificate bound.The certificate argument covers both positive-count and zero-count cases.Source: §C.5.8 · pp. 48–50

C.5.9 Summed Bonus Bound

• For any window length 1 ≤ W ≤ K, an overlapping-window counting argument gives a pathwise bound on the realized bonus along the true trajectory.The proposition is used pathwise in Theorem C.1.Source: §C.5.9 · pp. 50–51

C.5.10 Pathwise Martingale Control; Flatten the episode-stage indices

• Flattening episode-stage indices makes the unified remainder a bounded martingale-difference sequence, enabling high-probability control by concentration.The filtration includes the episode’s planning objects and current state before the stage outcome is generated.Source: Flatten the episode-stage indices · p. 52

C.5.11 Unified High-Probability Event; C.5.12 Proof of Theorem C.1

• The proof combines concentration and remainder-control events into Etot, then establishes all pathwise regret bounds and proves Theorem C.1 on that event.The supplied proof states P(Etot) ≥ 1 − δ.Source: §C.5.11 · p. 54, §C.5.12 · p. 55

C.5.12 Proof of Theorem C.1

• Theorem C.1’s proof decomposes dynamic regret into three contributions, controls summed bonuses and variation, and absorbs the martingale term into a logarithmic factor.The variation contribution is bounded by W(Br + HBp).Source: §C.5.12 · p. 55

C.6 From Fixed-Window Regret to the Main-Text Informal Statement

• The known-variation tuning statement chooses the window length before the run as a deterministic function of the variation budget, then applies the fixed-window theorem.Source: §C.6 · p. 55

C.6.1 Proof of Corollary C.2

• The proof yields A ≍ W^3/2(VK + 1); with fixed H, Sξ, and A, VK = o(K) suffices for DynReg(K) = o(K).A single deterministic window is selected before the run by minimizing the deterministic upper-bound expression; no simultaneous guarantee over window lengths is required.Source: §C.6.1 · p. 56

D.1 Correspondence between SWOP and MPCR

• Constrained shortlist reranking adds 4.31 points of success over the unmodified proposal prior, and feasibility evidence adds a further 2.14 points.The frozen skill library and base executor remain unchanged: adaptation uses current-page feasibility evidence within an episode, soft scoring retains uncertain candidates, and regularization limits reranking drift. Enlarging the candidate set reduces skill invocation, while the pressure-dependent gain pattern also appears with a structurally different learned library.Source: §D.1 · p. 57

D.2 Executor Interface and Execution Loop

• MPCR returns one executable next action per decision, feeding observed page changes, errors, blockers, or timeouts into the next step through proposal, verification, scoring, and reranking.The appendix describes fallback behavior for malformed generations, uncertain probes, ambiguous rankings, and weak shortlists.Source: §D.2 · p. 57

D.3 Candidate Interface and Proposal

• MPCR expands the base executor’s next-action preferences into exactly Ncand ranked candidates, then constrains all later decisions to that parseable candidate set.Each candidate is a complete executable action with a concise rationale; the common interface covers primitive actions and fixed-library skills without allowing unstructured direct actions.Source: §D.3 · pp. 58–59

D.4 Local Feasibility Annotation

• Feasibility annotations check whether targets are identifiable, visible, enabled, in the viewport, and unobstructed, while page-level reachability distinguishes hidden content from unavailable content.These signals remain soft: a scrollable page can make an unseen target a reason for an intermediate exposure action, rather than grounds to discard it.Source: §D.4.1 · p. 60, §D.4.2 · p. 60

D.5 Conservative Feasibility Scoring

• MPCR scores candidates using relevance, progress, readiness, exposure, stability, and risk; blocked or scroll-dependent actions are downweighted rather than categorically filtered.The scoring rule is fixed across reported evaluations, so gains do not rely on per-condition tuning or a hidden adaptive learner.Source: §D.5 · pp. 60–61

D.6 Constrained Prior-Guided Reranking

• A constrained LLM comparator reranks a verified shortlist using task state, candidate rationales, heuristic scores, and feasibility annotations, and may select only a provided candidate.Reported experiments use M = 2; the pairwise prompt compares the next action locally rather than replacing the executor with an open-ended planner.Source: §D.6 · pp. 61–62

D.7 Totality, Fallbacks, and Recovery

• Explicit safety-guard fallbacks keep MPCR well-defined and ensure it always returns an action under parser uncertainty, incomplete probes, or ranker ambiguity.These recovery rules are not additional adaptation mechanisms and preserve the fixed-library, candidate-constrained design.Source: §D.7 · p. 62
• MPCR preserves deterministic action selection when structured judgments are noisy, using extra deliberation only when feasibility signals are informative.Its fallbacks cover uncertain probes, ambiguous ranking output, and weak shortlists, avoiding skill relearning or unconstrained exploration.Source: §D.7 · p. 62

D.8 Design Validation

• Design validation tests the necessity of all four MPCR stages and the default candidate budget Ncand = 2.Both studies use Magento, GPT-5 mini, a fixed ASI library, and the same six evaluation conditions; full MPCR averages 39.18% on this single-site subset versus 41.26% pooled across three sites, so values are not directly comparable.Source: §D.8 · p. 62

D.8.1 Component Ablation

• 39.18% success with full MPCR falls to 8.09% with random selection, below the base executor’s 32.73%, indicating that shortlist structure and extra compute alone do not explain the gain.The randomized control shares the pipeline, feasibility annotations, and shortlist, but replaces comparison with a fixed-seed coin flip and uses no reranking call.Source: §D.8.1 · p. 63, Table 7 · p. 63
• Success rises from 32.73% to 37.04% with blind reranking, then to 39.18% with feasibility evidence; the 2.14-point increment holds in five of six conditions.Full MPCR exceeds both “Rerank only” and “Verify + Score,” so neither component subsumes the other.Source: §D.8.1 · p. 63, Table 7 · p. 63
• Full MPCR uses 0.24 skill calls per task versus 0.68 for blind reranking, and only 0.04 calls under Execution Shift.The authors interpret this as avoiding skill invocation when a runtime blocker makes execution infeasible.Source: §D.8.1 · p. 63, Table 7 · p. 63
• “Verify + Score” reaches 27.93% success versus 32.73% for the proposal prior, as hard top-1 selection can discard task-context judgment.Feasibility annotations are designed to inform the comparator, rather than replace it with a fixed-coefficient choice.Source: §D.8.1 · p. 63, Table 7 · p. 63

D.8.2 Candidate-Budget Selection

• Overall success is 39.18%, 35.92%, and 37.73% for candidate budgets of 2, 3, and 4, respectively; the authors favor 2 as a lightweight operating point.With one run per budget, they draw no conclusion about the exact ordering of budgets 3 and 4.Source: §D.8.2 · pp. 63–64, Table 8 · p. 64
• Under Execution Shift, pooled success rises from 24.56% to 31.25% to 32.08% as the candidate budget grows; under clean and perceptual conditions, pooled success falls from 46.49% to 40.55%.The execution figures are pooled across Src-Exec and Tgt-Exec; the non-execution pool includes Src, Src-Perc, Tgt, and Tgt-Perc.Source: §D.8.2 · p. 64, Table 9 · p. 64
• Skill calls per task fall from 0.24 to 0.05 to 0.02 as the budget grows, indicating dilution of the skill-library preference.The paper suggests a shift-adaptive budget as future work because the favorable trade-off reverses across shift types.Source: §D.8.2 · p. 64

E Experimental Details and Supplementary Analyses

• The supplementary studies run on Magento, whereas the main tables pool GitLab, Magento, and WordPress, so their absolute values are not comparable.The appendix covers transfer to a second skill-learning method, random-seed stability, stronger Execution Shift schedules, and comparison with target-environment relearning.Source: E Experimental Details and… · p. 64

E.1 Compared-Method Cards

• The compared baselines—AWM, ASI, SkillWeaver, and WALT—differ in skill forms, learning signals, agent frameworks, and action-selection strategies.The shared tasks and perturbations enable representative end-to-end comparisons, but do not remove all execution-stack confounds.Source: §E.1 · pp. 64–65

E.2 Skill-Library Supplements

• Across nine website–backbone combinations, SkillWeaver averages 18.0 library items, AWM 14.4, ASI 11.7, and WALT 6.0.The counted units differ in meaning across methods, so library size is descriptive rather than a larger-is-better comparison.Source: §E.2 · p. 66, Table 10 · p. 66
• Across methods and websites, retained-item totals are 199 for DeepSeek-V3.2, 142 for GPT-5 mini, and 110 for Claude Haiku 4.5.The paper distinguishes acquisition scale from transfer robustness, which also depends on executor-side selection, grounding, and execution under changed conditions.Source: §E.2 · p. 66
• Aggregated across methods and backbones, GitLab and Magento yield 174 and 171 retained items, respectively, versus 106 for WordPress.The authors use this site variation to avoid reducing the benchmark to a single library-density regime.Source: §E.2 · p. 66
• Larger libraries do not establish greater transfer robustness; transfer also depends on selecting, grounding, and executing retained items under changed conditions.The library counts reflect acquisition scale and method design, and the retained units are not semantically identical across methods.Source: §E.2 · p. 66

E.3 Backbone Analyses

• 46.30% (GPT-5 mini, Src) falls to 38.72% (Tgt), then 27.14% (Src-Exec) and 20.23% (Tgt-Exec), matching the degradation trend across all three backbones.DeepSeek-V3.2 and Claude Haiku 4.5 reach 16.01% and 9.89% success, respectively, on Tgt-Exec.Source: Table 11 · p. 67, §E.3 · p. 67
• GPT-5 mini has the highest overall success (SR = 35.23%), lowest skill failure rate (SFR = 13.52%), and lowest trajectory cost (AS = 8.77).DeepSeek-V3.2 reaches SR = 31.80% with AS = 11.93; Claude Haiku 4.5 reaches SR = 27.86% with SFR = 20.95%.Source: §E.3 · pp. 66–67, Table 11 · p. 67
• On Tgt-Exec, AS reaches 14.84 for DeepSeek-V3.2, 10.49 for GPT-5 mini, and 10.25 for Claude Haiku 4.5, while SFR reaches 33.78%, 19.77%, and 25.36%, respectively.GPT-5 mini combines higher success with lower SFR rather than relying on longer trajectories.Source: §E.3 · p. 67
• SkillWeaver remains weakest across backbones, while ASI's SFR ranges from 13.86% with GPT-5 mini to 39.51% with DeepSeek-V3.2.WALT reaches 40.46% success with GPT-5 mini, but its SFR remains 20.98%.Source: §E.3 · p. 67
• Similar skill-call frequencies (ASC = 0.80 for GPT-5 mini and 0.74 for DeepSeek-V3.2) accompany different success and SFR, so frequency alone does not indicate reliable skill use.Claude Haiku 4.5's ASC = 0.05 does not imply stronger robustness alongside its lower success.Source: §E.3 · p. 68

E.4 Generalization to a Second Skill-Learning Method

• With AWM's workflow library frozen, MPCR raises success in all six Magento conditions, peaking at +53.4% on Src-Exec and +84.2% on Tgt-Exec.AWM uses prompted textual workflows rather than executable skill invocations.Source: Table 12 · p. 68, §E.4 · p. 68

E.5 Stability across Random Seeds

• 18.9% pooled Magento success gain over ASI accompanies per-condition standard deviations no higher than 2.32 points across three seeds.MPCR's mean success exceeds ASI's in every condition; this is a variance characterization, not an independent replication.Source: §E.5 · p. 68, Table 13 · p. 69
• +66.7% is MPCR's largest relative gain on Src-Exec, while gains are near zero on Tgt and Tgt-Perc, where the fixed library already transfers well.MPCR is not worse than ASI in any condition.Source: §E.5 · p. 69, Table 13 · p. 69

E.6 Robustness to Repeated and Randomized Interruption

• MPCR exceeds ASI in all six interruption-regime and context cells, including schedules where blockers return after every navigation or appear at randomized times.Absolute success is sensitive to injection frequency: the multiple regime reduces MPCR from 30.70% to 20.35% in the source context and from 18.42% to 8.77% in the target context.Source: Table 14 · p. 69, §E.6 · p. 69
• The source-side margin narrows from +81.0% to +13.4% under the most aggressive schedule; rapid blocker reinstatement leaves transfer-time action selection limited headroom.The authors identify complementary recovery mechanisms as a future direction.Source: §E.6 · pp. 69–70

E.7 Comparison with Target-Environment Re-Learning

• 35.09% (target-native library, Tgt-Perc) and 14.91% (Tgt-Exec) trail MPCR's 18.42% under Execution Shift and fixed ASI's 16.67%.The target-native library was induced in Tgt and then evaluated under Tgt-Perc and Tgt-Exec.Source: Table 15 · p. 70, §E.7 · p. 70
• Re-learning's inference remains unmodified after its one-time induction cost, yet its fixed library still inherits the runtime feasibility gap; MPCR requires no target training data.The comparison therefore does not attribute re-learning's result to a compute-starved configuration.Source: §E.7 · p. 70

E.8 Compute Budget and Infrastructure

• Experiments used self-hosted browser environments and hosted LLM APIs; MPCR used at most two candidate actions per step and a top-two shortlist.No local GPU training or fine-tuning was used; infrastructure supported browser orchestration, logging, and automatic grading.Source: §E.8 · p. 70