PaperlayerView exact revision on arXiv

Source-linked AI summary

GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution

Geyi Yang, Zikun Qu, Xiang Li, Zhiyong Wang, Min Zhang, Shipei Zeng, Zhongxiang Dai

arXiv:2610.00948v1cs.LGcs.AI

TL;DR

GUI harness optimization must reconcile model intent with visual effects, variable execution outcomes, and recurring failures across tasks. GUI-HARVEST uses repeated multimodal evidence to make and validate bounded harness edits around frozen backbones. It improves held-out OSWorld-Verified performance across six backbones and transfers to WindowsAgentArena without further optimization.

  • Problem

    GUI harness optimization must address visual mismatches between model intent and observed effects, variable outcomes across repeated runs, and the challenge of making task-specific findings reusable across tasks.

    Source: §1 · p. 2

  • Method

    GUI-HARVEST aligns actions and model outputs with before-and-after screenshots, compares repeated executions within tasks, and validates bounded code edits derived from cross-task failure patterns.

    Source: §1 · p. 2

  • Results

    All six frozen backbones improve on held-out OSWorld-Verified; Qwen3-VL-32B-Instruct reaches 50.94% on the full suite (+12.33 points), while frozen GPT-5 gains 13.87 points on WAA at 50 steps.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    These results support harness adaptation as a mechanism for self-improving GUI agents with frozen backbones.

    Source: §6 · p. 10

    Remaining errors increasingly involve successfully executed actions aimed at the wrong object, route, or evaluator-relevant state, making further harness-only correction harder.

    Source: §4.6 · p. 9

Abstract

from arXiv · show

The executable harness surrounding a GUI model determines how observations are assembled, actions are executed, and verification, recovery, and termination are controlled. Compared with harness optimization for non-GUI agents, automatically optimizing this harness poses three coupled challenges: reconciling model intent with observed visual effects, diagnosing failures under variable execution outcomes, and identifying recurrent failure patterns across tasks and translating them into reusable runtime changes. We introduce GUI-HARVEST, an automatic harness optimizer that enables self-improving GUI agents with frozen backbone models. First, to ground diagnosis in observed action effects, it aligns model outputs and executed actions with before-and-after screenshots, tying findings to specific interface transitions. Second, to account for execution variability, it treats repeated runs of the same task as a joint evidence unit, using within-task comparisons to locate outcome-relevant behavioral differences. Third, it consolidates verified findings across tasks into recurring failure patterns, maps them to bounded source-code edits with predictions recorded before evaluation, and checks the predicted behavioral effects alongside task performance through repeated execution. Experiments on OSWorld-Verified show consistent held-out gains across six general-purpose open, GUI-specialized open, and proprietary backbone models; Qwen3-VL-32B-Instruct gains 12.33 points on the full suite. Frozen-harness transfer improves GPT-5 by 13.87 percentage points on WindowsAgentArena at 50 steps without further optimization. With the same backbone and initial harness, GUI-HARVEST outperforms Self-Harness and Meta-Harness, suggesting that GUI-specific diagnosis and validation help harness improvements generalize to unseen tasks. The code is available at https://github.com/GaryYang12345/GUI-HARVEST.

1 INTRODUCTION

GUI-HARVEST addresses GUI-specific harness optimization challenges by grounding edits in repeated visual execution evidence while keeping backbone models frozen. It improves held-out performance across six backbones and transfers to WindowsAgentArena without further optimization.

1 INTRODUCTION

  • GUI harness optimization must reconcile intent with visual effects, diagnose variable outcomes, and convert task-specific findings into reusable changes verified against recurring failures.Across backbones, 11.8–20.4% of Full tasks have both zero- and positive-score runs among three executions.Source: §1 · p. 2
  • GUI-HARVEST aligns model outputs and executed actions with before-and-after screenshots, compares repeated runs within tasks, and turns cross-task failure patterns into bounded code edits with predicted behavioral effects.The Validator checks repeated-run scores and predicted behaviors before promoting edits; the test set stays sealed until optimization ends.Source: §1 · p. 2, Figure 2 · p. 3
  • At 15 steps, Qwen3-VL-32B-Instruct reaches 50.94% on the full suite (+12.33 points), while all six frozen backbones improve on held-out OSWorld-Verified tasks.Frozen-harness transfer also raises GPT-5 by 13.87 points on WindowsAgentArena at 50 steps without WAA optimization.Source: §1 · p. 2
  • With the same backbone and initial harness, GUI-HARVEST outperforms Self-Harness and Meta-Harness; its final harnesses also outperform LFF’s released harnesses on both OpenCUA backbones.These comparisons suggest GUI-specific diagnosis and validation help produce improvements that generalize to unseen tasks.Source: §1 · p. 2

2 PROBLEM SETTING

The problem setting confines optimization to an executable harness around a frozen GUI model and specifies how tasks, optimization rounds, and candidate edits are controlled.

2 PROBLEM SETTING

  • The optimizer may edit only harness components; model weights, benchmark tasks, evaluators, environment infrastructure, and resource limits are outside its scope.A capability manifest specifies editable files and where new modules may be created.Source: §2 · p. 3
  • Tasks are split into search, validation, and sealed test sets, and only promoted edits advance the harness to the next bounded optimization round.Validation trajectories are hidden from the optimizer and contribute only aggregate scores; the sealed test set is evaluated after optimization stops.Source: §2 · p. 3
  • Performance on a task set is summarized by the repeated mean of benchmark raw evaluator scores.Source: §2 · p. 4

3 GUI-HARVEST

GUI-HARVEST adapts the executable harness around a frozen GUI backbone by connecting repeated task executions to evidence-grounded diagnoses, recurring behavior modes, bounded code edits, and validation. Its loop promotes edits only when they pass performance and behavioral checks, otherwise retaining the current harness and its verified evidence.

3 GUI-HARVEST

  • GUI-HARVEST connects repeated task executions to four optimizer roles that link observed failures to reusable harness changes and check their intended effects.A task comprises a user instruction, initial desktop-application state, and evaluator-scored outcome.Source: §3 · p. 4

3.1 THE SELF-IMPROVEMENT LOOP

  • Each round diagnoses the current harness, proposes an edit, and promotes it only after checks; rejection restores the prior harness, while an update ledger records edits, predictions, scores, and decisions.Runs are collected for search and validation tasks; search supplies detailed evidence, while validation contributes aggregate scores.Source: §3.1 · p. 4

3.2 EVIDENCE ANALYST: DIAGNOSIS FROM REPEATED EXECUTIONS

  • The Evidence Analyst compares repeated runs of a task, locates decisive divergences between intended actions, executed actions, and resulting screens, and verifies evidence consistency.A bundle enters diagnosis when at least one valid run scores zero; other runs provide comparison evidence, while behavioral interpretation remains the analyst’s assessment.Source: §3.2 · pp. 4–5

3.3 CROSS-TASK CLUSTERER: RECURRING BEHAVIOR MODES

  • The Cross-task Clusterer groups verified findings from independent tasks into recurring behavior modes, using outcome categories to constrain comparisons without prescribing mode names or patches.Modes require support from at least two tasks and retain their mechanism, behavioral target, and linked findings.Source: §3.3 · p. 5

3.4 HARNESS ENGINEER: SOURCE-AWARE INTERVENTIONS

  • The Harness Engineer selects compatible behavior modes and makes bounded, source-aware code edits, recording the evidence, proposed change, predicted effects, and invariants before GUI evaluation.Local tests are run on proposed changes.Source: §3.4 · p. 5

3.5 VALIDATOR: UTILITY AND BEHAVIORAL CHECKS

  • The Validator promotes a candidate only after code checks, no score decline on either search or validation with a strict gain on at least one, and a behavioral check of its predictions.Behavioral verdicts compare pre-edit findings and frozen predictions with post-edit trajectories; both splits contribute scores, but only search executions are available for diagnosis and behavioral validation.Source: §3.5 · pp. 5–6

4 EXPERIMENTS

Across six frozen backbones, GUI-HARVEST improves held-out OSWorld-Verified scores, with gains at larger step budgets, cross-benchmark transfer, and component ablations testing its design choices.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses a domain-stratified 361-task split, frozen target-model weights, 15-step search rollouts, and three independent runs per task.Search trajectories provide optimization evidence; Validation exposes aggregate scores, and Test remains sealed until selection.Source: §4.1 · p. 6

4.2 HARNESS ADAPTATION AND CROSS-BENCHMARK TRANSFER

  • Under the unified 15-step, K = 3 protocol, GUI-HARVEST improves Test and Full scores for all six frozen backbones; Qwen3-VL-32B-Instruct has the largest gains, including +12.33 points on Full.At a 50-step WAA budget, without using WAA trajectories for optimization, the Qwen3-VL-32B-Instruct harness gains 6.47 points and the GPT-5 harness gains 13.87 points.Source: §4.2 · pp. 6–7

4.3 MATCHED-BACKBONE OSWORLD COMPARISON

  • Matched-backbone comparisons show GUI-HARVEST leading every displayed comparison across six backbones and 15-, 50-, and 100-step budgets.At 15 steps, it exceeds CoAct-1 by 22.61 points with GPT-5 and VLAA-GUI by 21.68 points with Gemini 3.1 Pro.Source: §4.3 · p. 7
  • Increasing the budget from 15 to 100 steps adds 2.54 and 0.85 points for Qwen3-VL-8B and 32B, versus 5.77 and 6.00 for Gemini 3.1 Pro and GPT-5.The authors associate Qwen’s smaller gains with persistent long-horizon execution bottlenecks.Source: §4.3 · p. 8

4.4 ACCURACY–COST OPERATING POINTS

  • With GPT-5, the 15-step harness nearly matches Agent S3’s accuracy at approximately 72% lower full-suite API cost, while the compared systems use different step budgets.The reported pairs are 62.4%/$72 for GUI-HARVEST and 62.6%/$260 for Agent S3; Gemini reaches 79.1% at $125 versus Claude Sonnet 4.5 at 58.1% and $316.Source: §4.4 · p. 8

4.5 COMPONENT ABLATIONS

  • Removing visual evidence drops Test and Full to 44.54% and 40.69%; removing cross-task clustering yields 45.66% and 42.13%, below the complete method.Using one optimizer rollout per task trails the full method by 5.74 Test points and 7.82 Full points.Source: §4.5 · p. 9
  • The hard-gate-only variant selects a lower-scoring harness after nine rounds than GUI-HARVEST does after eight; K = 3 gives the strongest measured final harness at 50.9%.K = 5 raises rollout cost without improving the result, reaching 49.7%.Source: §4.5 · p. 9

4.6 INSIGHTS INTO HARNESS ADAPTATION

  • Adaptations differ by backbone, while residual failures involving valid-looking actions toward the wrong object, route, or evaluator state are harder to detect and correct through the harness alone.Qwen, frontier models, and OpenCUA receive different types of interventions.Source: §4.6 · p. 9

5 RELATED WORK

Related work spans GUI runtimes, trajectory diagnosis, and execution-driven agent improvement; GUI-HARVEST combines repeated visual evidence with cross-task pattern induction and post-edit validation.

GUI models and runtime frameworks

  • GUI runtime systems add varied planning, grounding, memory, verification, and recovery capabilities, but the cited frameworks are static at evaluation time.The paper studies execution-driven adaptation for a given frozen model.Source: §5 · p. 9

GUI trajectory diagnosis and improvement

  • GUI diagnosis work localizes errors from traces or screenshots, while reliability studies use repeated executions to separate sources of variability; GUI-HARVEST combines repeated visual evidence with cross-task edits and behavioral validation.LFF is described as an early GUI-domain effort to turn failed executions into runtime code changes.Source: §5 · p. 9

6 CONCLUSION

The conclusion presents GUI-HARVEST as a way to turn repeated multimodal execution evidence into reusable harness changes, and the appended material situates the method among prior optimizers and reports paired split results.

6 CONCLUSION

  • GUI-HARVEST improves held-out scores across six frozen backbones, including a 12.33-point full-suite gain for Qwen3-VL-32B-Instruct and a 13.87-point frozen-harness gain for GPT-5 on WindowsAgentArena.The WindowsAgentArena evaluation uses 50 steps without further optimization.Source: §6 · p. 10

A ADDITIONAL RELATED WORK

  • Prior self-improvement approaches retain execution feedback in memories, executable scaffolds, text, or source-code edits; GUI-HARVEST specializes harness updates to repeated visual evidence and validation.The related-work passage contrasts these approaches’ predominantly serialized source, language/tool traces, and score evidence with GUI-HARVEST’s GUI-specific evidence.Source: A ADDITIONAL RELATED WORK · p. 13

B.1 COMPLETE SPLIT-LEVEL ADAPTATION RESULTS

  • Table 4 reports paired 15-step results before and after optimization: Search and Validation guide selection, Test remains sealed until selection, and Full covers all 361 tasks.Parentheses report absolute percentage-point gains over the corresponding initial harness.Source: Table 4 · p. 13, §B.1 · p. 13

B.2 BENCHMARK VISUALIZATION AND DOMAIN-LEVEL RESULTS

The appendix compares OSWorld-Verified operating points across model families and reports domain-level results where published sources provide them.

B.2 BENCHMARK VISUALIZATION AND DOMAIN-LEVEL RESULTS

  • Tables 5–7 compare general open-weight, GUI-specialized open, and proprietary frontier systems while retaining published models, runtimes, budgets, and protocols.Domain entries are blank when the source reports only an overall score.Source: §B.2 · p. 13, Table 5 · p. 13, Table 6 · p. 14
  • At 50 steps, OS-Symphony with Qwen3-VL-32B-Instruct reports 46.86 Avg., with domain scores of 58.33 OS, 40.94 Office, 53.54 Daily, 75.10 Prof., and 31.24 Multi.The listed Qwen3-VL-32B-Thinking row reports 41.00 at 50 steps.Source: §B.2 · p. 14
  • Figure 5 contrasts frozen GUI-HARVEST harnesses optimized with 15-step Search rollouts against published systems at selected 15-, 50-, and 100-step operating points.Colored bars denote GUI-HARVEST; gray bars denote published systems in their reported configurations.Source: Figure 5 · p. 15

C REPEATED EXECUTION AND REPEAT-COUNT ABLATION

Repeated GUI runs can diverge because small environmental and rendering differences alter observations and subsequent actions. The repeat-count sweep weighs more stable estimates and diagnostic coverage against rollout cost and harness-selection outcomes.

C REPEATED EXECUTION AND REPEAT-COUNT ABLATION

  • Clean resets and in-task system or web events can change screenshots and action outcomes, with runs sometimes diverging after sharing several initial steps.Figure 6 illustrates runs that share four steps before diverging at step five.Source: C REPEATED EXECUTION AND REPEAT-COUNT… · p. 16, Figure 6 · p. 16
  • A K-run task bundle exposes both successful and failed paths when outcomes coexist, letting analysts compare where runs diverge rather than treat one branch as a stable mechanism.A failed run alone does not establish that a task is unreachable, and a successful run does not erase an informative failure path.Source: C REPEATED EXECUTION AND REPEAT-COUNT… · p. 16
  • Across backbones, 11.8–20.4% of tasks flip between zero and positive scores across three executions, making a single run an incomplete account of task outcomes.The highest reported rates are 20.4% for Qwen3-VL-32B-Instruct and 16.6% for Qwen3-VL-8B-Instruct.Source: C REPEATED EXECUTION AND REPEAT-COUNT… · p. 16, Table 8 · p. 16

C.1 CHOOSING THE REPEAT COUNT

  • Increasing K from 1 to 3 lowers the standard error from 1.61 to 0.93 points and reveals mixed zero- and positive-score outcomes in 20.4% of tasks.Increasing K to 5 lowers standard error further but raises rollout cost to 5×; the paper uses K = 3 for its main experiments.Source: §C.1 · pp. 16–17
  • 43.1% at K = 1 rises to 50.9% at K = 3, while K = 5 selects a weaker 49.7% harness despite higher rollout cost.The sweep uses Qwen3-VL-32B-Instruct; Figure 7 also shows the standard-error, flip-rate, and selected-score trends.Source: §C.1 · pp. 16–17, Figure 7 · p. 17

D.1 BEHAVIOR PROFILES ACROSS OPTIMIZATION

The behavior analysis links recurring observable patterns to runtime adaptations while distinguishing shared consequences from identical causes. The optimization paths and retained adaptations vary by backbone.

D.1 BEHAVIOR PROFILES ACROSS OPTIMIZATION

  • The cross-backbone view groups recurring patterns by runtime consequence, while explicitly not treating similar symptoms as identical underlying causes.The per-backbone behavior profiles retain model-specific quantities and residual bottlenecks.Source: §D.1 · p. 18, Table 10 · p. 19
  • Behavior labels describe observable patterns rather than unique causes; inefficient execution strategy excludes legitimate GUI work and short-budget overruns, unlike redundant loops without useful progress.Table 9 reports Search-set counts under initial and final harnesses; patterns can overlap and do not partition failures.Source: Table 9 · p. 18, §D.1 · p. 19
  • Infeasibility misjudgment appears across all six backbones, but premature-completion patterns recur across four and receive different runtime interventions.Interventions include feasibility checks or probes and terminal remapping for the first pattern, and save nudges, completion adjudication, persistence, or file normalization for the second.Source: §D.1 · p. 19
  • The adaptation granularity differs by model family: Qwen3-VL-8B uses action-level enforcement, while other families emphasize routing, persistence, completion policy, normalization, or terminal semantics.Qwen3-VL-32B combines enforcement with application-aware routing and state evidence; OpenCUA stays close to its coordinate-action interface.Source: §D.1 · p. 19
  • The remaining errors increasingly involve successfully executed actions aimed at the wrong object, route, or evaluator-relevant state, leaving less observable evidence for runtime guards.The paper links this lower observability to later candidate edits more often plateauing or trading gains across tasks.Source: §D.1 · p. 20

D.2 PER-BACKBONE HARNESS EVOLUTION AND FINAL CHANGES

  • Qwen3-VL-32B promoted seven updates in eight rounds with 25 candidate attempts; OpenCUA-32B promoted two updates in three rounds with 14 attempts.The optimization path depends on the backbone and initial harness; Figure 8 shows Search and Validation scores, promoted harnesses, candidate attempts, and final Test and Full scores.Source: Figure 8 · p. 20, §D.2 · p. 20

E.1 WINDOWSAGENTARENA BENCHMARK CONTEXT

The WindowsAgentArena section contextualizes transfer of harnesses optimized on OSWorld and identifies how its tables present transfer and comparison results.

E.1 WINDOWSAGENTARENA BENCHMARK CONTEXT

  • The reported transfer runs use GUI-HARVEST harnesses selected on OSWorld and frozen before WindowsAgentArena evaluation, with platform-specific observation, action, and logging adapters.The transfer setup states that OSWorld optimization precedes WAA access.Source: §E.1 · p. 20, Table 11 · p. 20
  • Table 11 reports same-backbone transfer at 50 steps, while Table 12 places results in the broader six-domain WAA context.The published Qwen operating point in Table 11 is attributed to Yang et al. (2026a).Source: Table 11 · p. 20, §E.1 · p. 20
  • Table 12 covers six WAA task domains and contrasts frozen OSWorld-derived harness transfers with published rows that retain their reported models, runtimes, and step budgets.For OS-Symphony, Avg. covers all 154 tasks, including a separately reported 13-task infeasible subset; the displayed domain columns cover WAA application domains.Source: Table 12 · p. 21

F.1 INFERENCE COST OF THE FROZEN HARNESS

The section reports frozen-harness accuracy and API-cost measurements, then details model settings, evaluation protocols, runtime starting points, optimization records, evidence roles, and data-split controls.

F.1 INFERENCE COST OF THE FROZEN HARNESS

  • At 15 steps, Qwen3-VL-32B-Instruct rises from 38.6%/$27.3 to 50.9%/$34.2; GPT-5 and Gemini improve accuracy while lowering cost.These are end-to-end measurements of frozen harnesses; Figure 9 compares full-suite API cost and score at 15, 50, and 100 steps.Source: Figure 9 · p. 21, §F.1 · p. 21

G.1 MODEL USAGE AND INFERENCE SETTINGS

  • Target backbones include Qwen3-VL-8B/32B-Instruct, OpenCUA-32B/72B, Gemini 3.1 Pro, and GPT-5, with target-model settings fixed within each experiment.GPT-5 uses the dated gpt-5-2025-08-07 endpoint, and optimizer roles use Claude Sonnet 5.Source: §G.1 · p. 22

G.2 ENVIRONMENT AND BENCHMARK DETAILS

  • The main OSWorld-Verified protocol uses 361 tasks, excluding eight Google Drive tasks, with environment configuration and task groups fixed before optimization.The final full-361 run uses the frozen harness and stopping decision.Source: §G.2 · p. 22

G.3 INITIAL HARNESSES AND ACTION CONTRACTS

  • Agent S3 is the starting runtime for Qwen and proprietary backbones, whereas OpenCUA starts from its official coordinate-action runtime.Agent S3 uses a separate grounding stage; OpenCUA directly represents coordinate-bearing PyAutoGUI actions.Source: §G.3 · p. 22

G.4 OPTIMIZATION AND ABLATION PROTOCOL

  • Search uses a 15-step budget and K = 3 runs per task; frozen harnesses are also evaluated at 15, 50, and 100 steps without further edits.The protocol allows at most ten rounds and five valid candidates per round; component ablations remove one element at a time.Source: §G.4 · pp. 22–23

G.5 PER-BACKBONE OPTIMIZATION RECORDS

  • The optimization records summarize promoted rounds and candidate-edit attempts across six backbones, while Tables 9 and 10 connect behavior changes to runtime mechanisms.The candidate-attempt counts and retained behavior changes are reported in the optimization records.Source: §G.5 · p. 23

G.6 ROLE INTERFACES, PROMPTS, AND SCHEMAS

  • Task-local analysts and validators work alongside a cross-task clusterer and bounded harness engineer; step-aligned evidence links model text and actions to before-and-after screens.Findings retain run and step evidence, while validators return task-level verdicts and citations.Source: §G.6 · p. 23

G.7 SPLIT AND LEAKAGE CONTROLS

  • Search, Validation, and Test are disjoint sets of 80, 80, and 201 tasks, with Test identifiers and outcomes withheld until the harness is frozen.The assignment is fixed before rollout and reused across backbones.Source: §G.7 · pp. 23–24

G.8 COMPARISON METHODS

The evaluation compares harnesses under controlled protocols and defines how GUI-HARVEST collects execution evidence, discovers recurring behaviors, and promotes edits. Its formal loop pairs task performance checks with verification of predicted behavioral changes.

G.8 COMPARISON METHODS

  • Harness comparisons pair initial and optimized systems under fixed backbones and protocols; published rows retain their reported settings, while LFF is evaluated through released patches because its optimization procedure cannot be rerun.Table 2 reports within-backbone comparisons without aggregating Qwen and OpenCUA blocks.Source: §G.8 · pp. 24–25

G.9 RUNTIME VALIDITY AND FAILURE HANDLING

  • Runs with startup, screenshot, evaluator, or unrecoverable runner failures are invalidated and rerun under the same predeclared retry policy for current and candidate harnesses.A task enters diagnosis when at least one valid run scores zero; zero-score runs are finding targets and other runs provide comparison evidence.Source: §G.9 · p. 25

G.10 EVALUATION METRICS AND COST SCOPE

  • The primary metric averages raw evaluator scores across runs within each task and then across tasks; reported gains are absolute percentage-point differences, and evaluation API costs exclude optimization.Repeated outcomes are classified as always positive, never positive, or a flip when runs include both zero and positive scores.Source: §G.10 · p. 25

H FORMAL DEFINITIONS FOR GUI-HARVEST

  • The formal optimization loop reuses or builds search bundles, diagnoses failures, clusters behavior modes, proposes and evaluates edits, records verdicts, and tests the frozen final harness on sealed data.The appendix also specifies utility and score-based acceptance conditions.Source: H FORMAL DEFINITIONS FOR GUI-HARVEST · pp. 25–26

H.1 EXECUTION EVIDENCE AND FINDINGS

  • Each task-round bundle separates shared task and harness information from repeated run trajectories, scores, and runtime metadata; findings link decisive steps to evidence and successful-run contrasts.Findings are checked against their cited run and step evidence before clustering.Source: §H.1 · p. 26

H.2 BEHAVIOR MODES AND UPDATE PLANS

  • The clusterer groups findings into recurring behavior modes, while update plans connect a target mode and evidence to a source edit, intervention, predictions, target tasks, and invariants.Predictions are recorded before candidate evaluation and checked in subsequent executions.Source: §H.2 · pp. 26–27

H.3 CANDIDATE SCORING AND PROMOTION

  • Promotion requires the utility hard gate and behavioral checks of every prediction; rejection preserves the current harness, while the ledger records edits, predictions, score changes, verdicts, and decisions.The utility gate requires both changes to be nonnegative and at least one to be strictly positive.Source: §H.3 · p. 27

I A COMPLETE EVIDENCE-TO-EDIT CHAIN

The example traces an edit failure from repeated runs through cross-task clustering to a bounded harness change and behavioral validation. It distinguishes text visible in a field from a committed edit before accepting task termination.

EDIT_NOT_COMMITTED_AT_DONE

I A COMPLETE EVIDENCE-TO-EDIT CHAIN

  • Two Chrome runs fail after calling done() while the requested name remains in edit mode, whereas a successful run exits the field before termination.The clusterer links this discrepancy to analogous unsaved edits in Impress and Calc, forming the EDIT_NOT_COMMITTED_AT_DONE mode.Source: I A COMPLETE EVIDENCE-TO-EDIT CHAIN · pp. 27–28, Figure 11 · p. 30

EDIT_NOT_COMMITTED_AT_DONE

J REPRESENTATIVE ROLE PROMPT TEMPLATES

  • Representative role prompts require evidence-linked failure diagnosis, cross-task modes spanning at least two tasks, pre-edit predictions, and behavioral checks against post-edit trajectories.The analyst compares runs to find outcome-relevant divergence, while the validator judges frozen predictions rather than task success or utility.Source: J REPRESENTATIVE ROLE PROMPT TEMPLATES · pp. 30–32

Related papers

Loading 2610.00948v1…