PaperlayerView exact revision on arXiv

Source-linked AI summary

Visible but Not Avoided: State-Dependent Re-Proposal of Rejected Candidates in Catalog-Based LLM Optimization Loops

Jianlin Chen, Wenhui Chen, Chi Man Vong

arXiv:2610.15738v1cs.AI

TL;DR

The paper asks whether avoidance measured in static probes predicts re-proposals in an optimizer’s own trajectory, where rejected candidates are its earlier choices. It compares probes with catalog-based optimization loops and finds that probes underpredict re-proposal, while explicitly stating rejection status lowers it across models.

  • Problem

    Static probes test avoidance of rejected candidates the proposer mostly had little reason to choose, leaving unclear whether they predict avoidance of its own rejected choices in a loop.

    Source: §1 · p. 1

  • Method

    Three open LLM proposers improved a GUI agent by naming edits from a 40-edit catalog; the study compared clean probes with their own loops and tested rejection-status wording.

    Source: §1 · pp. 1–2, §4.1 · p. 5

  • Results

    0.32 [0.27, 0.37] macro-averaged loop-minus-probe rate; stating rejection status lowered re-proposal by −0.16 [−0.22, −0.09] relative to a neutral list of the same IDs.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    For catalog-based loops with exact edit identities, measure avoidance on the rejected sets those loops produce.

    Source: §7 · p. 8

    The study uses one synthetic GUI agent and one 40-edit candidate set; it does not study free-text optimizers whose re-proposals cannot be matched exactly by name.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLM optimization loops often show the proposer the candidates it has already tried and rejected, but leave it to the proposer to avoid them. We ask whether avoidance measured in a static probe predicts avoidance in the states an optimizer's own trajectory produces. In catalog-based loops where three open LLMs improve a GUI agent by naming one of 40 edits, a re-proposal is identified exactly by the edit's name. Clean base-state probes underpredict re-proposal in the proposers' own loops (macro-averaged loop-minus-probe rate 0.32 [0.27, 0.37]), and proposals of already evaluated harness-edit pairs consume 28-63% of valid proposer iterations. The two settings reject different edits: a probe's rejected edits are mostly ones the proposer did not choose, whereas a loop's are its own earlier choices, which remain preferred when the history is removed. Ordinary history lowers re-proposal for two proposers but raises it for the third, and the effects of failure evidence are model-specific. Stating the rejection status in one sentence lowers re-proposal in every model (-0.16 [-0.22, -0.09] relative to a neutral list of the same IDs). For catalog-based loops with exact edit identities, rejection avoidance should be measured on the rejected sets the loop produces.

1 Introduction

Showing a proposer rejected candidates does not enforce avoidance, and static probes may not represent the loop-generated rejected sets the proposer must avoid. In catalog-based GUI-agent loops, clean probes underpredict re-proposal, while stating rejection status lowers it in every model.

1 Introduction

  • LLM optimizers often show failed attempts and leave avoidance to the proposer; under cached, deterministic evaluation, an already evaluated candidate spends an iteration on a known outcome.In a loop, rejected candidates are the proposer’s own earlier choices, unlike arbitrary rejected candidates shown in a fixed probe.Source: §1 · p. 1
  • The study measures re-proposal by exact edit name in loops where three open LLMs improve a screenshot-only GUI agent by choosing from a 40-edit catalog.The loop distinguishes choosing an edit currently shown as rejected from proposing a harness–edit pair already evaluated.Source: §1 · p. 1
  • 0.32 [0.27, 0.37] macro-averaged loop-minus-probe rate; duplicate-pair proposals consume 28–63% of valid proposer iterations.Clean base-state probes showed no re-proposals for two models and rare re-proposals for the third, while all three re-proposed in their own loops.Source: §1 · p. 1
  • Without history, each proposer assigns more probability to its own loop’s rejected edits than a uniform choice would, while ordinary history lowers re-proposal for two models and raises it for Mistral.The comparison with other earlier choices was post hoc; those choices were enriched about as much as the rejected edits.Source: §1 · p. 2
  • −0.16 [−0.22, −0.09] re-proposal rate difference when rejection status is stated versus a neutral list of the same IDs; effects of failure evidence vary by model.Adding a prohibition had a further negative estimate (−0.05), but it did not pass the multiplicity-corrected test.Source: §1 · p. 2

2. Evidence across three proposers that clean-

The evidence indicates that clean probes do not capture the rejected sets produced by the proposers’ own loops. Matched-state interventions find model-specific failure-evidence effects but a consistent reduction when rejection status is stated.

2. Evidence across three proposers that clean-

  • Clean-probe avoidance does not transport to the rejected sets generated by the proposers’ own loops, where rejected edits remain preferred without history.The preference comparison is against a uniform choice over the menu.Source: §2 · p. 2
  • A static probe and a loop test avoidance on different rejected sets: the latter are generated by the proposer’s own trajectory.Source: §2 · p. 2
  • Matched-state interventions find model-specific effects of failure evidence and a consistent effect of rejection-status wording.Source: §2 · p. 2

2 Related Work

Prior work studies optimizer scores and trajectories, novelty enforcement, instruction following, and repetition from failure records. This paper focuses on rejection avoidance over trajectory-generated sets at the states optimization loops reach.

2 Related Work

  • Many LLM optimizers show prior candidates and expect the proposer to avoid them, whereas classical search and some LLM systems enforce exclusion through tabu lists, novelty archives, or rejection sampling.Source: §2 · p. 2
  • Instruction-following research tests constraints within a prompt, while this paper measures pressure toward a proposer’s own earlier choices at states reached by its loop.Related work also reports violations of negative constraints and prior pressure toward prohibited entities.Source: §2 · p. 3
  • Prior studies connect failure records with repeated calls, but this paper’s target is an evolving rejected set measured by exact candidate identity in an optimization loop.The related work describes repetition through surface form in tool-using agents and notes that a “do not repeat” instruction left that effect unchanged.Source: §2 · p. 3
  • The paper relates its findings to a gap between accessible memory and action, noting that two of three models show little measurable effect from failure content despite citing failures in their stated rationale.Source: §2 · p. 3

3 Measurement and Behavioural Accounting Framework

The framework defines re-proposal over a loop state and compares it with a uniform-menu reference. It separates observed changes in re-proposal from causal attribution to attraction or exclusion, which depends on explicit assumptions.

3.1 States and repeated proposals

  • A loop state contains the current harness, history, and rendered prompt; its visible rejected set consists of menu edits shown as tried with that harness and not kept.A duplicate-pair proposal instead counts an already evaluated harness–edit pair, including one whose rejection is no longer visible under a truncated history.Source: §3.1 · p. 3

3.2 A reference for re-proposal

  • Rejection-enrichment log-odds compare the proposer’s probability of choosing the rejected set with the probability under a uniform menu choice.The reference adjusts for the fraction of the menu that is rejected, but not for other state differences.Source: §3.2 · p. 4

3.3 Preference without history

  • The no-history comparison tests whether a reached state’s rejected edits remain attractive after removing the full history, but measures preference over a self-selected set rather than a causal effect.Source: §3.3 · p. 4

3.4 Accounting for changes between states

  • Under the accounting assumption, rejection enrichment decomposes into attraction toward the target set minus exclusion strength; the history contrast is a net change and does not separate those channels.Source: §3.4 · p. 4

3.5 What matched-state edits identify

  • Matched-state edits identify changes in observed re-proposal directly, while attributing those changes to attraction or exclusion requires channel-separation assumptions.The framework estimates enrichment with logistic regression and reports matched-state rate differences and stratified log odds ratios.Source: §3.5 · pp. 4–5

4 Experimental Design

The experiments compare static probes with optimizer-reached states, then separately edit failure evidence and rejection wording while holding prompt features fixed. The design also includes supporting analyses, preregistration, and state-level inference procedures.

4.1 Optimization setting

  • 36 loops cross three open proposers, two equivalent catalog realizations, two history views, and three seeds while improving a screenshot-only GUI agent.Each loop runs for 30 iterations; edits are kept when accuracy rises by more than δ = 0.02.Source: §4.1 · p. 5

4.2 Does static avoidance transport to reached states?

  • Clean probes use rejected base-harness edits, while reached-state tests sample eligible loop states and reconstruct them with selected history retained or removed.The clean probe varies window length, preferred-edit position, and catalog order; reached states are sampled in two rounds.Source: §4.2 · p. 5

4.3 Edits of the failure evidence

  • Failure-evidence conditions alter presence, text bulk, type, and attachment while controlling which history entries contain failures.Some conditions use matched neutral text or failure swaps; the strict non-ordinal within-app swap is available only on 13 states.Source: §4.3 · p. 5

4.4 Edits of the rejection signal

  • Rejection-signal conditions compare a neutral list, a sentence stating rejection status, an explicit prohibition, and the original prohibition wording.F − N tests the status wording and P − F tests the incremental explicit prohibition, with IDs and prompt position held fixed.Source: §4.4 · pp. 5–6

4.5 Boundary and supporting analyses

  • A supporting regime comparison tests instruct and thinking endpoints on 40 fresh states, with unpinned and provider-pinned runs.Two blinded annotators coded 60 replies using a preregistered eight-code taxonomy.Source: §4.5 · p. 6

4.6 Registration and inference

  • Experiments were preregistered before paid calls, with registered hypotheses, state-level units, clustered resampling, sign-flip tests, and reported call costs.The first 120 states and initial conditions were registered before analysis; later state edits were registered in subsequent rounds.Source: §4.6 · p. 6

5 Results

Across the main experiments, clean probes understate re-proposal in loops, failure-evidence effects vary by model, and stating rejection status lowers re-proposal. Supporting analyses compare proposer regimes and classify the rationales given for re-proposals.

5.1 Static avoidance does not transport to the rejected sets loops produce

  • With history removed, every proposer assigns more probability to its own rejected set than uniform choice; ordinary history lowers re-proposal for Gemma-4 and Qwen3-VL-32B but raises it for Mistral.The post hoc comparison finds other earlier choices similarly enriched, so it does not isolate a complete causal mechanism.Source: §5.1 · p. 6
  • 0.32 [0.27, 0.37] loop-minus-probe rate, positive in all six cells; duplicate-pair proposals consumed 28–63% of valid iterations.In clean probes two proposers never chose a shown rejected edit and the third did so only rarely.Source: §5.1 · p. 6, Table 1 · p. 7
  • Rejected-set composition may contribute to the probe–loop gap, but the post hoc comparisons do not quantify its causal contribution.The probe’s window contains one preferred edit, unlike loops where frequent probe choices can themselves enter the rejected set.Source: §5.1 · pp. 6–7

5.2 Failure evidence has model-specific text-bulk and content effects

  • −0.19 [−0.25, −0.13] and −0.17 [−0.23, −0.10] re-proposal changes followed deletion of failures; neutral replacement and attachment tests show model-specific effects and limit a pure semantic-content interpretation.Qwen3-VL-32B retained a substantial effect under neutral replacement, while most of Mistral’s deletion effect was recovered by restoring a neutral span.Source: §5.2 · p. 7

5.3 Restating rejection status lowers re-proposal across all three main proposers

  • −0.16 [−0.22, −0.09] re-proposal change: stating rejection status lowered re-proposal versus a neutral list in every model.Adding an explicit prohibition lowered the rate further, but its registered test was not supported.Source: §5.3 · pp. 7–8

5.4 Boundary conditions and supporting evidence

  • −0.40 [−0.55, −0.25] was the thinking-minus-instruct endpoint re-proposal difference on 40 fresh states; the difference persisted with provider-pinned routing.The endpoints also differed in post-training and completion tokens.Source: §5.4 · p. 8
  • Failure-target attraction was the most frequent stated rationale, while explicit retry was rare; the label does not establish an attachment-level causal mechanism.The two blinded annotators coded 16 and 21 of 30 sampled re-proposals as failure-target attraction, versus 2 and 3 as explicit retry.Source: §5.4 · p. 8

6 Discussion

The discussion distinguishes the rejected sets used in static probes from those produced by a loop’s own trajectory. It limits the interpretation of the comparisons and recommends evaluating avoidance on loop-produced sets.

6 Discussion

  • A static probe tests avoidance of rejected candidates the proposer mostly had little reason to choose, whereas a loop tests avoidance of its own rejected favourites.The same-harness reconstruction meets the registered equivalence margin on average, but the comparison does not separate harness and rejected-set composition effects.Source: §6 · p. 8
  • The self-selection comparison is post hoc and rests on few states, and history moves the three proposers in different directions.The paper therefore does not claim a single mechanism describes all three proposers.Source: §6 · p. 8
  • For catalog-based loops with exact candidate identities, the discussion supports measuring avoidance on loop-produced rejected sets; deterministic filtering is suggested for hard non-repetition but is not evaluated.Source: §6 · p. 8

7 Conclusion

The conclusion reports that visible rejections are not necessarily avoided in catalog-based loops. It recommends measuring avoidance on the rejected sets those loops produce when candidate identities are exact.

7 Conclusion

  • A rejection visible to an LLM proposer is not necessarily avoided.Source: §7 · p. 8
  • Without history, loop proposers put more probability on their rejected set than a uniform choice, while clean probes only partially represent such self-selected sets.Source: §7 · p. 8
  • For loops with exact candidate identities, avoidance should be measured on the rejected sets those loops produce.Source: §7 · p. 8

Limitations

The study is limited to catalog-based optimization of one synthetic GUI agent, and its identification and serving conditions constrain how broadly its findings can be interpreted.

Limitations

  • Results cover one synthetic GUI agent and one 40-edit candidate set; the second catalog changes presentation, not candidates or domain.Free-text optimizers, where re-proposals cannot be matched exactly by name, are not studied.Source: Limitations · p. 9
  • The no-history comparison assumes preferences remain stable at a harness and removes the accuracy trajectory; post hoc checks cannot exclude a modest harness-specific component.Interpreting rejection-status wording as exclusion also depends on Assumption 2, and failure type is nearly confounded with app.Source: Limitations · p. 9
  • Only three open 24–32B models and one 30B instruct/thinking pair were studied, with measured rates including serving nondeterminism.Main state interventions did not pin provider routing; smaller provider groups are unstable, and several contrasts lack provider-specific replication.Source: Limitations · p. 9
  • All annotators were LLMs, and human annotation of a held-out sample remains pending.Source: Limitations · p. 9

Ethical Considerations

The ethical discussion notes the oversight implications of cheaper automated self-improvement. The appendix formalizes comparisons of rejected-set preference and describes estimators and their assumptions.

Ethical Considerations

  • Lower re-proposal makes automated self-improvement cheaper, which inherits the oversight concerns of such loops.The study uses synthetic phone screens and involves no personal data or human subjects.Source: Ethical Considerations · p. 9

A Proofs and Estimators

  • The identities separate preference enrichment from exclusion, but the split between them is not identified from the serving distribution alone.Proposition 1 compares edits that preserve the rejected and available sets; its sentence-edit comparison relies on Assumption 2.Source: A Proofs and Estimators · p. 12
  • The no-history contrast reports the net effect of ordinary history on rejected-set mass, identifying exclusion only if history leaves preference unchanged.Source: A Proofs and Estimators · p. 12
  • The rejection-enrichment estimate uses a working likelihood because steps within a run are not independent.The stated maximum-likelihood interpretation requires constant enrichment across steps and independent replies given their states.Source: A Proofs and Estimators · p. 12
  • For state edits, the Mantel–Haenszel estimate is a weighted average when state-level log odds ratios differ.Its consistency statement assumes independent calls within state and condition.Source: A Proofs and Estimators · p. 13

B Registration, Claims and All Registered Tests

The appendix documents registration, tests, protocol checks, and verification of the study’s claims. Registered outcomes support the probe-to-loop gap and several intervention effects, while checks identify model-specific patterns and limits on interpretation.

B Registration, Claims and All Registered Tests

  • Registrations specified hypotheses, endpoints, sampling, tests, intervals, and stopping rules, but the Round 3 run stopped too early for its registered tests.Round 4 registered tests on 360 fresh eligible states before its calls; Round 2 follow-ups were registered after A–E results were seen.Source: B Registration, Claims and All… · p. 13
  • H1 and H1b were supported, with the positive gap in all six model–catalog cells; H2b was supported at −0.30 [−0.36, −0.24].H15 passed the ±0.10 equivalence margin but not ±0.05.Source: B Registration, Claims and All… · p. 14
  • Per-model average ∆ was 0.19 [0.13, 0.26] for Gemma-4, 0.58 [0.46, 0.67] for Mistral, and 0.19 [0.11, 0.27] for Qwen3-VL-32B.The catalog difference included zero for Gemma-4 and Mistral; it was +0.18 [0.02, 0.34] for Qwen3-VL-32B.Source: B Registration, Claims and All… · p. 14

C Protocol, Models and Costs

  • Provider routing was uncontrolled in most calls, and the authors note that sensitivity checks reduce but cannot eliminate concern about confounding.For F − N, the provider check was descriptive because assignment was not controlled.Source: C Protocol, Models and Costs · pp. 14–15

D Verification Log

  • Neutral replacement restored most of deletion’s effect for Mistral, had little effect for Gemma-4, and left a substantial effect for Qwen3-VL-32B.The authors describe the Qwen3-VL-32B pattern as consistent with a content-sensitive component.Source: D Verification Log · p. 17
  • The history raised re-proposal for Mistral (+0.31 [0.12, 0.49]) but lowered it for Gemma-4 and Qwen3-VL-32B; strict type swaps were scarce.Only 73 of 2,294 shown ordinal failures had an eligible unshown non-ordinal match, and the strict type-swap analysis was descriptive on 13 states.Source: D Verification Log · p. 17

E Supplementary Results

Supplementary results compare instruct and thinking regimes and summarize the round-four contrasts. The appendix also documents the two catalogs, the reply-annotation codebook, and the scope of its positioning matrix.

E Supplementary Results

  • On instruct-loop states, the thinking model’s re-proposal rate was lower by −0.62, and on its own loop states it was lower by −0.46.The instruct model gave 10 invalid replies on eligible loop steps; the thinking model gave none.Source: E Supplementary Results · p. 18, Figure 3 · p. 18

F The Two Catalogs

  • Catalog B rewords the 40 edits and groups options under prompt instructions, rendering, and decoding headings.Its task text, question, and history-line format also differ from catalog A.Source: F The Two Catalogs · pp. 18–19
  • Round-four contrasts show small deletion and content effects for Gemma-4, mostly non-semantic text-bulk effects for Mistral, and a negative status-sentence contrast for all three models.Table 11 presents fresh-state rate differences by model.Source: Table 11 · p. 19

G Codebook

  • The codebook defines eight registered rationale codes and reports κ = 0.80 agreement between annotator 1 and blinded DeepSeek.All three annotators agreed on 43 of 60 replies.Source: G Codebook · p. 19

H Positioning Matrix

  • The positioning matrix organizes each related-work line by what it establishes and which history-implied exclusion constraint it tests.The supplied text identifies the matrix’s scope but does not report its individual entries.Source: H Positioning Matrix · p. 19

I Earlier Evidence on the Same Benchmark

Earlier evidence on this benchmark includes probe-versus-loop comparisons, a near-miss analysis, instructed-loop runs, and a replay on violation-selected states. The paper labels the replay as biased and says the later random-state design replaces it.

I Earlier Evidence on the Same Benchmark

  • Four earlier open proposers declined their own rejected favourite in 81–100% of probe positions but declined visible rejections in only 28–82% of loop steps.These earlier results are reported as exploratory.Source: I Earlier Evidence on the Same Benchmark · p. 19
  • Near-misses were re-proposed in 467 of 4,921 pairs (0.095), versus 254 of 4,626 clearly failed pairs (0.055), but within-run rates were not consistently ordered.The authors conclude that rational retry of near-misses does not explain re-proposal.Source: I Earlier Evidence on the Same Benchmark · p. 19
  • In earlier instructed loops, a sentence prohibiting re-proposal changed the pooled rate from 0.33 (330/988) to 0.29 (292/997) on the same seeds..Source: I Earlier Evidence on the Same Benchmark · p. 19
  • The violation-selected replay reproduced visible re-proposals in 67 of 119 states, but selection biases rates upward and removal effects partly toward regression to the mean.One call per state could not separate an intervention from temperature-0 variation; the random-state design replaces this analysis.Source: I Earlier Evidence on the Same Benchmark · p. 19
  • In the earlier reasoning-mode result, gpt-oss-120b re-proposed in 16 of 327 loop steps (rate 0.05) after honouring 290 of 290 probe cells.Source: I Earlier Evidence on the Same Benchmark · pp. 19–20

Related papers

Loading 2610.15738v1…