PaperlayerView exact revision on arXiv

Source-linked AI summary

PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

Zayx Shawn

arXiv:2606.08106v1cs.AIcs.MA

TL;DR

Repeatedly accepting changes because a reused dev score rose turns self-evolution into uncontrolled adaptive testing. PACE applies a paired, anytime-valid commit test to each candidate. In the reported settings, it suppresses false commits while preserving genuine gains, with per-decision rather than run-level control.

  • Problem

    Self-evolving agents repeatedly reuse a noisy dev estimate to decide whether to commit changes, creating adaptive multiple testing and false commits.

    Source: §3 · p. 4

  • Method

    PACE compares each candidate with the incumbent on paired instances and uses a betting e-process to commit only when evidence crosses a threshold.

    Source: §4 · pp. 4–5

  • Results

    PACE reduces greedy’s 13–21 noise commits per run to approximately zero while preserving genuine gains at lower evaluation cost.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The paper concludes that reliable self-evolution requires attention to the acceptor as well as the proposer.

    Source: §7 · p. 9

    PACE is demonstrated in the authors’ prompt-evolution loop on one model family and one proposer; cross-system validation remains future work.

    Source: §6 · p. 9

Abstract

from arXiv · show

Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small held-out set. Almost all effort has gone into the proposer that generates candidates; we argue the weak point is the acceptor, the rule that decides whether to commit a change. Applied hundreds of times against the same noisy dev estimate, the ubiquitous "keep it if the score went up" rule is uncontrolled adaptive multiple testing: the agent effectively p-hacks itself, accumulating false commits that make it churn and drift rather than improve. We recast committing as a sequential hypothesis test and propose PACE (Paired Anytime-valid Commit Evaluation), a training-free, anytime-valid commit gate. Each candidate is compared to the incumbent on identical instances and committed only when a testing-by-betting e-process accumulates decisive evidence, stopping early to save evaluations and controlling each candidate's false-commit probability at a user-set level even under optional stopping (a per-decision guarantee). On Qwen2.5 agents (0.5B-3B) self-evolving at the prompt level on GSM8K, SVAMP, and ARC-Challenge, greedy acceptance commits 30-42% false and 10-33% harmful edits when a genuine improvement is hidden among noisy proposals, while PACE commits the real one and essentially nothing else, matching greedy's held-out accuracy at sharply lower variance and about 18% lower evaluation cost. With no real gain available, greedy commits 13-21 spurious self-modifications per run (72-100% false) and degrades the most fragile agent by 4.9 points, while PACE holds at baseline. Reliability of self-evolution depends on the acceptor, not only on the proposer.

1 Introduction

Self-evolution has focused on proposing changes, while the commit decision is often an unexamined rule to keep any change whose reused dev score rises. The paper frames that rule as adaptive multiple testing and introduces PACE, reporting fewer false commits while preserving genuine gains.

1 Introduction

  • A good acceptor should retain genuine gains while rejecting unhelpful proposals, addressing a decision step that has received less attention than candidate generation.The authors argue that better acceptance cheaply keeps proposer-found gains and rejects the rest.Source: §1 · p. 1
  • Keeping candidates whenever a reused dev score rises turns hundreds of noisy evaluations into adaptive multiple testing and can accumulate false commits that cause churn or degradation.Greedy acceptance is described as an unattended, uncorrected sequence of trials against one small validation set.Source: §1 · pp. 1–2
  • PACE is a training-free paired sequential gate that commits only when a betting e-process crosses a calibrated threshold, stopping early to limit evaluations.It compares candidates and incumbents on shared instances and uses paired outcomes rather than proposer internals.Source: §1 · p. 2
  • PACE drives greedy’s 30–100% audit-labelled false-commit rate to ≈0% and its 13–21 noise commits per run to ≈0 while preserving genuine gains at lower evaluation cost.The reported evidence covers controlled and stochastic regimes.Source: §1 · pp. 2–3
  • The evidence demonstrates PACE on prompt-level self-evolution; its applicability to richer agent loops is argued from its interface, not demonstrated.The paper explicitly distinguishes interface-level generality from system-level validation.Source: §1 · p. 3

2 Related Work

Prior work studies agent modification, prompt and workflow optimization, self-improvement, and sequential testing, but the paper identifies commit acceptance as a distinct gap. PACE applies anytime-valid testing machinery to that decision.

Self-evolving agents

  • Self-evolving-agent systems retain modifications by empirical performance, but prior work leaves acceptance as a statistical decision untreated.The cited systems span code, logic, prompts, skills, and workflows.Source: §2 · p. 3

Prompt and workflow optimization

  • Prompt and workflow optimizers search fixed development-set fitness, while this paper addresses the separate question of whether to commit a candidate.The passage describes this as an orthogonal question to optimizing the score.Source: §2 · p. 3

Self-improvement and its pathologies

  • Prior self-improvement work examines unreliable evaluation signals and their pathologies, whereas this paper focuses on the decision rule that consumes those signals.The cited examples include degradation, reward hacking, and diversity erosion.Source: §2 · p. 3

Sequential and anytime-valid testing

  • PACE imports e-process and testing-by-betting machinery, whose validity under optional stopping follows from Ville’s inequality, into agent commit decisions.The passage distinguishes this use from online error-control procedures over streams of tests.Source: §2 · p. 3

3 The Commit Decision in Self-Evolution

Self-evolution repeatedly compares proposed configurations using a small, reused held-out set, so greedy acceptance makes many adaptive decisions against the same noisy estimate. The paper motivates a commit rule that retains reliable gains without training or excessive evaluation.

Setup

  • A self-evolving agent maintains a configuration, receives a proposed modification each round, and uses a small held-out set to estimate configuration quality.The configuration here is a system prompt, and the accept rule either commits the proposal or retains the incumbent.Source: §3 · p. 3

Greedy acceptance is adaptive multiple testing

  • Because the same small dev set is reused across rounds, its noisy estimate is repeatedly queried as the agent makes adaptive comparisons.The number of comparisons grows with the run.Source: §3 · p. 4
  • When no genuine improvement is available, greedy acceptance can still accept candidates that win by chance, accumulating false commits and potentially harmful changes.The passage characterizes this as unregistered, uncorrected testing on a reused dataset.Source: §3 · p. 4

Why the obvious fixes fall short

  • A larger dev set only postpones reuse problems, fixed multiplicity corrections require a known test count, and refreshing the holdout assumes fresh labeled data.The paper argues that open-ended loops instead need an anytime-valid sequential test.Source: §3 · p. 4

What we want

  • The desired accept rule commits only with reliable evidence of improved true accuracy, preserves genuine gains, minimizes evaluation, and requires no training.The next section presents a rule intended to meet these goals.Source: §3 · p. 4

4 PACE: An Anytime-Valid Commit Gate

PACE evaluates the incumbent and candidate on the same instances, uses discordant outcomes in a betting e-process, and commits once evidence crosses a threshold. This gives a per-candidate false-commit bound under optional stopping, not a run-level familywise guarantee.

Paired evaluation

  • PACE compares incumbent and candidate on identical instances, using discordant outcomes and discarding ties to reduce instance-level variation.A candidate-only win is coded as a win in the paired comparison.Source: §4 · p. 4

Testing by betting

  • Starting from wealth 1, the betting process grows when candidate wins outnumber losses and remains near 1 on average under the null.The method bets a fraction of its wealth on each discordant pair being a candidate win.Source: §4 · p. 4
  • PACE commits as soon as its e-process reaches 1/α, bounding each candidate’s false-commit probability at α under optional stopping.If the evaluation budget ends before the threshold is crossed, the candidate is rejected; the described settings use α = 0.05 and λ = 0.5.Source: §4 · p. 4

Guarantee

  • Under the paired null, the e-process is a nonnegative supermartingale, and Ville’s inequality bounds the probability of ever committing a non-improving candidate by α.The guarantee holds for data-dependent stopping and is per candidate, not a run-level familywise bound.Source: §4 · p. 5
  • The guarantee does not control power: genuine improvements are committed only if evidence crosses the threshold within the evaluation budget.The loop-level adaptivity is treated empirically rather than claimed as a theoretical guarantee.Source: §4 · p. 5

Why so simple, and why anytime-valid

  • PACE adapts the number of evaluated pairs to the evidence, unlike a fixed-n test, and typically stops early with fewer model calls than continued evaluation.The paper reports that fixed-n tests require choosing the evaluation count in advance.Source: §4 · p. 5

5 Experiments

The experiments test whether PACE can retain genuine prompt improvements while avoiding noise-driven commits across controlled and stochastic settings. Across the reported runs, it preserves gains with fewer false commits, and in stochastic settings holds accuracy at baseline while greedy acceptance churns.

5 Experiments

  • The study evaluates frozen Qwen2.5-Instruct models from 0.5B to 3B on GSM8K, using a reused 40-problem dev split and a disjoint 120-problem audit pool.The controlled regime plants one known beneficial edit among noisy proposals; the stochastic regime samples at temperature 0.7 without a large available gain.Source: §5 · p. 6

5.1 Controlled: keep the real gain, reject noise

  • In the controlled regime, statistical gates keep the single planted improvement and reject noise, while greedy makes 3–3.4 commits per run with 30–42% false and 10–33% harmful commits.At 1.5B, all rules reach ∆≈+0.57; at 3B, PACE reaches ∆=+0.74±0.04 versus greedy’s +0.54±0.30, with about 18% fewer dev evaluations.Source: §5.1 · p. 6, Table 1 · p. 7

5.2 Stochastic: don’t chase sampling noise

  • 13.3–20.7 self-modifications per run and 72–100% false commits for greedy contrast with fewer than one commit per run and baseline-held accuracy for every statistical gate.Greedy’s edits lower the 0.5B agent’s held-out accuracy by 4.9±3.0 points; the 1.5B mean drop is 3.1±7.0 points, while 3B avoids net damage despite 100% false commits.Source: Table 2 · p. 7, §5.2 · p. 7, Figure 2 · p. 8

5.3 Sensitivity

  • Across α=0.01–0.10 and dev sizes n=20–80, the gate commits the planted gain at every setting with 0% false commits and ∆≈+0.57, while greedy remains 33–53% false.Online-FDR loses the gain at α=0.01 and recovers only part at n=20.Source: §5.3 · pp. 7–8

5.4 Generalization across tasks and domains

  • On SVAMP and ARC-Challenge, the same pattern holds: greedy makes 12–16 edits per stochastic run with 67–93% false commits, while the gate commits essentially none and holds baseline.In controlled runs, the gate captures planted gains with 0% false commits, including matching greedy’s +0.45 held-out gain on SVAMP-3B.Source: §5.4 · p. 8

6 Discussion and Limitations

PACE is most useful when evaluation noise makes spurious commits a risk; its power depends on effect size and the dev set’s coverage. The authors limit their claims to the stated evidence and per-decision guarantee.

6 Discussion and Limitations

  • In noisy, reused-dev settings, PACE suppresses noise-driven churn without sacrificing genuine gains; when gains are tiny, its power may be insufficient.The gate kept 78% of a +0.18 improvement in the milder-handicap case.Source: §6 · p. 9
  • When gains are too small for the evaluation budget to resolve, PACE may reject them; higher α or a larger dev budget trades precision for recall.The gate kept 78% of a +0.18 improvement rather than all of it.Source: §6 · p. 9
  • PACE inherits dev-set coverage and claims neither higher power nor system-level generality beyond its interface.The authors state that the false-commit bound is per decision, not run-level.Source: §6 · p. 9

7 Conclusion

The paper frames self-evolution’s commit step as uncontrolled adaptive testing on a reused dev set and presents PACE as a training-free paired anytime-valid test. It bounds each candidate’s false-commit probability per decision while preserving genuine gains at lower cost than greedy.

7 Conclusion

  • The unguarded commit step accumulates false acceptances when agents repeatedly reuse a dev set.The paper frames this as uncontrolled adaptive testing.Source: §7 · p. 9
  • PACE bounds each candidate’s false-commit probability at a user-set per-decision level while preserving genuine gains.It is a training-free paired anytime-valid test.Source: §7 · p. 9
  • PACE does so at lower evaluation cost than greedy, making the acceptor an overlooked part of the self-evolution loop.Source: §7 · p. 9

A Additional Results

The additional results report sensitivity to PACE’s α and dev size, plus cross-task results on SVAMP and ARC-Challenge. Across these settings, the gate retains planted gains and avoids greedy’s noise-driven commits.

A Additional Results

  • 0% false commits and an essentially constant ∆ across a 10× α range and 4× dev-size range show PACE retaining the planted improvement as greedy remains 33–53% false.The sensitivity results use the 1.5B controlled regime and three seeds.Source: Table 4 · p. 12
  • On SVAMP and ARC-Challenge, PACE keeps the planted controlled-regime gain with 0% false commits.Source: Table 5 · p. 12
  • On SVAMP and ARC-Challenge, greedy commits 12–16 edits per run with 67–93% false in the stochastic regime, while PACE commits approximately none and holds.Source: Table 5 · p. 12

Related papers

Loading 2606.08106v1…