PaperlayerView exact revision on arXiv

Source-linked AI summary

Safe Harness Self-Evolution: A Theoretical Analysis of Feasibility and Limits

Qianshu Cai, Yonggang Zhang, Jun Nie, Maohao Ran, Huajiang Zheng, Jun Song, Xinmei Tian, Yike Guo, Wei Xue

arXiv:2609.08175v1cs.AI

TL;DR

Persistent harness self-evolution can improve some tasks while harming retained tasks, raising questions about when modifications can be safely generated, certified, and adopted. This paper develops a theoretical analysis showing that generation and certification impose distinct limits, so stagnation can persist despite improvement opportunities and current safe improvement does not ensure future progress.

  • Problem

    Assessing modifications only on previously failed tasks is insufficient because persistent changes may degrade performance on other user tasks.

    Source: §3.1 · p. 4

  • Method

    The paper analyzes generation, finite-data certification, selection, safe adoption, and successor improvement under failure-task and retained-task requirements.

    Source: §1 · p. 2, §2 · p. 3, §3.1 · p. 5

  • Results

    Generation and certification impose distinct constraints: larger candidate pools need not resolve qualification uncertainty, while finite-run improvement guarantees do not ensure further successor improvement.

    Source: §1 · p. 3, §9 · p. 19

  • Takeaways & Limitations

    The analysis provides a basis for diagnosing stagnation and designing generation, certification, and update mechanisms for safer self-evolution.

    Source: §1 · p. 2, §9 · p. 19

    The guarantees assume a frozen language model, fixed reward function, specified user-task distribution, and average retained-task change control.

    Source: §8 · p. 18

Abstract

from arXiv · show

Harness self-evolution is the process by which an agent modifies its prompts, tools, code, or orchestration in response to task feedback while keeping the underlying language model frozen, with changes persisting across subsequent tasks. We provide a systematic theoretical analysis of the feasibility and limits of safe harness self-evolution, connecting modification generation, finite-data certification and selection, safe adoption, and behavior after an update. Under a fixed user-task distribution, we establish conditions guaranteeing overall expected-reward improvement while controlling changes on retained tasks, characterize the probability of generating qualified modifications, and derive finite-data bounds for safe selection and adoption. Our analysis shows that generation and certification impose distinct constraints: current task performance does not determine the probability of generating qualified modifications, and generating more candidates need not improve the guarantee of a successful update when evaluation is limiting. Stagnation may therefore arise even when improvement opportunities remain. We further show that worst-case evaluation cost for recognizing genuine improvements diverges as expected reward approaches its upper bound. Across successive updates, certified improvement guarantees accumulate over a finite run, but a successful update does not by itself guarantee that further improvement remains possible. These results provide a basis for diagnosing bottlenecks and designing safer self-evolution mechanisms.

1 Introduction

The paper develops a theoretical analysis of safe harness self-evolution, linking qualified-modification generation, finite-data certification and adoption, and successor-agent improvement. It separates the conditions and limits governing safe current improvement from those governing continued progress across updates.

Analysis scope

  • The analysis connects modification generation, finite-data certification, safe adoption, and successor-agent behavior to characterize when harness self-evolution is feasible.Harness self-evolution modifies prompts, code, tools, or orchestration while keeping the underlying language model frozen and changes persistent across tasks.Source: §1 · p. 2, Figure 1 · p. 2

Expected reward and reachability

  • Under a fixed user-task distribution, a target combining failure-task improvement with tolerated retained-task change can guarantee a prescribed expected-reward increase.Reachability—the probability of generating a modification satisfying the target—is not determined by current task performance, even with identical frozen models, task behavior, and qualified-modification sets.Source: §1 · p. 2

Certification and adoption

  • Finite-data bounds connect reachability and candidate-pool size to qualified-modification selection, while simultaneous coverage and abstention provide guarantees for safe adoption.Uncertainty from finite generation samples is distinct from uncertainty about whether evaluated modifications satisfy the target; generating more candidates need not resolve the latter.Source: §1 · p. 3

Limits to safe improvement

  • Safe improvement is impossible when the current state cannot supply the target’s required failure-task contribution, and worst-case evaluation cost diverges near the reward upper bound.Certification thresholds can also conflict with current performance and evaluation precision, although accounting for a modification’s retained-task loss can certify positive improvement beyond a baseline rule’s limit.Source: §1 · p. 3

Further improvement after updates

  • Certified improvement guarantees accumulate across a finite run, but a successful update does not ensure further improvement or preserve the rate of qualified modifications.The paper constructs a first-step success followed by no additional improvement and reports a descriptive association between positive-estimate selection and a lower fraction of subsequent qualified modifications.Source: §1 · p. 3

2 Problem Setting

This section formalizes harness self-evolution as generating, evaluating, and safely adopting harness modifications while the underlying language model remains frozen. It defines the reward and finite-evaluation framework and introduces questions about improvement, certification, continuation, and bottlenecks.

Process definition

  • Harness self-evolution generates harness modifications from task failures, evaluates valid candidates, and adopts a selected modification to define the successor agent.The harness comprises evolvable prompts, code, tools, and orchestration, while the language model remains frozen.Source: §2 · pp. 3–4

Candidate generation

  • The framework models generation as a finite pool of valid modifications and failed runs, with only valid modifications undergoing behavioral evaluation.At a fixed history and context, N generation runs produce candidates in M⊥, where ⊥ denotes failure to return a valid modification.Source: §2 · p. 3

Evaluation framework

  • Agent quality is measured by expected reward V(A, t) on each task and overall reward Juser(A) under the fixed user-task distribution Duser.Finite evaluation uses m independent rollouts for each evaluated agent-task pair.Source: §2 · p. 3

Safe adoption

  • The certifier uses finite evaluation results to determine whether a generated modification has sufficient evidence for safe adoption; otherwise, the incumbent is retained.After an update, the successor processes later user tasks and begins the next self-evolution step.Source: §2 · p. 4

Research questions

  • The paper studies improvement and generation, finite-data certification, limits of continued evolution, and bottlenecks in generation, certification, selection, and post-update continuation.These questions are organized across Sections 3–6 and summarized in Figure 1(b).Source: §2 · p. 4

3 Qualified Modifications and Reachability

This section defines qualified modifications by requiring failure-task improvement while limiting retained-task change, then characterizes reachability and limits on selecting qualified candidates. It also derives necessary conditions for positive reachability and shows that target and generation process, not benchmark performance or the frozen model alone, determine qualification probability.

3.1 Qualified Modifications

  • A qualified modification must improve failure-weighted tasks by at least λ while keeping retained-task change within δ, yielding an expected-reward gain of at least λ − (1 − Z_k)δ.Thus, positive overall improvement requires λ − (1 − Z_k)δ > 0.Source: §3.1 · pp. 5–6

3.2 Reachability and the Candidate-Pool Upper Bound

  • For a fixed candidate pool, no selector can exceed the probability that the pool contains a qualified modification; evaluation can realize this bound but cannot create a qualified candidate.The bound applies to selectors that choose a pool member or retain the incumbent, and also covers independently tested and repaired modifications when reachability is computed after repair.Source: §3.2 · p. 6

3.3 A Necessary Condition for Positive Reachability

  • Positive reachability requires the incumbent’s failure-task reward shortfall to satisfy the target’s necessary inequalities; if either fails, reachability is zero under every generation, selection, or evaluation process.Satisfying the inequalities is not sufficient: a qualified modification may still not exist or receive positive probability from the generation process.Source: §3.3 · p. 7
  • The frozen language model and user-task benchmark score alone do not determine qualified-modification probability, which also varies with the target, Agent state, context, and generation process.The proposition constructs configurations with identical frozen models and user-task behavior but different generation probabilities, and different targets with different qualified-set probabilities under the same process.Source: §3.3 · p. 7

4 Finite-Data Certification and Safe Agent Updates

The section develops finite-data certification and finite-horizon update protocols that support safe modification selection while accounting for uncertainty, incomplete data, and retained-task changes. It shows that certification probability can worsen with larger candidate pools, while accepted updates receive probabilistic reward guarantees without guaranteed improvement at every step.

4.1 Finite-Data Certification and Selection

  • Finite-data certification guarantees that certified modifications are qualified and bounds the probability of selecting one from a conditionally i.i.d. candidate pool.The construction uses simultaneous confidence intervals and pre-pool interval-width bounds, with coverage conditional on the realized candidate pool.Source: §4.1 · p. 8
  • Increasing candidate-pool size need not improve the successful-update guarantee because simultaneous interval widths scale as O(sqrt(log(N/βstat)/neff)) and can shrink the certifiable qualified subset.More candidates can increase pool coverage, but at fixed effective evaluation sample size they can reduce P+_k(T), so the lower bound need not increase with N.Source: §4.1 · p. 8
  • Shared generation context means mean reachability does not generally determine the probability that a candidate pool contains a qualified modification.Concavity yields C_k,N(T) ≤ 1−(1−P̄_k(T))^N, which is a valid but generally overestimating upper bound.Source: §4.1 · p. 8

4.2 Safe Agent Updates under Finite Task-Arrival Horizons

  • The finite-horizon protocol abstains when required generation or evaluation data are unavailable, and otherwise estimates failure gains, retained-task changes, and failure rates using separate or stored samples.Two-Gate applies simultaneous error radii and accepts only when its thresholds and observable check hold; rejection or abstention retains the incumbent.Source: §4.2 · p. 9
  • Theorem 4 bounds the probability of accepting a modification whose expected-reward gain falls below the certified bound by βstep,k, including the incomplete-data abstention branch.The guarantee controls unsafe acceptance over the entire update attempt but does not ensure improvement at every step.Source: §4.2 · pp. 9–10
  • Measured-margin is less conservative than Two-Gate when retained-task increases are not counted as losses, and every matched-tolerance Two-Gate acceptance is also accepted by Measured-margin.The converse can fail when a small estimated retained-task decrease permits a positive certified margin cM_k.Source: §4.2 · p. 10

5 How Far Can Safe Self-Evolution Proceed?

The section characterizes when safe-improvement acceptance remains feasible and shows that certifying further improvements can require diverging evaluation effort as incumbent shortfall vanishes. It also separates infeasible targets, rare generation, and evaluation uncertainty as distinct causes of stagnation.

5.1 Limits on Improvement

  • Acceptance is possible exactly when δmax > 0 and 0 < δ < δmax, while δ ≥δmax cannot be rescued by changing τ or generating more modifications.More accurate evaluation relaxes finite-data restrictions, but the retained-task tolerance term remains even with exact evaluation.Source: §5.1 · p. 11
  • The threshold constraints impose a failure-rate lower bound that combines retained-task tolerance with finite-evaluation error; increasing nF alone does not remove the limitation.Figure 2 separates the exact-evaluation restriction from the additional finite-data restriction and shows that other evaluation error bounds must also decrease.Source: §5.1 · p. 11, Figure 2 · p. 12

5.2 Evaluation Cost of Further Improvement

  • Algorithm 3 can certify improving modifications near the reward ceiling, but every positive margin is bounded by the incumbent’s shortfall uk.The sufficient threshold contains no sampling radius at a fixed state, whereas certification using a modification’s own retained-task loss can reach the stated boundary.Source: §5.2 · pp. 12–13
  • E−[Neval] ≥ Zk 10uk: on a hard two-task family, arbitrary level-β certifiers require a worst-case expected rollout budget that diverges as uk ↓0.The lower bound applies to certifiers with certification probability at least 1/2 and permits arbitrary task-selection policies and data-dependent stopping rules.Source: §5.2 · p. 13
  • A low acceptance rate alone cannot identify whether stagnation reflects an absent target, rare qualified-modification generation, or evaluation uncertainty.The section assigns corresponding remedies to reconsidering the target or state, improving generation, or changing evaluation design, budget, or certification.Source: §5.2 · p. 13

6 Diagnosing Self-Evolution

The section separates generation, certification, and selection as distinct bottlenecks, develops finite-data reachability bounds, and shows empirically that certification can fail despite qualified modifications. Across updates, accepted-step improvement guarantees accumulate, but they do not ensure future improvement opportunities remain.

6.1 Acceptance Rates

  • Certification probability can lower-bound reachability under coverage assumptions, but low certification remains compatible with either rare qualified generation or insufficient evidence.If nonreturn produces full-range rectangles, the resulting inner set may be empty and the bound uninformative.Source: §6.1 · pp. 13–14
  • Qualified modifications can be lost at certification or selection: audit-qualified candidates may lack certified evidence, while available certified candidates may be rejected or not selected.Changing the certifier or selector does not change generation reachability, enabling diagnosis of where adoption fails.Source: §6.1 · pp. 13–14

6.2 Estimating Reachability

  • Uniform confidence bounds estimate reachability from inner and outer rectangle counts, separating uncertainty from finite evaluation from uncertainty in the generation sample.More generated candidates reduce sampling error but may not resolve evaluation ambiguity; comparisons across targets still require additional context samples for population-level means or quantiles.Source: §6.2 · p. 14

6.3 Empirical Studies

  • In DS-1000, all pools contained audit-qualified modifications, but only 2/96 independent-generation modifications were certified and none of 192 revision modifications was certified.High-budget estimates often met the target, while simultaneous intervals remained too wide; under the stricter target, reachability differences remained unresolved and feedback did not establish higher reachability.Source: Table 1 · p. 16, §6.3 · p. 16

6.4 Self-Evolution over Multiple Steps

  • Across K updates, accepted-step lower bounds add to a final expected-reward guarantee under controlled total risk, without requiring independence or guaranteeing how many updates are accepted.A successful first update does not imply later qualified modifications exist: unrestricted constructions permit Pk(T) = 0 thereafter, and four DS-1000 sequences saw target-qualified generation fall from .906 initially to .438 after two opportunities.Source: §6.4 · p. 17

7 Relation to prior work

Prior work spans persistent harness evolution, theoretical self-improvement, generation-versus-selection analysis, and safe policy improvement. This paper connects these strands by analyzing persistent modifications, finite-data certification, retention, adoption, and later improvement opportunities under a common user-task distribution.

Harness self-evolution

Theoretical analyses of self-improvement

  • Theoretical self-improvement work studies improver utility, reachable hypothesis classes, validation-controlled risk improvement, statistical adoption, and sequential acceptance, while this analysis unifies generation, certification, selection, retention, and post-adoption improvement [Zelikman et al., 2024, Wang et al., 2025b, Wang et al., 2025a, Wu et al., 2025, Shawn, 2026, Sengupta, 2026, Scrivens, 2026].SEA leaves safety under endogenous updates as an open composition problem, whereas this paper analyzes finite-data certification and further improvement after adoption.Source: §7 · p. 18

Generation, selection, and iterative improvement

Safe policy improvement

  • Unlike policy-improvement methods that constrain action distributions, use off-policy tests, or preserve poorly covered baseline actions, this analysis compares incumbent and candidate agents under a common user-task distribution with separate improvement and retention requirements [Schulman et al., 2015, Thomas et al., 2015, Laroche et al., 2019].It distinguishes limits imposed by the target and current state from limits imposed by a particular certification rule.Source: §7 · p. 18

8 Limitations and Future Work

The analysis is limited by its fixed reward and task-distribution assumptions, finite-data resolution, and studied experimental settings. Future work should improve generation, certification, and adoption while preserving opportunities for later improvement and validating design choices empirically.

Limitations

  • The guarantees assume persistent modifications with a frozen LLM, a fixed reward function and user-task distribution, and average retained-task reward-change control.Tail-level and cumulative-change protections are included, but stronger subgroup guarantees, initial-agent retention tolerances, task-distribution response, and ground-truth utility require additional bounds or conditions.Source: §8 · p. 18
  • Finite generation samples do not resolve uncertainty about qualification when evaluation is imprecise or modifications lie near the target boundary.The studies illustrate these distinctions, but short DS-1000 update sequences and a fixed WorkBuddy–DSH modification bank limit broader conclusions about design choices across updates.Source: §8 · p. 18

Future Work

  • Future work should analyze adaptive search and successive updates to determine how feedback, modification structure, and current improvement affect later qualified-generation opportunities.Existing results cover conditionally i.i.d. complete-generation runs, with extensions for fixed repair and finite predeclared assembly; unrestricted updates show that current improvement alone is insufficient to predict later behavior.Source: §8 · pp. 18–19
  • Improving safe adoption requires sharper reachability measurements and more efficient finite-evaluation certification, including boundary-mass conditions and extensions of variance-adaptive and pooled rollout analyses.These refinements should connect achievable certification power at a fixed budget to the paper’s worst-case lower bound on evaluation cost while addressing stronger retention requirements and changing task distributions.Source: §8 · p. 19
  • Empirical work should compare certifiers, generation procedures, and update rules on common or newly sampled candidate pools while tracking whether later improvement opportunities persist.Repeated independent-task rollouts, more generation contexts, and longer update sequences could test safe adoption and transfer from reused suites to fresh tasks.Source: §8 · p. 19

9 Conclusion … C Assumptions and evaluation conditions

This section consolidates the paper’s framework for analyzing safe harness self-evolution, distinguishing generation, certification, and post-update improvement limits. It defines the notation, finite-sample quantities, assumptions, and evaluation conditions used to study reachability, drift, task-distribution response, selection, and confidence.

9 Conclusion

  • Safe self-evolution is limited by distinct bottlenecks: generation cannot reach targets beyond the current state, while certification limits can sometimes be overcome by accounting for retained-task loss.Reliable recognition of genuine improvement may still require increasing evaluation effort, with worst-case cost diverging as expected reward approaches its upper bound.Source: §9 · p. 19
  • Uniform confidence bounds separate finite-generation uncertainty from unresolved qualification under finite evaluation, explaining why adoption rates alone can obscure stagnation mechanisms.Certified improvement guarantees accumulate across a finite run under per-step conditions and risk allocation, but a safe update does not ensure that its successor can improve further.Source: §9 · p. 19

A.1 Formal common setting (§2)

  • The formal model freezes the language model while allowing persistent edits to prompts, tools, orchestration, templates, and sub-agent topology, with validity checked before behavioral certification.Generation runs return either a valid modification or failure, and candidate pools record complete runs including failures rather than sampling until enough valid modifications appear.Source: §A.1 · p. 22
  • Agents are measurable Markov kernels over user and self-modification tasks, and comparisons use fixed expected reward under the user-task distribution rather than the architectural self-task quantities.The target reward lies in [0, 1]; optional uniform fidelity to ground-truth utility converts an expected-reward gain γ into a ground-truth gain of at least γ −2ϵtar.Source: §A.1 · pp. 22–23

Gates and estimators

  • The notation formalizes failure-task improvement, retained-task deviation, certification gates, evaluation horizons, plug-in estimators, concentration radii, and per-step and run-level confidence budgets.These quantities support the paper’s finite-data guarantees for safe selection and adoption, including the accumulated run-level bound βrun.Source: Appendix A · p. 24, Advantage and deviation · p. 24, Gates and estimators · p. 25

Target, reachability, and selection probabilities (§3.2–§3.3; §4.1; §6.1)

Reachability measurement and fixed suites (§6)

  • Reachability measurement and fixed suites (§6) define probability weights, reward-change measures, qualified modification sets, empirical frequencies, confidence terms, measurement sizes, and reward-variance bounds.These quantities specify how reachability and fixed-suite outcomes are measured and certified.Source: Reachability measurement and fixed… · p. 26

Multi-step and drift (§6)

  • Multi-step and drift (§6) introduces per-step distribution drift and tracks accepted reward changes, lower bounds, and expected increments incorporating drift terms.The notation includes a uniform positive lower bound over a declared interval and expected drift-adjusted improvement quantities.Source: Multi-step and drift (§6) · p. 26

Task-distribution response (Appendix H)

  • Task-distribution response (Appendix H) distinguishes self-induced expected reward, comparative and realized improvement, response-modulus bounds, and the probability of substantial per-task reward changes.The appendix also disambiguates overloaded symbols, including β, target and margin variables, and N versus Nmeas.Source: Task-distribution response (Appendix H) · p. 27

B Guide to the appendices

  • B Guide to the appendices maps common notation, evaluation bounds, reward fidelity, proofs, extensions, task-distribution response, and measurement details to their respective appendices.Numbered results retain their identifiers when restated.Source: B Guide to the appendices · p. 27

C Assumptions and evaluation conditions

  • C Assumptions and evaluation conditions registers which assumptions and protocol conditions support the results, including common task distributions, data separation, stored-sample coupling, failure-task coverage, and candidate-pool restrictions.It separates reward, task-distribution, and stochastic-process assumptions from evaluation-protocol conditions, while listing unused assumption families separately.Source: C Assumptions and evaluation conditions · p. 27

C.1 Reward and task structure … D.4 Sequential stopping (always-valid confidence sequences)

The framework assumes stable task and reward-comparison conditions, separates generation from evaluation, and permits broad modification distributions while making reachability and scope limits explicit. Its optional evaluation methods use paired rollouts, variance-adaptive bounds, and always-valid confidence sequences to improve efficiency without changing the main guarantees.

C.1 Reward and task structure

  • Cross-Agent comparisons use the same user-task distribution, with bounded rewards and uniform pointwise reward fidelity preserved across successor agents.Rewards lie in [0, 1], advantages in [−1, 1], and adaptive generation, selection, and updating do not weaken the uniform fidelity condition.Source: §C.1 · p. 28

C.2 Task stream, sampling, and data hygiene

  • The task stream is modeled as stationary β-mixing, while disjoint generation and evaluation data, stored-sample coupling, and fresh-rollout separation prevent reuse-induced dependence.Common-offset records yield a coupling defect bounded by 2nRβ(ℓ), with additional conditions controlling failure-indexed batches and candidate evaluation.Source: §C.2 · p. 29

C.3 Additional assumptions for selection and reachability

  • Selection draws a fixed number of i.i.d. terminal outputs and chooses a valid modification or retains the incumbent, without conditional redrawing, repair, or composition.A local boundary condition supports continuity of reachability probabilities as interval widths vanish, but is not required by the main bounds.Source: §C.3 · p. 30

C.4 Scope conditions (not assumptions)

  • The analysis studies frozen LLM weights and persistent harness modifications; numerical informativeness depends on reward variance, declared failure-rate ranges, and returned-sample radii.The stated [0.01, 0.3] failure-rate range is an example design rather than a full-theory assumption.Source: §C.4 · p. 30

C.5 Assumption families not required by the main results

  • The main results require neither monotonicity in expected reward, small-step dynamics, nor factorized generation, and they allow general modification distributions.Stronger subgroup guarantees impose additional task-level reward-change constraints.Source: §C.5 · p. 30

C.6 Scope note on unlisted assumptions

  • The failure detector is separate from reward, no additional geometric assumptions are imposed, and task-distribution drift is handled explicitly; positive reachability for a fixed target requires Juser(Ak) ≤1−λ.Changing the target, generation context, or update rule requires reevaluating reachability rather than extrapolating this condition.Source: §C.6 · p. 30

D Paired evaluation and variance-adaptive bounds; D.1 Paired sampling setup; D.2 Variance under paired sampling

  • Paired rollouts share external randomness while keeping LLM decoding independent, and positive within-pair covariance reduces evaluation variance relative to independent sampling.For equal variances and correlation ρt ∈[0.7, 0.95], the variance ratio is 1−ρt, corresponding to a reduction factor from 3.3 to 20.Source: §D.1 · pp. 30–31, §D.2 · p. 31

D.3 Empirical Bernstein for ˆδ; D.4 Sequential stopping (always-valid confidence sequences)

  • Empirical-Bernstein bounds adapt to task-specific paired-estimator variance, while sequential evaluation requires time-uniform confidence sequences before stopping at a threshold within a declared maximum budget.Fixed-budget radii cannot simply be reevaluated after data-dependent stopping, and no particular stopping boundary or cost bound is derived.Source: D Paired evaluation and… · p. 30, §D.3 · p. 31, §D.4 · p. 31

D.5 Evaluation cost for certification

Paired empirical Bernstein certification can reduce fixed-budget evaluation requirements when paired differences have low variance, but the analyzed fixed-m gate cannot certify positive improvement at m = 50. Alternative sequential stopping mechanisms are noted, although their sample complexity and admissible regions are not derived.

Fixed-budget certification

  • m = 833 versus m ≈2.8 × 10^3 reduces the matched-risk evaluation requirement by about 3.3× with paired Bernstein.The calculation compares gate-side fixed-budget radii, not empirical amortized costs.Source: §D.5 · p. 32

Sequential alternatives

  • Mechanism (M3) uses pre-specified rollout budgets, but its amortized sample reduction depends on the unknown probability near the target boundary and is not estimated.Sequential reachability measurement and always-valid confidence sequences could replace the fixed radius while preserving coverage, but their stopping complexity and admissible regions remain undetermined.Source: §D.5 · p. 32

Fixed-budget certification

  • m = 50 is outside the fixed-m gate’s numerical admissible region and cannot certify a positive paired difference under the analyzed radius.At this budget, the paired-radius additive term is approximately 1.38 > 1, while the fixed-m Hoeffding bound fails separately.Source: §D.5 · p. 32

E Reward fidelity and ground-truth utility · F Proofs and supporting statements

The section shows how guarantees based on fixed reward r transfer to ground-truth utility r∗, including finite-sample improvement bounds, and organizes the supporting proofs and conditions for Sections 3–6.

E Reward fidelity and ground-truth utility

  • Under Assumption 2, expected-reward guarantees established for fixed reward r transfer directly to the corresponding guarantees under ground-truth utility r∗.The certification rule and finite-sample evaluations use r, while the transfer applies for every agent A.Source: E Reward fidelity and ground-truth… · p. 32
  • The transfer proof compares the reward- and ground-truth-based definitions and bounds the integrand’s absolute value by ϵtar under Assumption 2.This establishes the stated fidelity relationship between the two utility measures.Source: E Reward fidelity and ground-truth… · p. 32
  • Finite-sample lower bounds on expected-reward differences transfer on the same confidence event after subtracting 2ϵtar.No additional probabilistic event is required for this transfer.Source: E Reward fidelity and ground-truth… · p. 32
  • When Theorem 4 provides a confidence-event improvement bound ∆k, Corollary 3 applies with γ = ∆k.This instantiates the ground-truth utility transfer using the lower bound already supplied by Theorem 4.Source: E Reward fidelity and ground-truth… · p. 32
  • The fixed-reference endpoint identity decomposes utility change along any update sequence through finite telescoping, with rejected steps contributing zero.The zero contribution follows because rejected steps leave Ak+1 = Ak, while the bound follows from Proposition 6.Source: E Reward fidelity and ground-truth… · p. 33

F Proofs and supporting statements

  • The supporting-statement groups provide the formal conditions and proofs underlying Sections 3–6, with the general assumption register located in Appendix C.This subsection serves as the organizational overview for the paper’s formal details.Source: F Proofs and supporting statements · p. 33

F.1 Decomposition and retained-task changes · F.2 Generation and reachability · F.3 Certification and selection from a candidate pool

This section analyzes retained-task changes, modification reachability, and finite-data certification for selecting qualified candidates in a single self-evolution step. It develops guarantees accounting for generation limits, pilot uncertainty, dependence, and candidate-pool effects, without bounding eventual reward across multiple updates.

F.1 Decomposition and retained-task changes

  • Reward-level retained-task distance can be zero even when outcome distributions differ, because it ignores reward-preserving remappings that distribution-level KL detects.The distance is bounded, reflexive, reference-asymmetric, and obeys a fixed-reference difference bound; the overall reward guarantee satisfies J_user(Ã) − J_user(A_k) ≥ L_Ak(Ã) − (1 − Z_k)δ.Source: §F.1 · pp. 33–34

F.2 Generation and reachability

  • Near the attainable improvement endpoint, qualified-and-safe generation probability follows a Weibull-type tail governed by the endpoint index ϑ_k,δ and a slowly varying factor.The endpoint s_k,δ is bounded by the incumbent’s global reward shortfall, s_k,δ ≤ Z_k ā_k ≤ 1 − J_user(A_k).Source: §F.2 · pp. 35–36
  • Reachability is not determined by the frozen language model or current user-task behavior: changing self-task composition can alter qualified-modification probabilities while preserving user-task performance and target sets.For a target with qualified-set probability p, N independent draws have any-hit probability 1 − (1 − p)^N; distinct targets can have different reachability under the same distribution.Source: §F.2 · p. 36

F.3 Certification and selection from a candidate pool

  • Under Theorem 3’s pre-pool separation, interval-width, independence, and risk conditions, every certified modification satisfies the target, while inner-target candidates are certified and yield lower and upper selection bounds.Evaluation bundles can achieve deterministic interval-width bounds of order log(N/β_stat)/n_eff; finite-order Markov structure alone does not imply the required independence.Source: §F.3 · pp. 36–37
  • Pilot uncertainty reduces certified gains through an added margin and failure allowance, with any separate event validating the pilot-based bound requiring its own allocated risk.The margin is w̄_Zδ, while pilot invalidity contributes β_Z; an additional environmental-validity event contributes β_env unless already included.Source: §F.3 · p. 38
  • Increasing candidate count raises the any-hit probability but can enlarge simultaneous confidence widths, so the certified lower bound need not be monotone in N.An optimal N is meaningful only after fixing the feasible budget, evaluation allocation, and dependence model.Source: §F.3 · p. 38
  • Applying the any-hit formula after averaging candidate reachability over contexts is generally optimistic because the formula is concave in the context-specific probability.Reporting dispersion or quantiles alongside the mean captures the context variation hidden by context averaging.Source: §F.3 · p. 39
  • The reachability and candidate-pool coverage quantities characterize one self-evolution step and do not bound the reward an agent can eventually attain across multiple updates.The distinction limits what single-step certification guarantees imply about long-run self-evolution.Source: §F.3 · p. 39
  • Under the stated certification conditions, a candidate pool containing a qualified modification yields a positive expected-reward gain, while failure events incur an additive bounded loss allowance.Proposition 12 uses g_T := λ − (1 − Z_k)δ > 0; for Corollary 5, g_T ≥ γ on the pilot-validity event, with its risk and mixing residual included in the final failure term.Source: §F.3 · p. 39

F.4 Task-stream evaluation and updates … G.20 Further directions for harness self-evolution (§8)

This section develops a theory of dependence-aware certification and repeated harness updates, covering task-stream evaluation, finite-data guarantees, reachability, distribution drift, and retained-task constraints. It also characterizes computational, statistical, observation-lag, recollection, and feasibility limits while outlining extensions for safer self-evolution.

F.4 Task-stream evaluation and updates

  • The finite-horizon protocol freezes candidate statistics before evaluation, requires valid returned failure and retained samples, and abstains whenever quotas or readiness conditions fail.Generation and evaluation must be separated by the stated coupling conditions; elapsed time alone is insufficient.Source: §F.4 · pp. 41–42
  • After acceptance, incumbent-labeled stored failures may not represent the successor, so the protocol marks samples for recollection and abstains if a valid retained subset cannot be recovered.Relabelling alone does not restore the required coupling between the successor state and stored records.Source: §F.4 · p. 42
  • 1/(2m) bounds the excess conservatism of finite-rollout absolute-value estimation, with E|ˆδ(t)| ≥ |δ(t)| and strictness when rollout estimates take both signs.Under i.i.d. arrivals, the retained-task estimator is conservative; under β-mixing, Lemma 9 supplies the corresponding mixing-aware comparison.Source: §F.4 · pp. 42–44
  • Measured-margin acceptance controls the joint risk of accepting insufficient expected-reward improvement, contains Two-Gate acceptance under matched conditions, and supports finite-run improvement accumulation.A positive threshold γ yields a finite-run lower bound γ|A| on the common coverage event, but requires Juser(Ak) < 1 − γ.Source: §F.4 · p. 43
  • Block-based retained samples preserve marginal distribution Duser while controlling pairwise dependence through β(ℓ), and first-failure sampling yields i.i.d. retained tasks conditionally under the stated i.i.d. construction.The finite protocol treats stored-sample availability and failure-batch return separately; 2nF/Zk is only an infinite-horizon nominal planning scale.Source: §F.4 · pp. 41, 43
  • Returned-task conditioning preserves an i.i.d. failure-task distribution, enabling concentration bounds for declared statistics and a return-probability expression B_HF,k,nF(Z_k).The finite-horizon protocol also bounds conditional nonreturn probability using the declared tail condition and the event Z_k ≤ Z̄_k.Source: §F.4 · p. 45
  • The Two-Gate protocol accepts with certified improvement margin Δ_k except on a bounded failure event, and Δ_k > 0 guarantees non-degradation.The guarantee combines ready-side estimator coverage with the gate conditions, yielding I_k ≥ Δ_k on accepted paths outside the bad event.Source: §F.4 · p. 45

F.5 Constraints on further improvement

  • Once J_user(A_k) ≥ 1 − (1 − Z_k)δ−, no candidate can satisfy the Two-Gate sufficient condition, regardless of candidate-pool size, sample sizes, or the frozen LLM.The result gives a sufficient-condition threshold for stagnation, not a proof that all positive actual improvement margins have disappeared.Source: §F.5 · p. 46
  • Positive actual improvement can remain possible beyond the sufficient-condition threshold: a two-task construction achieves J_user(A_k) ≥ B_k with R(eA; A_k) = 0 and M_Ak(eA) = L_Ak(eA) = u/2 > 0.Thus positive candidate margins may coexist with zero retained-task change even when no modification meets the sufficient condition.Source: §F.5 · p. 46
  • A nonempty admissible region requires Θ_k > ε_Σ + ε_L, while feasibility also imposes measurement and expected-reward lower bounds on retained-task mass and evaluation accuracy.The analysis notes that increasing the returned-failure quota can eventually make return impossible at a fixed finite arrival horizon.Source: §F.5 · pp. 47–48
  • As expected improvement u_k approaches zero, every level-β certifier with power at least 1/2 requires expected evaluations diverging at least as Z_k/(10u_k).This lower bound holds even with adaptive task selection and stopping, for fixed Z_k and δ_- under the stated range of u_k.Source: §F.5 · p. 49

F.6 Fixed-suite guarantees and measurement

  • Fixed-suite certification combines failure and retained tasks, and Theorem 8 guarantees that every modification passing the empirical gate satisfies the corresponding true suite conditions on E_suite.The suite uses a designer-chosen failure fraction and simultaneous estimation-error control across evaluated modifications.Source: §F.6 · p. 49
  • The measurement protocol freezes the candidate pool before collecting disjoint audit data, uses count-adaptive rectangles, and abstains with vacuous intervals when either label group is empty.This separation makes valid modifications independent of measurement material, while failed outputs retain deterministic singleton rectangles.Source: §F.6 · pp. 50–52
  • Conditional reference sampling yields direct retained-task concentration without estimating a normalization factor, whereas failure-task bounds still require estimating the failure probability Z_k.The retained-task result avoids both ε_Z and 1/(1−Ẑ_k), while the failure bound retains dependence on probability estimation.Source: §F.6 · p. 52
  • Disjoint measurement samples support uniform confidence bounds for all reachability targets with probability at least 1 − β_rect − β_emp, extending under whole-window β-mixing coupling with an added β_mix loss.The result relies on VC control of lower-right-orthant target sets and applies conditionally under exact reference implementations before marginal transfer under mixing.Source: §F.6 · pp. 51, 53
  • High certification probability lower-bounds reachability, but low certification probability cannot distinguish poor reachability from poor certification of qualified modifications.The inner sufficient conditions are not necessary because qualified modifications outside the inner set may also be certified.Source: §F.6 · p. 53, §F.7 · p. 53

F.7 Multiple updates and changing task distributions

  • Across accepted updates, certified margins accumulate into a finite-run improvement guarantee, while rejected or abstained steps retain the incumbent.A uniform positive margin Δ⋆ over accepted steps yields an explicit count bound; later qualified modifications are not guaranteed without additional constraints on generation or qualified sets.Source: §F.7 · pp. 54, 56
  • As reward shortfall vanishes, maintaining the stated certification probability and error control can require a diverging worst-case rollout budget.The remaining-shortfall bound limits failure-task contributions but does not imply monotonicity of separate failure-rate or evaluation-error terms.Source: §F.7 · p. 54, §G.3 · p. 57

G.1 Pool-wise confidence accounting (§4.1)

  • Pool-wise confidence accounting controls familywise error once, but increasing the candidate pool enlarges interval widths and can make the certified bound have an interior optimum.At fixed width, the effective evaluation sample size grows only logarithmically with pool size, whereas a fixed total budget creates a samples-per-candidate trade-off.Source: §G.1 · p. 56

G.2 Optional tail description at a fixed retained-task tolerance

  • Tail parameters are step-local and statistically harder to estimate than reachability, while regular variation and boundary-atom assumptions address distinct locations and neither is required by the main finite-budget results.The necessary relation sk,δ ≤ Zkāk ≤ 1 − Juser(Ak) can force zero reachability at a fixed target without making zero reachability absorbing under changing targets or states.Source: §G.2 · pp. 56–57

G.3 Additional consequences of the finite-sample conditions

  • Finite-sample feasibility conditions constrain when certification is non-vacuous, not whether a better agent exists, and additional samples reduce evaluation error at Θ(n−1/2).At fixed state and tolerance, the sufficient threshold is unchanged by more samples; exact evaluation still requires the incumbent reward to satisfy the stated condition.Source: §G.3 · p. 57

G.4 Comparison with HCPI and SPIBB (§7)

  • Unlike HCPI and SPIBB, this framework compares fresh incumbent and candidate rollouts while constraining retained-task reward changes through failure-weighted task distributions.The retained-task threshold can limit a baseline certification rule even when further improvements remain certifiable, as shown by Proposition 2.Source: §G.4 · pp. 57–58

G.5 Finite-sample bound for ˆDR: statement, proof and remarks (§4.2)

  • The finite-sample ˆDR analysis separates estimation error in candidate values from errors in absolute per-task differences using concentration bounds and triangle inequalities.The supplied derivation introduces a stored-sample mixing Hoeffding event and bounds each absolute-difference estimate through the corresponding value-estimation error.Source: §G.5 · p. 58

V , for each j:

  • The certification proof separates stored-sample dependence from fresh-rollout error, using coupling, conditional concentration, and a union bound without requiring event independence.Dependence contributes an additive 2n_Rβ(ℓ) coupling defect, while fresh rollouts are controlled separately under the stated assumptions.Source: V , for each j · pp. 59–60

G.6 Finite predeclared repair and assembly (§3.2)

  • Finite predeclared repair and assembly may combine candidate tuples through fixed kernels and terminal outputs, but cannot redraw or adapt kernels after observing the pool.The resulting assembly probabilities must be measured on terminal outputs; pool-adaptive redraws and post-pool shared state fall outside the framework.Source: §G.6 · pp. 60–61

G.7 Numerical detail for the fixed-suite guarantee (§6.2; Appendix F)

  • 0.061 is an approximate lower bound on the fixed-suite guarantee’s right-hand side under the reference design, and the gate becomes supportable only for δ ≲0.08 with w = 0.0909.The bound cannot be removed by shrinking the retained suite; increasing the failure suite reduces it, while smaller w can leave no admissible τ.Source: §G.7 · p. 61

G.8 Estimating observation-lag drift

  • The bound uses an L2 argument supplying the √m rate, with constant 0.219 when m = 50, nP = nD = 2000, and β = 0.05.The analysis also states a within-cell variation condition and requires accounting for coverage failures when estimated bounds are used in an unconditional expectation guarantee.Source: §G.8 · pp. 62–63

G.9 Reachability with multiple generation models

  • With multiple generation models, the target is reachable exactly when at least one constituent has positive reachability, while uniform mixing averages constituents and loses at most a factor E relative to the best.Mixing can improve on a particular constituent even when supports coincide, but it changes generation rather than the subsequent certification rule.Source: §G.9 · p. 63

G.11 The tail-level CVaR gate and subgroup protection (§3.1)

  • CVaR constraints preserve the expected-reward guarantee and provide subgroup control, but estimating tail risk costs an additional 1/α factor and rollout errors require separate accounting.The guarantee is informative for sufficiently large subgroups; subgroup absolute reward change does not imply subgroup non-degradation.Source: §G.11 · p. 64

G.12 The L1 choice: four desiderata and the TRPO contrast (§3.1)

  • The L1 retained-task constraint is the most permissive fixed-radius Lp choice because it avoids extra relaxation, while stronger exponents require separate finite-data certification.The framework treats retained-task reward changes as a service requirement distinct from norm selection.Source: §G.12 · p. 65

G.13 The KL-based bound and the bound based on DR (§3.1)

  • The DR constraint and KL-based constraint yield the same sufficient expected-reward bound after parameter matching, but DR can admit reward-preserving output changes that violate KL.DR controls retained-task reward changes, whereas KL compares full output distributions; neither enters the analyzed Two-Gate rule.Source: §G.13 · p. 65

G.14 Entropy regularization as sequence-level tempering (§3.3)

  • Entropy regularization performs sequence-level reweighting that preserves support, so it preserves which targets have positive reachability while changing their reachability values; normalization may fail for heavy-tailed distributions.Sequence-level reweighting need not match token-level temperature scaling.Source: §G.14 · p. 66

G.15 Comparing the task distributions represented by evaluation data and served after an update (§6.4)

  • The task distribution represented by certification data can differ from the distribution served after an update, requiring an observation-lag bound in addition to inter-step drift and dependence assumptions.The one-step inequality pays the observation-lag term at both incumbent and updated agents, producing a factor of two.Source: §G.15 · p. 66, §G.15.1 · p. 66, §G.15.2 · p. 67

G.15.3 Cumulative bound under the two task distributions

  • Across K steps, the cumulative bound limits sustained positive update contributions: if νk ≥ c > 0 throughout, then Kc ≤ 1, ruling out arbitrarily long such intervals.Accepted-step contributions are zero on nonaccepted steps and can be lower-bounded through the acceptance probability when each accepted increment is at least γ.Source: §G.15.3 · p. 67

G.16 Drift estimation and recollecting the stored sample (§6.4)

  • Drift detectors can trigger recollection during rejection runs, but accepted steps always require recollecting the stored sample because agent-indexed labels and selection effects invalidate reuse.If recollection data are unavailable by the horizon, the algorithm retains the incumbent; collecting nR records requires roughly nRℓtask arrivals.Source: §G.16 · pp. 67–68

G.17 Comparison with TRPO (§7) / G.18 Comparison of self-evolving systems (§7)

  • Unlike TRPO’s continuous parameter updates and policy-KL constraint, harness self-evolution generates discrete modifications whose qualification probability must be analyzed separately; Table 2 organizes related systems by persistence, self-modification, adoption, evaluation, and path dependence.The review covers 24 papers and distinguishes reported, narrower, and absent properties using Y, P, and N.Source: §G.17 · p. 68, §G.18 · p. 68

G.18 Comparison of self-evolving systems (§7)

  • The analysis separates the probability of generating a qualified persistent modification from certification and selection success, while unrestricted updates can leave further improvement unguaranteed.Evaluation procedures support claims on tested tasks, whereas guarantees under the user-task distribution additionally require representative evaluation data.Source: §G.18 · p. 69, §G.20 · p. 71
  • The 24-paper comparison finds no prior work jointly addressing reward-independent verification, user-task targets, uniform generation guarantees, and two-sided generation-to-selection bounds.Prior approaches instead provide distinct evidence for updating, including learnability, repeated-decision error control, anytime-valid tests, or formal utility-improvement proofs.Source: §G.18 · p. 70

G.19 Sample complexity for estimating reachability (§6.2)

  • 6.6 × 10^8 total rollouts are required by the reference sample-complexity design, comprising 5.5 × 10^8 Candidate-Agent and 1.1 × 10^8 baseline rollouts.The total can be reduced under (M3), although the reduction is not estimated; reachability intervals also depend on evaluation rectangles crossing target boundaries.Source: §G.19 · p. 70

G.20 Further directions for harness self-evolution (§8)

  • Adaptive search across candidate outputs requires tracking changing qualified-generation probabilities, while estimating repair and assembly dependence could sharpen reachability bounds.Theorems 2 and 3 cover conditionally i.i.d. candidate pools, and Proposition 9 and Theorem 9 cover fixed repair and finite predeclared assembly.Source: §G.20 · pp. 70–71
  • Future work targets certification power, stronger retention, dependent task sampling, task-distribution response, and transfer from reused suites to fresh user tasks.Theorem 11 gives a worst-case evaluation-cost lower bound near the reward upper bound, while later propositions address subgroup retention and repeated-update control.Source: §G.20 · p. 71

G.21 Sample-complexity lower bound for certification near the reward upper bound (§5.2)

Theorem 11 shows that any sound certifier with fixed nontrivial power requires evaluation cost growing inversely with the incumbent’s reward shortfall from the upper bound. Thus no worst-case rollout budget independent of user reward can preserve certification power above B_k.

Sample-complexity lower bound

  • For shortfall u ≤ Z/2 and soundness β ≤ 1/8, certifying an improving modification with probability at least 1/2 requires Ω(Z/u) expected candidate rollouts on a non-improving instance.The hard pair differs only in candidate rewards on failure tasks, with improvement and degradation magnitudes ±u/2.Source: §G.21 · p. 72
  • As J_user approaches 1, no fixed worst-case expected-rollout budget can maintain certification probability at least 1/2 for improving modifications above B_k.The lower bound diverges as the reward shortfall u approaches zero.Source: §G.21 · p. 72
  • The lower bound follows from indistinguishability: only failure-task candidate rollouts differ between the improving and non-improving instances, so adaptive stopping and arbitrary side computation cannot avoid the information requirement.A KL and Pinsker argument converts the per-rollout information bound into the required expected sample complexity.Source: §G.21 · p. 72
  • Empirical-Bernstein concentration matches the hard-pair scale ˜O(u^-1) on the informative reward coordinate, while the current full gate’s Hoeffding-based allocation does not yield a J_user-uniform budget.Extending variance-adaptive control to failure-task and stored-sample components is identified as a direction for the full gate.Source: §G.21 · p. 73
  • The certification lower bound composes with candidate-pool selection limits, so both pool coverage and evaluation budget constrain safe improvement above B_k.Selection can only succeed when the pool contains a qualified modification, while certification still requires shortfall-dependent evaluation.Source: §G.21 · p. 73

H When the task distribution responds to an Agent update … I.1 Conditional rates and the i.i.d. numerical example (§5)

The appendices distinguish comparative improvement on the pre-update task distribution from realized improvement after distributional response, and extend the analysis to response estimation, drift, and numerical evaluation. Safe realized-improvement claims require an additional response bound, while response effects may be difficult to estimate and can accumulate only after accepted updates.

H When the task distribution responds to an Agent update

  • Comparative certification evaluates agents on the incumbent distribution, whereas realized improvement evaluates the updated agent after task-distribution response; absolute expected reward is a distinct quantity.Paired re-rollouts estimate the comparative quantity, while the response modulus bounds the gap to the realized reading.Source: H When the task distribution responds… · p. 73
  • A single-step realized-improvement guarantee subtracts the response modulus from the comparative lower bound, changing the gate to τ > (1 −ˆZk + εZ)(δ + εD) + εL + ϱ.The estimators of L, DR, and Z remain unchanged, but a valid setting-specific response bound is required.Source: H When the task distribution responds… · p. 74

H.1 Comparative estimation and recollecting the stored sample

  • After each accepted modification, Algorithm 2 recollects the stored sample because old labels need not match successor failure probabilities or restore Condition 1.New records and fresh evaluations define bZk+1, bLk+1, and bDR,k+1 relative to the successor’s current distribution.Source: §H.1 · p. 74

H.2 Bounding the reward response after an update

  • Assumption 5 alone gives no informative pre-update response bound; estimating reward response through task partitions requires substantial samples, including approximately 3.5×10^5 to 2×10^6 records per source.At spacing ℓ=50, the latter requirement costs approximately 10^8 raw task arrivals per source.Source: §H.2 · pp. 74–75
  • A small average reward change need not mean few tasks change, and kernel-preserving modifications yield zero response but cannot provide positive failure-task improvement when τ > 0.The kernel-preservation implication requires the outcome space to include every channel through which the agent affects user tasks.Source: §H.2 · p. 75

H.3 Response and distribution drift

  • Response occurs only at acceptance, so it accumulates over accepted updates, while its reward effect may be favorable or unfavorable and conservative bounds can reject favorable modifications.The multi-step drift analysis includes endogenous response alongside ambient movement, avoiding double-counting the same distribution shift.Source: H When the task distribution responds… · p. 74, §H.3 · p. 75

I Evaluation, Measurement, and Further Results

  • The evaluation appendices collect the detailed derivations, numerical readings, and implementation records supporting the body’s §5–§6 results.Each body result is paired with corresponding supporting material below.Source: I Evaluation, Measurement, and Further… · p. 75

I.1.1 Conditional sample-complexity relations and numerical example … I.3.1 Scope of reachability confidence bounds

The section develops conditional sample-complexity, subgroup-retention, reachability, and confidence-bound analyses for safe harness evolution. It separates declared tolerances, sampling and evaluation radii, generation reachability, candidate-pool selection, and the scope of guarantees across updates and evaluation suites.

I.1.1 Conditional sample-complexity relations and numerical example

  • At m = 2 × 10^4 and nR = 2000, the relaxed lower bound reaches approximately 0.1526 at nF = 38208, with a later rise from reallocating fixed rollout risk.The bound is conditional on returned samples and full readiness, not an optimization of arrival horizons, abstention, or throughput.Source: §I.1.1 · pp. 76–77
  • Positive-tolerance feasibility retains a declared boundary term, while additional samples reduce only measurement error; conditional failure quotas grow quadratically with inverse distance to the zero-failure boundary.The returned-sample scaling is distinct from finite-horizon batch arrival, which is governed separately by the binomial return relation.Source: §I.1.1 · pp. 76–77

I.1.2 Subgroup protection and retention across updates

  • For subgroups with probability at least w0, Cauchy–Schwarz yields a sufficient condition for positive improvement, whereas the tail criterion instead guarantees bounded average absolute reward change.The signed condition ensures positive improvement; the tail-level protection permits bounded loss and has its own i.i.d. sampling cost.Source: §I.1.2 · p. 77
  • Per-step retention tolerances need not preserve the initial retained-task distribution, because that distribution is reweighted by the incumbent and cumulative evaluation residuals can accumulate.Persistent reference evaluation or constraints on retained-task weights could address this limitation.Source: §I.1.2 · p. 77

I.1.3 Dual-track widening and failure-radius attribution

  • Dual-track widening depends on declared tolerances and concentration radii; failure-task radii govern descent toward the zero-εF boundary, while (m, nR) determine that boundary.In one branch εL cancels, while in another εF enters through εµ and is weighted by Zk.Source: §I.1.3 · p. 78

I.2 Reachability: scope and interpretation (§3.2–§3.3; §4.1)

  • Reachability varies continuously with candidate probability, while context means can hide difficult cases; diagnostics therefore require context distributions or low quantiles, not only averages.Selection may use arbitrary evaluation or verification but must choose from the fixed i.i.d. candidate pool, and shared evaluation material can invalidate a simple any-hit identity for QG.Source: §I.2 · pp. 78–79
  • Reachability is the generation probability of persistent modifications meeting a target under Duser, transformed for fixed candidate pools by 1 −(1 −p)^N rather than by capability after multiple updates.Its scope depends on the specified generation experiment and conditioning context; positive reachability also requires reward and tolerance upper-bound conditions.Source: §I.2 · pp. 78–79

I.3 Measuring reachability: mechanisms, cost, and the drawn-pair protocol (§6)

  • Reachability measurement separates rollout noise, incumbent-estimate sharing, and risk allocation, while confidence bounds remain conditional on the current history, context, and evaluation suite.A fixed reused suite yields suite-specific reachability that may not represent a fresh suite; estimating context means or quantiles requires a second hierarchical context sample.Source: §I.3 · p. 79, §I.3.1 · pp. 79–80

I.3.2 Evaluation cost of reachability estimation … I.6.3 One-sided deviations and the dual-track radius

This section develops the evaluation, certification, and safety limits of harness self-evolution, covering reachability measurement, finite-sample and rollout costs, cumulative change, and data-separation requirements. It also distinguishes related evaluation criteria and shows how threshold, margin, and dual-track constraints shape certification, drift control, and acceptance of modifications.

I.3.2 Evaluation cost of reachability estimation

  • Pooled reachability estimation reduces the reference radius to approximately 0.0083 versus 0.035 for the corresponding per-task union-bounded quantity, but shared incumbent error can change threshold comparisons.The suite calculation pools candidate rollouts and reuses the incumbent estimate; a shared threshold shift need not cancel and may reverse comparisons.Source: §I.3.2 · p. 80
  • Reachability measurement remains budget-limited because Jensen bias does not average over the suite, an additive term is O(1/|Rev|), and the reference calculation totals approximately 6.6 × 10^8 rollouts.At |Rev| = 5000, the additive term is 0.0053, while increasing generation or evaluation budgets alone cannot guarantee the desired total resolution because confidence-bound ambiguity and simultaneous radii interact.Source: §I.3.2 · p. 81

I.3.3 Sample complexity for the drawn measurement construction

  • The drawn-pair construction estimates reachability under Duser, whereas fixed-suite gates concern the suite and require separate representativeness or transfer conditions for user-distribution interpretation.Its displayed counts use expected random counts rather than fixed-quota guarantees; the actual protocol uses realized counts and reports informativeness probability.Source: §I.3.3 · p. 82

I.3.4 DS-1000 studies

  • Many DS-1000 modifications meet the audit target, but simultaneous certification rarely succeeds under the studied evaluation budgets.Independent-audit point estimates found at least one target-satisfying modification in every primary-target candidate pool, while simultaneous bounds remained mostly vacuous.Source: §I.3.4 · p. 84, Figure 3 · p. 84
  • In DS-1000, fresh-generation target membership was measured with paired sequence-level statistics, and Figure 4 reports lower measured target-membership fractions after adoptions.Each time point averages fractions from 16 new terminal outputs across four sequences; pairing is by sequence rather than modification.Source: §I.3.4 · p. 86, Figure 4 · p. 86

I.3.5 WorkBuddy–DSH study

  • In WorkBuddy–DSH, all four modifications met T0 under audit point estimates, but unresolved intervals and zero gate certifications caused incumbent retention and bQpt = 0.Every LB interval crossed λ, every DB upper bound was below δ, and the simultaneous bank-fraction bounds remained [0, 1].Source: §I.3.5 · p. 88, Table 4 · p. 88

I.3.6 Training objectives and reachability

  • Optimizing self-task training reward can reduce qualified-modification reachability because no assumption links training reward to user-task qualification.For fixed p, the example’s reachability tends to zero as β ↓0; with finite normalization, positive reference mass remains positive but need not increase.Source: §I.3.6 · pp. 88–89

I.3.7 Fixed-suite and task-distribution comparison

  • Certification and independent reachability measurement use disjoint samples and distinct guarantees, while fixed-suite bounds differ from task-distribution bounds and interval width also depends on unresolved target membership.Independent reachability samples never enter certification decisions, and its mixing defect is separate from the gate’s net-risk constraint.Source: §I.3.7 · p. 89, Table 5 · p. 89

I.4 Multi-step guarantees under changing task distributions (§6)

  • Under changing task distributions, step-0 information does not ensure later improvement because the same later modification can have different effects under admissible composition maps.A prescribed structured composition map therefore requires separate analysis of realizability and subsequent reachability.Source: §I.4 · p. 89

I.4.1 Multi-step setup, accumulation, and drift

  • For K steps, the finite-run guarantee accumulates per-step bad-acceptance risks by a union bound and lower-bounds net improvement by certified evolution minus cumulative distribution drift, without requiring cross-step independence.When drift exceeds evolution, long-run net improvement is not guaranteed; the long-run K →∞ case remains open without an additional generation assumption.Source: §I.4.1 · pp. 90–91

I.4.2 Cumulative expected-reward change

  • Across fixed-distribution updates, cumulative absolute expected-reward change and endpoint change are bounded by sums over accepted-step changes, with a sharper terminal-state factor when its hypotheses hold.Without the terminal-state condition, the factor 1 −ε(K) is replaced by 1.Source: §I.4.2 · p. 91
  • Raising τ increases each accepted step’s improvement and reduces its failure-task contribution, but small thresholds can make improvement arbitrarily small; accepted-step counts require a common positive lower bound.The stronger count bound additionally requires the stated terminal-state condition.Source: §I.4.2 · p. 92

I.5 Task-stream evaluation sample sizes and implementation details (§4.2; §5.1)

  • 1.12 × 10^8 complete rollouts cost $5.6M–$56M for paired evaluation of one candidate at the reference coordinate, excluding arrival-horizon return probabilities.The protocol uses fresh rollouts, evaluates only when samples are ready, abstains otherwise, and recollects the stored sample after acceptance.Source: §I.5 · p. 93

I.5.1 Sample-complexity calculation for the split protocol

  • The split protocol requires separate generation and evaluation failure batches, with finite-horizon non-return treated as abstention rather than an exact nF/Zk arrival cost.The evaluation batch has the conditional failure-task distribution under i.i.d. marked streams, and its return probability is a binomial tail.Source: §I.5.1 · p. 93

I.5.2 Reusing the stored sample after an update

  • After acceptance, Algorithm 2 recollects the stored sample to restore current-agent labels and the joint-to-product separation required by Condition 1; after rejection, reuse remains conditional.Relabelling the existing stored sample alone does not establish the required coupling.Source: §I.5.2 · p. 93

Outcome model and conditional independence; Trial worker consistency

  • The analysis assumes local failure detection, conditional independence of observed outcomes from prior history, and independent fresh evaluation rollouts under fixed trial-worker settings.These assumptions distinguish observed task outcomes from repeated evaluator rollouts and their cross-task dependence.Source: Outcome model and conditional… · p. 94, Trial worker consistency · p. 94

Trial worker consistency

  • The pooled radius removes the √nF factor only under cross-task rollout independence; shared environment state violates that assumption, so otherwise the union-bound radius is required.Evaluation-order randomization and task interleaving are recommended when using the pooled bound.Source: Trial worker consistency · p. 94

I.5.4 The common-offset construction; I.5.5 Stored-sample and failure-batch radius accounting

  • A common offset preserves the sequential β(ℓ) coupling, while retaining the first nR records keeps the schedule fixed; dependence contributes an additive 2nRβ(ℓ) defect.When β(ℓ)=0, the bound becomes the standard Hoeffding bound at full effective size nR.Source: §I.5.4 · p. 94, §I.5.5 · p. 94

I.6.1 Failure weighting and retained-task reward change; I.6.2 KL and Hellinger comparisons between output distributions

  • Small source edits can cause total-variation change 1, while large edits can leave output distributions unchanged; reward-based retained-task distance therefore does not constrain full output distributions.KL and Hellinger comparisons can be infinite or nonzero even when expected rewards are identical, and they do not enter certification.Source: §I.6.1 · p. 95, §I.6.2 · p. 96

I.6.3 One-sided deviations and the dual-track radius

  • Threshold-based certification is capped by B_k = 1 − (1 − Z_k)δ−, whereas measured-margin certification requires inverse-square samples and has an inverse-linear rollout lower bound near the reward ceiling.The threshold bound is unaffected by larger measurement budgets at fixed state and δ−; measured-margin certification instead depends on the shortfall 1 − J_user. The sufficient condition also applies to externally or human-generated modifications.Source: §I.6.3 · p. 97
  • Both B_k and cumulative-change control depend on δ−, while δ+ can alter the passing event and drift bound without changing B_k; splitting risk budgets adds about 7% to each task-sampling term at β_R = 0.02.The two-sided design replaces log(2/β_R) with log(4/β_R), and mixing additionally requires subtracting the coupling defect while preserving positive net risk.Source: §I.6.3 · p. 98
  • With R = 0.001, LAk = 0.005, and margin M = 0.005 − 0.9 × 0.001 = 0.0041 > 0, Algorithm 3 certifies a strict improvement.Certification requires resolving a combined radius of 4 × 10−3, involving roughly 10^5–10^6 samples under the Hoeffding bound.Source: §I.6.3 · p. 98
  • The upward constraint can reject substantial improvements under D_R,k, retaining the incumbent and forgoing improvement, even though it is unnecessary for the one-sided improvement bound.In the local oscillation example, the candidate changes reward by +0.5 and −0.5 on equal retained-task halves, yielding D−_R = 0.25 and one-sided rejection.Source: §I.6.3 · p. 98
  • The symmetric baseline uses one tolerance and one retained-task concentration event, whereas the dual-track criterion separates decreases and increases at the cost of splitting the retained-task risk budget.The numerical illustrations use the symmetric constraint; the corresponding dual-track drift bound is provided in Appendix I.4.2.Source: §I.6.3 · p. 98

Related papers

Loading 2609.08175v1…