PaperlayerView exact revision on arXiv

Source-linked AI summary

The Red Queen G\"odel Machine: Co-Evolving Agents and Their Evaluators

Alex Iacob, Andrej Jovanović, William F. Shen, Daniel Burkhardt, Meghdad Kurmanji, Nurbek Tastan, Lorenzo Sani, Niccolò Alberto Elia Venanzi, Ambroise Odonnat, Zeyu Cao, Bill Marino, Xinchi Qiu, Nicholas D. Lane

arXiv:2606.26294v2cs.LGcs.AIcs.MAcs.NE

TL;DR

Self-improving agents usually rely on fixed evaluation criteria, limiting recursive improvement when benchmarks are unavailable, weak, saturated, or vulnerable to bias. RQGM co-evolves agents and learned evaluators through fixed-criterion epochs with controlled utility transitions. Across coding, writing, reviewing, and proof tasks, it improves performance or efficiency over fixed-evaluator systems while using adversarial objectives to correct reviewer over-leniency.

  • Problem

    Existing self-improvement methods depend on stationary verifiers, benchmarks, or datasets, limiting use on benchmark-free tasks and exposing search to saturation, reward hacking, and evaluator bias.

    Source: §1 · p. 1

  • Method

    RQGM co-evolves task agents and learned evaluators by freezing evaluation within epochs, promoting challengers at boundaries using fixed anchors, and erasing invalidated utility records.

    Source: §1 · p. 1, §3.2 · p. 4, §3.4 · p. 5

  • Results

    RQGM improves coding, paper writing, reviewing, and proof-task outcomes over prior self-improving or fixed-evaluator systems, including 71.7% versus 69.9% coding pass rate and 40.5% versus 21.8% paper acceptance.

    Source: §1 · p. 2, §5 · p. 8, §6 · p. 13

  • Takeaways & Limitations

    Co-evolving evaluators can provide signals for hard-to-verify tasks, improve search efficiency, and correct evaluator bias while retaining per-epoch self-improvement guarantees.

    Source: §1 · p. 3, §6 · p. 13

    The investigation is preliminary and based on short search horizons; longer co-evolution and improvements beyond the anchor remain unresolved.

    Source: §1 · p. 2, §6.1 · p. 13

Abstract

from arXiv · show

Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains. However, their search methods generally assume a stationary evaluation criterion: a fixed verifier, benchmark, or labeled dataset that remains valid as the agent improves. This ignores a central feature of evolution: species adapt as their environments change with them. We aim to bring the same principle to recursive self-improvement, making evaluation part of the improvement loop and opening search to evolving evaluators, adversarial objectives, and dynamic utilities that may surpass static benchmarks. We introduce the Red Queen Godel Machine (RQGM), an evolutionary framework for recursive self-improvement under non-stationary utilities. The RQGM makes this possible through controlled utility evolution: search is organized into epochs with a fixed within-epoch evaluation criterion, while the utility can be updated at epoch boundaries, so self-improvement guarantees hold per epoch as the objective evolves across them. We begin by showing that even on verifiable coding tasks, the RQGM improves test pass rate over the prior SOTA by adding a complementary agent-as-a-judge code-review signal. This signal is cheaper and the RQGM uses 1.35x-1.72x fewer tokens. We then turn to scientific paper writing and reviewing, and Olympiad-level proof writing and grading, where the RQGM improves performance over prior self-improving agents: co-evolved writers reach 1.78x-1.86x higher acceptance rates under a diverse agent-as-a-judge panel, while co-evolved graders reach 9% higher ground-truth accuracy. In paper reviewing, the strongest baseline reviewer over-accepts AI-generated papers at up to 1.91x the human rate. The RQGM corrects this by introducing an adversarial objective that discovers reviewers equally stringent on AI and human work.

1 Introduction

RQGM makes evaluation part of recursive self-improvement by co-evolving learned evaluators with task agents under controlled utility evolution. Across coding, writing, reviewing, and proof tasks, it improves performance or efficiency over fixed-evaluator systems while addressing evaluator bias.

Motivation

  • RQGM uses learned evaluators to supply signals for tasks without direct benchmarks and to add criteria such as code maintainability or cheaper proxy evaluation.This extends self-improvement beyond settings with fixed, readily available verifiers.Source: §1 · pp. 1–2

Empirical results

  • 71.7% held-out pass rate versus 69.9% for the prior SOTA on verifiable coding tasks, using an agent-as-a-judge code reviewer.The reviewer complements test execution and reduces search cost.Source: §1 · p. 2, Figure 1 · p. 2
  • 40.5% paper acceptance versus 21.8% for the prior SOTA, while co-evolved graders achieve lower search cost than static baselines.The reported grader comparison uses 3× lower search cost than the prior SOTA.Source: §1 · p. 2

Evaluator improvement

  • Adversarial objectives let later epochs correct reviewers that over-accept AI-generated papers, producing reviewers equally stringent on machine- and human-written work.The method replays accepted AI-generated samples as adversarial examples while retaining the ground-truth anchor.Source: §1 · p. 2

Limitation

  • RQGM loosens convergence guarantees compared with static evaluation criteria as a cost of allowing evaluators and utilities to evolve.The authors describe the empirical investigation as preliminary and the framework as a direction toward recursive co-evolution.Source: §1 · p. 2

Core contribution

  • RQGM treats evaluation as part of the search process, allowing learned utilities and objectives to evolve across controlled epochs.Within each epoch, the evaluator is fixed; utility changes only at epoch boundaries.Source: §1 · pp. 2–3

2 Related Work

Prior self-improvement systems search over agents while holding evaluation fixed, limiting adaptation to reward hacking, judge bias, and domains without direct benchmarks. RQGM instead makes the learned evaluator itself part of the evolving search objective.

Self-Improving AI

  • Darwin-Gödel Machines, Huxley-Gödel Machines, and HyperAgents use archive search but retain a fixed evaluation criterion throughout each run.These systems differ in search utility and domain coverage while sharing the fixed-evaluator assumption.Source: §2 · p. 3
  • Fixed evaluation leaves prior systems exposed to reward hacking and model-judge biases, while changing benchmarks is costly and forces search to restart [11] [16] [29].The cited biases include position, verbosity, and self-preference effects.Source: §2 · p. 3

LLM-as-a-Judge and Automated Scientific Discovery

  • LLM-as-a-judge and automated discovery pipelines generally keep judges fixed per run, whereas RQGM allows the judge to improve during search [6] [29] [31] [32] [34].RQGM builds on learned evaluation but makes evaluator evolution explicit.Source: §2 · p. 3

Multi-Agent Co-Evolution

  • Classical and modern co-evolution methods adapt agents against moving targets, but recent LLM-agent approaches generally keep evaluation fixed; RQGM co-evolves the learned utility itself [40] [41].The target of adaptation is therefore the evaluator rather than only another task agent.Source: §2 · p. 3

Open-Endedness

  • Open-ended search commonly maintains novelty or quality-diversity under static objectives, making co-evolution of the objective RQGM’s missing piece [45].The paper frames evaluator evolution as the addition needed for a closed self-referential loop.Source: §2 · p. 3

3 RQGM: Co-Evolving Agents and Their Evaluators

RQGM extends archive-based self-improvement to multi-agent workspaces containing task agents and learned evaluators. Controlled utility evolution preserves fixed-criterion search within epochs while enabling evaluator replacement, anchor-guided improvement, and bounded transition costs.

3 RQGM: Co-Evolving Agents and Their Evaluators

  • RQGM changes archive nodes into multi-agent workspaces, makes evaluators learned processes, permits epoch-boundary utility changes, and replaces evaluators during search.The search still builds on prior archive-based self-improvement methods.Source: §3 · p. 3

3.1 Preliminaries: Self-Improvement as Tree Search

  • Archive search expands or evaluates nodes using clade metaproductivity and Thompson sampling, returning the highest-scoring node after the evaluation budget is exhausted.Clade utility pools successes and failures over a node’s descendant subtree.Source: §3.1 · p. 4

3.2 Search Space

  • Each workspace contains editable task-agent and evaluator roles with separate task pools, while the scoring and orchestration harness remains fixed.Evaluator-dependent roles use replaceable evaluator slots and epoch indices; evaluator-independent roles retain fixed benchmarks.Source: §3.2 · p. 4

3.3 Three-Level Sampling Hierarchy

  • The scheduler balances expansion and evaluation, then selects nodes, roles, and tasks to equalize sampling; under a fixed epoch, the pooled posterior converges to balanced role-task utility.Role- and task-level counters load-balance evaluations while node outcomes drive Thompson sampling.Source: §3.3 · p. 5

3.4 Co-Evolving Evaluation

  • Epoch-local stationarity and selective erasure turn each epoch into a fixed binary-outcome search problem, so HGM’s self-improvement guarantees apply within epochs.Records tied to a displaced evaluator are removed rather than mixed with utilities from the replacement.Source: §3.4 · p. 5, §3.5 · p. 6

3.5 Controlled Utility Evolution

  • Exponentially spaced checkpoints reduce transition bookkeeping from O(B^2) to O(B) over B evaluations by delaying re-scoring until affected nodes are revisited.Cached agent outputs can be reused during re-evaluation.Source: §3.5 · p. 6
  • Together, the framework provides per-epoch convergence, cross-epoch evaluator improvement, and amortized transition cost without asymptotic overhead beyond archive search.The same transition structure can support time-varying benchmarks and adversarial objectives.Source: §3.5 · pp. 5–6
  • Evaluators replace incumbents only when they improve an epsilon-best-belief score on a fixed ground-truth anchor, preserving an anchor-guided improvement criterion across epochs.The promoted evaluator maximizes the same lower-bound score used for agent selection, with ties favoring the incumbent.Source: §3.5 · p. 6

4 Experimental Design

The experiments compare RQGM with frozen-evaluator HGM-H baselines across coding, writing, reviewing, and proof tasks using ground-truth anchors and matched search budgets. Results are reported through specialist/generalist choices, held-out evaluation, cost-to-quality comparisons, and epoch-aware visualizations.

4 Experimental Design

  • RQ1–RQ4 test benefits on expensive verification, benchmark-free generation, evaluator-guided search, and evaluator improvement.The questions span coding, paper writing and reviewing, and proof writing and grading.Source: §4 · p. 6
  • Paper writing uses APReS anchors, proof grading uses IMO-GradingBench, and coding combines executable Polyglot tests with CRAVE reviewer labels.The paper writer and proof prover lack direct anchors and are scored post-hoc with fixed external judges.Source: §4 · pp. 6–7
  • Specialists maximize best-belief on one target utility, whereas generalists maximize average best-belief across active utilities.Reporting both distinguishes task-specific from joint optimization.Source: §4 · p. 7
  • HGM-H is the learned baseline with frozen external evaluators, supplemented by published HyperAgents patches where available.Baselines include Sakana’s paper reviewer, ProofAutoGrader, and IMO25 prover.Source: §4 · p. 7
  • Epoch-boundary objectives are also used to penalize rejection failures on reviewer-accepted AI papers while preserving accuracy on human APReS data.This intervention targets self-preference bias through an adversarial pool formed after the first evaluator replacement.Source: §4 · p. 7
  • Search budgets are matched, and runs are compared by best agent at matched compute, cost to matched quality, and best overall.Blended tokens include input plus output, with output weighted at 5× input cost.Source: §4 · p. 7

4.1 Visualization Guide

  • Figure 1 plots held-out pass rate against search cost and separately shows best-belief utility trajectories across evaluator epochs.Evaluator replacements are marked by crowned dashed rules; selective erasure can cause drops before recovery under the replacement.Source: §4.1 · p. 7

5 Results

Across coding, writing, proof, and evaluation tasks, RQGM improves search by co-evolving learned evaluators and changing utilities at controlled epoch boundaries. Evaluator replacement re-ranks search while preserving productive lineages, and hybrid models reduce cost with domain-dependent trade-offs.

5.1 Learned evaluation helps even where ground truth exists (RQ1)

  • In the Polyglot coding domain, RQGM exceeded HGM-H’s held-out pass rate for both specialist and generalist at 1.35×–1.72× lower token cost, with a cheaper, complementary agent-as-judge surrogate objective.In the Polyglot run, 90% of accepted patches modify shared task-agent functionality or infrastructure used by both the coder and reviewer.Source: §5.1 · p. 8

5.2 Learned evaluation improves agents on domains without objective evaluation (RQ2)

  • 1.78× higher reviewer-panel acceptance at matched compute and 1.86× across the full run show that co-evolved reviewers improve paper writers.The reviewer objective is regularized to favor harsher evaluation of AI-generated text, providing a stricter signal for writer search.Source: §5.2 · p. 8, Table 1 · p. 9
  • The RQGM specialist prover attains the best panel-mean score and Pass@6, while longer-horizon gains emerge as HGM-H stagnates under its frozen evaluator.The specialist exceeds the best HGM-H prover on Pass@7, but the remaining gap to the IMO25 baseline is attributed to search budget.Source: §5.2 · pp. 8–9, Table 2 · p. 9

5.3 Evaluator replacements act as a curriculum on the search (RQ3)

  • Evaluator replacements permanently re-rank the archive, while selective erasure enables new candidates to contend and a resilient backbone lineage preserves progress.The no-erasure control remains near the displaced ordering, whereas the paper-run archive retains the winning lineage while re-ranking its top nodes.Source: Figure 3 · p. 9, §5.3 · p. 10, Figure 4 · p. 10

5.4 Co-evolution improves the evaluators themselves (RQ4)

  • The co-evolved grader becomes the strongest judge at 3× lower specialist search cost than HGM-H, and the adversarial reviewer accepts AI and human papers at similar rates with 80% ground-truth accuracy.The grader improves without adversarial objectives; the reviewer needs an adversarial pool at an epoch boundary to correct self-preference bias.Source: §5.4 · pp. 11–12, Figure 5 · p. 11, Figure 6 · p. 12

5.5 Reducing search costs via hybrid-model approaches

  • ≈13.0× lower search-token cost approaches GPT-5.5-only accuracy when Nemotron 3 Ultra handles search-time task-agent calls under final GPT-5.5 evaluation.The benefit is domain-specific: an insufficiently capable cheaper model may make expansions harder to distinguish and increase search costs.Source: §5.5 · p. 12, Figure 7 · p. 12

6 Conclusion

RQGM enables co-evolving evaluators and agents under changing utilities by freezing evaluation within epochs and replacing evaluators only at checkpoint boundaries. The paper concludes that this supports broader self-improvement, while current guarantees remain epoch-local and depend on the quality of fixed anchors.

6 Conclusion

  • Controlled utility evolution changes the objective only at epoch boundaries, preserving stationary evaluation and self-improvement guarantees within each epoch.Challenger evaluators replace incumbents only when they improve an epsilon-best-belief score on evaluator-independent ground truth.Source: §6 · p. 13, §6.1 · p. 13
  • RQGM co-evolves evaluators with agents, enabling non-stationary and adversarial objectives that fixed evaluation criteria cannot express.The framework supports interventions such as an adversarial term that corrects reviewer self-preference bias during search.Source: §6 · p. 13

6.1 Future Research Directions

  • The framework’s current investigation is preliminary, using short search horizons that the authors intend to extend.Longer co-evolution is presented as a future direction rather than an established result.Source: §6.1 · p. 13
  • RQGM’s evaluator quality is bounded by its ground-truth anchor, and its theoretical guarantees cover improvement only within individual epochs.Weak or biased anchors may yield uninformative evaluators, while improvements beyond the benchmark require objectives layered on top of the anchor.Source: §6.1 · p. 13

B The Algorithm in Full

  • The full procedure freezes evaluator slots between checkpoints, promotes anchor-best challengers, erases displaced-slot records, and returns the archive node with the highest epsilon-best-belief score.Search expands and evaluates agent work under frozen evaluators, while replacement and record erasure occur only at checkpoints.Source: B The Algorithm in Full · pp. 18–19

C Experimental Setup

The experimental setup pairs generator roles with learned evaluator roles and fixed ground-truth anchors across coding, paper, and proof domains. Runs use common model and search settings, with frozen evaluator slots and checkpoint-only replacement.

C.1 Domains and Anchor pairs

  • Each domain pairs a generator role with a learned evaluator role and a ground-truth anchor, separating durable anchor evidence from epoch-local evaluator records.Only evaluator-independent anchor evidence enters evaluator replacement, while displaced evaluator records are erased after replacement.Source: §C.1 · p. 19, Figure 8 · p. 20

C.2 Models and Run Identities

  • All eight main experiments use GPT-5.5 (low), identical total budgets, and best-belief selection, with HGM-H receiving the same exploration budget.The study covers coding, paper writing and reviewing, and proof writing and grading, while allowing cheaper delegated models in specified settings.Source: §C.2 · p. 20

C.3 Search and Scheduling Settings

  • Search uses frozen evaluator slots and checkpoint-only replacement under an epsilon-best-belief rule, with a 12,288-evaluation budget chosen between single-task and doubled-pair alternatives.Nodes require at least five accumulated evaluations for the shared replacement and selection rule.Source: §C.3 · p. 20

C.4 Held-Out Evaluation Protocols

Held-out protocols evaluate coding, writing, reviewing, grading, and proving with domain-specific anchors and fixed evaluation panels. The initial agents share a minimal template, while the meta-agent can modify the codebase and must optimize unseen validation outcomes rather than training feedback.

C.4 Held-Out Evaluation Protocols

  • Coding patches are evaluated by execution on 166 held-out tasks, while writers and reviewers use held-out APReS papers and fixed reviewer panels.Reviewer endpoints are evaluated on 100 held-out APReS papers, and writer endpoints generate 100 papers each for panel scoring.Source: §C.4 · pp. 20–21
  • Proof graders use 100 held-out IMO-GradingBench items, while provers write 20 proofs per problem set scored by three fixed graders.Pass@6 counts proofs scoring at least 6 of 7 points, whereas Pass@7 requires full credit.Source: §C.4 · p. 21
  • Writer-panel uncertainty uses 95% Jeffreys intervals over 100-paper outcomes, whereas prover-panel results use standard errors over 60 raw outcomes.The two panels’ interval widths therefore are not directly comparable.Source: §C.4 · p. 21
  • The cost-reduction ablation uses Nemotron 3 Ultra for search-time task-agent calls but GPT-5.5 for final reviewer scoring, reporting GPT-5.5-price-equivalent blended tokens.The conversion weights expansion at 20% and repeated evaluation at 80%, producing approximately 0.28B equivalent tokens from B Nemotron tokens.Source: §C.4 · p. 22

C.5 Initial Agents and Prompts

  • All roles begin from a shared minimal template, and evaluator behavior beyond output schemas is produced by search rather than engineered prompts.The meta-agent may modify its own loop, prompts, tool surface, and task-agent harness.Source: §C.5 · p. 22
  • Selection uses a held-out validation set unavailable to the meta-agent, so readable training predictions provide feedback but cannot be the optimization target.Outcomes are binary, and utility is the mean validation outcome for each role-task pair.Source: §C.5 · p. 23

D Results and Ablations

The supplied results appendix identifies the raw held-out counts and per-grader proof scores underlying the paper’s headline results. These tables support detailed evaluation of the coding, writing, reviewing, grading, and proof experiments.

D.1 Raw Held-Out Counts

  • Table 5 reports raw held-out counts underlying the results in Sections 5.1 and 5.2.Source: Table 5 · p. 24
  • Table 6 reports per-grader proof scores underlying Table 2.Source: Table 6 · p. 24

D.2 Mechanism Ablations

The ablations show that evaluator replacement escapes fixed-critic saturation, while selective erasure prevents stale evidence from repeatedly steering search toward obsolete nodes. Across the broader co-evolution analysis, validation dominates cost, shared code surfaces attract most accepted patches, and evolved evaluators become more evidence-bounded.

D.2 Mechanism Ablations

  • 52/57 anchor result: evaluator replacement alone escapes the fixed critic’s 75/75 acceptance plateau, while the adversarial pool mainly shapes reviewer selection.The fixed critic trivially raises writer acceptance beyond the human rate; replacement re-bases the writer after each evaluator change.Source: §D.2 · p. 24
  • Selective erasure prevents stale retired-critic evidence from causing A→B→A oscillations and repeated exploration of the same nodes.Without erasure, stale rows remain in official utility and eventually stop evaluator replacements.Source: §D.2 · p. 24

D.3.1 Cost decomposition

  • 65–69% of blended tokens go to validation, versus 18–23% for workspace expansion and 12–14% for train-time evaluation across the three main runs.The authors note that increasing train samples could make search more efficient, suggesting the current train-time evaluation share may be low.Source: §D.3.1 · p. 25, Figure 9 · p. 25

D.3.2 Patch surfaces

  • 59–90% of accepted patches in every GPT-5.5 run modify shared task-agent or infrastructure surfaces rather than role-specific code.The Polyglot run concentrates on shared and infrastructure surfaces, while the Nemotron 3 Super outlier targets the meta-agent instruction module in 42% of edges.Source: §D.3.2 · p. 25, Figure 10 · p. 26

E What the Co-Evolution Discovered

  • Across roles, the most common evaluator change rewrites a vague one-line instruction into a specific, criterion-bearing rubric.The shared workspace lets each evaluator’s behavioral history directly inform its lineage.Source: E What the Co-Evolution Discovered · p. 26

E.1 What the Evolved Evaluators Changed

  • The adversarial reviewer selects a conjunctive standard, while the grader evolves from strict milestone scoring toward calibrated tolerance for repairable gaps.The adversarial pool selects a lineage that rejects a greater fraction of AI-generated papers; the grader’s later rule reserves downgrades for fatal gaps, false central claims, or missed milestones.Source: §E.1 · p. 26, §E.2 · pp. 26–27

E.3 The Polyglot Code Reviewer

  • The code reviewer lineage moves toward evidence-bounded failure decisions, requiring concrete defects visible in the diff rather than style concerns or missing context.Successive reviewers prioritize intended behavior, production correctness, definite regressions, and specific broken scenarios.Source: §E.3 · p. 27

E.4 Winning Lineages

Winning lineages combine ordinary code repairs and evaluator changes within an archive-based tree search. RQGM preserves HGM-compatible fixed-criterion reasoning locally by freezing evaluators within epochs and replacing them only at checkpoints with stale evidence removed.

E.4 Winning Lineages

  • Node 19 filters untracked dependency artifacts and unsafe symlinks from submitted patches, making diff hygiene a consequential non-evaluator modification.The generalist chain’s largest gain is described as a code repair rather than a prompt change.Source: §E.4 · pp. 27–28
  • Both final Polyglot best-belief agents score 119/166 held out, while the paper-domain writer and adversarial reviewer follow distinct winning chains.The paper writer chain passes through reviewer 49 and generalist 78, whereas adversarial reviewer 31 descends separately.Source: Table 8 · p. 29, Table 9 · p. 29
  • Node 66 disables self-revision by default to preserve essentially sound detailed proofs instead of replacing them with shorter answers relying on obscure unproved theorems.The retraction records that revision could degrade correct or near-correct proofs into weaker citation-based proofs.Source: §E.4 · p. 29
  • The largest single-step proof-chain gain is the grader’s calibration reversal at node 22, which removes blanket strictness while retaining milestone-based scoring.The grader’s reported form is reached through two patches, with the later patch removing blanket strictness.Source: Table 10 · p. 30

F.1.1 Self-Improvement as Tree Search

  • HGM searches an archive tree by alternating node modification and binary evaluation, then returns the highest-scoring node after a fixed budget.RQGM generalizes the node workspace and outcome-generation process while preserving the flat HGM search algorithm.Source: §F.1 · p. 30, §F.1.1 · p. 30

F.1.3 Clade-Level Metaproductivity

  • HGM’s clade-level metaproductivity scores descendants collectively, and RQGM retains Thompson sampling, UCB-Air tree growth, and shared workspaces with role agents plus a meta-agent.A node’s immediate performance may not predict its long-run value because weak nodes can produce strong descendants.Source: §F.1.3 · p. 31

F.2.1 Evaluator replacement rule

  • Evaluator replacement selects a challenger by conservative ε-best-belief performance on evaluator-independent anchor evidence, then freezes the winner for the next epoch.If the selected evaluator is not better than the incumbent under the replacement rule, no transition occurs.Source: §F.2 · p. 31, §F.2.1 · p. 31

F.3 Working-Posterior Consistency

The theory shows that balanced role–task sampling makes the pooled HGM accumulator asymptotically mean-consistent, while epoch freezing and selective erasure restore a fixed-criterion search problem after evaluator changes. The pooled distribution remains a working, conservative approximation rather than an exact Bayesian posterior.

F.3 Working-Posterior Consistency

  • Under balanced role–task sampling and cell-wise strong-law assumptions, the pooled Beta accumulator preserves the correct role–task-balanced posterior mean asymptotically.The result retains HGM’s flat success/failure interface for Thompson sampling.Source: §F.3 · p. 32
  • The pooled working posterior is not an exact Bayesian posterior for a role–task mixture unless all cell success probabilities are equal.It is over-dispersed relative to the balanced stratified estimator, supporting a conservative lower-bound interpretation.Source: §F.3 · p. 33

F.4 Epoch-Local Fixed-Utility Validity

  • Within a fixed epoch, frozen criteria, workspaces, sampling protocols, and binary scoring rules make evaluator-dependent outcomes time-homogeneous.The assumption permits independent Bernoulli outcomes or fixed conditional means without requiring independence.Source: §F.4 · p. 33
  • Selective erasure removes utility records dependent on a displaced evaluator and recomputes all derived statistics from the retained criterion-valid records.Raw artifacts and audit logs may remain, but they contribute to utility only through newly valid evidence records.Source: §F.4 · pp. 33–34
  • After selective erasure, the archive remains criterion-consistent, and subsequent records generated under the new epoch are valid by construction.Transitions on disjoint evaluator slots commute because their filters remove the union of affected records.Source: §F.4 · pp. 34–35
  • Conditioned on a fixed epoch and criterion-consistent archive, RQGM reduces to an ordinary fixed-criterion binary-outcome search problem to which applicable HGM oracle-level theorems extend.The empirical Thompson-sampling implementation is only a working estimator, not an exact CMP-oracle implementation.Source: §F.4 · pp. 35–36

F.5 Supporting Results for Controlled Utility Evolution

The supporting results establish that controlled utility evolution preserves fixed-criterion validity within epochs while enabling evaluator replacement across checkpoints. The guarantees include calibrated lower bounds, non-worsening evaluator promotion, and linear replay overhead under exponential checkpoints.

Evaluator anchor lower bound

  • The checkpoint rule selects an evaluator maximizing its best-belief lower bound, breaking ties in favor of the incumbent.This rule underlies the non-worsening promotion result on the shared anchor evidence.Source: §F.5 · p. 36
  • At evaluator checkpoints, replacement is never worse than retention on the fixed anchor, because the incumbent remains eligible and ties favor it.The promoted evaluator’s anchor best-belief is therefore at least the incumbent’s at the moment of replacement.Source: §F.5 · pp. 36–37

Best-belief lower-bound trajectory

  • With calibrated working posteriors, a sequence of L selected candidates simultaneously exceeds its reported best-belief lower bounds with probability at least 1−Lε.The proposition is a calibration statement about the selected lower-bound trajectory, conditional on the stated posterior assumptions.Source: §F.5 · p. 36

Piecewise fixed-criterion validity

  • Each realized epoch is a fixed-criterion binary-outcome search problem with a fixed role–task balanced utility after criterion-consistent erasure.The result is conditional on the checkpoint history and selected criterion vector; stale records are removed and derived statistics recomputed.Source: §F.5 · p. 37
  • The formal guarantee is epoch-local: it does not establish monotonic multi-epoch performance or global convergence after criteria change and evidence is erased.The inherited tree is conditioned on at epoch boundaries, while the within-epoch outcome process remains criterion-consistent.Source: §F.5 · p. 37

Evaluator anchor lower bound

  • With calibrated anchor posteriors, every promoted evaluator’s true anchor utility exceeds its checkpoint lower bound jointly with probability at least 1−Kε.This is a per-checkpoint lower-bound statement, not a guarantee that realized anchor accuracy improves monotonically.Source: §F.5 · p. 37

Exponential checkpoint overhead

  • O(B) records are reprocessed under exponential checkpoints, with constant at most ρ/(ρ−1), whereas uniform checkpoints require Θ(B^2/h) work.For the common base-two schedule, the bound is at most 2B; bounded recovery remains the default for auditability and evaluation efficiency.Source: §F.5 · pp. 37–38

G Limitations

The paper’s limitations concern empirical scope, formal guarantees, evaluation validity, anchor quality, and the hand-crafted search loop. These constraints limit claims about generalization, long-term convergence, and recursive self-improvement beyond the tested settings.

Empirical scope

  • The empirical evaluation covers coding, paper writing and review, and IMO grading and proof writing, with each domain isolated under an HGM-H control.A unified cross-domain, all-role tree was not tested, and main experiments relied exclusively on GPT-5.5.Source: G Limitations · p. 38

Formal guarantees

  • Theoretical guarantees are epoch-local and do not bound transition counts, cumulative regret from erased evidence, or convergence to a globally optimal agent–evaluator pair.The guarantees instead characterize calibrated best-belief lower-bound trajectories under stated assumptions.Source: G Limitations · p. 38

Evaluation validity

  • Reviewer-panel metrics measure cross-reviewer acceptance behavior rather than objective scientific merit, and no human grading of generated papers or proofs was performed.Because the evidence comes from intellectual-artifact domains, the observed dynamics may not generalize to environments with different feedback structures.Source: G Limitations · p. 38

Anchor quality

  • Weak, noisy, or biased anchor datasets can allow evaluators to drift across epochs, especially because the APReS and IMO-GradingBench anchors are imperfect.The paper notes that this issue is fully avoidable only in domains with verifiable ground truth, such as mathematics.Source: G Limitations · p. 38

Search-loop design

  • The hand-crafted search loop limits recursive self-improvement to the selected benchmarks and task-agent outputs, while evolving the scheduler or replacement rules would require stronger guardrails.Those outputs are shaped by both the benchmarks and the foundation models’ pretraining distribution.Source: G Limitations · p. 38

Related papers

Loading 2606.26294v2…