Selection Is Not Learning: Choosing Model Futures

Motivation

• Self-training can raise a target metric and then collapse it within one training campaign, making the inheritance decision consequential.Each inherited update changes both the learner and the curriculum of later rounds.Source: §1 · p. 2
• Task-side selection leaves a selection-to-update gap: learnability signals do not reveal whether an optimizer update will improve the model.In the audits, 60 of 96 Skill-SP updates on exact-valid tasks have nonpositive correct-target NLL gain.Source: §1 · p. 2

Method

• HEIRCOMMIT freezes a successor and uses fresh probes to decide whether to inherit it or retain the current model.Fresh evidence separates confirmation from noise that favored the proposal during selection.Source: §1 · p. 2
• EVOHEIR selects the parameter recipient separately from the experience it absorbs, repairing failures with current and archived references while preserving successes.Qualified experience is archived whether or not the resulting parameter state is inherited.Source: §1 · p. 2

Results

• Across four backbones, EVOHEIR achieves the highest mathematics and general-reasoning averages, and beats Agent0 on 45 of 48 benchmark–backbone pairs.HEIRCOMMIT cuts the harmful share of accepted updates by 68–83% relative to random inheritance at the same acceptance rate on three hosts.Source: §1 · p. 2

Self-evolution and update acceptance

• Self-evolution methods generate supervision or tasks through self-play and student feedback, while R-Zero, Agent0, and Skill-SP adapt curricula (Huang et al., 2026a; Xia et al., 2026).Earlier curriculum methods select tasks using learning progress (Graves et al., 2017), while Willems et al. (2020) argue that signal is flawed and propose mastering rates.Source: §2 · pp. 2–3
• Update acceptance ranges from candidate fine-tuning in APE (Mar´ın, 2025) to agent-side skill verification and update certificates (Shang et al., 2026; Sengupta, 2026).SOAR and Actor-Curator use realized improvement (Gu et al., 2026).Source: §2 · p. 3

Experience persistence

• EVOHEIR distinguishes its approach by choosing the recipient and experience donors separately, then confirming the post-absorption successor.Related approaches include replay (Rolnick et al., 2019), policy combination, and internalizing accumulated experience (Ye et al., 2026a; Ren et al., 2026).Source: §2 · p. 3

Privileged-context distillation

• Recovery combines recipient-specific repair and preservation with cross-branch references, fixed recipient trajectories, and post-absorption confirmation.The related-work comparison places this alongside privileged-context distillation methods including GKD (Agarwal et al., 2024), SDFT, SDPO, and OPSD (Shenfeld et al., 2026; H¨ubotter et al., 2026; Zhao et al., 2026).Source: §2 · p. 3

Self-evolution environment

• Each round starts from learner state St = (θt, ωt), trains two native branches, and admits tasks based on pre-update observations such as pass rate over ten learner responses.The Evolution Controller admits tasks at the Evolving Learner’s frontier; training updates both agents in an R-Zero-style loop (Huang et al., 2026a).Source: §3 · p. 3
• Qualified branch responses enter a bounded archive, while training, recovery, screening, confirmation, and final evaluation use disjoint question pools.Online scores use frozen consensus labels.Source: §3 · p. 3

Value and inheritance

• For task distribution Pt and utility q, Vt(S) is the expected utility gain of state S over the root, with the root assigned value zero.Capability uses exact correctness or sampled pass@1, while update audits use correct-target NLL reduction, not an accuracy difference.Source: §3 · p. 3
• With binary correctness, value equals acquired successes minus lost successes relative to the root.The selection-to-update gap is the failure of task-side admission to identify positive value.Source: §3 · p. 3

Self-evolution environment

• Each EVOHEIR round explores native branches, screens a parameter recipient, recovers experience, and confirms the result against the root on fresh probes.The figure caption says the archive keeps qualified experience whether or not the round commits.Source: Figure 3 · p. 4

Hosts, backbones, and benchmarks

• The study evaluates HEIRCOMMIT on R-Zero, Agent0, and Skill-SP, and runs ten-round self-evolution experiments across four backbones and twelve benchmarks.The backbones are Qwen3-4B-Base, Qwen3-8B-Base, OLMo-3-7B, and Gemma-4-E4B-it; evaluation covers seven mathematics and five general-reasoning benchmarks.Source: §3 · p. 4

4 EVOHEIR: EXPLORATION, EXPERIENCE RECOVERY, AND HEIRCOMMIT

• EVOHEIR runs four steps: explore native branches, screen a parameter recipient, recover current and archived experience into it, and confirm the result against the root.The archive retains qualified experience whether or not the round commits.Source: §4 · p. 4

4.1 EXPLORE AND SCREEN A RECIPIENT

• Screening selects the parameter starting point for absorption from native endpoints and any predeclared training-prefix checkpoints, using paired root-relative scores.An Experience Donor contributes qualified responses without supplying inherited parameters or needing to remain available as a teacher model.Source: §4.1 · p. 4

4.2 RECOVER EXPERIENCE WITHOUT INHERITING THE DONOR

• Current exploration and a bounded archive provide candidate problems and references; qualification checks completeness, answer validity, and problem-level deduplication.Recovery material is disjoint from screening, confirmation, and final evaluation.Source: §4.2 · p. 4
• Recovery identifies recipient failures and successes, using qualified references to repair failures and preserving successful attempts in separate groups.An archived success becomes repair material when the current recipient cannot reproduce it; the recipient trajectories and groups remain fixed during absorption.Source: §4.2 · pp. 4–5
• A recipient-initialized student matches a repair teacher on failed attempts and a keep teacher on successful ones.The repair teacher conditions on a donor reference, whereas the keep teacher and student see only the problem; the resulting learner is deployed, not teacher conditioning.Source: §4.2 · p. 5

4.3 HEIRCOMMIT: CONFIRM THE FINAL PROPOSAL

• HEIRCOMMIT scores the frozen final proposal after absorption on a held-out confirmation pool and inherits it or retains the root based on paired root-relative value.Rejection restores the complete root state, while qualified experience is archived independently of the decision.Source: §4.3 · p. 5

• For any Δ > 0, identical pre-update observations and equally valid targets can yield update values +Δ or −Δ, forcing worst-case expected regret of at least Δ/2 against the better of the update and root.The result applies to persistence rules based only on pre-update behavioral observations.Source: §5 · p. 5
• Screening inflates the selected winner’s score even when each candidate’s score is unbiased (Cawley & Talbot, 2010; Russo & Zou, 2016), while reusing screening for inheritance pays a union bound over branches that fresh confirmation avoids.The analysis motivates screening and confirming on disjoint pools, including when absorption follows recipient selection.Source: §5 · p. 5
• The reported zero-threshold gate trades the positive-margin guarantee for greater acceptance sensitivity.Source: §5 · p. 6
• Within the realized candidate set, regret is bounded by twice the screening error plus absorption loss, confirmation error, and the margin; No Recovery has zero absorption loss.The bound is stated in Corollary 1 in Appendix C.3.Source: §5 · p. 6

6.1 TASK SELECTION DOES NOT PREDICT LEARNING

• 60 of 96 real Skill-SP updates (62.5%) do not reduce correct-target NLL, while 52.2% of Agent0 and 42.0% of R-Zero admitted tasks leave the native frontier under fresh responses.These audits distinguish pre-training task behavior from the value of the resulting update.Source: §6.1 · p. 6

6.2 HEIRCOMMIT CUTS HARMFUL COMMITS ON EVERY HOST

• 68–83% relative to matched random commits, HEIRCOMMIT lowers the harmful share of accepted updates from 47–77% to 13–16% across all three hosts.It commits 23–52% of updates and retains 78–87% of useful updates.Source: §6.2 · pp. 6–7

6.3 EVOHEIR LEADS ON EVERY BACKBONE

• After ten rounds, EVOHEIR (Full) has the highest Math Avg. and General Avg. on every backbone; it exceeds Agent0’s Math Avg. by 1.59–2.40 pp (6.0–7.0%) on three base models.Full still leads by 0.55 pp on Gemma-4-E4B-it, the only instruction-tuned start and the backbone with the least headroom.Source: §6.3 · p. 7, Table 1 · p. 7
• Full beats Agent0 on 45 of 48 benchmark–backbone pairs; the three exceptions are 30-problem AIME sets.Full also exceeds No Recovery on every backbone, isolating Recovery as a module; Figure 6 shows Full ending the ten rounds highest on all four backbones.Source: §6.3 · p. 7, Figure 5 · p. 7

6.4 EXPERIENCE OUTLIVES ITS MODEL

• Eight of ten donors weaker than the current root still improve the recipient, despite mean donor gain of −1.09 pp and mean absorption gain of +0.71 pp.The study tests 16 donor interventions on four fixed recipients using the same 1,024 evaluation items before confirmation.Source: §6.4 · p. 7
• With the same 64 references, Recovery gains 2.54 pp and loses 1.15 pp, for +1.39 pp net, while Cache SFT gains 2.08 pp and loses 2.61 pp, for −0.54 pp net.Recovery beats Cache SFT at all four checkpoints by 1.93 pp on average, though its net gain is −0.20 pp at round 7.Source: §6.4 · p. 7
• At a fixed 64-reference budget, admitting all qualified history, including rounds whose model was rejected, yields +1.39 pp versus +0.39 pp for current-only material.Matched replacements show non-inherited records add 0.34–0.73 pp at ages one to four, with pointwise 95% bootstrap intervals above zero.Source: §6.4 · pp. 7–8

6.5 TARGETED REUSE AND CONFIRMATION BOTH MATTER

• NoConfirm has the largest ablation drops: 2.02 pp in Math Avg. and 1.19 pp in General Avg.; other mechanism removals lower the averages by 0.97–1.28 and 0.48–0.65 pp, respectively.At equal budget, Full's mean is lowest at 0.8 of default but highest from 1.0 onward; λkeep = 1 gives the largest net absorption gain.Source: §6.5 · p. 8, Table 2 · p. 8, §6.6 · p. 9

7 CONCLUSION

• 60 of 96 updates on verified tasks show no gain, illustrating that admitting a learnable-looking task does not ensure a useful update.The conclusion argues that systems should judge the updates they produce, not only the tasks they choose.Source: §7 · p. 9
• 68–83%: HeirCommit cuts the harmful share of commits on three self-evolution hosts by requiring successors to beat the current model on fresh probes.The conclusion describes this as a reduction in harmful commits.Source: §7 · p. 9
• Up to 7.0%: EvoHeir raises the math average over Agent0, the strongest baseline, while keeping experience beyond the models that produced it.EvoHeir separates the parameters a round inherits from the experience it keeps.Source: §7 · p. 9

A EXPERIMENTAL SETUP AND HYPERPARAMETERS

• Four backbones run for ten rounds with three training seeds; native exploration trains two isolated branches from the same root, each on 64 questions.Screening uses 16 questions, while confirmation uses 32 fresh questions with four responses per question and state.Source: A EXPERIMENTAL SETUP AND HYPERPARAMETERS · p. 15

B.1 STATE CONSTRUCTION AND RECIPIENT SELECTION

• The controller freezes the root and branch endpoints before screening, selects a recipient by paired root-relative gain, and breaks ties in favor of the root.Each branch starts from the same resumable root state and follows the host’s training objective.Source: §B.1 · p. 16

B.2 RECIPIENT-CONDITIONED RECOVERY

• Recovery uses current and archived records to repair recipient-specific failures and preserve successes, verifying attempts and references for the selected recipient.The teacher and student use the same fixed recipient prefixes during absorption rather than resampling from the changing student.Source: §B.2 · p. 16

B.3 ARCHIVE, FEEDBACK, AND STATE INHERITANCE

• The completed proposal is compared with the original root on a fixed confirmation pool; qualified current records may enter the archive regardless of the commit decision.A flawed reference that passes verification can still steer repair, absorption can lose successes, and the empirical zero threshold can admit a harmful proposal through sampling noise.Source: §B.3 · pp. 16–17

B.4 IMPLEMENTATION CONFIGURATION

• Recovery uses response-token loss with equal per-question weighting and a bounded archive of qualified records, while each round’s recovery pass allows at most 16 optimizer updates.The archive holds at most 256 records; conflicting labels, known reasoning errors, incomplete responses, and non-training records are excluded.Source: §B.4 · p. 17

C.1 SETUP AND TASK-SIDE NON-IDENTIFIABILITY

• An observation-based rule has maximum expected regret of at least ∆/2 in the constructed worlds, whose identical root observations lead to native updates with opposite values.The two worlds share stipulated root behavior and target validity but have training gradients +1 and −1.Source: §C.1 · pp. 17–18

C.2 SELECTED-FUTURE OPTIMISM AND INDEPENDENT CONFIRMATION

• Under conditionally centered screening errors, selecting the maximum score creates nonnegative expected optimism, even when candidates’ errors are correlated through shared probes.The empirical mode uses a zero threshold; the stated confirmation-error proxy applies to exact correctness and consensus matching, while raw negative-NLL utility needs a separate tail assumption.Source: §C.2 · pp. 18–19

C.3 REGRET DECOMPOSITION AND ADAPTIVE ROUNDS

• The regret analysis defines absorption loss as decreases from recipient to proposal; prescribed positive confirmation-error budgets can sum to δ across an unbounded horizon.The unbounded-horizon allocation is δt = δ/[t(t + 1)].Source: §C.3 · p. 19

D AUDITS OF TASK SELECTION AND UPDATE INHERITANCE

• The task-selection audit measures selected-target optimism using an independent response batch and separately counts admitted tasks that leave the native frontier.The exact-valid audit compares paired held-out correct-target losses across 96 realized Skill-SP optimizer transitions.Source: D AUDITS OF TASK SELECTION AND UPDATE… · p. 19
• Figure 10 reports three-seed mean results for every benchmark, backbone, and method after ten rounds, with colors showing change from each backbone’s starting checkpoint.Base checkpoints appear as gray rows.Source: Figure 10 · p. 20
• The inheritance audit defines coverage, useful retention, harmful commits, and pool gain, comparing commit-all with matched random commits at HeirCommit’s coverage.Pool gain averages NLL reduction over the full pool and counts a rejection as zero.Source: D AUDITS OF TASK SELECTION AND UPDATE… · p. 20, Table 4 · p. 20

E EXTENDED SELF-EVOLUTION RESULTS

• The ten-round study covers seven mathematics and five general-reasoning benchmarks, with equally weighted averages and three-seed means.Evolved scores are measured after ten rounds.Source: E EXTENDED SELF-EVOLUTION RESULTS · pp. 20–21

F DISENTANGLING PARAMETER AND EXPERIENCE INHERITANCE

• Donor–recipient interventions fix the root, recipient, donor, qualification rule, and absorption budget, then measure donor and absorption gains using independently evaluated exact-answer correctness.The donor gain is measured against the current root, including for archived donors; each reference is attributed to its generating checkpoint.Source: F DISENTANGLING PARAMETER AND… · p. 21
• Recovery gains 3.57 ± 0.30 pp, versus 1.11 ± 0.86 for Cache SFT and 2.26 ± 0.45 for Repair only; λkeep = 1 gives the largest net gain.The fixed-recipient diagnostic uses 2,400 problems and three absorption-order seeds.Source: F DISENTANGLING PARAMETER AND… · p. 21, Figure 11 · p. 21
• Recovery beats Cache SFT at every recipient checkpoint in the matched-source comparison.The comparison matches current records, record counts, and repair and keep group sizes while varying whether qualified records come from non-inherited or same-age inherited sources.Source: F DISENTANGLING PARAMETER AND… · p. 21, Figure 11 · p. 21

G COMPUTE AND BUDGET ACCOUNTING

• A Full run takes 80–90 hours across ten rounds; normalized accounting assigns 123.7 units to Full versus 100.0 for the native baseline, not GPU-hours or FLOPs.The ten-round results use the same horizon and three-seed aggregation across methods.Source: G COMPUTE AND BUDGET ACCOUNTING · pp. 21–22

H PROMPT TEMPLATES AND INPUT FORMATS

• The Curriculum Scout proposes tasks, while the Evolving Learner trains, attempts problems, and receives Recovery updates; native admission uses the learner’s response consensus, not the Scout’s answer.The Scout uses the same two messages each round for native training and candidate-pool generation.Source: H PROMPT TEMPLATES AND INPUT FORMATS · p. 22, §H.1 · p. 22

H.1 NATIVE PROBLEM GENERATION

• The Scout prompt asks for an original competition-mathematics problem predicted to be solved by fewer than 30% of advanced high-school students, with a question and boxed answer only.The Scout is instructed to work out the answer privately before responding.Source: §H.1 · p. 22

H.2 EVOLVING LEARNER PROBLEM-SOLVING INPUT

• Native rollouts, admission, recipient attempts, screening, and confirmation use the same learner instruction, which requests intermediate reasoning and a boxed final answer.The problem is supplied as the user message.Source: §H.2 · p. 22

H.3 REFERENCE-CONDITIONED REPAIR INPUT

• The repair teacher receives one qualified reference and aligns its forward passes to the recipient’s fixed recorded response; the student and keep teacher see only the problem-solving input.A reference may come from an unselected branch or an earlier round.Source: §H.3 · p. 22

H.4 ANSWER VERIFICATION AND CONFIRMATION

• Recovery eligibility filters out contradictory problems, wrong labels, and erroneous references; screening and confirmation use numerical decisions on fresh responses, while benchmark data never enter training or memory.The runs use no natural-language judge prompt or model-generated accept/reject decision.Source: §H.4 · p. 23