PaperlayerView exact revision on arXiv

Source-linked AI summary

Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges

Jiayue Pu, Ang Gao, Jinyi Bai, Qipeng Wang, Tongyu Wen, Jiahao Zhao, Keyu Qu, Junhao Zhang, Chenglei Shen, Zhongxiang Sun, Jun Xu, Xiao Zhang

arXiv:2610.35193v1cs.AIcs.CR

TL;DR

Trustworthy recursive self-improvement requires evidence that gains persist, constraints hold, and oversight remains effective as systems and update mechanisms change. This survey defines RSI, organizes its risks and safeguards, and concludes that aligned, versioned evidence and controlled continuations are needed to assess these claims.

  • Problem

    Trustworthy RSI requires evidence that gains are genuine and sustained, constraints remain satisfied, and authorized oversight remains effective across system changes.

    Source: §8.4 · p. 56

  • Method

    The survey defines RSI through recursive improvement cycles and synthesizes risks, safeguards, and evidence requirements across updates, successors, and mechanism revisions.

    Source: Definition (Recursive… · p. 10, §8.4 · p. 56

  • Results

    Trustworthy RSI claims require capability, constraint, and intervention measurements aligned across versions under independent, comparable, and reproducible conditions.

    Source: §8.4 · p. 56

  • Takeaways & Limitations

    Versioned evidence and controlled continuations can make claims about trustworthy improvement testable while keeping unassessed requirements explicit.

    Source: §8.4 · p. 56

    Rollback assessments must distinguish recoverable system state from irreversible external effects.

    Source: §6.4.2 · p. 43

Abstract

from arXiv · show

AI systems built around large models increasingly use experience and feedback to make persistent changes to models, prompts, memories, tools, and workflows. Recursive self-improvement (RSI) extends this process to inductive biases and to the mechanisms that govern future updates, which can themselves be revised and reused across improvement cycles. Yet self-modification alone does not guarantee trustworthy improvement. Reported gains may reflect flawed evaluations, previously acquired capabilities may deteriorate, and compliance with behavioral constraints may weaken. Compromised artifacts or sensitive information may also persist in successor systems. Changes to improvement mechanisms may invalidate earlier verification results and make human oversight less effective. This survey presents an operational definition and process framework for RSI and organizes the requirements for trustworthy RSI into four interconnected themes: genuine and sustained improvement; behavioral constraints, alignment, and fairness; system security and privacy; and transparency and controllability. We examine these themes from three perspectives: Improvement Validity, Constraint Preservation, and Effective Oversight. This analysis covers improvement diagnosis, proposal generation, proposal evaluation, and update implementation (S1--S4), including the recursive application of these stages to improvement mechanisms (RI). We review risks, safeguards, and evaluation evidence for individual updates, effects inherited by successor systems, and revisions to improvement mechanisms, while distinguishing direct RSI findings from evidence in related systems. Finally, we examine evaluation settings, identify requirements for independent and longitudinal assessment, and discuss applications and open research problems.

1. Introduction

The survey frames recursive self-improvement as persistent changes that can extend to the mechanisms governing later updates. It develops a trustworthiness framework and evidence requirements spanning updates, successors, and revised improvement mechanisms.

1. Introduction

  • Schumacher's epigraph contrasts nature's tendency toward self-adjustment with technology.Source: §1 · p. 4, §1.1 · p. 4

1.1. Motivation

  • The motivation spans potentially misleading improvement scores, deteriorating capabilities, weakened behavioral constraints, inherited security risks, and oversight challenges.The framework groups these concerns into four trustworthiness themes and three analytical roles.Source: §1.1 · p. 5

Key Questions

  • The four research questions ask how to establish sustained genuine gains, preserve behavioral requirements, maintain security and privacy, and keep recursive updates under effective human control.Source: Key Questions · p. 6

1.2. Scope and Perspective

  • The paper examines risks, safeguards, and evidence across diagnosis, proposal generation, evaluation, implementation, and recursive updates to improvement mechanisms.It focuses on large-model systems with persistent changes and distinguishes RSI findings from evidence in related systems.Source: §1.2 · p. 6

1.3. Comparison with Existing Surveys

  • Prior surveys address trustworthy AI, self-evolution, RSI, and evolutionary safety, while this survey distinguishes evidence needs across updates, successors, and mechanism revisions.It also examines whether capability, constraint-preservation, and oversight evidence can be aligned within an evolving system.Source: §1.3 · pp. 6–7

1.4. Contributions

  • Trustworthiness claims must distinguish individual-update evidence from evidence about successor systems and revised improvement mechanisms.The survey links these levels to the need for independent, longitudinal evaluation and joint evidence across aligned trajectories.Source: §1.4 · p. 8, §1.3 · p. 8

1.1. Motivation

  • The survey organizes four trustworthiness themes under Improvement Validity, Constraint Preservation, and Effective Oversight.It distinguishes direct RSI findings from related-system evidence, potential risks, and safeguards not yet empirically validated.Source: §1.4 · p. 8

1.5. Paper Organization

  • The paper proceeds from RSI definitions and trustworthiness foundations through four themes, then evaluation requirements, findings, applications, and open problems.Source: §1.5 · p. 8

2. Preliminaries

The preliminaries define RSI as persistent policy learning with autonomous revision and reuse of the mechanism that governs future updates. They establish a four-stage process and a trustworthiness framework that assesses changes across updates, successors, and recursive revisions.

2. Preliminaries

  • The preliminaries connect the RSI definition and process to the trustworthiness requirements used throughout the survey.The following sections examine four trustworthiness themes, with the first addressing genuine and sustained improvement and the next addressing behavioral constraints, alignment, and fairness.Source: §2 · pp. 8–9

Definition (Machine Learning) (Mitchell, 1997)

  • Large-model systems may realize policy h through a model, prompts, memory, skills, and tools, while bias B governs how that policy is learned or revised.A prompt’s role depends on whether it controls task execution or governs construction or selection of policy updates; shared components can serve both roles.Source: Definition (Machine Learning)… · p. 10

Definition (Recursive Self-Improvement, RSI)

  • RSI updates policy from new experience, autonomously revises its inductive bias, and revises and reuses the bias-update mechanism to seek sustained task-performance gains.Components without accepted revisions retain their previous values.Source: Definition (Recursive… · p. 10

2.1.1. Definition of Recursive Self-Improvement

  • RSI extends ordinary learning by making inductive bias and its update mechanism revisable, with the mechanism reused to govern later bias updates.The survey distinguishes revising aspects of B under a fixed procedure from revising and reusing that procedure itself.Source: Definition (Recursive… · pp. 10–11

2.1. Recursive Self-Improvement: Definition and Process

  • The process framework distinguishes policy h, inductive bias B, and update function U as components that evolve across improvement cycles.It identifies diagnosis, proposal, evaluation, and implementation, plus their recursive application to U.Source: Definition (Recursive… · p. 10, §2.1.2 · p. 12

2.1.2. The RSI Process in the Era of Large Models

  • The S1–S4 cycle diagnoses limitations, generates and evaluates candidate policy or bias revisions, then implements accepted changes; RI applies the same stages to the update function U.The revised U is retained for subsequent cycles if accepted; otherwise it remains unchanged.Source: §2.1.2 · p. 12

2.2. Trustworthiness: Concepts and Requirements

  • The survey synthesizes trustworthiness requirements into eight groups, including performance validity, behavioral safety and alignment, fairness, security, privacy, transparency, human control, and accountability.It groups related performance requirements while treating behavioral safety, system security, fairness, and oversight-related requirements distinctly.Source: §2.2 · p. 13

2.3. Trustworthiness under Recursive Self-Improvement

  • Trustworthiness requirements are organized into four themes under Improvement Validity, Constraint Preservation, and Effective Oversight, with S1–S4 and RI locating where violations and safeguards may arise.The framework assesses accepted updates, inherited effects in successors, and changes to the update mechanism.Source: §2.3 · p. 14

3. Genuine and Sustained Improvement

Genuine and sustained improvement requires evidence that intended capabilities improve and persist, while recursive improvement requires showing that revised mechanisms produce better subsequent updates. The section traces risks and evidence requirements across diagnosis, proposal generation, evaluation, implementation, and recursive improvement.

3.1.1. Conceptual Scope and Boundaries

  • Genuine improvement strengthens intended capabilities, while sustained improvement preserves those gains through subsequent use and updates.The section assesses validity, reliability, robustness, and truthfulness after changes to h or B, and asks whether revised U produces better later updates.Source: §3.1.1 · p. 16

3.1.2. Claims for Individual Updates, Successors, and Meta-Improvement

  • Individual-update, successor, and meta-improvement claims require distinct evidence about capability gains, retention, and the quality of later updates.Meta-improvement requires comparing revised and unchanged mechanisms under comparable conditions, separating the revision's effects from resources and inherited starting states.Source: §3.1.2 · p. 16

3.2.1. Risks in Improvement Diagnosis (S1)

  • Narrow or unreliable experience and feedback can hide rare failures, misdirect diagnosis, or undermine the explanation for an improvement target.Generated-sample replacement can remove rare patterns, while feedback may become more favorable even as code quality deteriorates.Source: §3.2.1 · pp. 16–17

3.2.2. Risks in Proposal Generation (S2)

  • Search may favor proxy-reward shortcuts, while extra computation can make an unchanged proposal mechanism appear more effective.Selecting the best of more attempts may raise performance without improving proposal quality per attempt.Source: §3.2.2 · pp. 17–18

3.2.3. Risks in Proposal Evaluation (S3)

  • Acceptance scores can rise without capability gains when self-generated rewards share errors or evaluation conditions and standards change.Reliable comparisons require deployment-relevant conditions and stable evaluation standards.Source: §3.2.3 · p. 18

3.2.4. Risks in Update Implementation (S4)

  • Updates can interfere with earlier skills, and useful changes yield little benefit if agents do not invoke or follow them.Compatibility between a policy and its available skills also affects deployment robustness.Source: §3.2.4 · p. 18

3.2.5. Risks under Recursive Improvement (RI)

  • Revising an improvement procedure does not by itself establish greater future-improvement capacity; attribution requires linking revisions to later updates.Comparisons must separate the revision's contribution from additional computation and simultaneous system changes.Source: §3.2.5 · p. 18

3.3. Existing Safeguards and Their Limitations

The safeguards section reviews checks across S1–S4 and recursive improvement, emphasizing that evidence depends on coverage, reliable evaluation, and comparable conditions. It distinguishes practices that preserve evidence or alternatives from tests needed to establish actual gains.

3.3. Existing Safeguards and Their Limitations

  • Safeguards across S1–S4 and recursive improvement address genuine gains, retained capabilities, and revisions to the improvement mechanism.The evidence and conditions needed to establish those outcomes remain central.Source: §3.3 · p. 19

3.3.1. Safeguards for Improvement Diagnosis (S1)

  • Broad experience coverage, preserved records, validated outcomes, and controlled tests of suspected causes strengthen diagnosis, but checks can themselves be unreliable.Retaining real examples alongside generated data helps prevent earlier evidence from being erased; coverage of unseen conditions is not ensured.Source: §3.3.1 · p. 19

3.3.2. Safeguards for Proposal Generation (S2)

  • Diverse candidate retention preserves alternatives to score-favored search, while matched resource budgets and transfer tests help assess proposal quality.Separate evaluation is still needed to establish whether a proposed change improves the intended capability.Source: §3.3.2 · p. 19

3.3.3. Safeguards for Proposal Evaluation (S3)

  • Proposal evaluation uses reserved tasks, verification, calibrated criteria, and explicit acceptance rules, but protections depend on valid tests and stated assumptions.Repeated test use can expose held-out information, and automatically generated tests require independence checks.Source: §3.3.3 · pp. 19–20

3.3.4. Safeguards for Update Implementation (S4)

  • Retention criteria test earlier skills, while regression testing and preserved state support evaluation after reuse or recovery.Protection remains limited to the tested skill set; untested capability interference may go undetected.Source: §3.3.4 · p. 20

3.3.5. Safeguards for Recursive Improvement (RI)

  • Assessing recursive revisions requires comparing revised and unchanged procedures from matched starting states with matched resources and common evaluation.Table 4 uses study-specific horizon units; ✓ marks a revised updater reused later, while – marks artifact reuse under a fixed protocol.Source: §3.3.5 · pp. 20–21, Table 4 · p. 21

3.4. Evidence Synthesis and Research Gaps

The evidence synthesis distinguishes task-level gains from evidence that revised improvement mechanisms contribute to later updates. It identifies limits in study horizons and resource accounting and proposes controlled, longitudinal, independently checked evaluations.

3.4.1. Findings across Updates and Successors

  • Studies report task-level gains, but differ in which components they revise and whether those components are reused to produce later updates.Comparisons that hold the updater or related components fixed provide stronger evidence about a revised mechanism's contribution.Source: §3.4.1 · pp. 21–22
  • Evidence also distinguishes gain quality from persistence, with reversals, interference, transfer, and attempt-budget effects reported across studies.Reported horizons include different units of exposure to change and are not comparable measures of long-term retention.Source: §3.4.1 · p. 22, §3.4.2 · p. 22

3.4.2. Coverage, Limitations, and Research Directions

  • Long-term mechanism claims need repeated successor tests and complete resource accounting, including the training needed to obtain evolver weights.The cited inference-cost comparisons for trained evolver weights omit that training cost.Source: §3.4.2 · p. 22
  • Future evaluations should control starting states and budgets, retest earlier skills and new-task transfer across checkpoints, and independently check complete run histories.Keeping the updater or executor fixed in turn can clarify component contributions in self-editing and joint-training settings.Source: §3.4.2 · p. 22

Takeaway

Trustworthy improvement depends on checks that distinguish genuine gains from inflated scores and test whether gains persist. Future evaluation should track capability persistence and transfer and compare revised with unchanged procedures under matched conditions.

Takeaway

  • Biased experience, shared errors, and mismatched evaluation can inflate scores, while ineffective updates and regressions can prevent gains from persisting.Independent verification, matched-resource comparisons, and retained-capability tests assess whether gains are genuine and attributable.Source: Takeaway · p. 22
  • Future studies should track persistence and transfer across system versions and test revised procedures against unchanged ones under matched conditions.The conclusions depend on the accuracy and coverage of the checks.Source: Takeaway · p. 23

4. Behavioral Constraints, Alignment, and Fairness

The section asks whether behavioral constraints, authorized human goals, and fairness persist through individual updates, successor systems, and revisions to the improvement mechanism. It organizes risks and safeguards across diagnosis, proposal generation, evaluation, implementation, and recursive improvement.

4.1. Definitions and Trustworthiness Claims

  • Behavioral constraints cover permitted, prohibited, and required conduct; alignment concerns authorized goals, while fairness concerns unjustified group differences.Claims require requirement-specific evidence; successor claims also require evidence of inheritance and reassessment, and claims about revised mechanisms require testing after reuse.Source: §4.1.1 · p. 23, §4.1.2 · p. 23, Figure 5 · p. 24

4.2.1. Risks in Improvement Diagnosis (S1)

  • At S1, incomplete rules, skewed feedback, or missing groups can misdirect diagnosis; execution experience may increase unsafe actions, while refusal experience may increase over-refusal.Safeguards trace targets to behavioral rules, affected groups, and feedback sources, and evaluate execution and refusal examples together.Source: §4.2.1 · p. 24, §4.3.1 · p. 26

4.2.2. Risks in Proposal Generation (S2)

  • At S2, proxy optimization and editable components can favor unsafe changes or mask group harms; controls distinguish performance objectives from behavioral and fairness constraints.The section notes that safeguards depend on the edit surface and encoded requirements, and recommends evaluating current and later group outcomes.Source: §4.2.2 · p. 25, §4.3.2 · p. 27

4.2.3. Risks in Proposal Evaluation (S3)

  • At S3, context-sensitive harms, evaluator manipulation, or aggregate scores can make passing tests poor evidence of preserved behavior or fairness.Safeguards use tests independent of search scores, reference points beyond the candidate, and group-level results tied to specified outcomes.Source: §4.2.3 · p. 25, §4.3.3 · p. 27

4.2.4. Risks in Update Implementation (S4)

  • At S4, behavioral, goal-related, and group effects may persist or interact with inherited components, motivating evaluation of the deployed configuration and later outcomes.Studies include task gains paired with downstream safety failures, while generated-data effects and evaluator weaknesses can propagate across rounds.Source: §4.2.4 · pp. 25–26, §4.3.4 · p. 28

4.2.5. Risks under Recursive Improvement (RI)

  • Recursive improvement can alter preferences, search, or acceptance criteria; earlier guarantees for an updater must be checked again after revision and reuse.The section calls for transition and successor tests, independent standards, and distinguishing authorized goal changes from drift; Figure 5 pairs stage risks with safeguards and examines RI authorization.Source: Figure 5 · p. 24, §4.2.5 · p. 26, §4.3.5 · p. 28

4.4.1. Findings across Updates and Successors

  • Studies span different intervention surfaces and horizons; reported findings include contract checks, downstream safety failures, and behavioral repair over limited updates.The section identifies longer observation periods and comparisons across populations and environments as key evidence gaps.Source: §4.4.1 · p. 28, §4.4.2 · p. 29

4.4.2. Coverage, Limitations, and Research Directions

  • The section identifies limited evidence on longer horizons and cross-population or environmental comparisons, and calls for matched resources and independent longitudinal evaluation.Future studies should compare original and revised procedures from matched inherited states, then follow behavior, goal pursuit, and group outcomes across updates.Source: §4.4.2 · p. 29, Takeaway · p. 29

5. System Security, Resilience, Privacy, and Data Governance

The section examines security, resilience, privacy, and data governance across updates, successors, and revised improvement mechanisms. It synthesizes risks and safeguards for compromised experience, derived assets, evaluation evidence, and sensitive information, while identifying limits in recovery and longitudinal assessment.

5.1. Definitions and Trustworthiness Claims

  • Security protects system components and evidence from unauthorized influence; resilience concerns withstanding, containing, and recovering from compromise, while privacy and governance address sensitive information and authorized use.Claims require specified changes and access conditions; successor and meta-improvement claims require evidence of inheritance or mechanism reuse and reassessment.Source: §5.1.1 · p. 30, §5.1.2 · p. 31

5.2.1. Risks in Improvement Diagnosis (S1)

  • At S1, poisoned or poorly warranted experience can persistently influence diagnosis, while privacy failures can occur during acquisition or later disclosure.Safeguards distinguish authority to act from evidential support and assess acquisition, storage, and retrieval against task need and consent.Source: §5.2.1 · p. 31, §5.3.1 · p. 33

5.2.2. Risks in Proposal Generation (S2)

  • At S2, compromised experience can become persistent or combined skills, and derived persona skills can retain sensitive information without copying source records verbatim.Containment, package inspection, joint-use review, and privacy review address distinct risks in generated assets.Source: §5.2.2 · pp. 31–32, §5.3.2 · p. 33

5.2.3. Risks in Proposal Evaluation (S3)

  • At S3, candidates can alter evidence pathways, while repeated evaluation-time queries can expose candidate or memory contents under specified access assumptions.Admission evidence should bind candidate inputs, outputs, evaluator, and decision; extraction tests should state access conditions and report detection limits.Source: §5.2.3 · p. 32, §5.3.3 · pp. 33–34

5.2.4. Risks in Update Implementation (S4)

  • At S4, compromised assets may remain active after source changes, and deletion tests can miss cross-version exposure, relearning, or extraction.Recovery should address descendants and active components, test residual influence and restored benign service, and specify accessible versions and conditions.Source: §5.2.4 · p. 32, §5.3.4 · p. 34

5.2.5. Risks under Recursive Improvement (RI)

  • Under RI, changes to the updater can alter evidence, permissions, checks, or data-use rules, so protections require testing after revised-mechanism reuse.The section recommends external approval authority and renewed checks of changed enforcement boundaries.Source: §5.2.5 · pp. 32–33, §5.3.5 · p. 34

5.4.1. Findings across Updates and Successors

  • In a Hyperagents poisoning experiment, successors produced insecure code on benign held-out tasks, and the harmful instruction persisted after clean-benchmark evolution; penalizing the vulnerability removed it in the tested run.The section recommends separate tests of source removal, inherited-instruction removal, and secure behavior after reuse.Source: §5.4.1 · p. 35

5.4.2. Coverage, Limitations, and Research Directions

  • Evidence remains limited for mechanism revisions and longer-term recovery or deletion across successors; future comparisons should use matched inherited states, budgets, access, and external tests.The section calls for comparing source-only with source-and-descendant cleanup, testing benign-service restoration, and repeating deletion tests after revised-updater reuse.Source: §5.4.1 · p. 35, §5.4.2 · p. 35, Takeaway · p. 36

6. Transparency and Controllability

Effective oversight in RSI requires changes to remain understandable, traceable, attributable, and correctable across individual updates, inherited successor states, and revisions to the improvement mechanism. The survey maps oversight risks and safeguards across S1–S4 and RI, emphasizing that evidence and controls must be retested as systems and mechanisms change.

6.1.2. Claims for Individual Updates, Successors, and Meta-Improvement

  • Oversight claims require evidence matched to the change level: inspect and test an individual update, verify inheritance and use in successors, and retest oversight after a revised mechanism is reused.A claim that oversight improved also requires a controlled comparison with the unchanged mechanism under comparable conditions.Source: §6.1.2 · p. 37

6.2. Risks and Failure Evidence

  • Failures can begin with a mislocalized diagnosis, an unfaithful explanation or feedback target, dependent review evidence, or an intervention that fails to change behavior.Running state can also diverge from the accepted configuration, and recursively revised mechanisms may invalidate earlier oversight evidence or shift control boundaries.Source: §6.2.1 · p. 38, §6.2.2 · p. 38, §6.2.3 · p. 38

6.3.5. Safeguards for Recursive Improvement (RI)

  • Recursive revision can alter protected checks, assumptions, evidence production, or control boundaries, so affected proofs and intervention tests must be revisited after mechanism reuse.Examples include retained code, revised search policies, and revised guidance for generating tasks and rubrics.Source: §6.2.5 · p. 39, §6.3.5 · p. 42

6.3.1. Safeguards for Improvement Diagnosis (S1)

  • Faithfulness tests, matched replay, and provenance records address different diagnosis and proposal risks, while checks of intended and collateral effects track what edits actually do.Checks should be repeated on the state a successor inherits; textual memories, skills, and adapters also require representation-appropriate provenance and use checks.Source: §6.3.1 · pp. 39–40, §6.3.2 · p. 40

6.3.3. Safeguards for Proposal Evaluation (S3)

  • Review safeguards combine evidence access and verification with independent checks and explicit escalation rules, but their value depends on available evidence, feedback, and review resources.An acceptance procedure should assess claimed outcomes as well as reproducibility and be retested when its observation channel changes.Source: §6.3.3 · p. 41

6.3.4. Safeguards for Update Implementation (S4)

  • Enforceable intervention depends on trusted controls, coverage of relevant actions, and sufficient review resources; correction must reach inherited state and its effects must be behaviorally tested.Attribution across versions requires controlled comparisons, while irreversible external effects require separate remedies.Source: §6.3.4 · p. 41

6.4.1. Findings across Updates and Successors

  • Oversight safeguards address distinct dependencies: replay tests repair, explanation tests assess predictions and intervention effects, monitoring depends on observation channels, and intervention depends on trusted controls.Longitudinal assessment should track diagnosis usefulness, correction reach into retained and derived state, review effectiveness, and stopping after mechanism reuse.Source: §6.4.1 · p. 42

6.4.2. Coverage, Limitations, and Research Directions

  • Future evaluation should retest explanation validity and use matched inherited states, tasks, access, budgets, and external criteria to assess review, correction, stopping, and later utility.Investigators should reconstruct outcomes from incomplete or merged histories using revision ablations, interaction tests, intervention evidence, and separately recorded duties and authorization.Source: §6.4.2 · pp. 42–43, Takeaway · p. 43

7. Evaluating Trustworthy RSI

Evaluating trustworthy RSI requires evidence that distinguishes individual updates, inherited successor outcomes, and effects attributable to revised improvement mechanisms. The survey reviews existing evaluation coverage and proposes longitudinal comparisons and evidence conditions for assessing performance, constraints, and oversight together.

7.1. What Do Existing Evaluations Cover?

  • Existing evaluations span update, successor, and meta-improvement claims, but studies differ in whether they assess changes directly or provide component measures and protocols.The evidence map groups studies by primary use and distinguishes discussed cases, unevaluated dimensions, unclear evidence, and supporting evaluation designs.Source: §7.1 · p. 43, Figure 8 · p. 44

7.2. How Should Successive Versions Be Compared?

  • Successive-version comparisons should separate update development, acceptance, and independent reporting; track inherited state across common tasks; and compare revised and fixed updaters from matched starting states.Assess outcomes across task performance, constraint satisfaction, and oversight effectiveness, while accounting for resources and uncertainty.Source: §7.2 · pp. 46–47, Figure 9 · p. 46

7.3. What Supports Trustworthiness Claims?

  • Trustworthiness claims require evidence provenance, valid acceptance procedures, and joint assessment of requirements across the same declared trajectory.A trajectory fails the joint criterion if any mandatory requirement fails at a scheduled assessment, even when its initial and final versions pass.Source: §7.3 · pp. 47–48
  • Reviewed successor and meta-improvement studies provide evidence across multiple concerns, but joint longitudinal evidence for constraints and effective intervention remains limited.The survey calls for matched comparisons of reused improvement procedures and tests of constraints and authorized interventions across successive versions.Source: §7.3 · pp. 48–49

8. Applications, Open Problems, and Conclusions

Trustworthy RSI must be assessed across updates, successors, and revised improvement mechanisms, with evidence matched to changing tasks, feedback, and implementation effects. The survey identifies open questions about preserving gains, constraints, and effective oversight through recursive change.

8.1.1. Established Failure Modes

  • Adaptation can improve targeted tasks while degrading retained capabilities, weaken safety behavior, and pass contamination into successors; self-authored tests may also diverge from protected deployment results.These failures are established in tested settings, but their prevalence, persistence, attenuation, and sensitivity to updater revision remain open questions.Source: §8.1.1 · pp. 49–50

8.1.2. Three Levels of Trustworthiness Claims

  • Trustworthiness claims concern individual updates, successor sequences, or revised updaters, and each requires evidence appropriate to the level assessed.Contamination may persist through evolution, while self-authored tests may favor changes that deteriorate under independent evaluation; gains, constraints, and oversight should be examined across successors.Source: §8.1.2 · p. 50

8.1.3. Evidence across Tasks, Versions, and Use

  • Evidence should match the claim’s task scope, versions, deployment setting, and recovery needs, including reassessment of inherited changes and later candidates from revised updaters.A claim should state its covered update, successors, or updater, its assessment horizon, and untested conditions.Source: §8.1.3 · p. 50

8.2. How Application Context Changes the Evidence Required

  • Application evidence depends on what systems modify and who controls feedback: retained artifacts need checks for later effects, while self-authored or co-evolving tests need independent validation.Changing evaluators can alter standards across versions; protected evidence must cover test data, grading rules, and outcome access.Source: §8.2.1 · p. 51, §8.2.2 · p. 52

8.2.3. Effects of Update Implementation: What Happens after Acceptance?

  • After implementation, persistent state and consequential actions require checks beyond code rollback, including effects on derived artifacts, affected successors, and intended-use conditions.Memories and skills can retain influence after source removal, and privacy assessment should examine permitted use, retention, and disclosure of real user data.Source: §8.2.3 · pp. 52–53, Table 9 · p. 53

8.3. Open Problems and Future Directions

  • Open theory should specify which requirements remain invariant, permissible changes to Ut, and assumptions for guarantees, while verification should test whether revised mechanisms produce better later updates.Paired evaluator versions and protected references can reveal changed acceptance decisions; verification contracts should identify dependencies and trigger checks when affected assumptions change.Source: §8.3.1 · p. 54, §8.3.2 · p. 54

8.3.3. Long-Horizon Risk Dynamics and Adaptive Mitigation

  • Long-horizon studies should track failure recurrence and recovery across reused artifacts, while testing how revised safeguards and updater mechanisms affect protections over later updates.Comparisons can vary clean versus contaminated states, updater revision, and safeguard adaptation, measuring violations, unnecessary refusals, and correction success.Source: §8.3.3 · p. 54, §8.3.4 · pp. 54–55

8.4. Conclusions

  • Trustworthy RSI requires aligned, independent, comparable evidence for sustained gains, applicable constraints, and effective intervention across versions; trade-off and stakeholder studies should test those requirements under evolving acceptance criteria.The survey highlights false-acceptance versus missed-improvement costs, privacy versus provenance in logging, and whether agreed stakeholder constraints persist across versions.Source: §8.3.6 · p. 55, §8.3.7 · p. 55, §8.4 · p. 56

A.1. Subject Coverage and Evidence Boundaries

The appendix describes the survey’s literature scope, selection criteria, and distinctions among evidence types. It also states that findings are reported within their tested settings and assumptions.

A.1. Subject Coverage and Evidence Boundaries

  • The survey covers literature available through 26 October 2026, searched across scholarly records and relevant references using terms spanning RSI, self-evolution, and trustworthiness topics.Titles, abstracts, methods, experiments, and related-work discussions were examined to identify relevant studies and connected research.Source: §A.1 · p. 80
  • Included studies specify an improvement mechanism, measure a trustworthiness outcome, test a safeguard, or provide a relevant evaluation method; duplicates and works without substantive relevance were excluded.Foundational work, standards, related surveys, and system descriptions supply conceptual and technical context.Source: §A.1 · p. 80
  • The survey distinguishes direct recursive-improvement evidence from general self-evolution and related mechanism or evaluation evidence according to component function and observed reuse.Empirical findings retain their tested settings and comparisons, while formal findings retain their stated assumptions.Source: §A.1 · p. 80

A.2. Counting Unit and Source Normalization

The appendix defines how cited works are counted and how publication records are normalized. The survey reports 301 distinct cited works across publication sources spanning 1973–2026.

A.2. Counting Unit and Source Normalization

  • The survey counts 301 distinct cited works, including citations in the main text, tables, captions, and active literature figures, but excluding commented-out references and uncited bibliography entries.Each work is represented by one citation key.Source: §A.2 · p. 80
  • Preprint and archival versions are deduplicated using identifiers, titles, and author lists, with archival publication records taking precedence when available.A cited literature audit counts as one work; studies it analyzes count separately only when independently cited.Source: §A.2 · p. 80
  • Publication sources are grouped by venue conventions, with conference editions combined under parent venues, workshops separate, and journals, preprints, and other sources categorized distinctly.The cited publications span 1973–2026.Source: §A.2 · p. 80

A.3. Distribution by Recorded Publication Source

The appendix reports publication-source distribution for the survey’s 301 cited works. It treats publication source and evidential role as separate classifications.

A.3. Distribution by Recorded Publication Source

  • Table 10 reports the distribution of all 301 cited works by publication source, with percentages rounded independently to two decimal places.The supplied passages do not state the category values.Source: §A.3 · p. 81
  • The table’s scope is publication source rather than evidence type, so its categories should not be read as evidential-role counts.The appendix explicitly reports the two classifications separately.Source: §A.3 · p. 81
  • Each cited work is assigned to one publication-source category.Publication source is reported separately from evidential role.Source: §A.3 · p. 81

A.4. Evidence Selection and Coding Protocol

The protocol compares representative studies across coded trustworthiness dimensions, distinguishes evidence statuses and updater-revision criteria, and documents source checks and joint-assessment requirements.

A.4. Evidence Selection and Coding Protocol

  • Figure 8 compares 20 representative studies across 13 evidence dimensions, covering individual updates, successor inheritance, and improvement-mechanism revisions.Related benchmarks and fixed-model oversight studies contribute evaluation methods.Source: §A.4 · p. 81
  • Cells distinguish examined evidence, substantive discussion or cases, out-of-scope properties, and evidence that remains unclear.Task performance includes benign utility, while intervention covers tested stopping, correction, recovery, or blocking.Source: §A.4 · p. 81
  • Updater edits require retained changes to components governing later proposals, selection, or acceptance, with modification, reuse, and downstream evaluation coded separately.For DGM, the code-writing agent changes while archive selection stays fixed; SAV's updater revision and reuse remain unresolved because retained tests' role in later acceptance is unidentified.Source: §A.4 · p. 81
  • The audit provides source locations and verification status for all 260 coded cells, including 40 targeted primary-source checks and annotated locations for the other 220.A companion record documents functional components, successor evidence, and joint assessment across the three analytical roles.Source: §A.4 · p. 81
  • Joint evidence requires matched system versions, an identifiable improvement trajectory, and compatible tasks, threat models, and evaluation conditions.The record separates properties assessed within a study from those tested together on the same trajectory.Source: §A.4 · p. 81
  • Table 10 reports the publication sources of the 301 references cited in this survey.Source: Table 10 · p. 82

Related papers

Loading 2610.35193v1…