PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence

Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, Hongru Wang, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, Qihan Ren, Cheng Qian, Zhenhailong Wang, Minda Hu, Huazheng Wang, Qingyun Wu, Heng Ji, Mengdi Wang

arXiv:2507.21046v4cs.AI

TL;DR

LLMs remain static in novel, evolving, and interactive settings, motivating agents that can continually adapt. This survey organizes self-evolving agents by what, when, and how they evolve, providing a structured roadmap while noting substantial blind spots in current evaluation practices.

  • Problem

    LLMs remain unable to adapt their internal parameters to novel tasks, evolving knowledge domains, and dynamic interaction contexts.

    Source: §1 · p. 2

  • Method

    The survey synthesizes self-evolving agents through the dimensions of what, when, and how to evolve, covering mechanisms, evaluation, applications, and challenges.

    Source: §1 · p. 3, §9 · p. 53

  • Results

    The survey provides a systematic framework for understanding self-evolving agents and a roadmap for theoretical inquiry and practical deployment.

    Source: §1 · p. 3, §9 · p. 53

  • Takeaways & Limitations

    Self-evolving agents are framed as a basis for more adaptive, robust, and versatile agentic systems in research and real-world deployments.

    Source: §1 · p. 3, §9 · p. 53

    Current evaluation practices leave substantial blind spots and hinder fair comparisons across self-evolving agent methods.

    Source: §7.3 · p. 46

Abstract

from arXiv · show

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks but remain fundamentally static, unable to adapt their internal parameters to novel tasks, evolving knowledge domains, or dynamic interaction contexts. As LLMs are increasingly deployed in open-ended, interactive environments, this static nature has become a critical bottleneck, necessitating agents that can adaptively reason, act, and evolve in real time. This paradigm shift -- from scaling static models to developing self-evolving agents -- has sparked growing interest in architectures and methods enabling continual learning and adaptation from data, interactions, and experiences. This survey provides the first systematic and comprehensive review of self-evolving agents, organizing the field around three foundational dimensions: what, when, and how to evolve. We examine evolutionary mechanisms across agent components (e.g., models, memory, tools, architecture), categorize adaptation methods by stages (e.g., intra-test-time, inter-test-time), and analyze the algorithmic and architectural designs that guide evolutionary adaptation (e.g., scalar rewards, textual feedback, single-agent and multi-agent systems). Additionally, we analyze evaluation metrics and benchmarks tailored for self-evolving agents, highlight applications in domains such as coding, education, and healthcare, and identify critical challenges and research directions in safety, scalability, and co-evolutionary dynamics. By providing a structured framework for understanding and designing self-evolving agents, this survey establishes a roadmap for advancing more adaptive, robust, and versatile agentic systems in both research and real-world deployments, and ultimately sheds light on the realization of Artificial Super Intelligence (ASI) where agents evolve autonomously and perform beyond human-level intelligence across tasks.

1 Introduction

The survey frames self-evolving agents as a shift from static LLMs toward autonomous, continual adaptation, organizing the field around what, when, and how agents evolve. It also reviews evaluation, applications, and open challenges to provide a roadmap for robust and adaptive agentic systems.

Motivation

  • Self-evolving agents address static LLMs’ inability to adapt to novel tasks, evolving knowledge, and dynamic contexts by learning continuously from data, interactions, and experiences.The survey distinguishes self-evolution by autonomous adaptation rather than by using particular algorithms such as supervised fine-tuning or reinforcement learning.Source: §1 · p. 2

Related Work

  • The survey fills a gap left by prior work that largely treated evolution as isolated components or subsidiary parts of broader agent taxonomies rather than as a holistic agent paradigm.It positions itself as the first systematic and comprehensive survey dedicated to self-evolving agents while acknowledging that the field’s conceptual boundaries remain unsettled.Source: §1 · p. 3

Framework

  • The survey’s unified framework organizes self-evolution around what evolves, when adaptation occurs, and how evolution is implemented across agent components, temporal stages, signals, and architectures.It examines models, memory, tools, workflows, and architectures; intra-test-time and inter-test-time learning; textual feedback and scalar rewards; and single-agent and multi-agent evolution.Source: §1 · p. 3

Evaluation

  • It reviews benchmarks and metrics for adaptability, robustness, efficiency, safety, and real-world complexity, emphasizing that evaluation and agent evolution should coevolve.The survey also maps evaluation paradigms spanning static, short-horizon, and long-horizon settings.Source: §1 · p. 3, Figure 3 · p. 5

Applications

  • The survey highlights applications in autonomous software engineering, personalized education, healthcare, and intelligent virtual assistance as practical settings requiring continual adaptation.These applications illustrate the practical potential of self-evolving agents across mission-critical and real-world domains.Source: §1 · p. 3

Challenges

  • It identifies safety, personalization, multi-agent co-evolution, scalability, regulatory considerations, and broader societal implications as critical challenges and research directions.The survey aims to support systematic comparison and design as LLM-based agents enter mission-critical applications.Source: §1 · p. 3

2 Definitions and Foundations

This section formalizes self-evolving agents as systems that persistently adapt through experience and feedback, and positions them as a system-level solution paradigm distinct from problem-setting and targeted-editing approaches. It defines their environment, agent components, evolution objective, inclusion criteria, and progression from proto-evolution toward full autonomy.

2.1 Definitions

  • The formal framework models environments as POMDPs and agent systems as architectures Γ containing models, context, tools, policies, trajectories, and feedback.The agent acts over natural-language and tool spaces, while feedback may be environmental or internal and scalar or textual.Source: §2.1 · pp. 5–6
  • A self-evolving strategy repeatedly transforms agent systems across tasks to maximize cumulative utility derived from rewards, textual evaluations, and performance indicators.The utility function may incorporate completion time, accuracy, or robustness, and evolution is conditioned on each task’s trajectory and feedback.Source: §2.1 · p. 7
  • Self-evolving agents modify parameters, contextual state, tools, or architecture from trajectories or feedback to improve future performance persistently and autonomously.Updates must be experience-dependent, policy-changing, and accompanied by autonomous exploration or self-initiated learning; passive and active evolution are distinguished.Source: §2.1 · p. 7
  • The survey includes methods from proto-evolution through strong self-evolution because fully autonomous evolution without human intervention remains aspirational.This scope captures iterative bootstrapping and feedback-driven prompting alongside autonomous diagnosis and reconfiguration.Source: §2.1 · p. 7

2.2 Relationships with Other Works

  • Self-evolving agents differ from curriculum and lifelong learning by adapting in dynamic environments beyond model parameters, and from model editing by modifying memory, tools, and workflows through interaction-driven strategies.Curriculum learning organizes static data by difficulty, lifelong learning primarily uses training-time memory and externally provided tasks, and model editing applies targeted predefined procedures.Source: §2.2 · pp. 8–9
  • The two-lens framework classifies curriculum and lifelong learning as problem-setting paradigms, while model editing and self-evolving agents are solution paradigms that provide mechanisms for system change.Self-evolving agents generalize targeted updating into a first-class, system-level capability spanning parameters, runtime context, memory, tools, and workflow structures.Source: §2.2 · pp. 8–9

3 What to Evolve?

Self-evolving agents are defined by which internal states they can autonomously and persistently rewrite from trajectories, reflection, and feedback. The survey organizes these evolutionary loci into models, context, tools, and agentic architecture, spanning parameter updates, memory and prompt adaptation, skill development, structural optimization, and multi-agent co-evolution.

3 What to Evolve?

  • Self-evolution targets four loci—models, context, tools, and agentic architecture—whose states can be persistently rewritten through experience-driven feedback.Table 2 positions representative methods across these four evolutionary pillars.Source: §3 · p. 9, Table 2 · p. 10

3.1 Models

  • Models evolve through self-generated supervision, environmental interaction, and execution feedback that autonomously refine parameters and reasoning behavior.Examples include SCA and Self-Rewarding Self-Improving for self-generated data, while SELF, SCoRe, PAG, TextGrad, and AutoRule use execution or preference feedback for updates.Source: §3.1 · pp. 10–11

3.2.1 Memory Evolution

  • Memory evolution converts accumulated episodes into reusable knowledge by selecting, restructuring, summarizing, and distilling experiences into heuristics, workflows, and reasoning strategies.Systems use operations such as ADD, UPDATE, MERGE, and DELETE, while ReasoningBank, Agent Workflow Memory, MUSE, and related methods generalize experience into longer-term competencies.Source: §3.2.1 · pp. 11–12

3.2.2 Prompt Optimization

  • Prompt optimization evolves instructions and whole workflow or multi-agent prompt structures through search, textual feedback, evolutionary mutation, and self-generated preference data without changing model weights.Methods range from APE, ORPO, ProTeGi, and PromptBreeder to DSPy, TextGrad, MASS, MAS-ZERO, EvoAgent, and AgentSquare.Source: §3.2 · p. 11, §3.2.2 · pp. 12–13

3.3 Tools

  • Tool evolution progresses from autonomous discovery and creation to iterative mastery and scalable selection or composition, enabling agents to expand, refine, and manage executable skills.The resulting loop must address code safety through verification and sandboxing, while refinement can use compiler, API, environmental, user, or human-in-the-loop feedback.Source: §3.3 · pp. 13–14

3.4 Architecture

  • Architecture evolution treats internal logic, workflows, memory systems, and collaborative structures as optimizable components, extending self-improvement beyond individual parameters or tools.Single-agent methods optimize nodes, blueprints, source code, or memory architectures, while broader systems co-evolve organization and agent behavior.Source: §3.4 · p. 14, §3.4.1 · pp. 14–15

3.4.1 Single-Agent System Optimization

  • Single-agent optimization jointly searches agent structure and node behavior, including prompts, model assignments, planning, memory, tool use, and self-modifiable code.AgentSquare, EvoFlow, TextGrad, the Darwin Gödel Machine, AlphaEvolve, Gödel Agent, and MemEvolve exemplify this progression.Source: §3.4.1 · pp. 14–15

3.4.2 Multi-Agent System Optimization

  • Multi-agent optimization evolves workflows and internal policies through automated topology search, query-specific generation, lightweight performance prediction, and multi-agent reinforcement learning.ADAS and AFlow established search-based workflow optimization, while ReMA and GiGPO target co-evolving coordination, delegation, and group-level competencies.Source: §3.4.2 · pp. 15–16

4 When to Evolve

The survey distinguishes self-evolution by when adaptation occurs: intra-test-time during task execution and inter-test-time between tasks. These modes share in-context learning, supervised fine-tuning, and reinforcement learning, but differ in data, objectives, and computational constraints.

4 When to Evolve

  • Self-evolution has two temporal modes: intra-test-time adaptation during task execution and inter-test-time learning from completed-task experiences for future performance.Intra-test-time optimization targets the immediate problem and requires rapid feedback processing, whereas inter-test-time optimization targets generalization across task distributions using historical data and allows more sophisticated procedures.Source: §4 · pp. 16–17

4.1 Intra-Test-Time Self-Evolution

  • Intra-test-time agents use feedback, reflection, planning revision, temporary fine-tuning, or targeted reinforcement learning to improve solutions without or with immediate model modification.ICL methods revise behavior through context and episodic reflections; SFT can trigger self-edits or temporary updates; TTRL generates related variants and trains specifically on difficult problem classes.Source: §4.1 · pp. 17–18

4.2 Inter-Test-Time Self-Evolution

  • Inter-test-time agents consolidate rewards, gradients, metrics, and behavioral trajectories after execution to refine policies without real-time task constraints.This retrospective process supports identifying success and failure patterns while improving capabilities for subsequent problem-solving.Source: §4.2 · p. 18
  • Inter-test-time ICL adapts behavior from accumulated observations, actions, workflows, and feedback in context, producing progressive improvement through attention rather than parameter updates.ICRL relies on pre-trained networks implementing implicit reinforcement-learning procedures during forward passes.Source: §4.2 · p. 18
  • Inter-test-time SFT bootstraps training data through self-evaluation, critique, rationalization, refinement, and optional human feedback, reducing reliance on extensive human supervision.STaR and Quiet-STaR augment failed attempts with explanations, while SiriuS performs multi-stage refinement and ARIA incorporates expert guidance.Source: §4.2 · pp. 18–19
  • Inter-test-time RL uses extended interaction and curriculum design to develop robust policies for multi-turn, web-navigation, and device-control tasks before deployment.These methods exploit computational resources for thousands of interactions, adapting task difficulty or combining on-policy exploration, off-policy efficiency, and demonstrations.Source: §4.2 · p. 19

5 How to Evolve

Section 5 frames self-evolution as the ongoing transformation of agent systems through interaction, feedback, reflection, data generation, and strategy adaptation. It organizes methods into reward-based, imitation-based, and population-based paradigms, progressing from individual improvement toward collective intelligence.

Foundations

  • Self-evolution transforms an agent system through continual interaction, feedback, self-reflection, data generation, self-correction, knowledge transfer, and collaboration.Unlike static datasets or one-time fine-tuning, evolution adapts strategies within dynamic environments.Source: §5 · p. 19

Evolutionary paradigms

  • The field progresses through reward-based, imitation-based, and population-based evolution, addressing brittleness, limited exploration, and the need for diversity and coordination.These paradigms trace a trajectory from individual self-improvement toward collective intelligence.Source: §5 · p. 19

Reward-based evolution

  • Reward-based evolution guides iterative self-improvement using textual, implicit, internal, or external feedback signals.Methods vary by feedback type and source, learning paradigm, updated components, and update timing, including ICL, SFT, RL, and evolutionary algorithms.Source: §5 · p. 19, Table 3 · p. 20, Figure 6 · p. 21

Imitation and demonstration learning

  • Imitation and demonstration learning trains agents by mimicking complete, high-quality exemplars generated by humans, themselves, or other agents.This approach is especially effective when demonstrations are abundant or autonomously synthesized, supporting progress in reasoning and multimodal domains.Source: §5 · p. 20

Population-based evolution

  • Population-based evolution maintains agent variants or collaborating agents and uses selection, mutation, crossover, and competition to explore solutions, preserve diversity, and generate novel strategies.These methods draw on biological evolution and collective intelligence while enabling parallel exploration and architectural innovation.Source: §5 · p. 21

5.1 Reward-based Self-Evolution

Reward-based self-evolution organizes feedback into textual, internal-confidence, external, and implicit rewards, with reward design determining learning efficiency and effectiveness. These approaches enable increasingly autonomous self-improvement but remain sensitive to reward design, creating stability and safety trade-offs.

Reward-based self-evolution

  • Reward design guides iterative learning from model outputs and interactions, and the survey categorizes methods by textual, internal, external, and implicit feedback.The reward signal determines the nature, efficiency, and effectiveness of reward-driven evolution.Source: §5.1 · p. 21

Textual Feedback

  • Textual feedback provides interpretable critiques and actionable suggestions, enabling nuanced, flexible, and sample-efficient self-improvement through language-based reflection and correction.Reflexion stores verbal reflections as episodic memory, while AdaPlanner revises plans using in-plan and out-of-plan feedback; Self-Refine and SELF use iterative self-critique without additional supervised data or external reinforcement.Source: §5.1 · p. 21

Internal Rewards

  • Internal-confidence rewards use probability estimates or certainty to self-evaluate responses, filter high-quality candidates, improve efficiency, and reduce overconfidence bias.CISC weights reasoning paths by confidence, while Self-Ensemble groups choices before aggregating predictions to mitigate confidence distortion.Source: §5.1 · pp. 21–22

External Rewards

  • External rewards derive objective supervision from majority votes, environment and tool feedback, or explicit rules, but may require substantial engineering or lack expressiveness.Rule-based signals are particularly effective for mathematical reasoning, game play, and structured problem solving, whereas environment feedback supports learning through direct interaction.Source: §5.1 · p. 22

Implicit Rewards

  • Implicit rewards let models learn from unlabeled contextual signals or reward-like information encoded by language-model training objectives, without explicit reinforcement fine-tuning or additional human effort.Examples include in-context learning from scalar signals, extracting endogenous rewards from logits, and PIT’s preference-based improvement objective.Source: §5.1 · p. 22

Reward-based self-evolution

  • Reward-based evolution offers explicit optimization and strong autonomy but remains sensitive to reward design, trading stability and safety for adaptability and openness.This limitation applies across the reviewed reward mechanisms.Source: §5.1 · p. 22

5.2 Imitation and Demonstration Learning

Imitation and demonstration learning evolves agents by transferring high-quality exemplar behavior rather than relying on evaluative rewards or population selection. Demonstrations may be self-generated, shared across agents, or hybrid, but effectiveness depends on their quality and diversity.

5.2 Imitation and Demonstration Learning

  • Imitation learning directly guides individual-agent improvement from complete successful exemplars, including trajectories generated by the agent, other agents, or environmental interaction.Unlike reward-based methods, it provides prescriptive behavioral guides rather than scalar or textual feedback requiring trial-and-error; unlike population methods, it transfers knowledge directly from exemplars.Source: §5.2 · pp. 22–23

5.2.1 Self-Generated Demonstration Learning

  • Self-generated demonstration learning iteratively generates, selects, and trains on the agent’s own outputs to bootstrap reasoning, including verifier-guided, adaptive, web-grounded, and multimodal self-training.Zelikman et al. (2022) introduced iterative self-training without groundtruth reasoning paths; later methods improve reliability, task coverage, and visual-textual learning.Source: §5.2.1 · p. 23

5.2.2 Cross-Agent Demonstration Learning

  • Cross-agent demonstration learning transfers successful trajectories across agents through shared experience libraries, enabling collaborative improvement, specialization, diverse strategies, and domain-specific personalization.Zhao et al. (2025b) describes collective experience sharing, while Tang et al. (2025) uses successful recommendation interactions to fine-tune personalized recommendations.Source: §5.2.2 · pp. 23–24

5.2.3 Hybrid Demonstration Learning

  • Hybrid demonstration learning combines self-generated and external exemplars through recursive self-correction and confidence-guided filtering, but remains constrained by demonstration quality, diversity, and availability.Qu et al. (2024b) generates corrective demonstrations through reflection, while Li et al. (2024b) filters low-confidence examples; high-quality exemplars improve reliability and sample efficiency but can limit exploration and generalization.Source: §5.2.3 · p. 24

5.3 Population-based and Evolutionary Methods

Population-based and evolutionary methods evolve multiple agent variants through selection, mutation, crossover, competition, and self-play, enabling broad search and architectural change beyond parameter fine-tuning. The section distinguishes single-agent evolution from multi-agent evolution of coordination, architecture, and collective knowledge, while noting higher computational cost and lower interpretability.

5.3 Population-based and Evolutionary Methods

  • Population-based evolution maintains multiple variants, enabling parallel exploration, diverse capabilities, and discovery of solutions in complex or multimodal spaces requiring architectural changes.These methods complement reward-based optimization of individual agents and imitation learning from demonstrations through genetic variation and competitive interaction.Source: §5.3 · p. 25

5.3.1 Single Agent Evolution

  • Single-agent evolution divides into learning from evolution and self-play, improving agents through population mechanisms, self-modification, or interaction with stronger versions of themselves.Self-play creates an adaptive curriculum and learning signals from dynamic interaction, while evolutionary approaches can modify code, parameters, prompts, memory, tools, or model combinations.Source: §5.3.1 · pp. 25–26
  • Self-play evolves capabilities through competing past versions or interacting roles such as challengers and solvers, generating difficult tasks and continuous improvement without external labels.Variants include teacher-generator curricula, human-guided challengers to prevent concept drift, adversarial critics, and specialized solver-verifier roles.Source: §5.3.1 · p. 26

5.3.2 Multi-Agent Evolution

  • Multi-agent evolution optimizes teams and networks through system architecture evolution or knowledge-based evolution, targeting coordination, workflow, collective behavior, and accumulated experience.Architectural methods alter team composition, prompts, orchestration, or coordination policies, whereas knowledge-based methods retrieve successful cases, failure reflections, and prior trajectories.Source: §5.3.2 · pp. 26–27
  • Knowledge-based multi-agent evolution improves teams through structured memory and retrieval rather than parameter updates, accumulating successes, failures, patient interactions, or scientific principle-outcome pairs.Examples include CorrectKB and ChainKB for medical consultation, retrieval-augmented diagnostic improvement, and trajectory-guided hypothesis generation.Source: §5.3.2 · p. 27
  • Population-based and self-play methods enhance diversity and open-ended discovery but generally require more computation and provide less interpretability than single-agent paradigms.The tradeoff follows from maintaining populations or evolving interactions rather than optimizing a single agent directly.Source: §5.3.2 · p. 27

5.4 Cross-cutting Evolutionary Dimensions

This section organizes self-evolution methods around learning paradigm, policy consistency, and reward granularity, while comparing feedback, data sources, efficiency, stability, and scalability trade-offs. These dimensions distinguish offline versus online adaptation, on-policy versus off-policy experience use, and outcome-based, process-based, or hybrid rewards.

5.4 Cross-cutting Evolutionary Dimensions

  • Self-evolution methods are compared across learning paradigm, policy consistency, reward granularity, feedback type, data source, sample efficiency, stability, and scalability.The major families are reward-based, imitation/demonstration-based, and population-based methods.Source: §5.4 · p. 27

5.4.1 Online and Offline Learning

  • Offline learning uses pre-collected experiences for filtering, fine-tuning, or reinforcement learning, whereas online learning adapts continuously through live environmental interaction.Offline examples include Self-Instruct, WizardLM, OS-Genesis, UI-Genie, GUI-R1, and InfiGUI-R1; online examples include Voyager, AdaPlanner, SwiftSage, DigiRL, DistRL, and MobileGUI-RL.Source: §5.4.1 · pp. 27–29, Figure 7 · p. 28

5.4.2 On-policy and Off-policy Learning

  • On-policy methods preserve policy consistency and learning stability but require costly fresh data, while off-policy methods improve sample efficiency by reusing diverse historical, expert, or cross-agent experiences.Off-policy learning remains vulnerable to distribution shift, reward hacking, and instability unless controlled through mechanisms such as KL regularization or ranking-based supervision.Source: Figure 7 · p. 28, §5.4.2 · pp. 29–30

5.4.3 Reward Granularity

  • Outcome-based rewards evaluate only final task completion, offering simple supervision but sparse feedback that complicates learning from frequent failures in dynamic web and GUI environments.DPO, RRHF, AutoWebGLM, DigiRL, and WebRL address outcome-based learning through preference optimization, ranking, successful-trajectory selection, sparse final evaluation, or outcome reward modeling.Source: §5.4.3 · p. 30
  • Process-based rewards provide step-level guidance and can be more reliable for complex reasoning, but traditionally require expensive human annotations.Math-Shepherd, AlphaMath, rStar-Math, AgentPRM, and Agent Q reduce this bottleneck through automated search, value modeling, verification, and iterative policy refinement.Source: §5.4.3 · pp. 30–31
  • Hybrid rewards combine final success signals with intermediate progress feedback to reduce outcome sparsity while keeping stepwise reasoning aligned with task goals.GiGPO uses episode-level and localized step-level rewards, while SPA-RL distributes final-task progress across trajectory steps.Source: Figure 7 · p. 28, §5.4.3 · p. 31

5.5 Other Dimensions of Self-Evolution Methods

Self-evolution methods differ across feedback type, data source, sample efficiency, stability, and scalability. Together, these dimensions guide the selection and design of domain-appropriate agent learning pipelines.

Feedback Type

  • Feedback determines the information used for improvement: rewards use scalar, language, or confidence signals; imitation uses demonstrations and rationales; population methods use fitness or competition.Source: §5.5 · p. 31

Data Source

  • Data sources differ across paradigms: interaction or engineered rules support reward learning, expert demonstrations support imitation, and collective multi-agent or generational experience supports population methods.Population approaches enable diverse exploration but require significant coordination.Source: §5.5 · p. 31

Sample Efficiency

  • Imitation learning is generally most sample-efficient with high-quality demonstrations, reward-based methods are moderately efficient but reward-sparsity-sensitive, and population evolution is typically sample-inefficient.Source: §5.5 · p. 31

Stability

  • Stability risks arise from reward hacking, limited demonstration quality or diversity, and premature convergence in small or homogeneous populations.Reward-based methods depend on reward design, imitation on demonstrations, and population methods on population size and diversity.Source: §5.5 · p. 31

Scalability

  • Scalability depends on feedback collection and parallelization: automated reward feedback scales well, demonstrations create imitation bottlenecks, and population methods scale computationally but remain resource-intensive.Source: §5.5 · p. 31

6 Where to Evolve?

Self-evolving agents evolve either broadly across tasks as general-purpose assistants or deeply within specialized domains. General evolution relies on memory, curriculum-driven training, and model-agent co-evolution, while specialized applications span coding, GUI, finance, medicine, education, and other complex environments.

6 Where to Evolve?

  • Self-evolving agents divide into general domain evolution, which broadens capabilities across tasks, and specialized domain evolution, which deepens expertise within particular domains.General-purpose assistants transfer learned experience to broader tasks, whereas specialized agents target narrower task sets.Source: Figure 8 · p. 32, §6 · p. 32

6.1 General Domain Evolution

  • General-purpose agents continuously adapt through memory optimization, curriculum-driven training, and model-agent co-evolution to address increasingly complex user demands.These mechanisms support versatile assistants operating in dynamic and diverse digital environments.Source: §6.1 · p. 32
  • Memory mechanisms distill past successes and failures into reusable representations, enabling agents to abstract experience and refine behavior on unseen tasks.Examples include Mobile-Agent-E’s Tips and Shortcuts, MobileSteward’s cross-app execution memories, and Generative Agents’ episodic reflection.Source: §6.1 · p. 32
  • Model-agent co-evolution jointly improves agents and auxiliary reward or world models, while curriculum-driven training generates progressively suitable tasks from failures and agent progress.UI-Genie, WebE-volver, Absolute Zero, WebRL, and Voyager illustrate synthetic data, self-play, adaptive policy updates, and iterative task expansion.Source: §6.1 · pp. 32–33

6.2 Specialized Domain Evolution

  • Self-evolving coding agents improve software development through autonomous code editing, optimized multi-agent workflows, iterative testing feedback, and difficulty-stratified experience selection.SICA, EvoMAC, AgentCoder, and Zhang et al. (2025b) demonstrate iterative self-improvement across coding tasks.Source: §6.2 · p. 33
  • GUI agents evolve grounding, memories, and skills through self-reinforcement, failure replay, reflection, self-fine-tuning, interface mining, and hierarchical monitoring.Navi doubled task completion across 150 Windows challenges, while WebVoyager increased unseen-site success from 30 % to 59 % and ReAP recovered a further 29-percentage-point margin on previously failed queries.Source: §6.2 · pp. 33–34
  • Financial and medical agents build domain expertise through environmental feedback, reflection, reinforcement learning, simulation, reusable trajectories, evolving workflows, and tool or template expansion.These mechanisms improve trading decisions, diagnostic strategies, consultation success, multimodal diagnosis, and biomedical analytical processes without always requiring manual labels.Source: §6.2 · p. 34
  • Education and other domains use personalization, self-play, adversarial refinement, historical interactions, automatic curricula, and strategic self-improvement to support adaptive learning and complex task performance.Applications include tutoring, teacher development, lesson planning, academic assistance, Minecraft, strategic games, and diplomacy.Source: §6.2 · pp. 34–35

7 Evaluation of Self-evolving Agents

Evaluating self-evolving agents requires moving beyond fixed-task, single-shot assessment toward longitudinal, cost-aware evaluation of adaptation and improvement in dynamic environments.

Evaluation challenges

  • Evaluation must measure adaptation over time, knowledge accumulation and retention, long-term generalization, and transfer across sequential or novel tasks while mitigating catastrophic forgetting.These requirements reflect agents’ continuous learning and interaction with changing environments, rather than performance at one fixed point in time.Source: §7 · p. 35

7.1 Evaluation Goals, Metrics, and Benchmark Coverage

The survey evaluates self-evolving agents across adaptivity, retention, generalization, efficiency, and safety, while linking these goals to static, short-horizon, and lifelong evaluation paradigms. It also emphasizes trajectory-centric assessment of autonomous evolution, whose performance gains must be interpreted alongside safety risks and external guidance.

7.1 Evaluation Goals, Metrics, and Benchmark Coverage

  • Evaluation is organized around five dimensions—adaptivity, retention, generalization, efficiency, and safety—and spans static capability tests, short-horizon adaptation, and lifelong learning.The framework connects each goal to corresponding metrics, benchmarks, and coverage gaps, motivating trajectory-centric and cost-aware assessment.Source: §7.1 · pp. 35–36, Figure 9 · p. 36

7.1.1 Adaptivity

  • Adaptivity measures performance improvement over interaction steps or iterations, but existing benchmarks use narrow task distributions and preset learning protocols rather than autonomous strategy discovery.Coverage includes coding, web navigation, and reasoning benchmarks, while AgentBench reports weaknesses in sustained reasoning and strategic decision-making.Source: §7.1.1 · p. 36

7.1.2 Retention

  • Retention evaluates catastrophic forgetting through Forgetting and Backward Transfer, but episodic benchmarks that reset agent state largely preclude measuring knowledge accumulation or degradation.Replay buffers can improve learning yet degrade performance beyond optimal sizes because of context overflow and resource exhaustion.Source: §7.1.2 · p. 37

7.1.3 Generalization

  • Generalization measures transfer across domains and held-out tasks, but static snapshots do not reveal whether breadth persists during specialization or after prolonged learning.Older tasks and data contamination further reduce discriminative power, while no benchmark tests transfer after hundreds of learning episodes.Source: §7.1.3 · p. 37

7.1.4 Efficiency

  • Efficiency tracks tokens, time, interaction steps, tool calls, memory growth, and human oversight, with lower Cost-per-Gain indicating more efficient learning.Reporting remains sparse and rarely isolates evolution-specific costs, while standard protocols generally omit real-world token, iteration, and latency constraints.Source: §7.1.4 · pp. 38–39

7.1.5 Safety

  • Safety evaluation measures safe behavior, harm, policy adherence, violations, refusals, and information leakage, yet benchmarks rarely track risks across extended evolution.Agent-SafetyBench identifies tool-robustness and hazard-recognition deficiencies, while SwarmBench exposes coordination and shared-situational-awareness problems in multi-agent settings.Source: §7.1.5 · p. 39

7.1.6 Self-Directedness and Evaluation Trade-offs

  • Autonomous curriculum generation improved WebRL from 4.8% to 42.4% and SEAgent from 11.3% to 34.5%, but alignment-faking rates rose from 12% to 78%.The field therefore recommends reporting whether tasks and strategies are predetermined or autonomous, feedback sources, and external-intervention frequency.Source: §7.1.6 · p. 39

7.2 Evaluation Paradigm

The evaluation paradigm spans static, short-horizon adaptive, and long-horizon lifelong-learning assessments, tracing self-evolution across interaction episodes and changing environments. Standardized protocols further make trajectory-centric evaluation reproducible through persistence assumptions, evolution budgets, logging, and metrics.

7.2 Evaluation Paradigm

  • Evaluation is organized by temporal scope into static assessment, short-horizon adaptation, and long-horizon lifelong-learning ability, covering instantaneous performance through extended learning trajectories.Here, continuous evolution refers to system-level changes in policies, memories, skills, tools, or procedures across episodes, not necessarily parameter updates.Source: §7.2 · p. 40

7.2.1 Static Assessment

  • Static assessment provides instantaneous baseline comparisons on fixed tasks but cannot capture dynamic learning, long-term evolution, or continuous knowledge retention.It uses metrics such as accuracy, success rate, progress rate, completion rate, and domain-specific indicators to summarize isolated invocations or fixed task sets.Source: §7.2.1 · pp. 40–42

7.2.2 Short-Horizon Adaptive Assessment

  • Short-horizon assessment measures immediate adaptation through repeated attempts, new instances, temporal extensions of traditional benchmarks, or benchmarks with built-in test-time learning.Success-by-iteration curves, learning-curve analysis, and adaptation speed quantify improvement over limited interactions, but cannot establish long-term retention or lifelong learning.Source: §7.2.2 · pp. 43–44

7.2.3 Long-Horizon Lifelong Learning Ability Assessment

  • Long-horizon assessment evaluates continuous acquisition, retention, reuse, transfer, and efficiency across diverse task streams while addressing forgetting, resource management, data leakage, and evolving environments.Emerging benchmarks include LTMBenchmark, LifelongAgentBench, and AutoEnv; dynamic benchmarks can evolve datasets or task difficulty, with preliminary evidence that performance may degrade as benchmarks evolve.Source: §7.2.3 · p. 44

7.2.4 Standardized Evaluation Protocols

  • Standardized protocols separate short- and long-horizon persistence assumptions, evolution budgets, logging requirements, and metrics, while EvoAgent exposes remaining gaps in retention, cost-per-gain, token drift, and safety-drift reporting.Short-horizon protocols require per-iteration traces and success-by-iteration curves; long-horizon protocols require persistent checkpoints, retention probes, replayable trajectories, and evolving-behavior safety indicators.Source: §7.2.4 · pp. 45–46

7.3 Limitations of Current Evaluation Practices

Current evaluation practices leave major capability blind spots and do not support reliable apples-to-apples comparisons across self-evolving agents. Missing intersection benchmarks, inconsistent experimental conditions, and incomplete reporting limit synthesis and motivate standardized evaluation.

7.3 Limitations of Current Evaluation Practices

  • Evaluation practices leave substantial blind spots and hinder fair comparisons across self-evolving agents.The limitations concern both under-evaluated capability dimensions and factors complicating comparisons under shared settings.Source: §7.3 · p. 46

7.3.1 Underserved Capability Intersections

  • Key capability intersections remain untested, including long-horizon personalized retention with privacy guarantees, architecture adaptation under millisecond latency and token budgets, self-directed tool lifecycle evolution, and multi-agent safety during extended co-evolution.Existing evaluations do not establish whether agents can retain personalization without sensitive-data leakage, improve architecture-selection strategies under hard resource limits, discover and integrate tools, or experience social contagion of unsafe behaviors.Source: §7.3.1 · p. 46

7.3.2 Challenges for Fair Comparison

  • True apples-to-apples comparison remains infeasible because reporting practices, evaluation pipelines, backbone models, and architectural designs vary substantially and interact with one another.Table 11 is therefore an illustrative snapshot rather than a definitive comparison; latency, cost, and safety metrics are inconsistently reported.Source: §7.3.2 · p. 46, Table 11 · p. 47
  • Richer what structures and inter-test-time mechanisms often achieve stronger performance on feasible multi-step optimization tasks, whereas lightweight intra-test-time reflection is more cost-efficient but yields smaller gains.The observed qualitative trends, together with inconsistent latency, cost, and safety reporting, underscore the need for standardized reporting.Source: §7.3.2 · pp. 46–47

8 Future Direction

Future directions center on personalizing agents responsibly, improving generalization, controlling risks introduced by self-evolution, and enabling adaptive multi-agent ecosystems. The survey emphasizes dynamic evaluation, privacy and fairness governance, knowledge transfer, and safety lifecycle mechanisms as priorities for reliable deployment.

8.1 Personalize AI Agents

  • Personalized agents must overcome cold starts while managing long-term memory, tools, privacy, fairness, and evolving user preferences through adaptive evaluations and governance.Suggested measures include Personal Adaptation Gain, retention–forgetting balance, privacy–utility trade-offs, on-device learning, bias/drift monitoring, and longitudinal outcomes.Source: §8.1 · pp. 48–49
  • Safe deployment requires continuous, lightweight evaluation of personalization that captures adaptation, forgetting, privacy, on-device learning, bias drift, safety drift, and longitudinal user outcomes.Traditional intrinsic and extrinsic metrics do not adequately represent evolving personalization or long-tailed user data.Source: §8.1 · pp. 48–49

8.2 Generalization

  • Generalization remains limited by specialization, nonlinear adaptation costs, domain-specific tuning, catastrophic forgetting, model drift, and weak transfer of knowledge between agents.Promising directions include reflective memory, test-time scaling, inference-time adaptation, meta-learning, parameter-efficient tuning, selective memory, and robust world models.Source: §8.2 · pp. 49–50

8.3.1 Emergent Risks in Self-Evolving Systems

  • Self-evolution creates emergent risks including value drift, misevolution, reward hacking, alignment tipping, unsafe memories, vulnerable tools, privacy leakage, and offensive cyber-capabilities.These risks can arise across model, memory, and tool evolution rather than being present in the initial agent.Source: §8.3.1 · pp. 50–51

8.3.2 Prescriptive Guardrails and Mitigation Strategies

  • A comprehensive safety-aware evolutionary lifecycle should combine verification, continuous monitoring, auditability, rollback, sandboxing, and adaptive guardrails to preserve alignment as agents become more autonomous.The proposed approach includes a deployment compliance checklist and safeguards spanning tool execution, self-modification, long-horizon drift, and high-stakes actions.Source: §8.3.2 · p. 51

8.4 Ecosystems of Multi-Agents

  • Multi-agent systems need adaptive individual–collective reasoning, explicit knowledge updating, and dynamic benchmarks that measure continuous role evolution, collaboration, and diverse contributions.Static benchmarks and unclear knowledge-management mechanisms currently fail to capture long-term adaptation and evolving interactions.Source: §8.4 · pp. 51–53

9 Conclusion

The survey frames self-evolving agents as dynamic systems that continually adapt beyond static models, organizing the field around what, when, and how agents evolve. It also synthesizes evaluation, applications, and future challenges needed to build adaptive, trustworthy, and aligned agents.

Conclusion

  • The survey systematically reviews self-evolving agents through three questions: what should evolve, when evolution should occur, and how evolutionary processes should be implemented.It aims to provide a foundational framework for designing, analyzing, and advancing agents capable of continual learning and adaptation.Source: §9 · p. 53
  • Self-evolving agents move beyond static, monolithic models by adapting reasoning, tools, and behaviors to new tasks, knowledge, and feedback in open-ended environments.The survey identifies continual learning and adaptation as essential for the next generation of agentic systems.Source: §9 · p. 53
  • The survey covers evaluation metrics and benchmarks, applications, and future directions for assessing and advancing self-evolving agents.Progress will require advances in models, data, algorithms, and evaluation practices.Source: §9 · p. 53
  • Addressing catastrophic forgetting, human-preference alignment during autonomous evolution, and agent–environment co-evolution is central to making these systems adaptive, trustworthy, and aligned with human values.These challenges are presented as key to unlocking reliable self-evolving agents.Source: §9 · p. 53

Related papers

Loading 2507.21046v4…