Source-linked AI summary
Self-Evolving Coding Agents
Hao Zhou, Haichuan Hu, Ye Shang, Quanjun Zhang
TL;DR
Static coding agents struggle to adapt to changing repositories and feedback-rich development settings. This survey defines self-evolving coding agents, organizes them by what, when, and how they evolve, and finds that executable feedback and repository context enable adaptation while raising reliability, safety, cost, and generalization challenges.
Problem
Most coding agents remain largely fixed after deployment despite evolving codebases, changing dependencies, repeated repair cycles, and reusable software-specific feedback.
Method
The survey develops a layered synthesis with an object-centered taxonomy and orthogonal dimensions for evolution timing and software-specific evidence.
Results
The literature indicates that executable feedback, repository context, and coding trajectories support agent evolution but create challenges in feedback reliability, safety, cost, maintainability, and generalization.
Takeaways & Limitations
Self-evolving coding agents are best understood as a family of adaptation processes grounded in software artifacts and repository-level context rather than a single algorithmic technique.
Takeaways & Limitations
Current evaluations emphasize functional correctness and benchmark success more strongly than long-term maintainability, robustness, safety, and learning from incomplete or misleading feedback.
Abstract
from arXiv · showhide
Large language models are increasingly embedded in software engineering workflows as coding agents that can inspect repositories, invoke tools, execute tests, debug failures, and generate patches. Yet most existing agents remain largely static after deployment, even though software development is a dynamic, feedback-rich process in which repositories evolve, dependencies change, tests fail, and repair attempts leave reusable experience. This tension has motivated a growing body of work on self-evolving coding agents, where the agent improves its future behavior by updating its framework, memory, skills, tools, models, or collaboration structures from prior coding interactions. In this survey, we provide a systematic synthesis of this emerging area. We first define self-evolving coding agents and distinguish them from conventional coding agents and general self-evolving agents. We then develop an object-centered taxonomy that characterizes what evolves in these systems, and complement it with two orthogonal perspectives: when evolution occurs and what software-specific evidence drives it. Across the literature, we find that executable feedback, repository-level context, and coding trajectories give software engineering a distinctive role as a natural domain for agent self-evolution, but also introduce new challenges in feedback reliability, benchmark overfitting, safety, maintainability, cost, and generalization. By organizing existing work around these dimensions, this survey aims to clarify the conceptual boundaries of self-evolving coding agents and provide a foundation for designing more adaptive, reliable, and software-aware agentic systems. The papers we collect can be found at https://github.com/zhouhao1024/Awesome-Self-Evolving-Coding-Agents.
1 Introduction
The survey argues that static coding agents cannot adequately adapt to evolving repositories and software-specific feedback, motivating a structured study of self-evolving coding agents. It organizes the area by what evolves, when evolution occurs, and which software evidence drives it.
- Motivation: Interactive coding agents now interpret requirements, inspect repositories, edit multiple files, invoke tools, run tests, diagnose failures, and generate patches.
- Motivation: Static designs leave models, prompts, tools, memory, and control flow largely fixed despite evolving codebases, dependencies, APIs, and project conventions.
- Motivation: Executable software feedback and repeated repair cycles provide experience that agents can accumulate to avoid repeated mistakes and adapt to project-specific contexts.Examples include unit tests, compiler errors, runtime traces, lint warnings, continuous integration results, and human code reviews.
- Scope: The survey treats self-evolving coding agents as distinct from general evolving agents because they operate in repository-centered environments with software-specific interactions.
- Evaluation: The survey also asks how self-evolving coding agents should be evaluated for software engineering performance, reliability, and generalization beyond the evolved setting.
- Survey framework: Its taxonomy organizes evolution by agent frameworks, memory, skills and tools, models, and workflow or topology structures, complemented by timing and software-specific evidence.
2 Background and Definitions
Coding agents are tool-mediated systems that operate within software engineering environments through iterative interaction with repositories, tools, tests, and developers. Self-evolving coding agents extend this setup by adapting their behavior or internal components from execution trajectories and software-specific feedback.
- Coding agents: Coding agents shift software development from isolated code generation to environment-grounded action involving repository inspection, tool use, file editing, command execution, feedback, and iterative revision.They are studied across repository-level generation, code review, interactive debugging, and repository understanding.
- Coding agents: A coding agent couples a language model with context, tools, control logic, and verification, while its quality depends on the interaction loop rather than architectural complexity alone.The controller selects actions, context maintains repository state and task history, and tools support search, editing, testing, debugging, and dependency management.
- Self-evolving agents: Self-evolution is feedback-driven adaptation that changes an agent’s behavior or internal components over time, rather than repeated prompting, one-time optimization, or necessarily updating model parameters.The improvement locus moves from externally engineered updates to adaptation within the agent system itself.
- Self-evolving coding agents: Self-evolving coding agents turn software-engineering interactions into sources of persistent adaptation, distinguishing them from conventional coding agents and general self-evolving agents applied to code.Conventional coding agents act in software environments; self-evolving coding agents additionally use those interactions for persistent adaptation.
- Self-evolving coding agents: Executable software artifacts provide concrete and repeatable evolution signals, including unit tests, compiler diagnostics, runtime traces, static-analysis warnings, repository histories, continuous-integration logs, and code reviews.These signals distinguish the software-engineering loop from feedback based mainly on textual critiques, user preferences, or scalar rewards.
3 Taxonomy of self-evolving coding agents
The taxonomy classifies self-evolving coding agents by the primary artifact updated, including frameworks, memories, skills and tools, and workflows or topologies. These categories differ in how they transform software-specific experience into future behavior and in the reliability risks their adaptations create.
- Framework self-evolution: Framework self-evolution modifies the scaffold that orchestrates model calls, repository inspection, editing, commands, testing, tools, and control flow.Scaffold rewriting can let agents discover prompting schemes or tools and validate modified agents on coding benchmarks.
- Framework self-evolution: Framework evolution is especially risky because harmful modifications can break agent loops, degrade tool use, overfit benchmark feedback, or exploit evaluation weaknesses.The category therefore requires validation, rollback, and robustness checks in addition to performance-driven search.
- Memory self-evolution: Memory self-evolution continually constructs, refines, and reuses explicit records of issue-resolution trajectories, patches, tests, diagnostics, and repository history.Repository-centered memory uses historical commits, linked issues, and summaries of frequently modified regions to support future code localization.
- Memory self-evolution: Unlike generic experience storage, coding-agent memory is coupled to executable and repository-level evidence, including tests, compiler or runtime errors, and commit histories.Selective evolution is essential because noisy logs, misleading tests, brittle patches, and repository conventions can create harmful memories without filtering, abstraction, retrieval, and validation.
- Skill and tool self-evolution: Skill and tool self-evolution converts trajectories, repository conventions, execution failures, and repair patterns into reusable procedures and project-specific tools.Agents can synthesize editors, code-search utilities, and domain-specific analyzers during issue solving, then evaluate them through repository inspection, editing, execution, and testing.
- Workflow and topology self-evolution: Workflow and topology self-evolution changes global decisions about search, debugging, patch generation, test-failure routing, review, and communication costs.Its power is accompanied by risks including benchmark overfitting and unnecessary coordination overhead.
4 Evolving Time and Evidence
Self-evolution in coding agents varies by when adaptation occurs—during a task, after it, or across accumulated interactions—and by the software-specific evidence informing the update. These dimensions range from immediate environmental feedback to persistent experience and stage-wise policy improvement supported by executable tasks and verifiers.
- Evolving Time: The survey distinguishes task-time, post-task, and stage-wise evolution by when agents update behavior, components, or organization.Task-time updates occur during the current task, post-task updates reinterpret completed trajectories, and stage-wise updates follow accumulated interactions.
- Task-time evolution: Task-time evolution uses intermediate software feedback to revise ongoing patches, tool use, workflows, or collaboration structures before task completion.Relevant signals include failed tests, compiler diagnostics, runtime traces, tool errors, and unproductive repository searches.
- Post-task evolution: Post-task evolution converts completed issue-solving or repair trajectories into persistent experience, repository knowledge, repair heuristics, or reusable coding skills.Past work can support later tasks through issue-resolution memory, repository-specific knowledge, and vulnerability-repair experience.
- Stage-wise evolution: Stage-wise evolution aggregates software trajectories, executable outcomes, verifier judgments, or generated repair tasks to shape later agent policies.Self-play SWE-RL couples bug generation, bug solving, and executable verification, while SWE-Gym and R2E-Gym provide infrastructure for executable tasks, trajectories, and verifier signals.
- Evolving Evidence: The survey organizes evolutionary evidence into outcome evidence, environmental feedback, and trajectory-derived evidence, spanning coarse performance selection, local interaction signals, and complete coding records.Trajectory-derived evidence can be abstracted into reusable memories, summaries, skills, or skill registries, but is harder to process than outcome evidence and environmental feedback.
5 Benchmarks and Evaluation
Evaluation in self-evolving coding agents both measures performance and supplies evidence for future adaptation. Repository-level issue resolution is central, while evaluation increasingly extends beyond final outcomes to evolution process, efficiency, and generalization.
- Evaluation’s dual role: Evaluation results, failed tests, verifier judgments, and costly trajectories can guide retention, reuse, workflow revision, or model-component updates.This gives evaluation a dual role: measuring agents and supplying evidence for self-evolution.
- Repository-level issue resolution: Repository-level issue resolution is the central setting, requiring agents to understand issues, inspect projects, edit code, run tests, and revise patches using executable feedback.SWE-bench introduced this setting through real GitHub issues.
- Function-level and competition-style programming: Function-level and competition-style benchmarks evaluate complementary capabilities, from Python synthesis and short entry-level tasks to more difficult competitive programming.HumanEval and MBPP emphasize functional correctness and short tasks, while APPS and CodeContests target harder competitive-programming settings.
- Outcome-oriented metrics: Common outcome metrics include pass rate, solve rate, resolve rate, repair rate, benchmark score, and Pass@k, which indicate whether outputs satisfy benchmark validation.These metrics are used across repository-level issue resolution, workflow-based code generation, and competition-style programming.
- Evolution-aware evaluation: Evaluation should also expose evolution through performance under cost and time constraints, archives of improved variants, accumulated experience, efficiency, and transfer across held-out settings.Reported efficiency measures include cost, runtime, token usage, step counts, and retrieval overhead; generalization tests include held-out repositories, new benchmarks, different models, and programming languages.
6 Challenges and Open Problems
Self-evolving coding agents face distinctive reliability challenges because flawed feedback or benchmark shortcuts can be propagated into memory, skills, workflows, or model updates. Open problems include reproducibility, safety, quality control, long-term evaluation, and generalization beyond the settings where agents evolve.
- Core reliability challenge: Self-evolution can propagate unreliable tests, noisy trajectories, weak verifiers, or benchmark-specific shortcuts into memories, skills, workflows, and model updates.The central challenge is ensuring that evolutionary processes remain reliable, not merely improving benchmark performance.
- Reproducibility, contamination, and benchmark overfitting: Benchmark-selected self-modifications are vulnerable to evaluation noise, contamination, leakage, and benchmark overfitting, making reproducibility difficult across runs, tasks, repositories, environments, and model versions.These risks are amplified when systems select agent variants or self-modifications using benchmark outcomes.
- Feedback reliability, safety, and tool dependence: Imperfect tests, compilers, CI logs, generated tests, reward models, and learned verifiers can transmit biased or misleading feedback, especially when agents modify tools, workflows, or their own scaffolds.Reliance on unit-test validation, environment rewards, or learned verifiers exposes self-evolution to their blind spots.
- Long-term memory, skills, and coordination: Experience banks, repository memory, and skill libraries may become stale, redundant, repository-specific, or contaminated by failed trajectories, while evolving multi-agent roles and workflows create coordination challenges.Memory and skill reuse therefore requires quality control alongside coordination mechanisms for evolving teams and workflows.
- Evaluation beyond short benchmarks: Future evaluations must move beyond short-horizon pass rates, resolve rates, and benchmark scores to assess maintainability, security, reviewability, efficiency, long-term reliability, and robustness beyond original repositories or benchmarks.Existing evidence mainly concerns in-domain or near-domain transfer, while transfer from software-engineering feedback to non-coding domains remains largely unexplored.
7 Conclusion
Self-evolving coding agents represent a shift from static software engineering assistants toward systems that improve through sustained interaction with code, repositories, tools, tests, and human feedback. The surveyed literature frames this as a broad family of adaptation processes grounded in executable software artifacts and repository-level context, rather than a single algorithmic technique.
- Self-evolving coding agents improve through sustained interaction with code, repositories, tools, tests, and human feedback.
- The literature frames self-evolution as a broad family of adaptation processes rather than a single algorithmic technique.
- These adaptation processes are grounded in executable software artifacts and repository-level context.