PaperlayerView exact revision on arXiv

Source-linked AI summary

The View from Nowhere: Population-Centered Agent Evolution

Shuo Yan, Ningning Hou, Haoyu He, Haochen Pei, Shang Gao

arXiv:2610.36660v1cs.AI

TL;DR

Prior approaches largely use selected parents’ experience, while the gap between population coverage and best-agent performance remains underexamined. PCE distils population-wide records and modification histories to guide improvements to a selected parent’s harness. Under matched settings, it improves results over GEA and narrows or closes the coverage gap.

  • Problem

    Prior work had not systematically measured the gap between tasks solved by any evolved variant and the performance of the best individual agent, or examined how that gap could guide improvement.

    Source: §1 · p. 1

  • Method

    PCE distils population-wide task records and modification histories into a population block, then uses it to guide diagnosis and harness improvements for a selected parent.

    Source: §1 · p. 2

  • Results

    PCE exceeds GEA by 3.2 percentage points on SWE-bench Verified and 5.9 points on Polyglot, reducing the coverage gap by one-third on SWE-bench and closing it on Polyglot.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support treating the population as comparative evidence for improving leading agents, not only as a pool for parent selection.

    Source: §1 · p. 2, §6 · p. 9

    Published DGM, GEA, OpenHands, and Aider results use original models and protocols and are reference points, not controlled comparisons; the main conclusion rests on matched PCE and GEA runs.

    Source: RQ1: Does PCE evolve stronger agents… · p. 6

Abstract

from arXiv · show

Self-improving AI systems can revise the designs that govern their own behavior. Yet existing approaches center improvement on selected parent agents and their evolutionary experience, leaving the broader population largely underused. We identify a substantial gap between population coverage (i.e., the tasks solved by at least one evolved variant) and the performance of the best individual agent. This gap shows that useful capabilities and diagnostic evidence emerge across the population without being consolidated into its leading agents. These observations motivate the view from nowhere, a principle for agent evolution: an agent should be improved by interpreting its behavior through population-wide patterns of success, failure, and prior self-modification. We instantiate this principle as Population-Centered Evolution (PCE). PCE distils task records and modification histories from the population into a population block, which captures recurring failures, complementary successes, and the effects of previously attempted modifications to guide improvements to a selected parent's harness. Under matched models, initial harnesses, and evolution schedules, PCE achieves 72.1% success on SWE-bench Verified and 96.7% on Polyglot, improving over a strong group-evolution baseline by 3.2 and 5.9 percentage points. In our evaluation, PCE reduces the gap between population coverage and best-agent performance by one-third on SWE-bench and closes it on Polyglot. These results suggest that treating the population as evidence--not merely as candidates for selection--provides a complementary principle for agent evolution.

1 Introduction

Agent evolution seeks to turn discoveries made across self-modifying variants into stronger leading agents, but prior approaches mainly use parent-local evidence. PCE treats population-wide patterns as diagnostic evidence and improves selected parents’ harnesses.

1 Introduction

  • Prior approaches largely base modification proposals on selected parents or parent groups, rather than systematic comparisons across the full population.Other variants’ task successes, failures, and modification histories do not directly inform the current diagnosis.Source: §1 · p. 1
  • The GEA population solves 9 more of 60 held-in tasks than its best individual agent, a 15.0 percentage-point population-coverage gap.The gap indicates that successful outcomes remain distributed across variants rather than fully consolidated in the leading agent.Source: Figure 1 · p. 2, §1 · p. 2
  • The view from nowhere grounds agent improvement in population-wide patterns of success, failure, and prior self-modification.Comparing records on a task can distinguish isolated from recurring failures and show which remedies have already been attempted.Source: §1 · p. 2
  • PCE distils population task records and modification histories into a block that guides improvements to a selected parent’s harness.The block organizes outcome evidence, cross-agent observations, and candidate files for inspection; diagnosis checks its hypothesis against the parent’s records and source code.Source: §1 · pp. 2–3
  • 72.1% success on 200 SWE-bench tasks and 96.7% on 225 Polyglot tasks exceed GEA by 3.2 and 5.9 percentage points, respectively.Under matched models, initial harnesses, and evolution schedules, PCE also reduces the population-coverage gap by one-third on SWE-bench and closes it on Polyglot.Source: §1 · p. 2
  • PCE is presented as a way to use the population as evidence for agent improvement, alongside its role in selecting parents.The paper reports improved performance, a smaller population-coverage gap, and lower total token consumption than baselines.Source: §1 · pp. 2–3

2 Related Work

Prior work uses execution experience and populations to improve or preserve agents, but PCE differs in comparing evolving harness variants’ outcomes and modification histories to guide a selected parent’s improvement.

Learning from execution experience

  • PCE compares how a population of evolving harness variants handles the same task and relates outcomes to earlier self-modifications to diagnose and modify the current parent.This differs from reflection, memory, self-training, and harness-improvement approaches described as using other units or sources of experience.Source: §2 · p. 3

Population-based agent design and evolution

  • Prior population-based methods preserve diversity, select parents, share parent experience, or recombine designs, whereas PCE uses all available variants as comparative evidence.The related work describes DGM’s archive, HGM’s lineage-potential estimates, GEA’s parent sharing, and HarnessBank’s harness recombination.Source: §2 · p. 3

3 Population-Centered Evolution

PCE distils task records and modification histories across evaluated variants, then checks population-level findings against a selected parent before proposing a harness change. Its evolution loop retains evaluated children in the population and eligible children in the parent archive.

3 Population-Centered Evolution

  • PCE separates all evaluated variants in population V from archive A, the subset eligible for parent selection.A variant can inform future improvements even when it is not selected as a parent.Source: §3 · p. 3

3.1 Population Distillation

  • Population distillation compares available task records, execution logs, patches, harness source, and modification histories to identify recurring outcomes and prior remedies.Programmatic outcome counts are combined with agent-based analysis, which selects relevant records and files within an analysis budget.Source: §3.1 · p. 4
  • The population block separates measured outcome distributions from cross-agent hypotheses and candidate harness files, without prescribing a modification.Candidate files may be left empty when no plausible location is identified.Source: §3.1 · p. 4

3.2 Diagnosis and Harness Modification

  • Diagnosis checks the population-level hypothesis against the selected parent’s records and source code before proposing one harness modification.The modification is applied to a copy of the parent’s harness; the foundation model and inference limits remain fixed.Source: §3.2 · p. 4
  • For django-12209, records showed 6 solved patches, 10 empty patches, and 1 test-only patch; diagnosis led to an empty-patch check and bounded repairs.The example illustrates a population pattern being checked against the parent and implemented as a harness change.Source: §3.2 · pp. 4–5

3.3 Evolution Loop

  • PCE runs population distillation only when at least three records are available; otherwise, or without a usable observation, diagnosis falls back to the parent-local procedure.Evaluated children enter V, while children meeting retention criteria also enter A.Source: §3.3 · p. 5

4 Experiments Setup

The experiments evaluate PCE on SWE-bench Verified and Polyglot, compare it with GEA under matched conditions, and assess selected agents on held-in and held-out tasks.

4.1 Benchmarks and Data Splits

  • The evaluation uses 200 SWE-bench Verified tasks and 225 Polyglot tasks, with 60 held in for evolution and 140 and 165 held out, respectively.An additional set of 60 SWE-bench tasks spanning 12 repositories is also evaluated.Source: §4.1 · p. 5

4.2 Baselines and Settings

  • PCE and GEA use the same foundation models, initial harness, parent-selection procedure, diagnosis prompt, modifiable files, and evaluation protocol.Both run for ten generations and produce two children per generation; DeepSeek Pro performs diagnosis for both and population distillation for PCE.Source: §4.2 · p. 6

4.3 Evaluation Protocol

  • Evolution screens children on 10 tasks, advances those meeting the threshold to the remaining 50, and selects the highest-scoring agent across all 60 held-in tasks.Selected and initial agents are independently evaluated five times on the 60 held-in tasks, with mean success and sample standard deviation reported; held-out Polyglot and SWE-bench sets are each evaluated once.Source: §4.3 · p. 6

5 Results and Analysis

Under matched experimental settings, PCE outperforms GEA on both benchmarks and improves performance beyond parent-only distillation. It also brings leading agents closer to population coverage while using fewer tokens than GEA.

RQ1: Does PCE evolve stronger agents that generalize beyond the search evaluation?

  • 72.1% Overall success on SWE-bench and 96.7% on Polyglot exceed GEA by 3.2 and 5.9 percentage points, respectively.Both methods improve substantially over the shared initial agent under matched experimental settings.Source: RQ1: Does PCE evolve stronger agents… · p. 6
  • PCE achieves higher best search scores in both independent SWE-bench runs, retains a 4.0-point mean advantage on Held-in 60, and beats GEA on all three held-out sets.Held-out tasks make up the majority of Overall 200 and Overall 225.Source: RQ1: Does PCE evolve stronger agents… · p. 6

RQ2: Does PCE consolidate population-wide successes into stronger individual agents?

  • PCE improves the minimum score among the top three and top five variants on both benchmarks; its fifth-ranked agent matches GEA’s best on SWE-bench and beats it on Polyglot.The gains extend beyond a single exceptional agent.Source: RQ2: Does PCE consolidate… · p. 6
  • With similar SWE-bench population coverage, PCE reduces the mean coverage gap from 14.2 to 10.0 percentage points; on Polyglot, it closes the gap while GEA retains 10.0 points.The reduction reflects leading variants capturing more population-distributed successes, not substantially broader task coverage.Source: RQ2: Does PCE consolidate… · p. 7

RQ3: Does population-wide evidence provide gains beyond additional parent-only analysis?

  • 96.0% mean success for PCE exceeds the 92.0% mean for both GEA and parent-only distillation on Polyglot; parent-only distillation’s best search score is 91.7%, versus GEA’s 90.0%.Parent-only distillation matches PCE’s distillation model, tools, step budget, and output format, but uses only selected-parent evidence.Source: RQ3: Does population-wide evidence… · pp. 7–8
  • In three fixed-context SWE-bench probes, adding a population block changed the main diagnostic target in two contexts, but the proposals were not implemented or evaluated.The analysis characterizes diagnostic behavior rather than downstream performance.Source: RQ3: Does population-wide evidence… · p. 8

5.4 Performance Gains and Token Cost

  • PCE uses 36% fewer tokens on SWE-bench and 59% fewer on Polyglot than GEA across complete evolution runs, including task execution, diagnosis, self-modification, and population distillation.PCE also makes fewer model requests on both benchmarks, so its gains do not come from a larger inference budget.Source: §5.4 · pp. 8–9, Table 4 · p. 9
  • PCE evaluates more task episodes than GEA on both benchmarks while using fewer tokens; on SWE-bench, trajectories exceeding 500,000 characters fall from 158 to 45.PCE produces shorter trajectories for solved and unsolved attempts, and extremely long trajectories become less frequent.Source: §5.4 · p. 9

6 Conclusion

The paper frames the population as evidence for understanding how an individual agent should improve, not only as a pool of candidates. PCE distills population outcomes and modification histories into parent-specific diagnosis and reports stronger agents with lower token use.

6 Conclusion

  • The paper presents population-wide evidence as a source of knowledge for agent evolution, alongside using the population to select candidates.It identifies attribution, compression, and transfer of population evidence as topics for further study as evolutionary histories grow and systems move beyond coding agents.Source: §6 · p. 9
  • PCE improves performance on SWE-bench Verified and Polyglot, retains gains on held-out tasks, and brings the best agent closer to population coverage.The Polyglot ablation indicates that the gains do not arise from additional parent-only analysis.Source: §6 · p. 9
  • PCE produces stronger agents while using fewer total tokens, rather than purchasing improvement with a larger inference budget.The reported token reductions are 36% on SWE-bench and 59% on Polyglot compared with GEA.Source: §6 · p. 9

A Experimental Setup

The experiments use specified model settings and benchmark task sets, including a separately constructed SWE-bench held-out set. Table 5 reports child search scores across reported runs.

A Experimental Setup

  • Task execution and self-modification use DeepSeek-V4-Flash on SWE-bench and DeepSeek-V4.1-Flash on Polyglot, while diagnosis and distillation use DeepSeek-V4-Pro.The provider is pinned to CoreWeave through OpenRouter.Source: A Experimental Setup · p. 12
  • Table 5 lists two children per generation scored out of 60 tasks, with Min, Max, and Median calculated over fully evaluated children.A 0/10 entry marks evaluation stopped at the ten-task gate; a dash marks self-improvement that produced no patch.Source: Table 5 · p. 12
  • Held-in 60 uses DGM’s 60-task subsets; held-out sets include SWE-bench sets of 140 and 60 tasks and a Polyglot set of 165 tasks.The best agent is selected by its Held-in 60 search score and evaluated without further self-modification.Source: A Experimental Setup · p. 12
  • The additional SWE-bench Held-out 60 samples 60 validated instances with fixed seed, representing every candidate-pool repository and matching held-in difficulty as closely as possible.The set spans 12 repositories; the initial agent and best GEA and PCE agents are each evaluated five times.Source: A Experimental Setup · p. 12

Evolution loop.

The evolution loop selects parents and tasks, diagnoses each parent, implements harness modifications, and evaluates resulting children. Evaluated children enter the population, while only those meeting archive criteria enter the archive.

Evolution loop.

  • Across ten generations, two children are produced per generation; later generations select two parents using GEA’s performance-novelty rule and sample an eligible task for each.PCE supplies a population block for the parent’s task when available; the model, output cap, and tool loop remain fixed.Source: Evolution loop · p. 12
  • Evaluated children enter the population whether or not they qualify for archive admission; admission requires passing the gate, producing a non-empty task patch, and limiting changes to permitted paths.The archive is never pruned.Source: Evolution loop · p. 12
  • Table 6 reports proposal counts from five diagnoses per condition, with dashes marking untested trajectory controls lacking enough eligible generation-1 trajectories.Source: Table 6 · p. 13

Population distillation.

Distillation supplies an observation and candidate files to diagnosis when enough task records are available. The supplied comparison and evaluation passages describe the settings and selected-agent evaluations.

Population distillation.

  • Distillation has a 250-step budget and returns an observation of at most 60 words plus zero to three candidate files.Program-computed outcome counts accompany the observation; distillation is skipped with fewer than three task records, and diagnosis proceeds without a block if no usable observation is returned.Source: Population distillation · p. 13
  • GEA and PCE produce two children per generation, compared with one for DGM; Haiku/Sonnet denotes GEA’s Haiku 4.5-to-Sonnet 4.5 schedule.DGM’s model-transfer entries reuse the evolved harness without additional search.Source: Population distillation · p. 13
  • Table 7 reports repeated evaluations on Held-in 60 and single evaluations on Held-out 140 and Held-out 165, with solved counts and success percentages for held-out rows.Held-out sets are evaluated once; all tasks remain in the denominator, including never-graded attempts.Source: Table 7 · p. 13

B Diagnostic Proposals under Fixed Contexts

The diagnostic comparison fixes the focal parent, source code, modification history, and task record while varying the trajectory or failure evidence supplied. Five proposals per condition are classified by their main modification target.

B Diagnostic Proposals under Fixed Contexts

  • The conditions compare the original parent entry and a fixed population block with successful-trajectory and failure-based replacements, holding the diagnostic context fixed.The replacements use the paired parent’s successful trajectory on another task, three of its failures on other tasks, or three other agents’ failures on the focal task.Source: B Diagnostic Proposals under Fixed… · p. 13
  • Sphinx-7748, at generation 1, lacks enough eligible trajectories for the replacement conditions.The stated limitation applies to this task’s diagnostic comparison.Source: B Diagnostic Proposals under Fixed… · p. 13
  • Each condition has five proposals, classified by their main target as instructions, episode logic, or a new tool.The categories are abbreviated I, E, and T in Table 6.Source: B Diagnostic Proposals under Fixed… · p. 13

C Additional Results

Additional results report search-score and evaluation tables for SWE-bench and Polyglot, including repeated evaluations and selected held-out settings. The supplied passages describe table coverage but do not state outcome values.

C Additional Results

  • For SWE-bench, Table 5 reports per-child search scores from two independent runs of each method, including gate outcomes and fully evaluated-child summary statistics.Run 1 supplies the main performance and cost results; Table 2 summarizes both runs.Source: C Additional Results · p. 14
  • For SWE-bench, Table 7 reports five repeated evaluations on Held-in 60 and Held-out 60, plus a single evaluation on Held-out 140.The passage does not provide the evaluation outcomes.Source: C Additional Results · p. 14
  • For Polyglot, Table 7 reports five Held-in 60 evaluations for the initial and best evolved agents, including parent distillation, and Held-out 165 results for the initial agent, GEA, and PCE.The passage says these results support the main performance comparison and the population-distillation ablation, without giving their values.Source: C Additional Results · p. 14
  • For Polyglot, Table 5 reports per-child search scores for GEA, parent distillation, and PCE.The passage does not provide the search-score values.Source: C Additional Results · p. 14

Related papers

Loading 2610.36660v1…