PaperlayerView exact revision on arXiv

Source-linked AI summary

SePO: Self-Evolving Prompt Agent for System Prompt Optimization

Wangcheng Tao, Han Wu, Weng-Fai Wong

arXiv:2606.04465v1cs.CLcs.AI

TL;DR

System prompt optimization improves agents without changing their underlying models, but existing methods leave the prompt agent’s own prompt fixed and hand-engineered. SePO evolves both prompt layers through self-referential open-ended search with two-stage training, outperforming baselines across five benchmarks and generalizing beyond its pre-training mixture.

  • Problem

    Existing prompt optimization methods refine task-agent prompts while leaving the prompt agent’s system prompt hand-engineered and fixed, limiting accumulated experience.

  • Method

    SePO uses self-referential open-ended evolution with an archive, pre-training the prompt agent on multiple tasks before fine-tuning task-agent prompts.

  • Results

    SePO achieves the best accuracy on all five benchmarks and improves average accuracy by 4.49 points over Manual-CoT, while pre-training skill generalizes beyond the training mixture.

  • Takeaways & Limitations

    Prompt optimization becomes a learnable skill that accumulates across tasks rather than a fixed, hand-designed tool.

  • Takeaways & Limitations

    Search depths beyond G=5 yield only modest gains, and the evaluation covers five selected benchmarks spanning math, reasoning, science, code, and puzzles.

Abstract

from arXiv · show

System prompt optimization improves agent behavior without modifying the underlying model, yielding human-readable, model-agnostic instructions. Existing methods build a prompt agent that refines task agents' system prompts, yet leave the prompt agent's own system prompt hand-engineered and fixed. We propose Self-Evolving Prompt Optimization (SePO), which treats the prompt agent's own system prompt as an optimization target alongside task agents' system prompts. SePO adopts a self-referential design. A single prompt agent improves both task agents' system prompts and its own under an open-ended evolutionary search that maintains an archive of candidate prompts as stepping stones. Training proceeds in two stages: pre-training evolves the prompt agent on a multi-task pool, and fine-tuning then applies it to a target task. Across five benchmarks spanning math (AIME'25), abstract reasoning (ARC-AGI-1), graduate-level science (GPQA), code generation (MBPP), and logic puzzles (Sudoku), SePO consistently outperforms Manual-CoT, TextGrad, and MetaSPO, improving the average accuracy by 4.49 points compared to Manual-CoT. The prompt optimization skill from pre-training also generalizes to tasks beyond the pre-training mixture, rather than memorizing per-task prompts.

1 Introduction

Existing system prompt optimization improves task agents while leaving the prompt agent fixed and hand-engineered. SePO closes this loop by evolving both prompt layers through a self-referential, two-stage process, achieving stronger results across diverse benchmarks.

  • System prompt optimization improves agent behavior without modifying the underlying model, complementing approaches that retrain weights, augment memory, or redesign workflows.
  • Existing methods rely on a hand-engineered prompt agent that reads evaluation feedback and refines task-agent prompts, limiting accumulated improvement across tasks.
  • SePO treats the prompt agent as a special task agent, applying the same optimization procedure to its own system prompt and to task-agent prompts.
  • SePO uses open-ended evolution with an archive of candidate prompts, allowing earlier prompts to serve as stepping stones for later improvements.
  • SePO trains in two stages: pre-training evolves a prompt agent on multiple tasks, while fine-tuning applies it to target-task prompt optimization.
  • 4.49 points: SePO improves average accuracy over Manual-CoT across AIME’25, ARC-AGI-1, GPQA, MBPP, and Sudoku, while pre-training skill generalizes beyond its task mixture.

2 Related Work

Related work advances prompt optimization through black-box search, evolutionary populations, textual gradients, and meta-learning, while self-evolving agents modify outputs, skills, or architectures. SePO differs by placing the prompt agent itself inside the evolutionary search rather than keeping the search operator external and fixed.

  • Prompt Optimization: Prompt optimization progressed from hand-crafted cues to black-box feedback search, evolutionary candidate populations, textual gradients, and shared cross-task prompts.
  • Self-Evolving Agents: Self-evolving agents revise outputs, accumulate reusable skills, and increasingly modify agent code or architectures across episodes.
  • Evolutionary Search over Non-Agent Artifacts: Evolutionary systems such as FunSearch, Eureka, and AlphaEvolve optimize non-agent artifacts while using a fixed external agent to drive the search.
  • Evolutionary Search over Non-Agent Artifacts: SePO places the prompt agent itself inside the population being searched, making the search operator an optimization target rather than an external fixture.

3 Methodology

SePO optimizes both the task agent’s system prompt and the prompt agent’s own system prompt through a self-referential, two-stage evolutionary pipeline. Pre-training develops reusable prompt-optimization skill, while fine-tuning applies the resulting prompt agent to target tasks.

  • Self-Referential System Prompt Optimization: Existing prompt optimization methods improve task-agent prompts while leaving the prompt agent’s own system prompt hand-engineered and fixed.This asymmetric setup limits prompt-agent optimization to what a human author can produce.
  • Self-Referential System Prompt Optimization: SePO treats the prompt agent as a special task agent whose system prompt is optimized using the same procedure applied to task agents.The prompt agent’s prompt is optimized on a prompt task whose score measures whether proposed prompts improve task-agent accuracy.
  • SePO: Self-Evolving Prompt Optimization: SePO uses open-ended evolutionary search over candidate prompts, retaining an archive and admitting children that improve on their parents.Earlier prompts serve as stepping stones for later improvements.
  • Two-Stage Training Pipeline: Pre-training evolves the prompt agent’s system prompt across a task pool, then fine-tuning reuses the resulting prompt to optimize a task agent’s prompt on one target task.The two stages share the same algorithm but differ in which agent’s system prompt is optimized.
  • SePO-Specialist vs. SePO-Generalist: SePO-Generalist pre-trains on a multi-task mixture and can reuse one pre-training run across many fine-tuning tasks, unlike SePO-Specialist.SePO-Specialist pre-trains separately for each target task; SePO-Generalist is the default configuration.
  • Task Selection for SePO-Generalist: Task selection for SePO-Generalist uses a greedy heuristic, while more advanced task-selection algorithms could replace it.Task selection is explicitly outside SePO’s main focus.

4 Experiments

SePO is evaluated across five tasks and against three prompt-optimization baselines, using two-stage evolutionary prompt optimization with self-improvement. Experiments examine task selection, cross-task generalization, component necessity, model robustness, cost, and evolved prompt behavior.

  • Our Method: SePO evolves both the prompt agent’s and task agent’s system prompts through open-ended search during pre-training and fine-tuning.SePO-Generalist pre-trains on a greedily selected multi-task mixture and reuses the resulting prompt agent across fine-tuning tasks.
  • Main Results: SePO-Generalist achieves the best accuracy on all five tasks, raising average accuracy from 71.89 with Manual-CoT to 76.38.SePO-Specialist also beats Manual-CoT on every task but trails SePO-Generalist by 2.29 average-accuracy points.
  • Task Selection: 72.68 versus 71.14: the size-4 greedy task mixture outperforms random selection by the largest margin among mixture sizes below 8.The selected mixture is STEM+ARC-AGI-1+LIMO+MBPP; at size 8, both selectors return the full pool.
  • Cross-Task Generalization: Pre-training generalizes beyond its mixture: even without a related task, SePO-Generalist beats Manual-CoT on every task, including held-out Sudoku.Sudoku improves from 96.95 with Manual-CoT to 99.90 despite appearing in no pre-training mixture.
  • SePO Variants: Removing self-improvement lowers average accuracy by 1.44 points, while replacing open-ended evolution with linear search lowers it by 3.74 points.The largest task-specific drops are 3.63 points on ARC-AGI-1 without self-improvement and 6.98 points on AIME’25 without open-ended evolution.
  • Analysis with Varying Models: With swapped underlying models, SePO-Generalist again beats Manual-CoT on every task, improving average accuracy from 67.95 to 70.08.The swapped pair uses Gemini 3.1 Flash-Lite Preview for the task agent and Claude Opus 4.6 for the prompt agent.

5 Conclusion

SePO makes the prompt agent itself an optimization target through a self-referential design and two-stage training. Across five task types, it outperforms the compared prompt-optimization baselines and improves average accuracy by 4.49 points over Manual-CoT.

  • 5 Conclusion: SePO applies the same self-evolving procedure to the prompt agent’s own system prompt and to task agents’ prompts.Its two-stage training first evolves the prompt agent on a multi-task pool, then fine-tunes it for each target task.
  • 5 Conclusion: SePO consistently outperforms Manual-CoT, TextGrad, and MetaSPO on the five evaluated tasks.The reported tasks cover math, abstract reasoning, graduate-level science, code generation, and logic puzzles.
  • 5 Conclusion: The two-stage split amortizes pre-training across applications while accumulating prompt-optimization skill for cross-task generalization.The paper states that pre-training evolves the prompt agent on a multi-task pool and fine-tuning applies it to each target task.
  • 5 Conclusion: 4.49 points is SePO’s average-accuracy improvement over Manual-CoT across five tasks.The evaluation spans mathematics, abstract reasoning, graduate-level science, code generation, and logic puzzles.

A Open-Ended Evolutionary Search Details

SePO uses open-ended evolutionary search over an archive of candidate prompts. Parent selection balances score, exploration, and child-count penalties, while scoring and admission compare children with their parents under evaluation noise.

  • A Open-Ended Evolutionary Search Details: Open-ended evolutionary search maintains an archive from which candidate prompts are selected as parents.The section identifies parent selection, scoring, and archive admission as the procedure’s main components.
  • A Open-Ended Evolutionary Search Details: Parent selection samples archive candidates with a tempered, child-count-penalized softmax.The score is based on candidate quality, while the child-count penalty redirects computation toward less-explored stepping stones.
  • A Open-Ended Evolutionary Search Details: All candidates retain non-zero selection probability while adaptive temperature controls exploitation as scores spread.The child-count penalty favors less-explored candidates without eliminating any candidate from selection.
  • A Open-Ended Evolutionary Search Details: Children are scored by their per-sample accuracy delta against the parent on the same evaluation batch.The delta compares the child’s accuracy with the parent’s accuracy.
  • A Open-Ended Evolutionary Search Details: A child enters the archive when its score is at least its parent’s within evaluation-noise leeway ϵ; otherwise it is discarded.This is the keep_better admission policy.

B SePO-Generalist Task Selection Algorithm

SePO-Generalist constructs pre-training mixtures by greedily ordering candidate tasks according to target relevance and diversity. It evaluates prefixes of that order at candidate mixture sizes and selects the best one using proxy tasks.

  • B SePO-Generalist Task Selection Algorithm: The algorithm builds one greedy task order, adding the candidate that best balances target-suite relevance with diversity from already selected tasks.This avoids scoring all possible task subsets.
  • B SePO-Generalist Task Selection Algorithm: The candidate pool includes target-suite training tasks and auxiliary train-only tasks.An LLM judge scores task-skill similarity, answer-format similarity, and overall redundancy for task pairs.
  • B SePO-Generalist Task Selection Algorithm: Score(d | M) = λ U(d; T ) + (1 −λ) Div(d, M) combines transfer utility to the target suite with diversity from the current mixture.The utility term remains fixed for each candidate, whereas diversity changes as the mixture grows.
  • B SePO-Generalist Task Selection Algorithm: The algorithm repeatedly selects the highest-scoring remaining task and records prefixes of lengths 1, 2, 4, and 8 as candidate mixtures.The ordered list and candidate mixtures are then used to choose the selected mixture.
  • B SePO-Generalist Task Selection Algorithm: The best candidate mixture is selected by maximizing its proxy-task objective.Algorithm 2 returns the ordered task list, candidate mixtures, and selected mixture.

C Task Details

The evaluation uses train and test splits for five tasks, with additional train-only datasets in the multi-task pre-training pool. Repeated test runs use task-specific aggregation protocols, including pass@3 for ARC-AGI-1 and pass@1 for the other tasks.

  • C Task Details: Each task uses its train split for pre-training and fine-tuning, while evaluation uses its test split.The dataset construction is summarized by per-split sizes in Table 4.
  • C Task Details: The suite includes AIME’25, ARC-AGI-1, GPQA, MBPP, and Sudoku, with task-specific dataset sources and processing.AIME’25 uses verified-solution training entries, GPQA uses STEM MMLU training data as a proxy pool, and Sudoku training data is randomly subsampled.
  • C Task Details: Four train-only datasets—LIMO, Humanities, Social Sciences, and Other—contribute training examples but never appear in the reported evaluation.LIMO supplies a random subset of a mathematical reasoning dataset; the others use MMLU train subsets.
  • C Task Details: Test evaluation repeats each problem N times with N = 64 for AIME’25, 10 for ARC-AGI-1 and GPQA, 5 for Sudoku, and 4 for MBPP.ARC-AGI-1 reports pass@3; the other four tasks report averaged pass@1.
  • C Task Details: Table 4 reports the train and test split sizes for the evaluated task datasets.The table caption identifies its contents as train and test split sizes.

D Implementation Details

SePO uses two-stage evolutionary prompt optimization, with a prompt agent that analyzes balanced successes and failures, proposes tagged candidate prompts, and improves itself through pre-training. Its evolved prompts explicitly guard against regressions while preserving reasoning rigor.

  • SePO: SePO runs Algorithm 1 for G=5 generations with K=2 children per generation in both pre-training and fine-tuning.This produces ten candidate prompts per stage; the fine-tuning seed is the Manual-CoT prompt, while pre-training begins from a hand-written prompt-agent prompt.
  • Prompt-generation template: Each candidate is generated from the task-agent statement and a batch of failed and successful evaluation examples.The evaluation batch contains 16 train-split examples, approximately balanced 1:1 and prioritizing failures, subject to the prompt agent’s token budget.
  • Prompt-generation template: The prompt agent wraps its proposed system prompt in OPTIMIZED_SYSTEM_PROMPT tags, yielding the next candidate from the current prompt and evaluation results.The required output format makes the candidate prompt directly extractable from the prompt agent’s free-form response.
  • Prompt-agent evolution: The evolved prompt internalizes regression-avoidance heuristics, including preserving reasoning depth, domain rigor, and working behavior while preventing overfitting.The archive admission policy rejects children scoring below their parent on held-in evaluation, creating selection pressure for these safeguards.
  • Prompt-agent instructions: The prompt agent compares regressions and improvements across evaluation examples before proposing edits to the current system prompt.The instructions emphasize identifying added instructions that cause overthinking, hallucination, reduced rigor, or skipped reasoning, alongside modifications that resolve failures.

F Seed and Best-Evolved Task Agent System Prompts

The appendix contrasts concise task-specific seed prompts with longer SePO-evolved prompts that add domain-specific procedures, rigorous verification, and output constraints across the five benchmarks.

  • AIME’25: The evolved AIME’25 prompt emphasizes direct mathematical deduction and rejects guessing based on contest heuristics or perceived numerical patterns.It instructs the solver to backtrack when an approach produces a dead end or an unexpected irrational result.
  • ARC-AGI-1: The evolved ARC-AGI-1 prompt adds exhaustive object, coordinate, symmetry, connectivity, and hypothesis-testing procedures.It requires checking every training example and rejecting rules that fail an example or require arbitrary exceptions.
  • GPQA: The evolved GPQA prompt strengthens technical rigor by requiring formal definitions, exhaustive reasoning, and fundamental scientific mechanisms.It prioritizes deep physical, statistical, and thermodynamic correctness over pedantic terminological purity.
  • MBPP: The evolved MBPP prompt adds a workflow requiring hypothesis formation and exhaustive manual dry-runs on all supplied test cases before coding.The prompt also warns against infinite loops and requires faithful tracing of intermediate states, math, and values.

G Cost Details

The cost analysis reports token usage and monetary training costs for TextGrad and SePO variants, distinguishing per-task specialization from shared generalist pre-training.

  • Cost accounting: Tables 6 and 7 compute training costs from runtime token-usage files multiplied by published per-million-token model prices.The accounting includes separate input and output tokens for task and prompt agents at each stage.
  • Training configurations: TextGrad uses a single training stage, whereas SePO-Specialist trains both stages separately for each task.SePO-Generalist instead performs one shared pre-training stage followed by task-specific fine-tuning.

H Discussion

SePO’s reported gains are bounded by search depth and benchmark scope, while its interpretable natural-language archive remains dependent on evaluation design; future work proposes iterative training and broader artifacts.

  • Limitations: Search depths beyond G=5 yield modest rather than exponential gains in a preliminary sweep.The authors hypothesize, without verification, that gains may saturate near a ceiling imposed by the underlying model.
  • Limitations: The five-benchmark evaluation does not establish generality for tool-use agents, multi-turn dialogue, or long-horizon planning.The authors identify broader evaluation across these settings as necessary for a generality claim.
  • Broader impacts: SePO modifies inspectable natural-language prompts and admits children only when held-in scores improve, but its behavior remains dependent on evaluation choice.Safety-relevant deployment would require safety-aligned evaluation suites in addition to capability metrics.
  • Future work: Future work would alternate pre-training and fine-tuning so task-agent failures feed back into later prompt-agent pre-training.The authors also propose evolving system prompts alongside tool definitions, retrieval policies, and chain-of-thought scaffolds.
Loading 2606.04465v1…