PaperlayerView exact revision on arXiv

Source-linked AI summary

Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills

Chuan Xiao, Zhengbo Jiao, Shaobo Wang, Wei Wang, Bing Zhao, Hu Wei, Linfeng Zhang, Lin Qu

arXiv:2606.07412v1cs.SEcs.AI

TL;DR

High-quality SWE training tasks are scarce, and existing synthetic methods are largely disconnected from agents’ evolving weaknesses. Socratic-SWE distills solving traces into skills that guide validated repair-task generation, outperforming self-evolving baselines across four benchmarks and reaching 50.40% on SWE-bench Verified after three iterations.

  • Problem

    High-quality SWE training tasks are scarce, while existing synthetic pipelines generate data independently of agents’ training experience and weaknesses.

  • Method

    Socratic-SWE distills historical solving traces into structured skills that guide validated, gradient-aligned repair-task generation in a closed self-evolution loop.

  • Results

    Across four benchmarks, Socratic-SWE consistently outperformed self-evolving baselines under identical compute budgets, reaching 50.40% on SWE-bench Verified after three iterations.

  • Takeaways & Limitations

    Solving traces can serve as a reusable substrate for self-evolving SWE agents and support sustained curriculum adaptation without external annotation.

  • Takeaways & Limitations

    The method assumes executable verification and sandboxed repository interaction, and broader transfer beyond the evaluated SWE and terminal-agent settings remains unestablished.

Abstract

from arXiv · show

LLM-driven software engineering agents have become a central testbed for real-world language-model capability, yet their training remains limited by the availability of high-quality SWE tasks. Existing synthetic data methods typically create tasks through fixed mutation or bug-injection procedures, making the resulting distributions largely independent of the agent's own weaknesses and training progress. We introduce Socratic-SWE, a closed-loop self-evolution framework that reuses the agent's historical solving traces as a source of training signal. Rather than treating traces only as evidence for reward computation, Socratic-SWE distills them into structured agent skills that summarize recurring failures and effective repair patterns. These skills then guide the generation of targeted repair tasks in real repositories. Candidate tasks are checked through execution-based validation and scored with a solver-gradient alignment reward, so that the retained tasks are both verifiable and useful for improving the Solver. The updated Solver produces new traces, enabling the task curriculum to adapt over successive rounds. Across SWE-bench Verified, SWE-bench Lite, SWE-bench Pro, and Terminal-Bench 2.0, Socratic-SWE consistently improves over self-evolving baselines under the same compute budget, reaching 50.40% on SWE-bench Verified after three iterations. These results suggest that solving traces can serve as a scalable substrate for self-evolving SWE agents.

1 Introduction

Socratic-SWE addresses scarce, agent-independent SWE training data by converting solving traces into structured skills that guide validated, repository-grounded task generation in a closed self-evolving loop. Under a fixed compute budget, it improves across four benchmarks, reaching 50.40% on SWE-bench Verified after three iterations.

  • Motivation: Existing synthetic SWE pipelines operate independently of the agent’s training experience, leaving task distributions largely model-agnostic and non-adaptive.They use AST-level mutation, LM-guided rewriting, or learned bug injection to mitigate scarce high-quality SWE data.
  • Motivation: Solving traces expose recurring failures, regression-prone repairs, and ineffective exploration, but existing methods mainly use them for reward extraction or credit assignment.The traces include code search, file editing, command execution, and test runs during repository interaction.
  • Method: Socratic-SWE distills historical solving traces into an Agent Skill Registry that represents the Solver’s capability boundary and guides subsequent task construction.This trace-skill-task loop uses the agent’s own experience without requiring external annotation.
  • Method: Targeted repair tasks are generated in real repositories under skill constraints, filtered by staged execution-based validation, and scored with solver-gradient alignment.The pipeline aims to ensure reproducibility and non-triviality while selecting tasks useful for improving the Solver.
  • Results: 50.40% on SWE-bench Verified after 3 iterations, improving +7.80 over the base agent and +3.40 over SSR under the same compute budget.The framework is evaluated on SWE-bench Verified, SWE-bench Lite, SWE-bench Pro, and Terminal-Bench 2.0.

2 Related Work

Prior work spans reinforcement learning for LLM reasoning, repository-level SWE coding agents, and self-evolving systems that derive improvement signals through role-based interaction or verifiable environments. These directions motivate methods for handling sparse agentic-task feedback and executable software environments.

  • Agentic Reinforcement Learning: RL methods such as DAPO, GSPO, SAPO, and GDPO improve efficiency and stability but assume immediately verifiable rewards, unlike agentic tasks with sparse feedback across long trajectories.OpenAI o1 and DeepSeekMath’s GRPO established influential reasoning-RL directions and verifiable-reward frameworks.
  • SWE Coding Agents: SWE coding agents navigate executable repositories, edit code, run commands, and use regression tests, building on SWE-bench and systems including SWE-agent, OpenHands, and Agentless.SWE-Gym, SWE-RL, and SWE-Master extend training through executable environments, software evolution data, and post-training.
  • Self-Evolving LLMs: Self-evolving LLM methods use proposer, solver, evaluator, or teacher roles to create improvement signals from unlabeled data, co-evolution, or Teacher-Solver-Generator loops.Absolute Zero, SPIRAL, Socratic-Geo, and SpatialEvo reduce noisy feedback through verifiable environments such as code execution, zero-sum games, programmatic geometry, and deterministic spatial tasks.

3 Methodology

Socratic-SWE co-evolves a Generator and Solver with shared policy weights: traces become structured skills that guide repository-grounded repair-task generation, while execution-based validation and solver-gradient alignment filter and reward tasks. The Solver trains on accepted tasks, produces new evidence, and thereby updates the task curriculum over iterations.

  • Co-evolutionary framework: Socratic-SWE alternates a shared policy πθ between a Generator that constructs repository-grounded repair tasks and a Solver that produces patches.The framework uses an Agent Skill Registry distilled from historical traces and a staged execution-grounded validation pipeline as external signals.
  • Adaptive curriculum: At iteration t, the Solver trains on Dt, its trajectories expose capability gaps, and the Generator uses skills and Solver evidence to propose targeted tasks for Dt+1.Only candidates satisfying Valid(τ′, v′, r) = 1 are added to the curriculum.
  • Agent Skill Registry: The Agent Skill Registry contains structured skills with a name, description, applicability conditions, and ordered operations, enabling programmatic retrieval and conditioning.Skills are distilled from successful and failed traces, then deduplicated by semantic similarity and filtered by trace coverage.
  • Task validation: Generated tasks enter training only after format, grounding, execution, and semantics checks pass in the repository sandbox.The checks require well-formed task and verification data, existing repository artifacts, stable execution, separation of failing and repaired states, and at least one valid repair.
  • Generator reward: The Generator reward favors candidate tasks whose Solver-induced policy gradients align with a held-out validation gradient, while invalid tasks receive zero validation contribution.The target and candidate gradients are estimated from K Solver rollouts, and the validation gradient is recomputed periodically as the Solver evolves.

4 Experiments

Across four software-engineering benchmarks, Socratic-SWE achieves the strongest results after three iterations under a fixed 36k-instance budget and no pre-existing SWE training tasks. Its gains exceed those of self-evolving baselines, including on Terminal-Bench 2.0, where only Socratic-SWE and SSR show meaningful improvements.

  • Experimental Setup: All self-evolving methods ran for 3 iterations, generating 12k validated training instances per iteration, for 36k total.Baselines requiring seed tasks used 10% of SWE-smith, while Socratic-SWE required seed repositories.
  • Evaluation: Pass rate was the primary metric across SWE-bench Verified, Lite, Pro, and Terminal-Bench 2.0.SWE-bench agents submitted test-validated patches, whereas Terminal-Bench agents had to reach verifier-passing final states in a sandboxed terminal.
  • Baseline Comparisons: R-Zero fell below the base agent at Iteration 3 (−0.80), while SPIRAL and Absolute-Zero regressed after peaking at Iteration 2 (+1.80 and +2.40 on Verified).The passage attributes these outcomes to noisy vote-based rewards and self-play without execution-grounded validation.
  • Terminal-Native Generalization: +4.50 on Terminal-Bench 2.0 was achieved by Socratic-SWE, compared with +2.25 for SSR; these were the only meaningful gains on that benchmark.The passage links Socratic-SWE’s stronger transfer to trace-derived skills capturing file manipulation, command chaining, and environment navigation behaviors.

5 Analysis and Ablation Studies

Across five iterations, Socratic-SWE improves faster and reaches a higher final score than SSR before both plateau. Ablations show that the Skill Registry, trace-derived skills, and GDPO are key contributors, while performance remains robust across skill-extractor scales.

  • Long-Term Scaling: 52.00% for Socratic-SWE versus 48.00% for SSR at Iteration 5, after Socratic-SWE gains rapidly through Iteration 3 and still improves in Iteration 4.Socratic-SWE reaches 51.60% after Iteration 4; SSR improves more slowly and both methods nearly plateau by Iteration 5.
  • Long-Term Scaling: Socratic-SWE saturates 2 iterations later and at a higher ceiling than SSR because the closed-world setup increasingly covers the seed-repository space.The fixed repository pool also produces redundant training signal, while expanding the pool or enabling cross-repository transfer may extend the trend.
  • Component Ablations: Removing the Skill Registry causes the largest performance drop, indicating that curriculum design is the main driver of Socratic-SWE.The ablation is conducted on SWE-bench Verified at Iteration 3.
  • Component Ablations: Replacing trace distillation with manual skills degrades performance because hand-written taxonomies miss iterative grep-then-edit behaviors.The ablation compares trace distillation with manual skills on SWE-bench Verified at Iteration 3.
  • Component Ablations: Replacing GDPO with GRPO also degrades results, suggesting decomposed reward learning better handles partially correct patches than binary pass/fail signals.Table 2 reports these ablations on SWE-bench Verified at Iteration 3.
  • Skill Extractor Ablation: 49.80% on Verified is achieved by the smallest extractor, Qwen3.5-9B, only 0.60 points below the full system, while larger extractors provide small additional gains.Qwen3.6-27B gives a small gain, and Claude Opus 4.5 adds another +0.60; the passage attributes the core advantage to the framework rather than extractor strength.

6 Conclusion

Socratic-SWE is a practical closed-loop framework for self-evolving software engineering agents that reuses historical solving traces to generate targeted repair tasks addressing capability gaps. Across four benchmarks, it improves over five self-play baselines under identical compute budgets, gaining +7.80 points on SWE-bench Verified and +4.50 points on Terminal-Bench 2.0 after three iterations.

  • 6 Conclusion: Socratic-SWE provides a practical closed-loop framework for self-evolving software engineering agents.The framework is designed for settings with limited access to high-quality SWE task data.
  • 6 Conclusion: Historical solving traces are reused as training signal and distilled into an Agent Skill Registry.The registry guides targeted repair-task generation to address capability gaps and track the model’s frontier.
  • 6 Conclusion: +7.80 points on SWE-bench Verified and +4.50 points on Terminal-Bench 2.0 after three iterations, outperforming five self-play baselines under identical compute budgets.These gains are reported across four benchmarks.

Limitations · S via Eq. (14), update πθ via LS 26: end for 27: return Trained SWE agent πθ Appendix

Socratic-SWE’s limitations arise from its closed-world repository pool, validation-set-dependent reward, and reliance on executable verification. Its broader transfer remains uncertain across deployment settings, programming languages, repository styles, and software-engineering workflows.

  • Limitations: The fixed seed-repository pool makes later task generation increasingly redundant as the Agent Skill Registry covers more capability gaps.The scaling analysis therefore reflects curriculum behavior under a fixed repository distribution, not fully open-ended improvement with continuously refreshed data.
  • Limitations: Closed-world scaling results do not establish improvement with continuously refreshed data.The fixed repository distribution constrains what the reported curriculum behavior can show.
  • Limitations: The generator-gradient alignment reward depends on a held-out validation set and its trusted task selection.Although this improves curriculum quality, the design may be less robust when validation tasks do not represent the target deployment distribution.
  • Limitations: Validation-distribution representativeness remains a condition for robust deployment-oriented curriculum quality.The reward design may limit robustness if trusted validation tasks differ from the target deployment setting.
  • Limitations: The method assumes executable verification and sandboxed repository interaction.These assumptions may prevent direct transfer to settings without reliable tests, deterministic execution, or clear task-level validation.
  • Limitations: The evaluation covers four benchmarks in SWE and terminal-agent settings.This benchmark scope leaves broader transfer to other programming languages, repository styles, and software-engineering workflows unestablished.

A Training Algorithm … D.1 Reward Families

Socratic-SWE trains a role-conditioned Generator–Solver loop whose curriculum is driven by validation-gradient alignment rather than task difficulty alone. The method theoretically justifies this reward and compares it with alternative self-evolving SWE procedures under matched training conditions.

  • A Training Algorithm: Algorithm 1 defines Socratic-SWE’s role-conditioned Generator–Solver training loop.The Generator creates candidate tasks and the Solver trains on retained tasks.
  • B Baseline Adaptation Details: All methods use Qwen3.5-9B, matched SWE and Terminal-Bench harnesses, the same interaction budget, and 3 iterations with 36k validated instances total.Seed-dependent baselines use 10% of SWE-smith instances, whereas Socratic-SWE uses seed repositories without pre-existing SWE task instances.
  • C.1 Setup: Theoretical analysis models validation performance as expected solve rate and establishes that on-policy GRPO gradients with binary all-tests-pass rewards unbiasedly estimate its gradient.The setup uses a shared policy and a held-out trusted validation set; group-mean rewards provide an unbiased baseline estimate.
  • C.3 Direction Preservation Under Normalization: Advantage normalization preserves the validation-gradient direction because it rescales the raw gradient by a positive scalar, leaving cosine similarity unchanged.The result applies when the within-group standard deviation satisfies σA > 0.
  • C.4 Why Cosine Similarity Ranks by Validation Improvement: Cosine similarity provides a scale-normalized proxy for validation-aligned improvement, avoiding confounding from task length and patch complexity.Empirically, cosine reaches 50.40% versus 49.40% for the unnormalized inner product in Table 6.
  • D Discussion: Generator Reward Strategies: The Generator reward determines which candidate tasks enter the curriculum, motivating comparison among difficulty-aware, gradient-aligned, and hybrid reward philosophies.Difficulty-aware rewards use the Solver’s group pass rate and include uncertainty and hardness variants.
  • D.1 Reward Families: Difficulty-based rewards assume difficulty approximates learning value, but hard SWE tasks may reflect irrelevant domain knowledge and provide no downstream benchmark signal.This limitation motivates directly measuring optimization-direction alignment instead of using difficulty as a proxy.
  • D.1 Reward Families: The proposed gradient-aligned reward directly measures whether a candidate task’s optimization direction matches the direction improving validation performance, decoupling utility from difficulty.A moderate task teaching transferable repair patterns can therefore outrank a near-impossible task whose gradient is orthogonal to Gv.

D.2 Ablation Results … F Computational Overhead

The ablations show that gradient-aligned rewards outperform difficulty-only alternatives by selecting informative, transferable repair tasks. Validation gradients are most reliable with 100 held-out tasks and must be refreshed each iteration to track policy drift.

  • D.2 Ablation Results: −3.00 degradation makes Hardness the worst reward, since near-impossible tasks produce vanishing advantages when all rollouts fail.Variance performs best among difficulty-only baselines but remains 1.60 points behind gradient alignment.
  • D.2 Ablation Results: +0.20 is the hybrid reward’s improvement over pure gradient alignment, indicating cosine scores already encode appropriate difficulty.Trivial and impossible tasks receive low scores because their gradients are near zero when all pass or all fail.
  • D.2 Ablation Results: Gradient alignment selects common repair patterns, whereas Variance often selects obscure library-internal tasks that provide little transferable learning.Examples include off-by-one iteration bounds and incorrect argument ordering for gradient alignment, versus C-extension edge cases for Variance.
  • E.1 Design Principles: 100 held-out BeyondSWE tasks provide the validation reference direction without entering Solver training or final evaluation.The set spans repositories and difficulty levels, stratified across Python, JavaScript, TypeScript, and Go/Rust.
  • E.2 Sensitivity to Validation Set Size: 100 validation tasks stabilize the signal at correlation 0.61, while 20 tasks yield correlation 0.34 and a 1.20-point performance drop.Increasing the set to 200 tasks provides only +0.20, so |Vval| = 100 is fixed.
  • E.3 Recomputation Frequency: +0.20 is gained by recomputing the validation gradient every 4k instances, but this costs 3× more validation computation.The chosen schedule refreshes Gv once per iteration, every 12k training instances.
  • E.3 Recomputation Frequency: −1.40 degradation results from using one fixed validation gradient across iterations as the policy’s optimization landscape drifts.This supports refreshing Gv once per iteration rather than keeping it fixed throughout training.

F.1 Cost Breakdown · F.2 Alignment Metric Ablation · G Prompt Templates and Implementation Details

The framework adds a modest gradient-alignment cost while using cosine scoring to select directionally useful tasks. Its prompts constrain bug generation and solver behavior to minimal, verifiable source-code changes, with training hyperparameters documented separately.

  • F.1 Cost Breakdown: 1.3 h/iteration (8.4%) is the total gradient-alignment overhead, consisting of validation-trace rollouts and per-candidate gradient scoring.The mechanism rolls out Solver trajectories on 100 validation tasks once per iteration and scores each candidate against the resulting Gv.
  • F.1 Cost Breakdown: 800 rollouts for Gv computation are negligible beside the ∼96k Solver rollouts used for training.Per-candidate scoring reuses semantic-validation rollouts, and the cosine calculation takes <1s per candidate.
  • F.1 Cost Breakdown: ∼3.9 wall-clock hours across 3 iterations remain < 9% of the 45h wall-clock budget.The selection cost is described as modest relative to the overall budget.
  • F.1 Cost Breakdown: +3.40 higher Verified than SSR is achieved at comparable total FLOPs.SSR trains indiscriminately on all validated tasks, whereas Socratic-SWE recovers its selection cost through better data efficiency.
  • F.2 Alignment Metric Ablation: Cosine scoring isolates gradient direction, whereas the inner product conflates direction with magnitude and can favor complex but irrelevant multi-file tasks.Large gradients from multi-file patches can bias inner-product selection irrespective of task utility.
  • G.1 Generator System Prompt: The generator prompt requires exactly one realistic, targeted, reversible semantic bug in one production file, without modifying tests or introducing loading failures.It also requires a minimal diff free of unrelated cleanup and concealment cues such as comments, logs, TODOs, or revealing variable names.
  • G.1 Generator System Prompt: The generator workflow inspects the repository, identifies exposing tests, states the intended change, makes one contiguous edit, verifies failure, runs collateral checks when feasible, and inspects the diff.Interaction is restricted to a Linux Bash shell with at most one command per turn.
  • G.2 Mini-SWE-agent Prompt / G.3 Base Mini-SWE-agent Prompt: The solver prompt directs agents to inspect and reproduce issues, modify only necessary source files, run visible checks and edge cases when feasible, and leave a clean intended diff.Agents should not modify tests, generated files, build artifacts, or unrelated configuration unless directly required; the task template specifies /testbed/ and visible-test behavior.

H Worked Example of Skill-Guided Task Generation

The OAuthLib example turns trace-derived weaknesses into repository-specific bug-injection patterns across protocol helpers, constructors, OAuth1 plumbing, and OIDC delegation. Candidate tasks are retained only after targeted execution-based validation confirms reproducible, reversible failures without oracle leakage.

  • Target Overview: OAuthLib tasks target small semantic regressions requiring exact repository-specific behavior in protocol helpers, endpoint constructors, grant wrappers, and listed OAuthLib modules.The example focuses on utils.py, device authorization, OAuth1, and OpenID Connect grant-type modules.
  • Solver Weakness Analysis: Historical traces identify four Solver weaknesses: scope conversion, constructor storage, OAuth1 parameter plumbing, and OIDC inheritance or delegation.These weaknesses involve generic normalization, swapped or transformed fields, semantically distinct string-like parameters, and incorrect replacement or forwarding of inherited behavior.
  • Bug Injection Playbook: The playbook injects localized bugs by disrupting scope conversions, swapping device-endpoint fields, confusing OAuth1 parameters, or dropping and misrouting inherited OIDC behavior.Examples include mishandling None or empty values, storing interval incorrectly, confusing nonce with timestamp, and overriding methods without preserving parent behavior.
  • Validation and Anti-Patterns: Candidates are validated with existing targeted pytest tests rather than synthetic tests or full-suite execution.Acceptance requires a cleanly applying bug patch, failing target tests, passing tests after reversal, stable repeated runs, and no oracle-patch leakage.
  • Validation and Anti-Patterns: The generator rejects tasks that alter tests, create syntax or import errors, require network access, affect unrelated modules, reveal patches, or produce unreproducible targeted-test failures.These constraints keep generated tasks localized and reproducible.
Loading 2606.07412v1…