PaperlayerView exact revision on arXiv

Source-linked AI summary

EvoSyn: Generalizable Evolutionary Data Synthesis for Verifiable Learning

He Du, Bowen Li, Aijun Yang, Siyang He, Qipeng Guo, Dacheng Tao

arXiv:2510.17928v1cs.LGcs.AIcs.NE

TL;DR

Reliable verifiable data is costly to annotate, while synthetic data remains difficult to make reliable and generalizable. EvoSyn evolves a task-agnostic filtering strategy from human-verified seed data and uses it to synthesize executably-checkable training data. The paper reports gains on LiveCodeBench and AgentBench-OS, including distilled students surpassing their teacher on agentic tasks.

  • Problem

    Reliable verifiable data is costly to annotate, and synthetic data remains difficult to make reliable and generalizable beyond task-specific heuristics.

    Source: §1 · pp. 1–2

  • Method

    EvoSyn evolves a filtering strategy from human-verified seed data using a consistency-based evaluator, then applies the strategy to synthesize and filter executably-checkable data.

    Source: §1 · pp. 2–3, §3.1 · p. 4

  • Results

    EvoSyn-filtered data yields substantial gains and superior learning dynamics across LiveCodeBench RLVR and AgentBench-OS distillation, with distilled students surpassing their teacher on complex multi-turn agentic tasks.

    Source: §5 · p. 9

  • Takeaways & Limitations

    EvoSyn provides a task-agnostic route to assembling verifiable training instances from minimal seed supervision for use across domains.

    Source: §1 · p. 2, §5 · p. 9

    Synthesis scalability is constrained by low output diversity, which increases execution costs, as well as slow inference, time-intensive unit-test verification, and costly environment setup.

    Source: §4.2 · p. 7

Abstract

from arXiv · show

Reliable verifiable data has become a key driver of capability gains in modern language models, enabling stable reinforcement learning with verifiable rewards and effective distillation that transfers competence across math, coding, and agentic tasks. Yet constructing generalizable synthetic verifiable data remains difficult due to hallucination-prone generation, and weak or trivial verification artifacts that fail to separate strong from weak solutions. Existing approaches often rely on task-specific heuristics or post-hoc filters that do not transfer across domains and lack a principled, universal evaluator of verifiability. In this work, we introduce an evolutionary, task-agnostic, strategy-guided, executably-checkable data synthesis framework that, from minimal seed supervision, jointly synthesizes problems, diverse candidate solutions, and verification artifacts, and iteratively discovers strategies via a consistency-based evaluator that enforces agreement between human-annotated and strategy-induced checks. This pipeline upgrades filtering into principled synthesis: it reliably assembles coherent, verifiable training instances and generalizes without domain-specific rules. Our experiments demonstrate the effectiveness of the proposed approach under both RLVR and model distillation training paradigms. The results show that training with our synthesized data yields significant improvements on both the LiveCodeBench and AgentBench-OS tasks, highlighting the robust generalization of our framework.

1 INTRODUCTION

Reliable verifiable data is costly to annotate, while synthetic data remains difficult to make reliable and generalizable across tasks. EvoSyn evolves a filtering strategy from human-verified seed data and applies it to synthesis; the paper reports gains in both RLVR and distillation.

1 INTRODUCTION

  • Synthetic-data reliability and generalizability remain challenges: hallucinations undermine reliability, while task-specific handcrafted heuristics often fail beyond their intended domains.Reliably annotated verifiable data is costly, and large-scale manual labeling is described as infeasible.Source: §1 · pp. 1–2
  • EvoSyn evolves a filtering strategy using consistency with a small set of manually verified seed data, then applies it to synthesize reliable data without handcrafted task-specific heuristics.The framework targets executably-checkable tasks, where tests can verify a solution without requiring a complete solution.Source: §1 · pp. 2–3
  • The paper studies strategy evolution and its cost trade-offs alongside training results.It reports a detailed study of EvoSyn’s evolutionary process and validates the framework under RLVR and distillation.Source: §1 · p. 3
  • EvoSyn-generated data yields substantial improvements over baselines in both RLVR and model distillation.The paper validates the approach on LiveCodeBench and AgentBench-OS.Source: §1 · p. 3

2 RELATED WORK

Prior work uses verifiable feedback across multiple training paradigms, but constructing reliable synthetic data remains difficult and often depends on costly, task-specific techniques.

2 RELATED WORK

3 METHODOLOGY

EvoSyn derives a filtering strategy through evolutionary search, uses it to synthesize and filter problems, solutions, and tests, and trains models on the retained data.

3 METHODOLOGY

  • EvoSyn targets executably-checkable tasks and comprises strategy derivation, data synthesis and filtering, and model training.Its target tasks permit testing artifacts to determine correctness without first producing a correct solution.Source: §3 · p. 3

3.1 DERIVING DATA FILTERING STRATEGY

  • A filtering strategy ranks candidate solutions and tests generated from human-annotated seed problems; its quality is judged by agreement with annotated tests across candidate solutions.The evaluator requires the top-ranked solution to pass the annotated tests and checks that best and worst solutions behave consistently across annotated and strategy-selected tests.Source: §3.1 · p. 4
  • MAP-Elites with island-based populations searches user-defined strategy features while preserving population diversity, and the evaluator scores the share of seed instances meeting both criteria.The evolved best strategy scores solutions by passed-test count and tests by their power to separate passing from failing solutions.Source: §3.1 · p. 4
  • Evolutionary search discovers effective strategies that improve on manually designed baselines.The paper reports that evolved strategies demonstrate strong internal logic.Source: §3.1 · p. 5

3.2 DATA SYNTHESIS AND FILTERING

  • After deduplication, each synthesized problem receives candidate solutions and tests; Zero-Variance Pruning discards instances whose tests produce no ranking variation.The paper treats such cases as unreliable tests or trivial problems where tests perform equally well.Source: §3.2 · p. 5

3.3 MODEL TRAINING

  • The retained dataset contains problem descriptions and tests, with strongest generated solutions also kept, and can support RLVR or model distillation.Source: §3.3 · p. 5

4 EXPERIMENTS

Across coding and agent-task experiments, EvoSyn-filtered data improves training results, while the evolutionary process discovers stronger filtering strategies. The experiments also examine verification criteria, synthesis cost, and data quality.

4.2 EVOLUTIONARY PROCESS

  • On LiveCodeBench, the best evolved strategy surpasses initialization by over 10 percentage points within 20 iterations.Figure 4a's score is the ratio of seed instances satisfying consistency verification.Source: Figure 4 · p. 6, §4.2 · p. 6
  • Increasing M and N yields more usable instances but raises testing execution cost as O(MN).The study reports a log-linear relationship between usable instances and testing executions, alongside practical inference, verification, and environment-setup bottlenecks.Source: §4.2 · p. 7
  • Adding Criterion-1 to the K=1 consistency check is stricter and requires 8× fewer executions than omitting it at K=8.The table reports #Exec=4 versus 32; increasing K alone shows diminishing returns.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 EVOSYN FOR RLVR

  • The RLVR experiments use 51 seed instances; sampled solutions were often trivial or identical, motivating Zero-Variance Pruning.The observed solutions averaged about a dozen code lines, and trivial problems can prevent proper advantage computation in RLVR.Source: §4.3 · pp. 7, 9
  • Synthesized RLVR data averages 11.5 unit tests per instance, including edge cases; generating code from which tests are constructed preserves diversity while avoiding direct test generation.The adjustment also keeps the number of tests per problem sufficiently large.Source: §4.3 · pp. 7–8
  • EvoSyn-filtered data improves LiveCodeBench accuracy across models and outperforms random selection and the relaxed variant.The authors report a 14.1% improvement on Llama-3.1-8B.Source: Table 2 · p. 8, §4.3 · p. 8

4.4 EVOSYN FOR MODEL DISTILLATION

  • EvoSyn-filtered data yields large AgentBench-OS gains across student models, outperforming random selection; all students exceed the teacher's 30.1 score.The experiment synthesizes 673 instances and uses DeepSeek-R1 outputs for training.Source: Table 3 · p. 9, §4.4 · p. 9
  • The distillation experiment uses a curated 129-instance subset of AgentBench-OS as both seed data and evaluation set.The original 144 instances were filtered to remove samples with stringent time requirements or permission issues.Source: §4.4 · p. 9

5 CONCLUSION

EvoSyn evolves filtering strategies from minimal seed supervision to synthesize verifiable training data. The paper reports gains across coding and agentic tasks, alongside execution-cost and output-diversity limitations.

  • EvoSyn-filtered data yields substantial gains on LiveCodeBench and AgentBench-OS, with distilled students surpassing the teacher on complex multi-turn agentic tasks.Source: §5 · p. 9
  • EvoSyn uses a consistency-based evaluator to evolve robust filtering strategies from minimal seed supervision for executably-checkable tasks.Source: §5 · p. 9
  • The framework assembles coherent, verifiable training instances that transfer across domains by turning ad hoc filtering into strategy optimization.Source: §5 · p. 9
  • Ablations support strategy evolution and Zero-Variance Pruning, while the study characterizes cost–quality trade-offs in M ∗N execution.Source: §5 · p. 9
  • Verification and execution cost and output-diversity bottlenecks remain; future work targets larger population search, diversity-aware generation, and broader tooling and domains.Source: §5 · p. 9

A.1 PROMPTS

The appendix provides the synthesis prompts for LiveCodeBench and AgentBench-OS, with separate prompts for the two AgentBench-OS response types.

A.2 UNIT-TEST COUNT DISTRIBUTION

The synthesized problems carry a broad distribution of unit tests, including long-input edge cases, with a reported mean of 11.5 tests per problem.

  • 11.5 unit tests per problem is the reported mean for the synthesized-problem distribution.Source: §A.2 · p. 14, Figure 9 · p. 14
  • The broad test-count distribution includes both standard checks and long-input edge-case tests.Source: §A.2 · p. 14, Figure 9 · p. 14
  • The authors describe heterogeneous test coverage and difficulty as helping produce more stable, discriminative reward signals for ranking solutions and selecting tests.Source: §A.2 · p. 14

A.3 OPTIMIZED STRATEGIES

The appendix outlines five evolved scoring strategies that seek to improve discriminative power and solution-ranking consistency on seed data.

  • The TF-IDF-like approach gives higher scores to solutions passing difficult tests, defined as tests passed by few solutions.Source: §A.3 · p. 14, Figure 10 · p. 15
  • The coverage-based approach rewards solutions for passing more tests and measures test quality by the score gap between passers and failures.Source: §A.3 · p. 14, Figure 11 · p. 15
  • Inverse filtering treats tests passed by fewer solutions as better, reversing the initial strategy.Source: §A.3 · p. 14, Figure 12 · p. 15
  • Exclusion-based attribution measures a test's contribution without its own influence by weighting solutions that pass all other tests.Source: §A.3 · p. 14, Figure 13 · p. 16
  • Hardness-Aware ranking combines test strictness and pass count, penalizing all-or-none tests to select strong solutions and discriminative tests.Source: §A.3 · p. 14, Figure 14 · p. 16

A.4 TRIVIAL PROBLEM

Relaxing the final filtering condition adds 25 instances, nearly all overly simple; one example asks whether a set of numbers are all even.

A.4 TRIVIAL PROBLEM

  • 25 additional instances appear in DEvoSynrelaxed but not DEvoSyn, and nearly all were overly simple problems.Source: §A.4 · p. 16
  • The example problem asks whether some numbers are all even.Source: §A.4 · p. 16
  • Figure 15 shows the example's question description and solution code.Source: Figure 15 · p. 17

A.5 AI USAGE STATEMENT

AI tools assisted only with manuscript writing and polishing; the authors conceived, conducted, and validated the core research content.

A.5 AI USAGE STATEMENT

  • AI tools were used solely to assist with writing and polishing the main manuscript text.Source: §A.5 · p. 18
  • The authors conceived, conducted, and validated all core research content.Source: §A.5 · p. 18
  • No AI systems were involved in generating ideas, designing or running experiments, or producing core research results.Source: §A.5 · p. 18
Loading 2510.17928v1…