PaperlayerView exact revision on arXiv

Source-linked AI summary

Failure-Guided Co-Evolution of Prompts and Training Data

Tianyu Yuan, Zhuzhong Qian

arXiv:2609.15209v1cs.SEcs.AI

TL;DR

Most APO systems keep training data fixed, limiting feedback to failures represented in those instances. FORGE uses failures to co-evolve prompts and training data through reusable failure modes and verified mutations. Across eight benchmarks it improves over the unoptimized baseline, and its synthesized data also improve other prompt optimizers and GRPO comparisons.

  • Problem

    Most general-purpose APO systems derive feedback from fixed training data, leaving the question of how to adapt training evidence as prompts evolve.

    Source: Introduction · p. 1

  • Method

    FORGE uses imperfect executions to revise prompts and abstract reusable failure modes that guide synthesis of verified positive, negative, boundary, and stress instances.

    Source: Introduction · p. 2

  • Results

    Across eight benchmarks, FORGE improves average performance by 16.52 points over the unoptimized baseline and outperforms competing APO methods by 4.92–8.82 points.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    FORGE's synthesized instances improve other prompt optimizers and GRPO comparisons, supporting the usefulness of failure-guided data beyond the loop that produced them.

    Source: Conclusion · p. 7

    Validation data may guide search and candidate selection but cannot supply training or synthesis instances; test scores are used only for reporting.

    Source: Problem Formulation · p. 3

Abstract

from arXiv · show

Automatic prompt optimization (APO) improves language-model programs by revising prompts from task feedback, yet it typically holds its training data fixed. Repeatedly optimizing against the same instances confines feedback to weaknesses already represented in those data, leaving related failure conditions unexplored. We therefore view each failure as a dual signal: it indicates both how the prompt should be revised and what new training evidence should be synthesized. We introduce FORGE, a failure-guided framework that co-evolves prompts and training data. FORGE abstracts imperfect executions into reusable failure modes and synthesizes new training data through four complementary mutation strategies. Verified instances are fed back into prompt search, allowing updated prompts to expose the next data needs. Across eight heterogeneous benchmarks, FORGE improves the aggregate score over the unoptimized baseline by 16.52 percentage points and outperforms all evaluated APO baselines. The synthesized data also transfer beyond FORGE: in a transfer study, they improve all nine APO comparisons by 2--9 points and all three GRPO comparisons by 4--8 points under matched optimization budgets. These results establish failures as a shared interface between prompt optimization and data synthesis, and show the benefit of jointly adapting what a model is instructed to do and what it learns from.

Introduction

Most general-purpose automatic prompt optimization (APO) systems keep training data fixed, even as prompts evolve and repeatedly expose weaknesses in the same instances. FORGE instead uses failures to guide both prompt revision and new training-data synthesis, improving results across benchmarks and transferring synthesized data to other optimizers.

Introduction

  • Most general-purpose APO systems derive feedback from fixed training data, while repeated optimization can leave nearby failure conditions unexplored.This motivates treating training data as an adaptive optimization state.Source: Introduction · p. 1
  • FORGE consistently outperforms compared methods across representative multi-hop retrieval, domain knowledge, reasoning, and agent benchmarks.Figure 1 summarizes results across these four task categories.Source: Figure 1 · p. 1
  • FORGE treats failures as signals for both prompt revision and data synthesis, using recurring weaknesses to guide the prompt–data loop.When aggregate validation performance stops improving, active failure modes guide synthesis.Source: Introduction · p. 2
  • Four mutation intents generate positive, negative, boundary, and stress examples, which a verifier checks before accepted instances enter training.These instances shape later prompt search, whose failures refresh the memory and guide subsequent synthesis.Source: Introduction · p. 2
  • 16.52 points over the unoptimized baseline: FORGE improves average performance across eight benchmarks and exceeds competing APO methods by 4.92–8.82 points.Its synthesized instances also improve all nine prompt-optimizer and all three GRPO comparisons.Source: Introduction · p. 2

Related Work

Prior work has optimized prompts, generated or selected training data, and in some cases adapted examples alongside prompts. FORGE is positioned as combining execution-failure-guided synthesis with discrete prompt reflection across heterogeneous language-model programs.

Automatic prompt and context optimization

  • Despite differing search strategies, many APO methods keep training data fixed, whereas FORGE lets failures expand the data used in subsequent search.The reviewed approaches include textual feedback, optimizer models, prompt evolution, and multistage program optimization.Source: Related Work · p. 2

Data-augmented prompt optimization

  • Data-augmented prompt systems adapt examples or evaluation artifacts, but FORGE targets recurring failures for tool-grounded synthesis and verification.The cited systems include joint prompt and living-test-set revision, difficulty-tiered synthesis, and failure-case augmentation.Source: Related Work · p. 2

Learner-aware training-data synthesis

  • Training-data synthesis ranges from learner-independent bootstrapping to methods that expand errors, summarize error clusters, or search for failure-inducing queries.Executable self-play also validates generated tasks through code execution or environment interaction.Source: Related Work · p. 2
  • FORGE couples execution-failure-based prompt reflection with data synthesis across heterogeneous programs, then tests whether its instances transfer to other prompt optimizers and GRPO.The related work describes this as distinct from synthesis focused on weight optimization alone.Source: Related Work · p. 2

Problem Formulation

The formulation contrasts fixed-data prompt search with adaptive-data optimization while retaining the same validation objective. The adaptive update appends accepted synthetic instances, while validation and test data remain outside training and synthesis.

Problem Formulation

  • An LM program has frozen model weights and optimizable module prompts; an execution is imperfect when its normalized score is below 1.Executing a task instance also produces a message trace and task feedback.Source: Problem Formulation · p. 2
  • Conventional fixed-data APO draws update minibatches from initial training data and keeps that data unchanged throughout search.Validation selects candidates but supplies no training or synthesis instances, and test scores are used only for reporting.Source: Problem Formulation · pp. 2–3
  • Adaptive-data APO retains the validation objective but appends accepted synthetic instances to the training multiset for later prompt updates.Existing training instances are neither removed nor modified; fixed-data APO is the special case with no accepted synthesis.Source: Problem Formulation · p. 3

Method

FORGE uses imperfect executions both to revise prompts and to build reusable failure modes for later synthesis. When validation improvement stalls, failure-guided mutations produce verified instances that re-enter training and expose new weaknesses.

Method

  • Each imperfect execution guides an immediate prompt revision and updates failure memory, so the same observed weakness informs both branches of the loop.Prompt search changes program behavior, while synthesis changes the evidence available to later updates.Source: Method · p. 3

Optimization Loop

  • A prompt proposal enters the candidate pool only if it improves the aggregate score on its minibatch, while validation scores guide parent selection and synthesis scheduling.Synthesis begins after p prompt-search steps without strict improvement in the best aggregate validation score; verified instances are then appended to training.Source: Optimization Loop · p. 3

Failure Memory

  • A failure mode abstracts a reusable weakness from imperfect executions while retaining supporting task context and trace evidence for grounded generation.For example, incomplete paginated search grouped failures involving different apps because both missed qualifying records beyond the default page.Source: Failure Memory · p. 4
  • Active modes come from failures since the previous synthesis event, and modes with more supporting observations receive more generation budget.Generation uses both a mode description and a supporting failed execution, linking the target weakness to a concrete task setting.Source: Failure Memory · p. 4, Failure-Guided Data Synthesis · p. 4

Failure-Guided Data Synthesis

  • Positive, negative, boundary, and stress mutations probe transfer, contrast, decision conditions, and robustness around a failure mode.Together, the operators define the scope and robustness of a correction rather than merely diversifying surface wording.Source: Failure-Guided Data Synthesis · pp. 4–5
  • In the HotPotQA example, the failure was a correct entity returned in a noncanonical form; mutations varied the required answer form and added misleading name matches.The example illustrates how the four mutation directions create distinct cases from one recorded weakness.Source: Failure-Guided Data Synthesis · pp. 4–5
  • Task-specific tools ground generation and verification in benchmark evidence, and only proposals valid for the task and faithful to the failure mode and mutation direction enter training.On HotPotQA, generator and verifier agents access the same full-Wikipedia ColBERT index as the task program.Source: Failure-Guided Data Synthesis · p. 5

Experiments

Experiments compare FORGE with prompt-optimization baselines across diverse benchmarks, test the contributions of failure-guided synthesis, and assess performance across task models and transfer to other optimizers.

Experiments

  • The evaluation asks whether Forge improves prompt optimization, whether failure-guided synthesis is necessary, and whether its instances benefit other optimizers, using eight heterogeneous benchmarks and matched search budgets.The benchmarks span retrieval, verifiable instructions, structured prediction, legal judgment, mathematics, broad knowledge reasoning, and interactive agents; comparisons use shared models, splits, program structure, and evaluators.Source: Experiments · p. 5, Experimental Setup · p. 5

Main Results

  • 72.18 aggregate with Qwen3.5-35B-A3B is 16.52 points above Baseline and 4.92–8.82 above competing APO methods; Forge leads six of eight benchmarks.The gains span single-module prediction, multi-hop retrieval, mathematical reasoning, and interactive tool use.Source: Main Results · p. 5, Table 1 · p. 6

Synthesis Accounting and Failure Dynamics

  • Failure memory shifts both mode identities and frequencies; the most-to-least frequent mode ratio falls across all four benchmarks, while mode counts rise on two and fall on two.Ratios change from 11:1 to 4:1 on AIME-2025, 20:1 to 6:1 on AppWorld, 22:1 to 11:1 on HotPotQA, and 8:1 to 4:1 on FiNER-139; the figure also accounts for synthesis outcomes.Source: Figure 3 · p. 6, Synthesis Accounting and Failure… · p. 6

Component Ablation

  • Prompt search raises mean Score from 55.66 to 64.17, random synthesis lowers it to 62.38, failure memory raises it to 68.24, and mutation guidance reaches 72.18.The ablation is cumulative, so the reported deltas are conditional contributions rather than isolated main effects.Source: Component Ablation · p. 5, Table 2 · p. 6

Generalization Across Task Models

  • 50.97 aggregate with dense Qwen3-8B exceeds Baseline by 14.10 points and Gepa by 5.38, with Forge leading six of eight benchmarks.The authors report that the advantage is consistent across dense and mixture-of-experts task models.Source: Generalization Across Task Models · p. 6, Table 3 · p. 7

Transferability of Synthesized Data

  • Forge augmentation improves all twelve dataset–optimizer pairs: nine prompt-optimization gains span 2–9 points, and GRPO gains are 8, 4, and 8 points on HotPotQA, FiNER-139, and LawBench.With the same 100-step GRPO budget, augmented-data training finishes above original-data training on all three test sets; validation gains emerge at different stages and need not be monotonic.Source: Table 4 · p. 7, Transferability of Synthesized Data · p. 7, Figure 4 · p. 7

Conclusion

FORGE uses failures to guide both prompt updates and targeted data synthesis, with synthesized examples also transferring to other optimizers. The authors conclude that prompt search and data construction should be treated as a coupled process.

Conclusion

  • FORGE outperforms the unoptimized baseline and evaluated APO methods across eight benchmarks, with ablations indicating gains require failure- and mutation-guided rather than random synthesis.The conclusion reports no aggregate score for this comparison.Source: Conclusion · p. 7
  • FORGE-generated instances improve three competing prompt optimizers and GRPO across three transfer benchmarks, showing usefulness beyond the optimization loop that produced them.The authors call for broader evaluation across tasks and model families, and for disentangling dataset size, example selection, and sampling-distribution effects.Source: Conclusion · p. 7
  • The authors argue that observed failures reveal both how prompts should change and which training examples should be added next.Source: Conclusion · p. 7

A Algorithm and Implementation Details

The appendix specifies the operations and model-facing contracts behind FORGE’s prompt–data loop, along with its benchmark split and comparison setup. It details how candidates, minibatches, failure memory, prompt revisions, and verified synthetic instances are managed.

A Algorithm and Implementation Details

  • The implementation instantiates six loop operations, while maintaining a candidate pool, mutable training data, per-instance validation scores, and epoch-partitioned failure memory.Only failure modes collected since the preceding synthesis event are active for the next batch.Source: A Algorithm and Implementation Details · p. 11, §A.1 · p. 11

A.2 Candidate Selection

  • Candidate selection excludes candidates dominated across validation instances, then samples uniformly from the multiset of remaining per-instance front members.A candidate leading on more validation instances appears more often in that multiset.Source: §A.2 · p. 11

A.3 Minibatch Sampling

  • Minibatches follow shuffled training-instance orders; after synthesis, new instances and unconsumed examples are placed ahead of already-consumed examples.Only the final minibatch may be padded with duplicate identifiers when the dataset size is not divisible by the batch size.Source: §A.3 · p. 11

A.4 Failure-Memory Construction

  • Imperfect executions update failure memory through assign, create, or rename actions, and the next active catalogue starts empty after synthesis while stored history is retained.The assigner sees the current-epoch mode catalogue and a structured rendering of the new observation.Source: §A.4 · pp. 11–12

A.5 Reflective Prompt Update

  • Prompt reflection proposes a complete replacement for one selected module; it is screened against the parent on the same minibatch and accepted only if its total score is strictly higher.Unselected module prompts are copied from the parent, and accepted proposals are evaluated on the full validation set.Source: §A.5 · pp. 12–14

A.6 Failure-Guided Synthesis and Verification

  • Synthesis allocates a fixed attempt budget across active failure modes, cycles through four mutation intents, and admits only parsed instances that pass independent validity and faithfulness checks.The generator receives a task signature and mutation seed; the verifier receives the parsed instance and flattened context, but not the source trace or hidden generator reasoning.Source: §A.5 · p. 14, §A.6 · p. 14

B Experimental Setup

  • The experimental setup specifies benchmark data, evaluators, models, budgets, and a separate paired GRPO transfer protocol; comparisons use disjoint train, validation, and test partitions unless a benchmark fixes its split.Source: B Experimental Setup · p. 14, §B.1 · p. 14

B.2 Benchmark Evaluation, Tools, and Synthesis

The evaluation uses shared benchmark protocols and reports test performance across eight tasks, with specified model configurations and aligned search-effort ceilings. The budget protocol aligns task-program evaluations rather than exact realized compute.

B.2 Benchmark Evaluation, Tools, and Synthesis

  • Test percentages are means of per-example scores, and the aggregate is the unweighted mean of the eight benchmark percentages; validation selects optimized states while test outcomes are reporting-only.Methods share ordered train, validation, and test examples for each benchmark and task model, and use the same adapter score for a given execution.Source: §B.2 · p. 14

B.3 Model Configuration

  • The primary task model is Qwen3.5-35B-A3B, with Qwen3-8B used for cross-architecture comparison; task-model weights remain fixed across methods.GPT-5.5 is the optimizer model, and all methods share the stated Qwen inference settings, LM-program structure, and evaluator.Source: §B.3 · p. 17

B.4 Optimization-Budget Alignment

  • Search budgets use completed per-example task-program evaluations as a common ceiling, but do not ensure exact realized compute alignment.GEPA may overshoot at iteration boundaries, MIPROv2 estimates exclude proposal-stage calls, and ACE may stop early under its native rule.Source: §B.4 · p. 18

B.5 GRPO Transfer Protocol

The section reports a matched-budget GRPO transfer protocol and optimization artifacts, including an AppWorld prompt improvement and a HoVer candidate lineage. The artifacts illustrate stored states rather than additional evaluations.

B.5 GRPO Transfer Protocol

  • Both GRPO arms used the same checkpoint, validation and test files, seed, and 100 optimizer steps, differing only in training-set composition.The original-data arm used the original split; the augmented arm appended fixed verifier-admitted Forge examples.Source: §B.5 · p. 18
  • HotPotQA GRPO used continuous search-agent trajectories, regenerated a 64-completion batch per step, and evaluated the step-100 test endpoint using answer EM.The retriever corpus and top-seven interface were fixed across arms and checkpoints.Source: §B.5 · p. 18

C Optimization Artifacts

  • The optimization-artifact figures show stored prompts, candidate lineage, and admitted examples; they are illustrations, not additional evaluations.The longer optimized AppWorld prompt spans three figures.Source: C Optimization Artifacts · p. 18, §C.1 · p. 18

C.1 AppWorld Prompt Evolution

  • AppWorld’s aggregate validation score rose from 60.20% to 92.50% between seed candidate 0 and evaluated candidate 25.The run optimized the system prompt of a single appworld_agent module; Figures 9–12 reproduce the seed and optimized prompts.Source: §C.1 · p. 18, Figure 9 · p. 19

C.2 HoVer Candidate Lineage

  • HoVer’s candidate tree marks candidate 11 and its ancestry for test evaluation; its ancestry need not improve monotonically because admission uses minibatch scores.Node labels report full-validation-set means, while admission compares a parent and proposal on the current minibatch.Source: §C.2 · p. 19

C.3 Admitted Synthetic Examples

Figures 14–16 present verifier-admitted synthetic examples from non-AppWorld benchmarks. AppWorld examples are not shown as self-contained JSON cards because their task records depend on a larger executable environment.

C.3 Admitted Synthetic Examples

  • Figures 14–16 show one admitted synthetic record from each non-AppWorld benchmark’s run.The examples cover AIME and FiNER-139, HotPotQA and HoVer, and IFBench, LawBench, and MMLU-Pro.Source: §C.3 · p. 19, Figure 14 · p. 23, Figure 15 · p. 24
  • Each displayed record completed generation and verification and was admitted only after validity and faithfulness decisions were both true.The cards provide qualitative evidence, not an independent human correctness audit.Source: §C.3 · p. 19
  • AppWorld’s admitted record resolves through a task directory containing environment state, executable source, evaluator, and tests, so no equivalent self-contained JSON card is shown.Showing only the task identifier or instruction would omit the environment and executable supervision defining the example.Source: §C.3 · p. 19

D.1 Optimizer-Model Ablation

With Qwen3.5-35B-A3B fixed as the task model, the study compares Forge’s test performance across six benchmarks using two optimizer models. Score differences were small on average and at most 3.93 points.

D.1 Optimizer-Model Ablation

  • The ablation used DeepSeek-V4-Pro and GPT-5.5 as alternative optimizer models while keeping Qwen3.5-35B-A3B as the task model.For each benchmark, it reports test performance of the prompt selected on validation.Source: §D.1 · p. 26
  • Across six benchmarks, changing the optimizer model produced a 1.66-point average absolute score difference and a 3.93-point maximum difference.Table 7 reports test performance with Qwen3.5-35B-A3B fixed as the task model.Source: Table 7 · p. 26, §D.1 · p. 26
  • The authors report similar optimization effectiveness across the two optimizer models and characterize Forge as largely insensitive to optimizer-model choice.Source: §D.1 · p. 26

D.2 Synthetic-Data Leakage Audit

A two-stage audit screened generated examples for semantic overlap with fixed validation and test partitions. Neither automated screening nor manual inspection found a match.

D.2 Synthetic-Data Leakage Audit

  • Codex with GPT-5.5 screened every generated example against the validation and test partitions, followed by manual inspection of a random 10% sample.Source: §D.2 · p. 26
  • The audit counted a match only when questions were semantically equivalent despite paraphrasing or formatting; topical or reasoning-pattern similarity alone did not qualify.Source: §D.2 · p. 26
  • Neither audit stage found a match, providing no evidence of data leakage.Source: §D.2 · p. 26

Related papers

Loading 2609.15209v1…