Source-linked AI summary
Batching the Reflective Optimization Loop: Parallel Proposals Make GEPA Faster and Better
Jialin Zhang, Lakshya A Agrawal, Shangyin Tan, Donghyun Lee, Lutfi Eren Erdogan, Dan Klein, Koushik Sen, Alex Dimakis, Matei Zaharia
TL;DR
Reflective optimization can improve agentic systems, but GEPA’s sequential proposal-and-evaluation loop makes runs resource-intensive. This paper batches parallel proposals in GEPA; across two tasks, batched settings cut wall-clock time by about 3 to 4× and often matched or exceeded single mutation on held-out tests.
Problem
GEPA’s sequential optimization loop waits for each proposal and evaluation before starting the next, making runs resource-intensive and potentially taking hours.
Source: §1 · p. 2Method
Results
Batched settings cut wall-clock time by about 3 to 4×, and the best settings reached 72.1% versus 68.9% on LiveBench-Math and 60.0% versus 49.0% on HoVer held-out tests.
Source: §4.1 · p. 4Takeaways & Limitations
Batched optimization can make GEPA more practical and let users spend more budget in the same amount of time.
Source: §1 · p. 2, §6 · p. 6The runtime analysis assumes proposal acceptance rates stay constant across widths and independent proposal outcomes.
Source: §3 · p. 3
Abstract
from arXiv · showhide
Reflective text optimization makes agentic systems better, but the optimization loop is itself a resource-hungry workload whose sequential structure leaves parallel evaluation capacity idle. We extend the open-source GEPA optimizer to propose and evaluate a batch of candidates on each optimization step instead of one candidate at a time. In our sweep on two tasks, most batched runs finished in half the wall-clock time or less, and the fastest in about a quarter to a third. Batched settings also achieved higher held-out test scores, from 68.9% to 72.1% on LiveBench-Math (with $2\times2$) and from 49.0% to 60.0% on HoVer (with $8\times1$). A simple runtime model explains the measured speedups under a finite worker pool, and batched settings also spend their metric-call and dollar budgets more efficiently. We frame the axes along which a reflective optimization step can scale in parallel, a first look at allocating a fixed optimization budget across them.
1 Introduction
Reflective optimization can improve agentic systems, but GEPA’s sequential loop consumes wall-clock time while waiting for proposals and evaluations. The paper introduces batched parallel proposals to shorten optimization and improve held-out results.
1 Introduction
- SelectBestCandidate extends the current best, GEPA samples from a broader Pareto frontier one proposal at a time, and parallel proposals extend several frontier candidates at once.This broadens the search beyond a single best candidate or one-at-a-time frontier extensions.Source: Figure 2 · p. 2
- GEPA’s sequential iterations wait for each proposal and its evaluation before beginning the next, making runs take hours and consuming calls, dollars, and wall-clock time.Each iteration samples a parent, evaluates a mutation on a mini-batch, and evaluates it on the full validation set if it improves on its parent.Source: §1 · p. 2
- With batched parallel proposals, GEPA reduced wall-clock time by roughly 3 to 4× while raising held-out test scores from 68.9% to 72.1% on LiveBench-Math and 49.0% to 60.0% on HoVer.Source: §1 · p. 2
- Batched parallel proposals evaluate several candidates concurrently, allowing the same metric-call budget to finish sooner while exploring more of the search tree per step.The paper also frames parallel scaling axes as a potential area for future study.Source: §1 · p. 2
2 How parallel proposals work
Parallel proposals adapt GEPA to extend multiple frontier candidates and evaluate their mutations in batches. This reduces how often the search adapts to validation results, a property the paper connects to transfer beyond the validation set.
2 How parallel proposals work
- The authors relate parallel proposals to batch Bayesian optimization and argue that committing to more proposals simultaneously should transfer better beyond a fixed validation set.They cite evidence that repeatedly steering decisions with one fixed holdout can inflate its apparent performance.Source: §2 · p. 2
- Rather than extending one Pareto-frontier candidate at a time, parallel proposals sample several parents, draw multiple reflective mutations per parent, and score the proposals concurrently.This is the paper’s batched alternative to standard GEPA’s one-proposal step.Source: §2 · p. 2
- Each iteration samples P parents, draws N mini-batches per parent, dispatches P · N reflection requests concurrently, screens proposals on their mini-batches, then evaluates accepted candidates on the full validation set in parallel.The frontier is updated after full-validation evaluation.Source: §2 · p. 2, Figure 3 · p. 3
3 Runtime analysis
The runtime analysis models how batching changes iteration counts and latency under a fixed proposal budget. Its predicted gains are qualified by full-validation work, slower wide reflection steps, and finite worker capacity.
3 Runtime analysis
- Each iteration pays concurrent reflection and mini-batch evaluation latency, plus full-validation latency whenever at least one proposal beats its parent.Accepted proposals are evaluated on the validation set in parallel.Source: §3 · p. 3
- With the candidate acceptance rate held constant, a width-k run uses about 1/k as many iterations as single mutation for the same metric-call budget.The analysis assumes independent proposal outcomes with acceptance probability a; an iteration triggers full validation when one or more proposals beat their parent.Source: §3 · p. 3
- Wide steps can lose speed because reflection latency grows with the slowest concurrent call and validation can be limited by the worker pool.A validation stage may include up to k accepted candidates evaluated across V examples with W workers; the predicted curve uses the expected ak accepted candidates and holds Lstep at its measured single-mutation value.Source: §3 · p. 3
- For fixed W, evaluating a budget of B metric calls takes at least B · Te/W wall-clock time, while increasing the worker pool can lower this bound in workloads with expandable inference capacity.The bound is most relevant to system-bound workloads with genuinely fixed CPU or cluster capacity.Source: §3 · p. 3
4 Results
Across LiveBench-Math and HoVer, batched settings reduced runtime and often improved held-out scores. Results also show that performance depends on how parallel work is allocated and that batching can improve budget efficiency.
4.1 Batching cuts the wall-clock time and scores higher on held-out tests
- At 16-way parallelism, runs were about 3–4× faster; best settings scored 72.1% vs 68.9% on LiveBench-Math (2×2) and 60.0% vs 49.0% on HoVer (8×1).Most batched settings matched or beat single mutation on held-out tests.Source: Figure 4 · p. 4, §4.1 · p. 4
- On LiveBench-Math, 2×2 reduced iterations from 219 to 45, but full-validation-triggering iterations rose from 17% to 53%, yielding a 1.9× speedup (7.7 to 4.1 hours).The experiments used up to 512 concurrent workers on LiveBench-Math and 64 on HoVer.Source: §4.1 · p. 4, §4 · p. 4
4.2 Scaling parents (P) vs. mutations (N)
- With P held at 2, increasing N raised test scores from 68.6 at 2×1 to 71.6–72.1 at 2×2 through 2×8 on LiveBench-Math, and from 52.5 to 55.0 on HoVer.The highest-scoring split remained task-dependent: 2×2 on LiveBench-Math and 8×1 on HoVer; further ablations on other budgets and parallelisms were left for future work.Source: §4.2 · p. 4
4.3 Batched settings are more budget-efficient
- For both tasks, 2×4 dominated small metric-call budgets; on LiveBench-Math, single mutation needed about 2,000 calls to match its validation score, while 2×4 led on HoVer at every budget.The comparison was between 2×4 and single mutation on both tasks.Source: §4.3 · p. 5, Figure 5 · p. 5
- On held-out tests, 2×4 won 3.0pp over single mutation on LiveBench-Math and 6.0pp on HoVer; LiveBench-Math’s generalization gap was nine points for single mutation versus two for 2×4.On HoVer, both settings’ test scores exceeded their validation scores, and 2×4 finished higher on both.Source: §4.3 · p. 5
- 2×4 reached its final LiveBench-Math validation score within about $4 and passed single mutation’s final HoVer validation score within about $2.6; total spend stayed comparable at $12–$16.The spend measure combined solver and reflection calls.Source: §4.3 · p. 5
5 Related work
The related work situates parallel proposals among batch optimization and reflective optimization approaches. It also frames the paper’s spend-efficiency result as validation quality reached within the first few dollars.
5 Related work
- Batch Bayesian optimization commits to several evaluations per round, and the paper adapts that idea to reflective optimization.The paper relates its observed generalization behavior to adaptive data analysis and its budget allocation question to test-time compute scaling [Dwork et al., 2015, Blum and Hardt, 2015; Snell et al., 2024].Source: §5 · p. 5
- The batched setting reaches strong validation quality within the first few dollars of combined solver and reflection-call spend.Figure 6 plots best validation quality against total LLM spend.Source: Figure 6 · p. 6
6 Conclusion
The conclusion reports that parallel proposals shorten GEPA optimization while usually matching or improving held-out performance. It presents runtime modeling, budget efficiency, and parallel-scaling allocation as the paper’s main implications and open question.
6 Conclusion
- Across LiveBench-Math and HoVer, batched settings cut optimization wall-clock time by about 3–4×, with most matching or beating single mutation on held-out tests.A batched setting also reached strong validation quality with fewer metric calls and fewer dollars.Source: §6 · p. 6
- A simple runtime model predicts the measured speedups and identifies where the remaining runtime goes.Source: §6 · p. 6
- The open allocation question is how to distribute a fixed optimization budget across parents, mutations, reflection mini-batches, and reflection compute.The implementation is a plug-in sampling strategy built on the open-source gepa package.Source: §6 · p. 6
A Axes of parallel scaling
Reflective optimization can scale along four independently tunable dials, and the paper frames choosing among them as a fixed-budget allocation problem.
A Axes of parallel scaling
- P varies the number of programs selected for mutation per step, N the mutations per program, and ComBEE the reflection mini-batch size.Figure 7 depicts the first three scaling axes.Source: A Axes of parallel scaling · p. 7, Figure 7 · p. 8
- Reflection compute is a fourth axis, increased through stronger reflection programs that spend more compute to produce a higher-information mutation.Examples range from a single LM call to chain-of-thought, multi-step pipelines, and agentic workflows.Source: A Axes of parallel scaling · p. 7
- GEPA exposes four tunable dials for spending more optimization compute per step: P, N, reflection mini-batch, and reflection compute.The paper presents allocating a fixed optimization budget across these dials as an open question and a first look at that trade-off.Source: A Axes of parallel scaling · p. 7
B Task setup details
The experiments use LiveBench-Math and HoVer, with different tasks, evaluation setups, and metric-call budgets.
B Task setup details
- LiveBench-Math evaluates competition-math problem solving with LiveBench scorers on the Terrarium3 split of 100 training, 100 validation, and 168 test problems.Its budget is 5,000 metric calls, each a gpt-4.1-mini solution attempt.Source: B Task setup details · p. 8
- HoVer evaluates retrieval of Wikipedia pages needed to verify multi-hop claims using a four-hop gpt-4.1-mini agent with a query writer and note taker.The retrieval index contains 5.2 million 2017 Wikipedia abstracts; the three-hop claims are split into 200 training, 150 validation, and 200 test claims.Source: B Task setup details · p. 8
- HoVer optimization scores each rollout by top-5 recall of the claim’s three gold pages, while headline test numbers measure claims with all three pages retrieved.The budget is 3,000 metric calls, each a full program rollout, and one rollout makes about eight calls.Source: B Task setup details · p. 8
C Using parallel proposals
Parallel proposals are configured through GEPA’s sampling and selection strategies, while users control evaluation concurrency to match available capacity.
C Using parallel proposals
- The sampling strategy sets how many candidates are proposed per step, while the selection strategy chooses which improved candidates to keep.Two parents with two mutations each produce four candidates per step.Source: C Using parallel proposals · p. 8
- The implementation plugs into the open-source GEPA sampling-strategy interface, with an example using PxNSampling(p=2, n=2) and AllImprovements().The example also configures gpt-5-mini as the reflection LM.Source: C Using parallel proposals · p. 8
- GEPA calls the user-supplied evaluator in parallel by default, so max_concurrency should match evaluator and inference-provider capacity.System-bound evaluations may be limited by compute capacity; API-backed runs may use larger pools subject to rate limits and budget.Source: C Using parallel proposals · p. 9
- Users may provide a custom batch_evaluate function or choose concurrency and distributed-execution frameworks without changing GEPA itself.Other sampling and selection strategies may also be chosen or defined.Source: C Using parallel proposals · p. 9