DUET: Co-Evolving Solver and Grader Agents

1 INTRODUCTION

• As a solver improves, its plausible errors and finer quality differences can become harder to evaluate, especially when open-ended tasks allow multiple valid outcomes.For these tasks, evaluation may depend on the execution process and resulting artifacts, not just a reference answer.Source: §1 · p. 1
• Existing optimization methods typically keep the grader fixed, risking less informative feedback when the solver develops new failure modes or finer distinctions.The introduction argues that effective optimization may therefore require the grader to evolve with the solver.Source: §1 · p. 2
• DUET selects training tasks, runs the solver, and has the grader score and critique outcomes before alternating updates to either agent.Its design includes adaptive task selection and bounded component updates.Source: §1 · p. 2, Figure 1 · p. 2
• Across four benchmarks, DUET improves both solver and grader performance and outperforms baselines that optimize the solver with a fixed grader.The reported results suggest DUET can reduce reliance on carefully designed initial graders and reference answers.Source: §1 · p. 2

Agent Optimization

• Agent optimization methods such as GEPA and SkillOpt typically keep the evaluator fixed, whereas DUET treats the grader itself as an optimization target.The cited methods use execution trajectories and evaluator feedback to update prompts or skills; DUET lets task execution and evaluation evolve together.Source: §2 · p. 3

LLM-Based Evaluation

• Agentic evaluators can inspect trajectories, actions, environment states, and artifacts, but designing their criteria and instructions can require substantial manual effort.DUET allows the grader to develop its evaluation policy from a minimal initialization.Source: §2 · p. 3

Co-Evolution of Task Solving and Evaluation

• Unlike ECHO, DUET jointly optimizes the solver and the grader that directly supplies evaluation feedback, without a separate external reward model.Unlike CoEvoSkills’ executable tests, DUET’s grader can assess task-quality aspects not captured by tests alone.Source: §2 · p. 3

3 PROBLEM FORMULATION

• The solver is parameterized by an optimizable component and produces an outcome containing its execution trajectory and resulting outputs or artifacts.The formulation denotes the task distribution by D, tasks by x, and solver component by θ.Source: §3 · pp. 3–4
• The grader takes a task and solver outcome and returns a scalar score and natural-language critique, with its own optimizable component.The formulation denotes the grader component by ϕ; the underlying models, tools, and environments are fixed.Source: §3 · p. 4
• DUET uses reference-free training updates because open-ended tasks can have multiple valid outcomes and gold references or labels can be costly and incomplete.In the default setting, a separate labeled validation set is used only for checkpoint selection and early stopping; a fixed-budget variant omits validation-based selection and stopping.Source: §3 · p. 4
• The optimization seeks higher-quality solver outcomes and grader evaluations that faithfully reflect outcome quality and provide informative critiques, despite unavailable true quality.As solver outcomes change, the ideal grader also changes; the formulation targets joint evolution without true-quality access during training.Source: §3 · p. 4

4 DUET ALGORITHM

• DUET selects tasks, runs the solver, evaluates outcomes, alternates agent updates, and records each round’s evidence and components in training history.The procedure evaluates updated agents on a separate labeled validation set and tracks their best checkpoints independently.Source: §4 · p. 4, §4.3 · p. 6

4.1 STEP 1: ADAPTIVE TASK SELECTION

• DUET prioritizes tasks with larger recent grader-score changes while retaining exploration, using a sliding window of score differences to estimate which tasks remain informative.It first selects tasks round-robin until each has been executed twice.Source: §4.1 · pp. 4–5

4.2 STEP 2: SOLVER EXECUTION AND GRADER EVALUATION

• For each selected task, the solver produces an outcome that includes its execution trajectory and artifacts, and the grader returns a normalized score with evidence-grounded feedback.Source: §4.2 · p. 5

4.3 STEP 3: ALTERNATING AND BOUNDED UPDATES

• DUET updates only one agent per round—grader on odd rounds and solver on even rounds—so the other agent remains fixed during each revision.The update agent uses round evidence and training history; solver updates draw on grader feedback verified against artifacts, while grader updates examine reports and artifacts.Source: §4.3 · p. 5
• An edit budget limits discrete changes per update and shrinks over optimization, allowing larger early revisions and smaller later refinements.The update agent is instructed to make the smallest evidence-supported revision or leave the component unchanged, while discouraging task-specific rules.Source: §4.3 · p. 6
• DUET tracks the best solver and grader checkpoints separately on a labeled validation set and stops when neither validation metric improves for a fixed number of rounds.The paper also reports comparable performance with a fixed training budget and without validation-based checkpoint selection or early stopping.Source: §4.3 · p. 6

5 EXPERIMENTS

• The experiments span spreadsheet manipulation, multi-turn tool use, and retrieval-augmented question answering across four benchmarks.The main setup uses Claude Opus 4.8 and empty prompts, except for a minimal protocol prompt for the solver on τ 2-Bench; model weights, tools, and environments remain fixed.Source: §5 · pp. 6–7

5.1 UNDERSTANDING THE PERFORMANCE OF DUET

• 96.5% rubric score on Finance Agent Benchmark and 89.4% success rate on τ 2-Bench: DUET outperformed fixed-grader methods, including oracle-grader variants.On Finance Agent Benchmark, SkillOpt (G⋆) reached 95.7% and GEPA (G⋆) 95.9%; on τ 2-Bench, their oracle variants reached 83.9% and 82.0%.Source: §5.1 · p. 8, Table 1 · p. 8
• Grader pairwise agreement rose from 67.9% to 75.0% on Finance Agent Benchmark and from 68.4% to 78.5% on GDPval, but changed only slightly on two binary-label benchmarks.Agreement moved from 88.7% to 89.3% on τ 2-Bench and from 71.9% to 72.8% on SpreadsheetBench; the authors note that these comparisons capture coarse pass–fail differences.Source: §5.1 · p. 8

5.2 UNDERSTANDING THE PROPERTIES OF DUET

• Joint prompt-and-skill optimization reached 91.9% success on τ 2-Bench; DUET also retained strong performance across evaluated underlying models.The joint approach exceeded prompt-only DUET (89.4%), SkillOpt (83.3%), and GEPA (84.4%); tests with Claude Sonnet 4.6 and GPT-5.6 Sol also reported solver or grader improvements.Source: §5.2 · p. 9, Table 2 · p. 9, Table 3 · p. 9
• DUET improved solver and grader performance across empty, informed, and adversarial prompt initializations, raising the adversarial solver from 91.1% to 95.8%.The first update removed harmful adversarial instructions, while the final prompt retained about 60% of the informed solver prompt.Source: §5.2 · p. 9, Table 4 · p. 9

5.3 VALIDATING THE DESIGN OF DUET

• On Finance Agent Benchmark, co-evolving solver and grader reached 96.5% solver score and 75.0% grader pairwise agreement, the strongest performance among the compared ablations.With the solver fixed, grader optimization moved pairwise agreement only from 67.9% to 68.8%; the text connects grader feedback to continued solver improvement.Source: §5.3 · pp. 9–10, Figure 3 · p. 9
• Adaptive task selection raised solver performance from 95.9% to 96.5% and grader pairwise agreement from 62.7% to 75.0% on Finance Agent Benchmark.The authors suggest that revisiting tasks as evaluations change provides more useful training signals.Source: Figure 3 · p. 9, §5.3 · p. 10
• Alternating updates raised solver performance from 95.4% to 96.5% and grader pairwise agreement from 70.1% to 75.0% on Finance Agent Benchmark.Updating one agent at a time makes subsequent performance changes easier to attribute than updating both simultaneously.Source: Figure 3 · p. 9, §5.3 · p. 10

6 CONCLUSION

• Across four benchmarks, DUET improves both solver and grader performance and outperforms baselines that optimize the solver with a fixed grader.The authors note that benchmark reference evaluators, rather than human judgments, were used to assess agents because large-scale human evaluation is costly.Source: §6 · p. 10

A ADDITIONAL RELATED WORK

• DUET makes the grader an optimization target, unlike prompt- and skill-optimization work that typically keeps its evaluator fixed, including GEPA and SkillOpt.GEPA reflects on execution trajectories and evaluator feedback; SkillOpt accepts skill edits when held-out validation performance improves.Source: A ADDITIONAL RELATED WORK · p. 16
• DUET’s grader can develop its evaluation policy from a minimal initialization, reducing reliance on manually specifying criteria, rubrics, references, and judging instructions.The related work notes that such manual design can be demanding, especially when relevant failure modes are hard to anticipate.Source: A ADDITIONAL RELATED WORK · p. 16
• Unlike ECHO, DUET jointly optimizes the solver and the grader that directly supplies its evaluation signal without a separate external reward model.ECHO co-evolves a policy and critic using an external outcome reward to train the policy and assess critic feedback.Source: A ADDITIONAL RELATED WORK · p. 16
• Unlike CoEvoSkills’ skill generator and surrogate verifier, DUET’s grader can assess task-quality aspects not captured by executable tests alone.CoEvoSkills uses its verifier to provide executable tests and diagnostic feedback.Source: A ADDITIONAL RELATED WORK · p. 16

B.1 ADDITIONAL DETAILS OF ADAPTIVE TASK SELECTION

• A round-robin warm-up executes every training task at least twice, ensuring each has a score difference before adaptive selection begins.Source: §B.1 · p. 17
• After warm-up, DUET prioritizes tasks using SW-UCB: recent average score change favors changing evaluations, while the confidence term favors fewer observations and revisits unseen tasks.Score changes are associated with the later execution and added to task history after selected batches run.Source: §B.1 · p. 17

B.2 UPDATER SYSTEM PROMPTS

• The solver updater grounds revisions in real run artifacts, infers durable habits rather than symptoms, and compares the same batch across epochs rather than scores from different graders.Source: §B.2 · p. 18
• The grader updater similarly inspects artifacts, infers general rubric or calibration mechanisms, weighs same-batch history, and installs the smallest evidence-supported general change.Both role-specific prompts caution against overfitting and prompt bloat.Source: §B.2 · pp. 18–19

B.3 BOUNDED UPDATE DETAILS

• A cosine schedule decreases each agent’s edit budget from βmax to βmin; oversized revisions get one reduction attempt and are rejected if they still exceed 2βj edits.The first update for each agent is exempt from the edit limit; the stated budget endpoints are βmax = 4 and βmin = 2.Source: §B.3 · p. 19

• Experiments used one Azure VM with an Intel Xeon Platinum 8370C CPU, 64 GB RAM, and Windows 11 Enterprise; spreadsheet runs used local Excel.DUET took approximately 15 hours on Finance Agent Benchmark, with runtime varying by benchmark task complexity and interaction length.Source: §C.1 · p. 20

C.2 BENCHMARKS

• SpreadsheetBench uses 400 expert-verified tasks, split into 60 training, 20 validation, and 320 test tasks; reported test accuracy covers 315 evaluable tasks.A task succeeds only when all evaluated cells match the reference workbook.Source: §C.2 · p. 20
• GDPval uses 60 Excel-deliverable tasks across nine sectors, split into 8 training, 3 validation, and 49 test tasks, evaluated by pairwise win rate against human gold deliverables.Twenty tasks require creating a workbook from scratch.Source: §C.2 · p. 20
• τ 2-Bench uses 150 airline, retail, and telecom tasks, split into 22 training, 8 validation, and 120 test tasks, evaluated by task success rate.Success requires all specified components, including database state, actions, communicated information, and natural-language assertions.Source: §C.2 · p. 20
• Finance Agent Benchmark uses 126 finance QA questions, split into 20 training, 6 validation, and 100 test tasks, scored by an LLM judge across seven rubric dimensions.The final score is the mean over each task’s dimensions.Source: §C.2 · p. 20

C.3 AGENT CONFIGURATIONS

• Main experiments use Claude Opus 4.8 for both agents; prompts start empty except that the τ 2-Bench solver receives a minimal required protocol prompt.For τ 2-Bench, the grader prompt starts empty; prompts for both agents start empty on the other three benchmarks.Source: §C.3 · p. 20

C.4 EVALUATION DETAILS

• Solver performance uses each benchmark’s held-out test split and reference evaluator, with benchmark-specific metrics for spreadsheet accuracy, τ 2-Bench success rate, GDPval win rate, and Finance rubric score.SpreadsheetBench correctness requires every designated answer-region cell to match the reference workbook.Source: §C.4 · pp. 20–21
• Held-out grader pairs use benchmark reference judgments or native binary outcomes, with pair selection designed to expose misleading quality cues and hide model identity and native labels.GDPval pairs contrast workbook quality with apparent completeness; Finance pairs contrast quality and groundedness with citation quantity.Source: §C.4 · p. 21
• Grader performance is mean pairwise agreement, scoring 1 for ranking the reference-preferred output higher, 0.5 for a tie, and 0 for an inversion.The grader sees outputs individually and does not see pair labels.Source: §C.4 · p. 22

C.5 BASELINE DETAILS

• SkillOpt and GEPA optimize only the solver while holding the grader fixed; comparisons include reference-free G0 and benchmark reference-evaluator settings.All methods share initialization, task splits, and held-out test evaluator; DUET jointly optimizes solver and grader in the reference-free setting.Source: §C.5 · p. 22

C.6 HYPERPARAMETERS AND TRAINING SETTINGS

• Training settings use the same exploration coefficient c = 0.01 and stopping patience p = 10 across benchmarks, with bounded-update parameters βmax = 4 and βmin = 2.Batch size, SW-UCB window size, and stopping settings are chosen heuristically based on benchmark size and computational budget.Source: §C.6 · p. 22, Table 6 · p. 22

C.7 JOINT OPTIMIZATION OF SYSTEM PROMPTS AND AGENT SKILLS

• DUET extends unchanged to joint prompt-and-skill optimization, with separate textual learning-rate budgets for the system prompt and skills library.On τ 2-Bench, learned skills are composed directly into solver context because the benchmark has no separate skill-reading tool.Source: §C.7 · p. 23
• 91.9% success rate under joint prompt-and-skill optimization exceeds SkillOpt (83.3%) and GEPA (84.4%), and improves DUET’s prompt-only result of 89.4%.The authors attribute part of the improvement to encoding τ 2-Bench domain-specific procedures as reusable, narrowly scoped skills.Source: §C.7 · p. 23

C.8 RESULTS WITH GPT-5.6

• 89.6% solver success rate on τ2-Bench follows DUET optimization, up from 83.3%.The evaluation uses GPT-5.6 Sol with system prompts as the optimization target.Source: §C.8 · p. 23
• 55.0% pairwise agreement on SpreadsheetBench follows DUET optimization, up from 45.0%.The τ2-Bench grader agreement remains similar, while SpreadsheetBench solver accuracy is nearly unchanged.Source: §C.8 · p. 23
• DUET improves the component being optimized in some GPT-5.6 Sol settings, rather than improving every measured solver and grader metric.The reported changes differ by benchmark and agent component.Source: §C.8 · p. 23

C.9 PROMPT INITIALIZATION ABLATION

• The adversarial solver initialization reverses guidance on tool use, retrieved-data grounding, timeframe, organization, acronyms, depth, and stating assumptions.The adversarial grader prompt likewise reverses the informed prompt’s judging guidance.Source: §C.9 · p. 24
• DUET improves solver performance under empty, informed, and adversarial prompt initializations.The informed prompts use published solver and grader prompts; adversarial prompts reverse their main guidance while retaining role, structure, and ordering.Source: §C.9 · pp. 23–24
• DUET improves solver performance under all three initializations; with informed initialization, performance rises from 93.3% to 95.0%, while adversarial initialization improves from 91.1%.Source: §C.9 · p. 24
• 73.3% grader performance under informed initialization improves on its 68.2% starting result.DUET also improves the grader under all three initializations.Source: §C.9 · p. 25
• Adversarial initialization leads DUET to remove all harmful guidance clauses in its first update, while about 60% of informed-initialization solver prompt tokens remain in the final prompts.The authors report that DUET improves both agents across good and poor starting points.Source: §C.9 · p. 25

C.10 INFERENCE COST COMPARISON

• On held-out SpreadsheetBench tasks, DUET achieves the highest task accuracy while maintaining inference cost comparable to baselines.Inference cost is reported as average tokens and LLM calls required per task.Source: §C.10 · p. 25, Table 8 · p. 25

C.11 ABLATION STUDIES ON HYPERPARAMETERS

• Across the two ablations, the reported best settings are a moderate edit budget and calibrated exploration at c = 0.01.Both comparisons were conducted on the Finance Agent Benchmark.Source: §C.11 · p. 25
• The 4 →2 edit-budget schedule gives the best solver and grader performance; 8 →4 is similar, while removing the limit substantially degrades grader agreement.The reported results suggest a moderate edit budget supports stable optimization.Source: Table 9 · p. 25, §C.11 · p. 25
• The default SW-UCB coefficient c = 0.01 achieves the best solver and grader performance; greedy selection (c = 0) and stronger exploration (c = 0.1) perform worse.The differences are modest relative to observed variance.Source: §C.11 · p. 25

C.12 CHECKPOINT SELECTION ABLATION

• DUET’s fixed 50-round setting on GDPval achieves comparable performance without validation-based early stopping or checkpoint selection.The default instead uses a labeled validation set for checkpoint selection and early stopping; the ablation uses a fixed training budget and returns the final solver and grader checkpoints.Source: §C.12 · pp. 25–26, Table 11 · p. 26

C.13 EXAMPLES OF PROMPT EVOLUTION

• Prompt examples show solver updates promoting root-cause diagnosis, fix verification, and efficient tool use, while grader updates improve correctness checks, evidence assessment, and score calibration.The examples are described as reusable behaviors rather than solutions to individual training tasks, with the solver and grader serving complementary roles.Source: §C.13 · p. 26

C.14 EXAMPLES OF DUET-LEARNED PROMPTS AND SKILLS

• The Finance Agent Benchmark solver prompt prioritizes exact requested metrics and periods, grounding load-bearing figures in retrieved source text, and withholding claims unsupported by readable evidence.It prescribes gathering evidence before composing, tracking figures in an evidence ledger, and labeling or omitting values that remain unverified.Source: §C.14 · pp. 26–27
• The example grader prompt directs independent verification against run files and retrieved sources, with accuracy and source grounding weighted above calculations, completeness, analytical soundness, and clarity.It also calls for recomputing derived values and checking whether conclusions follow from verified evidence.Source: §C.14 · pp. 29–30
• The finance grader prioritizes verifiable correctness over polish, sharply penalizes load-bearing errors, and distinguishes genuine reliability defects from traceability gaps.It anchors scores to task completion and evidence, while requiring a delivered verdict even when investigation ends under uncertainty.Source: §C.14 · p. 31
• The telecom solver prompt requires policy or tool evidence for claims, treats unverified conditions as unknown, and takes exactly one action per turn.It also directs agents to verify that an offered path can advance the user’s goal and to resolve available paths before escalating.Source: §C.14 · p. 32
• The mobile-data skill diagnoses failures by testing the symptom, checking carrier, device, and app layers, reconciling conflicts, and retesting after each supported change.It allows for multiple faults and stops only when the requested outcome succeeds or an evidence-based escalation is needed.Source: §C.14 · p. 32
• The suspended-line skill identifies causes from line, contract, restriction, and billing records, follows only supported resolution paths, and verifies service restoration.It escalates only when resolution is unavailable or required by policy, reporting observed facts and unresolved blockers.Source: §C.14 · pp. 32–33
• The pointwise grader judges outcomes against run evidence and authoritative state, distinguishes observed facts from causal inferences, and scores substance over style.Its procedure includes checking the trajectory and final state, testing causal claims, and evaluating terminal decisions against available alternatives.Source: §C.14 · pp. 33–34
• The Excel agent manual prescribes a checklist-driven workflow: inspect the live workbook, plan and build changes, verify results, and report assumptions or limitations.It also advises batching workbook operations and isolating failed or unsupported cosmetic steps rather than repeatedly retrying them.Source: §C.14 · p. 35
• The solver prompt prioritizes complete deliverables and exact requirements, using live formulas, deliberate boundary logic, and readable formatting.It recommends breadth-first completion before polish and requires checks for errors, reconciled totals, and correctly rendered values.Source: §C.14 · pp. 35–37
• The grader prompt requires artifact-based verification, including workbook diffs, formula checks, independent recomputation, and screenshots for visual properties.It warns that the agent’s chat claims are not evidence and asks the grader to inspect the run log and actual workbook.Source: §C.14 · pp. 37–38
• The grader builds a weighted rubric from explicit task requirements, judges achievable substitutes within the agent’s environment, and anchors scores across the full 0–10 range.Core correctness and completeness receive the greatest weight; impossible out-of-environment artifacts are judged by the best achievable substitute, while achievable work remains accountable.Source: §C.14 · p. 38
• The grader prompt warns that malformed output can discard judgments, requiring plain-data verdicts, an explicit numeric score, and concrete strengths and weaknesses.It says the critique, score, strengths, and weaknesses must agree and tie claims to observed evidence.Source: §C.14 · p. 39
• The solver prompt directs agents to inspect the real workbook, make the requested edits, verify formulas and results, and preserve unrelated content.It frames the modified workbook—not advice or code in chat—as the deliverable.Source: §C.14 · pp. 39–40
• The grader independently derives success criteria from the request and input workbook, then verifies the deliverable rather than trusting the agent’s summary.It treats the summary as a hypothesis and checks claims against the actual workbook.Source: §C.14 · pp. 40–41
• It triangulates workbook evidence with input-output diffs, screenshots and underlying properties, formula results, and trajectory logs used only for context.Visual checks cover formatting, charts, and layout; diffs also expose collateral changes, while formulas are checked against evaluated values.Source: §C.14 · pp. 40–41
• It scores the end state rather than the method, crediting valid alternative approaches and treating harmless artifacts or moot subfeatures as minor.Defects are weighted by their effect on whether the user’s goal was met.Source: §C.14 · p. 41
• Its anchored 0–10 rubric assigns 9–10 to fully accomplished tasks, 7–8 to minor shortfalls, 4–6 to partial success, and 1–3 to failed tasks, with critical breaks for core failures.Critical breaks include an effectively unchanged workbook, a wrong primary result, or destroyed data; clarification earns credit only for genuine ambiguity.Source: §C.14 · p. 41
• Reports state whether the user’s goal was met and support specific strengths and outcome-bearing weaknesses with artifact evidence, excluding harmless issues.Evidence can include exact cells or ranges, diff counts, and computed-versus-expected comparisons; a correct deliverable without outcome-bearing shortfalls should have no weaknesses listed.Source: §C.14 · pp. 41–42