PaperlayerView exact revision on arXiv

Source-linked AI summary

Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction

Mingzhe Du, Luu Anh Tuan, Tianyi Wu, Renyang Liu, Zhijiang Guo, Dong Huang, See-Kiong Ng

arXiv:2607.01764v1cs.AI

TL;DR

Repository-scale vulnerability reproduction remains bottlenecked by choosing effective investigation strategies, even when agents can execute commands and edit code. Mastermind learns reusable strategies separately from frozen executors and achieves an 84.5% pass rate on held-out CyberGym tasks, improving across three executors.

  • Problem

    Repository-scale vulnerability reproduction challenges capable executors to choose effective investigation strategies, making strategy selection a central bottleneck for SE agents.

  • Method

    Mastermind trains a planner on compact natural-language strategies while explicitly storing task-specific experience, separately from frozen executors.

  • Results

    84.5% pass rate: Mastermind improves three frozen executors and outperforms independent sampling, iterative experience, and open-book context on held-out CyberGym tasks.

  • Takeaways & Limitations

    Strategy-level learning transfers across execution backbones and is useful when repository tasks require choosing among qualitatively different approaches.

  • Takeaways & Limitations

    Vulnerability reproduction does not capture every aspect of software engineering capability, so the evaluation’s scope is narrower than general SE capability.

Abstract

from arXiv · show

Repository-level vulnerability reproduction is a demanding software engineering (SE) task: an agent must inspect a codebase, infer the input grammar that reaches a vulnerable path, construct a proof-of-conceptv(PoC), and verify that the crash disappears on the patched build. Recent LLM agents can often execute these steps when the approach is correct, yet they still fail by choosing the wrong strategy. This paper argues that strategy, rather than the full action trajectory, is the right learning unit for such SE agents: it is compact enough to optimize, concrete enough to guide execution, and stable enough to store and reuse across attempts. We present Mastermind, a dual-loop framework that separates transferable strategy learning from task-specific experience. A trainable planner learns reusable vulnerability-reproduction strategies through SFT and milestone-based GRPO, while an experience loop maintains task-local strategy records that guide subsequent attempts. The planner is trained independently of the executor, allowing strategy learning to improve multiple frozen executors without modifying their action-generation capability. We evaluate Mastermind on CyberGym using 260 training tasks and 200 held-out evaluation tasks. With GPT-5.5 as the frozen executor, Mastermind achieves an 84.5% pass rate, outperforming open-book PoC context (60.0%), Best-of-8 sampling (63.0%), and iterative improvement (77.0%). The same planner also improves GPT-5.4 mini and GLM~5.1 from 45.0% and 58.5% to 60.0% and 71.0%. These results demonstrate that learning high-level strategies is an effective and transferable mechanism for improving repository-scale SE agents.

I. INTRODUCTION

Repository-scale vulnerability reproduction requires agents to discover vulnerable paths, infer input formats, build PoCs, and verify crashes, yet stronger execution still leaves strategy selection as a bottleneck. Mastermind addresses this bottleneck by learning compact, transferable strategies separately from frozen executors while retaining task-local experience.

  • Problem: Vulnerability reproduction requires repository exploration, build interaction, hypothesis revision, PoC construction, and execution-based verification on vulnerable and patched builds.Success is judged by whether a working PoC crashes only the vulnerable build.
  • Motivation: 23.5% of tasks are solved by one-shot GPT-5.5, versus 63.0% with independent Best-of-8, exposing a strategy gap despite stronger execution.Agents can navigate repositories, run commands, edit files, and submit PoCs, but may still pursue unproductive directions or fail to revise from feedback.
  • Framework: Mastermind separates transferable strategy learning from task-specific experience through a Curator, trainable Planner, frozen Executors, and Verifier.The Planner emits compact guidance about inspection targets, input structure, and PoC validation; Executors instantiate it as workspace actions.
  • Results: 169/200 GPT-5.5 tasks are solved by Mastermind, compared with 126 for independent Best-of-8 and 154 for iterative task-local experience.Mastermind uses fewer evaluation rollouts and also improves GPT-5.4 mini and GLM 5.1 without modifying either executor.
  • Contributions: Mastermind’s contributions identify strategy selection as a bottleneck, train reinforcement learning over compact strategies, and improve multiple frozen executors on held-out CyberGym tasks.The reported comparisons include independent sampling, iterative task-local experience, static-analysis guidance, and ground-truth-provided context.

II. PROBLEM CHARACTERIZATION … C. Task-Local Experience Beats More Context

Repository-scale vulnerability reproduction is bottlenecked by strategic decisions about investigation, input inference, exploitation, and revision rather than command execution alone. Controlled comparisons show that task-local strategy refinement and feedback outperform null strategies, independent sampling, and richer static context.

  • II. PROBLEM CHARACTERIZATION: Agents must choose where to inspect, which input structure to infer, when to exploit, and how to revise after verifier feedback.These choices are strategic rather than mechanical, so an executor can perform actions correctly yet fail under the wrong investigation plan.
  • A. Strategy Quality Changes Outcomes: 39.5% M7 pass rate for Soft Oracle exceeded 23.5% for Null Strategy and 24.0% for a Zero-shot Planner under a fixed GPT-5.4 mini executor.Even Hard Oracle reached only 32.0%, showing that task-relevant strategy signals materially changed outcomes while the executor and interface stayed constant.
  • C. Task-Local Experience Beats More Context: 154/200 tasks solved by sequential iterative improvement surpassed 126/200 for independent Best-of-8 using 753 rather than 1,600 rollouts.Iterative improvement therefore achieved the higher result with less than half as many executions.
  • C. Task-Local Experience Beats More Context: 120/200 tasks solved with single-pass Level 3 open-book context were exceeded by sequential iterative improvement, despite Level 3 exposing sanitizer output, patch information, and patched source.The comparison supports the claim that feedback changes the next hypothesis rather than merely adding text.
  • C. Task-Local Experience Beats More Context: Mastermind treats high-level strategy as the primary learning object while preserving task-local experience for iterative improvement.This premise follows from the observed bottleneck in choosing and refining strategies rather than merely executing commands.

III. MASTERMIND DESIGN · A. Mastermind Overview · B. Design Principles

Mastermind separates strategy learning from repository-specific execution through a Curator–Planner–Executor–Verifier pipeline. Its design keeps planning trainable and acting frozen, stores volatile task facts externally, and optimizes compact, diverse strategies using execution feedback.

  • A. Mastermind Overview: Mastermind organizes vulnerability reproduction as a Curator–Planner–Executor–Verifier pipeline linking task experience, strategy generation, repository actions, and executable feedback.Training and inference share the pipeline but differ in which components update.
  • A. Mastermind Overview: The Curator maintains task-local records of repository facts and outcomes, activating prior records before attempts and appending new records after verification.Stored facts include file locations, parser behavior, failed hypotheses, and useful partial progress.
  • A. Mastermind Overview: The Planner emits compact natural-language strategies covering inspection targets, input structure, PoC construction, and validation, without issuing shell commands.It chooses the approach from the task and curator experience.
  • A. Mastermind Overview: The Executor remains frozen while instantiating strategies through repository navigation, source inspection, edits, commands, and CyberGym submissions.This isolates gains attributable to better planning rather than changed action generation.
  • A. Mastermind Overview: The Verifier supplies execution-based ground truth through CyberGym feedback and milestone scoring, routing results to the Curator and, during training, the Planner.This replaces preference-based feedback with executable validation.
  • B. Design Principles: 39.5% M7 pass rate versus 23.5% shows that, with the same GPT-5.4 mini executor, Soft Oracle strategy outperforms having no strategy.Mastermind therefore improves the Planner without modifying the acting Executor.
  • B. Design Principles: Mastermind updates compact Strategy tokens while frozen Executors run asynchronously, avoiding direct GRPO training over long, noisy, executor-specific trajectories.A single rollout can take minutes and consume millions of tokens, coupling full-trace updates to costly executor behavior.
  • B. Design Principles: 154/200 tasks using 753 rollouts versus 126/200 with 1,600 rollouts shows the benefit of storing volatile repository facts as curator experience rather than model weights.Task-local facts include file locations, parser quirks, failed inputs, and verifier milestones.

C. Strategy as the Learning Unit · D. Two Substrates for Two Kinds of Knowledge · E. Training and Inference Algorithms

Mastermind treats strategy as the compact, task-facing learning unit linking planning to execution, while separating transferable planner knowledge from task-specific curator experience. Its dual-loop training updates planner weights from verifier rewards and accumulates task-local evidence, whereas inference follows the same path without planner updates.

  • C. Strategy as the Learning Unit: A Strategy tells the Executor where to look, what evidence to seek, and which vulnerability mechanism to test without prescribing the exact interaction sequence.Strategies are capped at 2,000 tokens and serve as the intermediate contract between high-level planning and low-level execution.
  • C. Strategy as the Learning Unit: Median Strategy lengths range from 700 to 1.1k tokens, with P90 at most 1.5k tokens across executor settings.These observed lengths remain comfortably below the 2,000-token cap, supporting compact strategy-level learning.
  • D. Two Substrates for Two Kinds of Knowledge: Transferable strategy instincts belong in Planner weights, whereas task-specific facts such as vulnerable files, fixtures, failed inputs, and verifier milestones belong in curator experience.The two substrates reflect knowledge with different lifetimes: strategy knowledge transfers across tasks, while task-specific facts expire after one task.
  • D. Two Substrates for Two Kinds of Knowledge: Each verified Rollout appends a strategy/outcome record to curator experience, and later attempts retrieve quality-filtered priors to condition the next Strategy on task-local evidence.This experience loop preserves concrete evidence from the current task for subsequent attempts.
  • D. Two Substrates for Two Kinds of Knowledge: During training, SFT/GRPO updates the Planner toward Strategies with high Verifier-derived rewards relative to comparable same-task rollouts.Better Planner weights improve rollouts, improved rollouts enrich curator experience, and richer experience sharpens later Planner context.
  • E. Training and Inference Algorithms: The training pipeline runs Curator, Planner, Executor, and Verifier stages, with verifier feedback updating both curator experience and Planner parameters.Task-local experience grows across visits while Planner weights update from milestone rewards.
  • E. Training and Inference Algorithms: Inference follows the same Curator–Planner–Executor–Verifier path but removes the Planner update.Algorithm 1 formalizes this evaluation stream while preserving the execution and verification flow.

IV. EXPERIMENTAL DESIGN · A. Experiment Setup

The experimental design evaluates Mastermind on CyberGym through four research questions spanning strategy causality, learned planning, task-local experience, and comparisons with alternative approaches. The setup combines frozen executors with a trainable planner, supervised warm-starting, GRPO training, and sequential inference with curator feedback.

  • IV. EXPERIMENTAL DESIGN: The experiments ask whether strategy quality affects reproduction with the executor and workspace fixed, whether learned planning improves frozen backbones, whether task-local refinement beats repeated sampling, and how Mastermind compares with alternatives.The comparison scope includes stronger benchmark context, static-analysis guidance, and different executor backbones.
  • IV. EXPERIMENTAL DESIGN: Each inference attempt activates curator experience, samples a strategy, executes it, verifies the rollout, and updates curator records.The inference stream omits Planner-parameter updates and retains the highest-scoring rollout, stopping early when the reproduction milestone reaches 7.
  • A. Experiment Setup: CyberGym contains 1,507 real-world vulnerability-reproduction tasks from ARVO and OSS-Fuzz across 188 open-source projects.Each task provides a repository and benchmark interface and requires a raw PoC input reproducing a specified sanitizer crash.
  • A. Experiment Setup: Mastermind uses Qwen3.6-35B-A3B as its trainable Planner and delegates execution to frozen GPT-5.4 mini, GPT-5.5, and GLM 5.1 Executors.The Planner emits only compact strategies, while the frozen Executor handles command execution and PoC submissions.
  • A. Experiment Setup: Before reinforcement learning, SFT trains the Planner to change strategies after feedback using stepwise examples derived from Claude Code and Codex trajectories.At step t, the prompt contains task context and accumulated curator experience, while the target is the strategy used at step t+1.
  • A. Experiment Setup: GRPO samples sixteen candidate Strategies per task from one policy snapshot, partitioned into two fixed 8-way groups with complementary slot-conditioned hypotheses.The frozen Executor runs each Strategy independently, and the Verifier converts each rollout into a milestone reward.

B. Training Reward

Mastermind trains with a compact, milestone-dominated reward that combines task progress, concise strategy length, and penalties for invalid or timed-out rollouts. Milestone feedback densifies CyberGym’s sparse binary success signal, while normalization compares matched same-task rollouts.

  • Reward design: The reward combines milestone progress, bounded Strategy length, and status penalties; diversity is handled through curator experience and slot-conditioned prompting.The status penalty applies only to timeout or invalid-format rollouts, and no learned reward model is used.
  • Milestone reward: Milestones map verifier feedback to the highest reached level m(y) ∈ {0, . . . , 7}, providing dense credit beyond CyberGym’s sparse binary success criterion.Milestones 0–3 use the mastermind execution trajectory, whereas milestones 4–7 use CyberGym verification output.
  • Milestone reward: 4.0 reward points are assigned to the final transition from m=6 to m=7, keeping milestone 7 dominant over earlier progress.This final step is worth more than the entire milestone-3 reward.
  • Reward design: 2,000 tokens define the strategy-length cap Tstrat, yielding a small preference for valid, concise Strategies without rewarding full execution traces.The length function is bounded in [0, 1], and its coefficient γℓ remains fixed across experiments.
  • Reward normalization: 10^-8 is used as ε in within-group reward normalization to guard against degenerate groups and stabilize comparisons among matched rollouts.Each group contains Strategies from the same task and policy snapshot, while bounded reward components keep ratios stable during training.

C. Evaluation Metrics · D. Baselines

Mastermind is evaluated primarily by CyberGym’s milestone-7 pass rate, requiring successful crash reproduction on the vulnerable build and normal execution on the patched build. The study compares it with independent sampling, iterative refinement, and static-analysis guidance baselines.

  • C. Evaluation Metrics: CyberGym uses eight progressively stronger milestones, from repository inspection through complete dual-build verification.Milestones 1–3 come from agent trajectories, while milestones 4–7 are assigned automatically after CyberGym executes PoCs on vulnerable and patched builds.
  • C. Evaluation Metrics: The primary metric is milestone-7 pass rate, counting a task as solved only when the PoC crashes the vulnerable build and the patched build exits normally.This follows CyberGym’s official strict dual-build evaluation protocol.
  • C. Evaluation Metrics: The milestone reward schedule gives full reproduction the highest reward while providing dense intermediate training signals.Milestones 0–3 are trajectory-level signals, whereas milestones 4–7 are server-level signals based on CyberGym execution output.
  • C. Evaluation Metrics: Evaluation also reports milestone distributions and executor rollout counts to characterize effectiveness and computational cost.Sequential methods stop after milestone 7, while Best-of-N uses the highest milestone reached across attempts.
  • D. Baselines: Best-of-8 performs eight independent Level-1 attempts without information sharing and solves a task if any attempt reaches milestone 7.It represents an independent-sampling baseline.
  • D. Baselines: Iterative Improvement executes attempts sequentially and supplies compact task-local experience after failures to enable strategy refinement.This baseline improves reproduction through repeated, experience-informed attempts.
  • D. Baselines: PAGENT adapts a static-analysis pipeline by adding recovered vulnerability targets when analysis succeeds, otherwise reverting to the standard Level-1 prompt.Its optional AFL-based coverage guidance is disabled because Mastermind receives no execution coverage feedback.

V. RESULTS · A. RQ1: Strategy Quality Causally Affects Reproduction · B. RQ2: Learned Planning Transfers across Executors

The results show that strategy quality causally changes reproduction success for a fixed executor, while a planner trained on GPT-5.4 mini trajectories transfers gains to GPT-5.5 and GLM 5.1. Mastermind improves strict milestone-7 reproduction by reducing unresolved and wrong-crash outcomes, though benchmark-verifier exceptions remain.

  • A. RQ1: Strategy Quality Causally Affects Reproduction: Under a fixed GPT-5.4 mini executor, strategy quality substantially changes strict milestone-7 reproduction success.The sensitivity study varies only the strategy signal under a 900-second timeout.
  • A. RQ1: Strategy Quality Causally Affects Reproduction: The RQ1 comparison supports dedicated planning as a causal bottleneck rather than a cosmetic prompt prefix.All four conditions use the same executor while varying the supplied strategy signal.
  • A. RQ1: Strategy Quality Causally Affects Reproduction: Soft Oracle reaches 39.5%, versus 23.5% for Null Strategy, 24.0% for Zero-shot Planner, and 32.0% for Hard Oracle.The comparison shows that direct Level 3 solution context is not interchangeable with an executable investigation strategy.
  • B. RQ2: Learned Planning Transfers across Executors: GRPO raises pass rates from 45% to 60% for GPT-5.4 mini, from 72.5% to 84.5% for GPT-5.5, and from 58.5% to 71% for GLM 5.1.The Strategy Planner is trained only on GPT-5.4 mini Executor trajectories and reused for the other frozen executors.
  • B. RQ2: Learned Planning Transfers across Executors: With GPT-5.5, Mastermind exceeds independent Best-of-8 at 63%, PAGENT-guided Best-of-8 at 70.5%, and iterative task-local improvement at 77%.Mastermind uses 560 rollouts, compared with 753 for iterative improvement and 1,600 for Best-of-8.
  • B. RQ2: Learned Planning Transfers across Executors: Across three Base Planner rows, m≤5 outcomes fall from 186 to 134, m6 wrong crashes from 62 to 35, and m7 full reproductions rise from 352 to 431.For GPT-5.5 and GLM 5.1, m6 drops from 20 to 5 and from 31 to 18, while m7 rises from 145 to 169 and from 117 to 142, respectively.
  • B. RQ2: Learned Planning Transfers across Executors: Three apparent benchmark exceptions reach m6 but not m7 because candidate PoCs crash on both vulnerable and patched builds or on a non-target stack.CyberGym’s dual-build verifier rejects arvo:16972, arvo:52317, and arvo:52430; CyberGym ground-truth solutions similarly remained below m7.

C. RQ3: Experience Beats Independent Sampling

Task-local experience outperforms independent sampling by feeding prior failures into subsequent strategies, while Mastermind’s trained Strategy Planner further improves both initial attempts and refinements across executors.

  • Independent sampling: Independent Best-of-8 reaches 63.0% for GPT-5.5 and 42.5% for GPT-5.4 mini, but gains flatten as rollouts revisit similar hypotheses.The comparison uses cumulative milestone-7 pass rates on the same 200 tasks.
  • Task-local experience: GPT-5.5 iterative improvement reaches 50.0% by round 2 versus 38.5% for Best-of-N, and ends at 77.0% versus 63.0%.It uses 753 rollouts instead of 1,600 because solved tasks stop early.
  • Task-local experience: GPT-5.4 mini iterative improvement ends at 53.0% versus 42.5% for Best-of-N, showing the same advantage from task-local feedback.Feeding prior failures into the next strategy changes the search trajectory rather than treating each attempt as a fresh sample.
  • Mastermind: Mastermind reaches 84.5% for GPT-5.5 and 60.0% for GPT-5.4 mini by round 8, while starting at 47.0% and 35.0%, respectively.The dual-loop design combines curator-maintained task-local experience with a trained Strategy Planner to improve revised strategies.

D. RQ4: Baseline Parity and Competing Sources of Help · E. Cost Analysis · F. Failure Analysis

Mastermind outperforms context, sampling, and iterative baselines while reducing rollout cost through strategy selection and early stopping. Remaining failures primarily involve semantic misses, repetitive search, or system-interface artifacts.

  • D. RQ4: Baseline Parity and Competing Sources of Help: 169/200 tasks are solved by Mastermind’s RL planner, versus 120/200 with open-book context, 126/200 with Best-of-8, and 154/200 with iterative improvement.PAGENT raises GPT-5.5 Best-of-8 to 141/200, but the supplied passage indicates this still trails Mastermind’s RL planner.
  • D. RQ4: Baseline Parity and Competing Sources of Help: Mastermind’s dual-loop ablation distinguishes independent Best-of-8 from variants removing either the Experience loop or the Policy loop.Independent Best-of-8 carries no curator experience across attempts and uses no trained Planner; the other variants retain only one loop component.
  • E. Cost Analysis: 55–70h is Mastermind’s estimated GPT-5.5 serial time, down from roughly 160h for Best-of-8 and 75h for iterative improvement.The reference cost per pass falls from about $29 and $11 to about $8, while subscription interfaces make these dollar values estimates rather than billed costs.
  • E. Cost Analysis: 407.6 seconds is the average GPT-5.5 Codex rollout time, with roughly 211k net tokens and 1.66M gross context tokens per trajectory across 517 valid rollouts.GLM 5.1 Zhipu trajectories are heavier, averaging 811.9 seconds on the Best-of-8 evaluation slice, with the median near the 900-second timeout.
  • E. Cost Analysis: Thousands of environment evaluations make full command-level trajectories prohibitively expensive, so Mastermind scores compact Strategy outputs through repository-scale rollouts.The passage characterizes full-trajectory policy optimization as requiring billions of tokens and thousands of agent-runtime hours.
  • F. Failure Analysis: 54/200 Codex GPT-5.5 Best-of-8 tasks are semantic failures at M4, where the server accepts the PoC but the vulnerable build does not crash.Examples include FLAC bitreader, njs parsing, and libarchive RAR5 parsing tasks whose malformed inputs miss the target crash.
  • F. Failure Analysis: Eight independent attempts on arvo:49427 repeat minor variants of one hypothesis and remain at M4, while iterative refinement solves arvo:60268 on attempt 8.The passage also notes that search can regress, as illustrated by arvo:38764.
  • F. Failure Analysis: System-interface failures produce M0 results despite claimed PoC creation, often after submit.sh connection failures or auto-submit misses; safety filtering is another artifact.Examples include arvo:38766, arvo:6796, arvo:38764, and arvo:38870, alongside a Claude Code / Opus 4.8 refusal on arvo:60268.

VI. DISCUSSION AND TAKEAWAYS · VII. RELATED WORK · A. Planner-executor decomposition.

Mastermind argues that strategy selection is the bottleneck in vulnerability reproduction and addresses it by combining a trainable strategy planner with task-local experience and a frozen executor. Its discussion also highlights safety constraints and positions the framework against complementary planner–executor approaches.

  • VI. DISCUSSION AND TAKEAWAYS: Strategy-level reinforcement learning targets strategy selection because execution capability is assumed sufficient, especially when viable approaches are numerous.CyberGym Level 0–1 tasks expose only source code and a brief description, requiring agents to discover an effective approach.
  • VI. DISCUSSION AND TAKEAWAYS: 60.0% with Level 3 single-pass context versus 77.0% with Level 1 iterative improvement shows that context and strategic exploration address different subsets of tasks.The disagreement between Level 3-only and Best-of-8-only tasks further supports their distinct roles.
  • VI. DISCUSSION AND TAKEAWAYS: Mastermind uses two complementary knowledge substrates: GRPO compresses recurring cross-task patterns, while the Curator explicitly stores per-task facts whose value expires after one task.The Curator is therefore presented as an intentional division of labor rather than a retrieval cache attached to reinforcement learning.
  • VI. DISCUSSION AND TAKEAWAYS: Mastermind amplifies vulnerability-reproduction capability through planner training, but it retains existing executor capabilities, requires many scaffolded rollouts, and evaluates PoCs within CyberGym.These three factors are presented as mitigations for the framework’s stronger capability amplification than inference-time methods.
  • VII. RELATED WORK · A. Planner-executor decomposition.: Existing work covers separate parts of the design space: some methods train planners without cross-run experience, others retain experience without trainable planners, and others emphasize strategy diversity or process-level credit.The cited categories include MPO, PilotRL, Plan-and-Act, CoDA, A-Mem, AlphaEvolve, SGE, Tree-GRPO, Turn-PPO, and ASearcher.
  • VII. RELATED WORK · A. Planner-executor decomposition.: Mastermind combines these axes through a trainable strategy Planner, a frozen Executor, task-local curator experience, and milestone-based planner updates.This combination distinguishes it from approaches that address only planner training, memory, strategy diversity, or trajectory-level credit assignment.

B. Reinforcement learning for LLM reasoning and agents. … IX. CONCLUSION

The paper situates Mastermind among reinforcement-learning and cybersecurity-agent methods, then evaluates its strategy-learning framework under controlled validity protocols. Its conclusion identifies strategy selection as a bottleneck and reports that an RL planner with GPT-5.5 solves 84.5% of held-out tasks, versus 63.0% for independent Best-of-8 sampling.

  • B. Reinforcement learning for LLM reasoning and agents.: PPO remains a standard LLM-RL backbone, while GRPO removes the learned value model by normalizing rewards within sampled groups.Recent agent-RL variants add tree-structured search, turn-level credit, and asynchronous long-horizon rollouts.
  • B. Reinforcement learning for LLM reasoning and agents.: Planner-training systems optimize high-level decisions for long-horizon agents, but most methods still operate within a single prompt or trajectory.This motivates treating strategy as a distinct learning unit for repository-scale agents.
  • C. LLM agents for cybersecurity and SE.: CyberGym evaluates agents on 1,507 real-world vulnerabilities across 188 projects, complementing CTF-style benchmarks such as Cybench, NYU CTF Bench, and InterCode-CTF.Cybersecurity-agent work also includes interactive tools, plan-and-execute methods, multiagent orchestration, and RL for exploit generation.
  • VIII. THREATS TO VALIDITY: Executor fixing and planner or experience variation isolate the effect of strategy learning within each comparison.All methods share the CyberGym interface, timeout, milestone definitions, and evaluation protocol.
  • VIII. THREATS TO VALIDITY: CyberGym success requires reproducing the vulnerability on the vulnerable build while verifying that the patched build no longer crashes.This dual-build criterion reduces false positives from measuring arbitrary crashes alone.
  • IX. CONCLUSION: 84.5% of held-out tasks are solved by the RL planner paired with GPT-5.5, compared with 63.0% for independent Best-of-8 sampling.The conclusion frames strategy selection as a bottleneck: strong executors often know how to act but not which strategy to try.
  • IX. CONCLUSION: Mastermind learns compact natural-language strategies, stores curator experience explicitly, and trains its planner with dense CyberGym milestone feedback.The framework is presented as complementary to systems that primarily improve per-trajectory quality or provide task-local analysis signals.
Loading 2607.01764v1…