PaperlayerView exact revision on arXiv

Source-linked AI summary

Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Traces

Simon Yu, Derek Chong, Ananjan Nandi, Dilara Soylu, Jiuding Sun, Christopher D Manning, Weiyan Shi

arXiv:2605.10913v3cs.AIcs.PLcs.SE

TL;DR

Existing agent substrates make runtime inspection and transformation of agent execution difficult because side effects resist treatment as manipulable values. Shepherd makes execution a reversible, first-class object for meta-agents, and three case studies demonstrate meaningful gains across supervision, workflow optimization, and long-horizon reinforcement learning.

  • Problem

    Existing agent substrates struggle to support runtime observation, interception, forking, reversion, and modification because agentic side effects resist treatment as inspectable values.

  • Method

    Shepherd represents agent tasks and executions as structured, reversible traces that meta-agents can observe, intercept, revert, fork, and rewrite.

  • Results

    Across three meta-agent applications, Shepherd produces meaningful uplifts; CooperBench joint pass rate rises from 28.8% to 54.7%.

  • Takeaways & Limitations

    Shepherd provides a substrate for higher-order agents to actively operate on agentic execution across runtime supervision, workflow optimization, and long-horizon reinforcement learning.

  • Takeaways & Limitations

    The case studies establish proof of existence rather than optimality, broad robustness, or superiority over every alternative substrate and policy.

Abstract

from arXiv · show

As LLM agent systems take on more complex tasks, they increasingly rely on meta-agents: higher-order agents that create, operate on and manage other agents. Meta-agent operations such as coordinating agents, halting risky actions before execution, or repairing failed runs, require runtime manipulation of agentic execution. Yet existing agentic substrates make this difficult: they expose only transcripts and environment snapshots, forcing meta-agents to build ad hoc tooling to reconstruct and operate over full execution state. Therefore, we introduce Shepherd, a Python substrate grounded in functional programming principles, where an agent's execution is itself a first-class object that a meta-agent can easily inspect and transform. Every model action, tool call, and environment change becomes a structured event in a reversible, Git-like execution trace, where any past state can be reverted 5x faster than docker commit and fork. Three example use cases show Shepherd's versatility: (1) a supervisor meta-agent prevents conflicts among parallel coding agents, lifting pair-coding pass rate from 28.8% to 54.7% on CooperBench; (2) a counterfactual optimization meta-agent repairs agent workflows by proposing edits and replaying runs from the point of changed behavior, outperforming MetaHarness on Terminal-Bench 2.0 by 12.8% with 58% lower wall-clock; (3) a training meta-agent picks fork points during rollouts to improve credit assignment in long-horizon agentic RL, doubling GRPO's uplift on Terminal-Bench 2.0. We open-source Shepherd to enable principled and efficient operations over agentic execution for both users and meta-agents.

1 Introduction

Shepherd makes agent execution a first-class, reversible object for meta-agents, enabling runtime inspection, intervention, reversion, and replay through a functional-programming-based Python substrate. Its trace-based design supports supervisors, counterfactual optimizers, and training meta-agents, with reported gains in coding success, benchmark performance, and rollout credit assignment.

  • Core approach: Shepherd treats agent executions as structured data that meta-agents can hold, execute, copy, rewrite, inspect, and transform.Its functional-programming grounding enables algebraic effect handlers and continuations for non-invasive observation and pause-inspect-decide-resume control.
  • Execution substrate: Every agent action becomes a commit in a reversible, Git-like execution trace, while forks create branches and checkouts restore prior agent-environment states.The trace records tool calls, filesystem modifications, and database operations as structured events.
  • Efficiency: 5× faster state forking than docker commit and over 95% LLM-provider KV-cache reuse make Shepherd lightweight for a 5.8 GB docker image.These measurements characterize the substrate’s execution-state efficiency.
  • Meta-agent applications: 54.7% CooperBench joint pass rate, up from 28.8%, was achieved by a runtime supervisor that intercepts conflicts among parallel coding workers.The supervisor watches execution traces and intervenes before conflicting actions occur.
  • Contributions: Shepherd provides a mechanized functional model, an efficient Python framework, and three meta-agents spanning runtime supervision, post-execution optimization, and training.The paper positions these components as its principal contributions and demonstrates them across the agent lifecycle.

2 Related Work

Related work develops meta-agents for self-improvement, multi-agent orchestration, and runtime state management. These approaches variously modify code or strategies, coordinate agents through messages and rollouts, or expose checkpointing at different infrastructure layers.

  • Meta-Agents: Meta-agent research maintains dynamic archives, optimizes task-agent strategies, and refines context or long-term retrieval through evolutionary search and memory augmentation.Examples include Darwin-Gödel Machines, Group-Evolving Agents, Hyperagents, and Meta-Harness.
  • Agentic Meta-Optimization: Multi-agent frameworks route natural-language messages among workers, while pipeline and test-time scaling methods use parallel rollouts, majority voting, and full end-to-end re-execution.CooperBench highlights coordination failures that can result from standard multi-agent frameworks.
  • Agentic Runtime and Infrastructure: Agentic runtime research places checkpointing at different software-stack layers, including cooperative workflow tools and kernel-level filesystem branching.AgentGit exposes version-control operations within LangGraph, while BranchFS isolates filesystem state independently of tool-call structure.

SHEPHERD Programming Model

SHEPHERD treats agentic execution as a first-class, functional-programming object that meta-agents can inspect, intercept, modify, fork, revert, and resume. Its model combines task values, typed effect streams, region-scoped execution, and navigable execution history to support these operations.

  • Tasks: Tasks are substitutable values, allowing meta-agents to accept other tasks as arguments and thereby form hierarchical meta-meta-agents.A task is specified by its signature and docstring, with outputs validated against the defined type; users may also provide deterministic bodies.
  • Effects: Every worker action becomes a typed effect in an immutable stream that meta-agents can observe, subscribe to, or intercept.Effects may represent LLM calls, tool calls, environment mutations, or custom actions.
  • Effects: Intent and outcome are separate effects, enabling a meta-agent to inspect an action before its outcome materializes and discard destructive tool calls.Observation is non-perturbing because the worker’s effect stream remains byte-identical whether or not a meta-agent watches.
  • Scopes: Scopes provide isolated, atomic execution branches through emit, fork, merge, and discard, allowing alternatives to be committed or abandoned without contaminating the parent.Copy-on-write scope forks capture the worker’s filesystem, processes, and bindings, while nested and parallel scopes support meta-agent and meta-meta-agent supervision.
  • Execution history: Persistent execution history keeps every past worker state reachable, so meta-agents can revert to an earlier moment, inspect it, or run forward under different conditions.The design draws on persistent data structures, where versions remain accessible after modification and share structure for efficiency.

4 Framework Performance

Shepherd’s performance depends on image-size-independent branching, trace scalability, and prompt-cache-preserving replay. On Terminal-Bench 2.0 images, copy-on-write forking is fast and image-size-independent, while replay achieves roughly 95% prompt-cache hit rates.

  • Cost properties: Shepherd targets three cost properties: image-size-independent forking, trace scaling with agent writes, and replay that reaches the LLM provider’s prompt cache.These properties are measured on real Terminal-Bench 2.0 images.
  • Fork and revert: 134–143 ms forks remain constant across 42 MB to 5.8 GB images because Shepherd creates copy-on-write layers instead of duplicating filesystems.On the 5.8 GB image, K forks cost K × 143 ms versus K × 53.5 s for full-rootfs copies.
  • Fork and revert: 192× per-branch slowdown separates full-rootfs copies from Shepherd forks on the 5.8 GB image.The comparison is K × 143 ms for Shepherd against K × 53.5 s for full-rootfs copies.
  • Replay: ∼95% cache-hit rate plateaus from K=2 onward across 8 Terminal-Bench 2.0 tasks using Anthropic Claude Haiku 4.5.The rate remains within 5% of the byte-identical ceiling because forking preserves the parent’s byte-identical LLM message prefix.

5 Experiments

Experiments demonstrate three meta-agent applications built on SHEPHERD: runtime supervision, counterfactual workflow optimization, and meta-agent-guided Tree-GRPO. These applications use non-perturbing observation, byte-identical replay, and cheap branching to improve coordination, optimization, and credit assignment.

  • 5 Experiments: The experiments span runtime execution, post-hoc workflow optimization, and training-time credit assignment through three SHEPHERD meta-agent applications.The applications are runtime supervision on CooperBench, counterfactual meta-optimization, and meta-agent-guided Tree-GRPO on Terminal-Bench 2.0.
  • 5.1 Runtime supervision: 54.7% pair pass rate: an Opus meta-agent raised CooperBench performance from the 28.8% coop baseline, reaching 91% closure of the coordination gap.The solo ceiling was 57.2%, while Sonnet reached 45.3%.
  • 5.1 Runtime supervision: 24.2 min per pair: Opus supervision incurred 4.3 min meta overhead while remaining below the 28.4-minute solo time.Supervision observes workers periodically and intervenes only when interception is required.
  • 5.2 Counterfactual meta-optimization: CRO improved performance by 4 pts on Terminal-Bench 2.0 while requiring the least wall-clock, and it achieved the best performance on four of five datasets.Across the reported datasets, CRO simultaneously exceeded MetaHarness’s held-out test scores and reduced wall-clock by 27–58%.
  • 5.3 Meta-agent-guided Tree-GRPO: 5.2 and 3.4 points: Tree-GRPO’s uplift over Flat GRPO was consistent for approximately 3B-active-parameter and 12B-active-parameter models, respectively.Meta-Agent Guided Tree-RL was strongest on both base models, with higher reward variance and more informative intra-tree gradient steps.

6 Conclusion · A Limitations and Future Works

Shepherd makes agent execution a first-class object that meta-agents can hold, inspect, fork, and modify. The substrate supports runtime supervision, counterfactual workflow optimization, and finer-grained credit assignment in long-horizon reinforcement learning.

  • 6 Conclusion: Shepherd treats another agent’s execution as a first-class object, analogous to functions in functional programming.Meta-agents can hold, inspect, fork, and modify the execution.
  • 6 Conclusion: Meta-agents in Shepherd operate over agentic execution as higher-order agents.The paper demonstrates this capability across three applications.
  • 6 Conclusion: Shepherd supports runtime supervision of parallel coding agents.This application uses meta-agents to manage execution during runtime.
  • 6 Conclusion: Shepherd supports counterfactual replay for workflow optimization.This application modifies and replays agentic execution to optimize workflows.
  • 6 Conclusion: Shepherd supports Tree-GRPO for finer-grained credit assignment in long-horizon reinforcement learning.Tree-GRPO is presented as an application of the substrate to long-horizon RL.
  • 6 Conclusion: Shepherd opens a path toward meta-agent systems centered on programmable operations over agentic execution.The passage frames this as the broader significance of the substrate.

A.1 Limitations · A.2 Future Works

Shepherd’s case studies establish proof of existence rather than optimality, with effectiveness and replay efficiency constrained by meta-agent cost and edit-induced side effects. Future work targets counterfactual interpretability, continual learning, native substrate use, and safety-critical reversible sandboxes.

  • A.1 Limitations: Shepherd’s three case studies demonstrate proof of existence, not optimality, broad robustness, or uniqueness of the reported headline results.The authors do not claim head-to-head comparison against every alternative.
  • A.1 Limitations: Meta-agent supervision can be economically unfavorable on short tasks because its token cost may exceed the worker’s.Favorability depends on task length and the worker-to-meta-agent cost ratio.
  • A.1 Limitations: Counterfactual replay loses its cache benefit when edits propagate widely, making the replayed suffix the entire trajectory.This occurs in every dataset’s cold first proposer session but amortizes within two to three sessions.
  • A.2 Future Works: Shepherd’s coupled forks enable testable agent-interpretability interventions by editing one component and replaying the affected suffix while holding other variance fixed.Candidate edits include tools, system-prompt spans, and sub-task definitions.
  • A.2 Future Works: Cheap filesystem and process-tree forks could support continual learning across computer-use, web-browsing, and large-codebase interactions over weeks.The execution trace maintains a coherent history across those interactions.
  • A.2 Future Works: Future agents could be post-trained to natively fork, discard, and replay their own typed effect streams.They would need to learn when to back up, retry, or spawn a sibling.
  • A.2 Future Works: Reversible sandboxes could support safety-critical deployments by enabling overrides before irreversible effects and rollback checks to the last materialisation point.The substrate assigns distinct rollback contracts to reversible, compensable, and irreversible effects.

B Mechanized Core and Proof Envelopes

Shepherd mechanizes a small algebraic-effects trace machine rather than the production runtime, using proof envelopes to mark how far executions are validated. These envelopes provide boundary discipline, not verification of arbitrary Python execution or surrounding infrastructure.

  • Mechanized boundary: The verified artifact is a small algebraic-effects trace machine with proof-backed profiles for static fragments whose lowered traces fall within its boundary.Production Python tasks, SDK calls, shell commands, sandbox operations, retries, scheduling, and storage remain unverified.
  • Proof envelopes: Each inspectable run may carry a proof envelope specifying a profile and explicit strength, while ordinary Python runs default to runtime_only.Profiles include runtime_only, reference_core_a, core0, core_a, core0h, and extension; strengths include runtime_only, reference_validated, forward_simulation, and semantic_adequacy.
  • Reference validation: The executable reference package validates Core-0/Core-A trace lifecycles, checks exact generated-trace agreement by rerunning static KernelProgram, and emits a ProofEnvelope.Public Run[T] values carry a proof field, placing the reference validator between production execution and the Lean development.
  • Evidence discipline: Proof-backed status derives from proof_strength and content-addressed evidence, so profile names alone cannot upgrade a trace.Non-runtime strengths must cite kernel-v3 reference provenance and a proof-evidence:sha256 identifier.
  • Non-claims: The proof envelope does not verify arbitrary Python control flow, external behavior, model outputs, filesystem correctness, carrier storage, scheduling, retries, recovery, or multi-branch replay.The main-paper meta-agent applications instead rely on the production substrate plus empirical validation; Lean supplies the semantic core and boundary discipline.

C Framework Performance: Extended Results · C.1 Measurement Protocol

The appendix details extended performance results and the measurement protocol underlying them. C.1 standardizes hardware, workloads, mutation patterns, operation timing, resource accounting, and repetition procedures across Shepherd and comparison systems.

  • C Framework Performance: Extended Results: The appendix organizes extended results covering measurement protocol, mutation size, latency, depth scaling, observation overhead, KV-cache reuse, and backend portability.These results support Section 4 and span Appendices C.1–C.7.
  • C.1 Measurement Protocol: Table 3 compares Shepherd, docker commit, and full rootfs copy on one Vultr instance, while Modal uses separate hosted gVisor hardware and Table 8 uses Firecracker micro-VMs.The Vultr setup has 2 vCPU, 16 GB RAM, SSD storage, Ubuntu 22.04, Docker 29.3.1, and overlay2 on extfs.
  • C.1 Measurement Protocol: Table 3 spans Terminal-Bench 2.0 images from 42 MB to 5.8 GB; full copy is O(n) in image size, whereas docker commit, Modal snapshot_filesystem(), and Shepherd’s overlay delta are O(1).The images are openssl-selfsigned-cert, caffe-cifar-10, and pytorch-model-recovery.
  • C.1 Measurement Protocol: Synthetic working directories contain 10 files and 30 KB, 100 files and 100 MB, or 100 files and 1 GB; KV-cache experiments use Terminal-Bench 2.0 tasks with a Haiku 4.5 agent.These workloads supplement the real Docker-image measurements.
  • C.1 Measurement Protocol: Each checkpoint uses 5 baseline random-file writes followed by one additional random write in each of K−1=3 sibling branches, with each write defaulting to 10,KB.Table 3 storage reports one sibling’s post-fork overlay delta, while disk at K=4 aggregates branch deltas.
  • C.1 Measurement Protocol: Fork and revert latency measures agent-perceived wall-clock under Pattern A, with a host Python agent driving the sandbox container through tmux after warming an OpenAI gpt-5.4-mini client.The setup matches production sandbox usage by mini-swe-agent, SWE-Agent, and smolagents.
  • C.1 Measurement Protocol: Method-specific timings include container creation and tmux readiness for docker commit and Modal, while Shepherd forks and reverts overlay layers within the same tmux session.Each operation runs from issuance until a probe bash command returns the readiness marker.
  • C.1 Measurement Protocol: Measurements discard 2–3 warm-up iterations, use time.monotonic() wall-clock timing, report ±1σ error bars, and apply method-specific repetition counts across latency and resource cells.Table 3 latency uses 10 repetitions for Shepherd, docker commit, and Modal; full-copy storage uses 5, and K=4 resources use 3.

C.2 Substrate scaling under varying mutation size … C.7 Realization across Sandbox Backends

Shepherd’s reversible substrate scales with mutation size and trajectory depth while keeping fork/revert latency nearly constant, adding no context tokens for supervisors. Its primitives span multiple sandbox backends, though gVisor and copy-based fallbacks introduce substantial latency or size-dependent costs.

  • C.2 Substrate scaling under varying mutation size: 8 KB to 800 MB disk deltas track agent writes 1:1, while fork and revert remain approximately 340 ms across mutation sizes.Across 12 repetitions, reverted workdirs exactly matched pre-fork states.
  • C.4 Scaling Behaviour: 157–252 ms overlay checkpoint/revert latency holds through 50 stacked layers, versus docker commit’s 451–558 ms baseline, while overlay chains are limited to approximately 60 layers.Trajectories exceeding the kernel page-size depth limit require periodic management.
  • C.4 Scaling Behaviour: Approximately 120 ms of per-event effect-stream overhead remains constant through 200 steps, while stream size grows linearly at approximately 130 B/event.The measured overhead is network-dominated on E2B.
  • C.5 Observe Overhead Detail: 113 ms remote recording/observation overhead is network-dominated, while subscribing a supervisor adds exactly 0 tokens to the worker context.Worker message lists were byte-identical with and without a supervisor across a 10-step trajectory.
  • C.6 KV-Cache Reuse Detail: Anthropic prompt caching serves eligible replay prefixes at 10% of normal input-token price, excluding prompts below Haiku 4.5’s 4,096-token cache threshold.The filter removed one Terminal-Bench 2.0 task, fix-git, from the sample.
  • C.7 Realization across Sandbox Backends: The single device-layer interface preserves identical application semantics across containerized sandbox providers, with measured fork latency varying by backend.Docker measured 72 ms median, while E2B measured 159–169 ms because of remote API roundtrips.
  • C.7 Realization across Sandbox Backends: Prime Intellect’s cp -a fallback scales O(n) with working-directory size, from approximately 100 ms for ≤5 MB to 57 s for a 6 GB rootfs.The fallback is usable for small repositories but unsuitable for large ones; Modal snapshot checkpointing instead takes 935–1137 ms.

D Trajectory Compression: Extended Results

Shepherd’s trajectory-compression meta-agent reads completed effect streams, chooses a fork point and hint, and reruns workers from that state. Across Terminal-Bench v2.0 and SWE-Bench Verified, hindsight often yields shorter passing trajectories, with larger absolute savings for Sonnet 4.6 on SWE-Bench Verified.

  • Setup: The meta-agent reads a completed worker trajectory and emits a fork_step, natural-language hint, and rationale before rerunning from a forked SHEPHERD scope.The rerun prepends the hint to the worker’s system prompt.
  • Results: 68% of Sonnet’s and 82% of GPT-5.4’s passing SWE-Bench Verified baselines admit a strictly shorter passing rerun.The evaluation covers 500 SWE-Bench Verified instances.
  • Results: 77% of the corresponding Terminal-Bench v2.0 trajectories admit a strictly shorter passing rerun under the meta-agent’s hindsight.Terminal-Bench v2.0 contains 88 tasks; the supplied passage reports the corresponding fractions but truncates the second model-family value.
  • Controls: The no-meta-agent best-of-N=5 control recovers only a small fraction of the compression gap, ruling out within-task baseline variance as the explanation.The control selects the shortest passing baseline among five independent samples on the same tasks.
  • Results: 12.5 model calls saved on average for Sonnet 4.6 on SWE-Bench Verified, versus 10.2 for GPT-5.4 high.The passage attributes the larger Sonnet reduction to longer baselines containing more excisable exploration.

D.1 Aggregate results … F.2.6 Handoff manifest example

Shepherd supports trajectory pruning, runtime supervision, and counterfactual workflow optimization by exposing execution traces that can be forked, replayed, and transformed. Across these uses, meta-agents reduce wasted exploration, intervene through graduated controls, and reuse unaffected execution state.

  • D.1 Aggregate results: Above 60% compression rates recur across cells, with mean baseline lengths dropping roughly by half on compressed trajectories.The largest absolute reduction is 12.5 model calls for Sonnet on SWE-Bench Verified.
  • D.2 Top compressions: Largest trajectory reductions target discardable diagnostic prefixes, using either full restarts or mid-trajectory cuts that retain useful early exploration.Examples include discarded searches, library-version probes, incorrect hypotheses, and fork steps such as f=8.
  • D.3 Hint examples: Hints identify target files, intended code-level changes, and detours to skip without providing complete code solutions.The rerun worker must translate the guidance into edits.
  • D.4 Best-of-N control: 5 fresh baseline rollouts rarely explain the meta-agent’s shorter reruns: on seven Terminal-Bench v2.0 tasks, shortest-passing-of-5 was 5.6 calls versus 4.4 for reruns.The per-task baseline mean on the same set was 5.7 calls.
  • D.5 Fork-step distribution: Roughly one third of compressed trajectories use f=0, one third fork in the first half, and one third fork at or beyond the midpoint.Forked reruns pay only suffix-call cost plus one diagnostic call; Sonnet’s SWE-Bench median rerun cost is roughly one-third of baseline cost.
  • D.6 Meta-agent prompt and output schema: The pruning meta-agent reads a completed trajectory and emits JSON specifying fork_step and a concise hint, while no_prune skips already-efficient trajectories.A compressed baseline must have a passing rerun using strictly fewer model calls.
  • E Runtime supervision: protocol, tools, and meta-agent prompt: Runtime supervision offers inject, handoff, and discard actions with increasing disruption, while coverage reaches 26.4%/18.7%/7.9% for Sonnet and 39.2%/31.5%/4.6% for Opus.Inject preserves session context, handoff resets the session while retaining files, and discard also rolls back the worker filesystem.
  • F.1 The CRO algorithm: CRO maintains workflow variants and execution traces in a shared store, proposes edits with fix and guard sets, and counterfactually replays only affected execution suffixes.Typed cache keys based on source and input hashes allow unaffected task-DAG subtrees to be reused, making replay cheaper than full reruns.

F.3 Per-dataset settings and optimised workflows

This section specifies benchmark splits and metrics, then describes how CRO transforms each baseline workflow through structured additions such as multi-branch solving, gated repair, and deeper retrieval. The settings include HoVer, MATH, LiveCodeBench, and Terminal-Bench 2.0, with MetaHarness-protocol evaluation on the latter.

  • HoVer: HoVer uses a 150/300/300 GEPA reproduction split and binary FullCoverage, while CRO adds summarisation, bridge resolution, deterministic grounding, recursive expansion, and a third hop.The baseline issues two LLM-generated queries, reranks TF-IDF top-100 candidates, and selects titles; CRO targets missing and ambiguous evidence.
  • MATH: MATH uses a uniformly sampled 100/50/50 L5 split and exact match, while CRO replaces one solver branch with plan-conditioned dual solvers, repair, and LLM selection.Repair runs only for non-compliant or errored branches, and the selector compares finalised branches before verification.
  • Audit-and-repair: CRO also introduces an explicit gated audit-and-repair stage: failed audits trigger repair, followed by constrained finalisation after a second failure.AuditResponse evaluates the original prompt’s constraints, while later rewrites condition on the verdict and audit history.
  • Terminal-Bench 2.0: Terminal-Bench 2.0 follows MetaHarness verbatim on the 25-task Stable25 subset for both optimisation and reporting, using avg@5 on the canonical Terminus-2 test suite.The overlapping split enables apples-to-apples comparison with MetaHarness without a held-out generalisation gap.

F.4 Per-dataset CRO results … F.5.3 Case 3: Recovered Bridge Pages Needed to Become New Evidence Sources

CRO reaches competitive or superior dataset-specific performance with substantially lower optimization cost, while its HoVer case studies progressively improve evidence retrieval through counterfactual workflow edits. The edits evolve from preserving recovered bridge pages, to resolving explicitly missing evidence, to recursively inspecting recovered pages for further bridges.

  • F.4 Per-dataset CRO results: On IFBench, MetaHarness leads by 1.0 pts at 0.523 versus CRO’s 0.512, but CRO reaches that frontier in 82 minutes versus 126 for MetaHarness.CRO cache reuse rises from 0% in the cold first session to ∼50% by session 5.
  • F.4 Per-dataset CRO results: CRO reaches the LiveCodeBench frontier with 0.510 held-out pass-rate, +11 pts over MetaHarness and +2.3 pts over GEPA, at roughly half MetaHarness’s wall-clock.Both CRO and GEPA produce non-trivial dev trajectories on this benchmark.
  • F.4 Per-dataset CRO results: CRO matches MetaHarness’s 0.80 dev pass-rate on MATH (Level 5) in ∼42 wall-clock minutes versus ∼100, although GEPA’s dev-history was empty on this run.This dataset is included for completeness and is not part of the main results table.
  • F.4 Per-dataset CRO results: 0.352 avg@5 is CRO’s held-out Terminal-Bench 2.0 result, while MetaHarness, GEPA, and the baseline each score 0.312 despite all three optimizers reaching 0.40 single-pass dev rate.Cache reuse saturates near 100% within three proposer sessions because each candidate evaluation set contains only 25 tasks.
  • F.5 Case Studies: Interpretable Counterfactual Workflow Edits on HoVer: The HoVer case studies use dev-set gold-document coverage over 300 examples, with the baseline retrieval workflow scoring 0.447 dev accuracy (134/300).CRO summarizes three counterfactual workflow edits discovered on HoVer.
  • F.5.1 Case 1: The Workflow Was Accidentally Candidate-Closed: Case 1 raises dev coverage from 0.447 to 0.693 (134/300 to 208/300) by adding bridge-title recovery and treating grounded non-candidate Wikipedia titles as first-class evidence candidates.The baseline discarded recovered bridge entities because later stages selected only from the upstream TF-IDF candidate list.
  • F.5.2 Case 2: The Workflow Could Diagnose Its Own Missing Evidence: Case 2 raises dev coverage from 0.737 to 0.787 (221/300 to 236/300), a +5.0% improvement, by resolving explicitly missing evidence and grounding short surface forms to exact local Wikipedia pages.Relative to the original baseline, the resulting workflow is +34.0% higher.
  • F.5.3 Case 3: Recovered Bridge Pages Needed to Become New Evidence Sources: Case 3 raises dev coverage from 0.787 to 0.797 (236/300 to 239/300), a further +1.0% improvement, by recursively inspecting recovered bridge pages for the next bridge.CRO selected this more general mechanism despite its lower targeted-train score, 0.833 versus 1.000, because its aggregate dev score was higher.

F.6 Meta-Agent Guided Tree-RL: full training configuration

Meta-Agent Guided Tree-RL trains GRPO policies with large-scale Modal infrastructure, bounded rollouts, and explicitly configured optimization and inference settings. Its procedure branches trajectories from meta-agent-selected fork points and assigns separate credit to shared prefixes and branched suffixes.

  • Full training configuration: Training uses Modal-managed 8×H100 nodes with SkyRL’s GRPO recipe, FSDP2, gradient checkpointing, and torch.compile.Each step processes 16 prompts with 8 samples per prompt, yielding 128 rollouts per step.
  • Full training configuration: Rollouts are capped at 8 turns, 1024 maximum generated tokens per turn, and 16,384 maximum input length, with overlong trajectories filtered.Overlong filtering uses SkyRL’s overlong-filtering mode.
  • Meta-Agent Guided Tree-RL: Meta-Agent Guided Tree-RL rolls out roots, selects a fork step, rolls the environment back, and creates K isolated branches sharing the trajectory prefix.Credit assignment uses a G-root group baseline for prefix actions and a (K+1)-sibling baseline for suffix actions before clipped GRPO updates.
  • Full training configuration: Optimization uses Adam with learning rate 5×10−7, weight decay 0.01, gradient clipping at max-norm 0.1, and a 20-step linear warm-up.Training then follows a constant schedule for ten epochs over the training set, totaling 1,120 steps; KL loss is disabled.

F.7 Meta-Agent Guided Tree-RL: meta-agent qualitative examples

This subsection shows how a stronger-model meta-agent selects meaningful fork points and executable corrective branches from full agent trajectories. Across three qualitative cases, it identifies earliest causal mistakes, traces symptoms back to causes, and targets deeply nested errors rather than recovery thrashing.

  • Method: The meta-agent receives the full inner-agent transcript and final reward, then returns a fork turn and bash command rendered through the policy’s own renderer.The command is injected as one of K branches at the selected turn, with tokens matching the policy’s emissions.
  • Evaluation: Across three hand-constructed trajectories, the meta-agent is evaluated on whether its selected branch matches the likely correct turn and whether its command addresses the failure.The cases cover an early single-cause failure, an ambiguous multi-branch failure, and a long trajectory with a deeply nested mistake.
  • Behavior: The meta-agent prefers the earliest downstream wrong choice, even when that turn is not a syntactically obvious failure.Examples include a package-name typo in Case 1 and an incorrect target-file choice in Case 3.
  • Behavior: The meta-agent traces symptoms back to their causes and emits executable commands rather than hand-wavy diagnoses.In Case 2, it links a defunct process at turn 5 to the launch issue at turn 3 and proposes a specific nohup invocation.
  • Long-trajectory case: The long-trajectory case shows the meta-agent selecting the wrong-file edit at turn 4 instead of any of the five subsequent recovery turns.It replaces the fragile edit with a multi-line heredoc that cleanly rewrites utils.py, concentrating branch budget on the causal mistake.
Loading 2605.10913v3…