PaperlayerView exact revision on arXiv

Source-linked AI summary

Proactive Agentic Orchestration

Xueyang Feng, Chaozhuo Li, Xiaohe Bo, Jiakai Tang, Chenxi Li, Qiwei Ye, Xu Chen

arXiv:2610.13485v1cs.CL

TL;DR

Existing orchestration methods can fix sub-agent goals and harnesses at creation, while sparse, delayed feedback limits policy optimization. POrchestra enables proactive harness revision during execution, and OPOD uses that feedback for process supervision; across three benchmarks, the paper reports improved task success and cost–performance.

  • Problem

    Existing orchestration fixes sub-agent goals and harnesses at creation, limiting adaptation to dynamic environments, while sparse, delayed rewards constrain orchestration-policy learning.

    Source: §1 · p. 2

  • Method

    POrchestra lets sub-agents request harness changes at any logical point, and OPOD attributes their feedback to prior decisions as process supervision alongside final reward.

    Source: §1 · p. 2

  • Results

    Across three benchmarks, task success improves by an average of 9.1% relative to the strongest baseline; with OPOD, small-orchestrator success improves by 10.0% relative to training baselines.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    POrchestra reports improved task performance and favorable cost–performance trade-offs across the evaluated long-horizon benchmarks.

    Source: §5 · p. 9

    The experiments use a single active sub-agent at a time; concurrent execution may improve efficiency on independent subtasks.

    Source: G LIMITATIONS AND FUTURE WORK · p. 41

Abstract

from arXiv · show

Agentic orchestration delegates tasks to sub-agents with independent contexts, providing a key path to extending the effective execution horizon of large language model agents. However, existing orchestration methods face two limitations: in execution, current methods fix sub-agents' goals and harnesses at creation time, limiting their adaptability to dynamic real-world environments; in optimization, sparse and delayed feedback constrains effective learning of the orchestration policy. To address these limitations, we introduce POrchestra, an on-the-fly proactive agentic orchestration framework. It allows sub-agents to proactively communicate with the orchestrator at any logical point to request harness adjustments while maintaining continuity of local execution. This mechanism not only enables sub-agents to adapt seamlessly to dynamic environments but also naturally inspires On-Policy Orchestration Distillation (OPOD) based on hindsight feedback. OPOD attributes sub-agents' proactive revision requests back to preceding orchestration decisions to derive targeted improvement signals, and leverages these signals as process supervision alongside the final reward to optimize the orchestration policy. We evaluate POrchestra on three long-horizon agent benchmarks spanning deep research, dynamic digital assistance, and software engineering. POrchestra improves task success by an average of 9.1% relative to the strongest baseline; combined with OPOD, it improves small-orchestrator success by an average of 10.0% relative to existing training baselines. POrchestra also achieves a more favorable cost-performance trade-off. Our code is publicly available at https://github.com/XueyangFeng/POrchestra.

1 INTRODUCTION

Existing orchestration can adapt too late for dynamic tasks, while sparse rewards offer little guidance for improving orchestration decisions. PORCHESTRA integrates harness revision into execution, and OPOD turns sub-agent feedback into process supervision; experiments report improved success and cost–performance.

1 INTRODUCTION

  • Existing re-orchestration may not detect or address harness mismatches until execution ends or predefined conditions trigger, while terminal rewards are sparse and hard to attribute to individual decisions.Mismatches can leave sub-agents pursuing outdated subgoals, wasting steps and cost and propagating local errors.Source: §1 · p. 2
  • PORCHESTRA lets sub-agents pause at any logical point to send evidence-backed revision suggestions, which the orchestrator uses to update harnesses while preserving continuous local execution.The update is recorded as an observation in the sub-agent’s local trajectory.Source: §1 · p. 2
  • OPOD attributes execution feedback to earlier orchestration decisions, distills improvement guidance, and combines process-level self-distillation with reward-weighted regression.The process feedback complements sparse final rewards with targeted supervision for orchestration decisions.Source: §1 · p. 2
  • The paper presents PORCHESTRA for adaptive execution and OPOD for converting sub-agent feedback into process-level supervision, evaluating them on three long-horizon benchmarks.The benchmarks span deep research, dynamic digital assistance, and software engineering.Source: §1 · p. 2
  • 9.1% average task success relative to the strongest baseline; with OPOD, small-orchestrator success improves by 10.0% relative to training baselines, and PORCHESTRA reaches the highest performance along the cost–performance frontier.Evaluations span deep research, dynamic digital assistance, and software engineering.Source: §1 · p. 2

2 RELATED WORK

Agentic orchestration has moved from manually configured and offline systems toward runtime re-orchestration. Existing optimization methods use sparse task rewards or local scores, whereas OPOD uses execution-grounded feedback to guide prior decisions.

2 RELATED WORK

  • Manual and offline orchestration configure agent roles, collaboration structures, or workflows before task execution, while re-orchestration permits runtime adaptation.The section describes progression across these orchestration paradigms.Source: §2 · p. 3
  • Runtime re-orchestration may revise workflows after specific errors or recreate later sub-agents after earlier sub-agents finish execution.These approaches use the sub-agent as the unit of orchestration.Source: §2 · p. 3
  • Prior optimization methods use sparse task-level rewards or process scores for compliance or decision quality, while OPOD grounds attribution and revision guidance in execution obstacles.The section distinguishes execution-grounded feedback from local reward signals.Source: §2 · p. 3

3 METHODOLOGY

The methodology formalizes the delays caused by fixed sub-agent harnesses and sparse, delayed supervision, then presents proactive runtime coordination and OPOD as its approach. Proactive feedback supports both in-execution harness revisions and process-level optimization of orchestration decisions.

3.1 PROBLEM FORMULATION

  • The formulation represents delegation as a harness containing a subgoal, context, tools, and budget; existing methods keep that harness fixed until the sub-agent returns or times out.Sub-agent feedback arrives at the end of its lifecycle, while terminal reward arrives after the task, making orchestration supervision sparse and delayed.Source: §3.1 · p. 3

3.2 ON-THE-FLY AGENTIC ORCHESTRATION VIA PROACTIVE SUB-AGENTS

  • Sub-agents can pause at any point to request harness revisions; structured status, request, and evidence signals guide changes to task scope or resources, after which execution can continue with its history.The orchestrator may revise the harness or terminate the sub-agent.Source: §3.2 · pp. 4–5

3.3 ON-POLICY ORCHESTRATION DISTILLATION

  • OPOD attributes feedback to a prior orchestration decision only when evidence supports the link, distilling the mismatch and suggested correction into a natural-language gradient.When attribution is unsupported, the attribution unit is excluded from distillation.Source: §3.3 · p. 5
  • OPOD combines process-level distillation with trajectory-level reward-weighted regression, using an augmented-context self-teacher and the same model backbone.The policy is initialized with supervised fine-tuning on expert trajectories, then optimized with OPOD on on-policy rollouts.Source: §3.3 · p. 6

4 EXPERIMENTS

The experiments assess proactive orchestration and OPOD across long-horizon benchmarks, examining success, optimization, cost–performance, and when proactive adaptation helps.

4.1 EXPERIMENTAL SETUP

  • Evaluation spans deep-research GAIA, dynamic read–write GAIA2, and GitHub-issue SWE-Bench, comparing single-agent, offline orchestration, and re-orchestration baselines.Task success rate is the evaluation metric.Source: §4.1 · p. 6, Table 1 · p. 7

4.2 EFFECTIVENESS OF PROACTIVE ORCHESTRATION IN EXECUTION (RQ1)

  • PORCHESTRA leads average GAIA and GAIA2 success and SWE-Bench success across three backbones; on GAIA2, it scores 55.5% with DeepSeek-V4-Flash, a 24.6% relative improvement over ReAct at 44.5%.Table 1 covers GAIA, GAIA2, and SWE-Bench across single-agent, offline orchestration, and re-orchestration baselines.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 EFFECTIVENESS OF OPOD IN OPTIMIZATION (RQ2)

  • With OPOD, PORCHESTRA reaches 77.9% on GAIA and 49.2% on GAIA2, improving over its SFT initialization by 5.5 and 5.4 percentage points, respectively.It achieves the highest average success reported in Table 2; PORCHESTRA has the highest or joint-highest average success before training and retains its lead under the same training methods.Source: Table 2 · p. 8, §4.3 · p. 8
  • Removing either process feedback or outcome supervision reduces average success on both benchmarks, with RWR removal lowering average success by 2.1 points on GAIA and 3.1 on GAIA2.Without OPD, GAIA2 Adaptability and Time task success drops by 7.8 and 7.9 percentage points, respectively.Source: §4.3 · p. 8

4.4 COST ANALYSIS (RQ3)

  • OPOD-trained Qwen3.5-9B lies on the Pareto frontier for both GAIA and GAIA2, extending it toward comparable performance at lower cost by reducing reliance on larger orchestrators.Figure 4 depicts cost–performance trade-offs on GAIA and GAIA2.Source: Figure 4 · p. 8, §4.4 · p. 9

4.5 TRACING THE GAINS FROM PROACTIVE ORCHESTRATION (RQ4)

  • Proactive feedback achieves 92% recovery with a mean delay of 1.67 steps; on GAIA2, it outperforms periodic polling and predefined triggers overall while preserving execution continuity.More frequent polling improves recovery and reduces response delay at higher cost, but can disrupt continuity and lower overall success; predefined triggers struggle to detect adjustment needs without explicit errors.Source: Figure 3 · p. 9, §4.5 · p. 9
  • PORCHESTRA’s advantage grows in longer task groups on GAIA and GAIA2, and is largest on GAIA2 tasks with two future events.Task length is estimated from the fewest environment actions among successful trajectories; dynamicity groups GAIA2 tasks by future exogenous events.Source: §4.5 · p. 9

5 CONCLUSION

The conclusion presents PORCHESTRA’s proactive harness adaptation and OPOD’s process-level optimization, alongside experimental gains in performance, cost–performance, and small-orchestrator training.

5 CONCLUSION

  • PORCHESTRA lets sub-agents communicate with the orchestrator at any logical point for seamless harness adaptation in dynamic, long-horizon tasks; OPOD converts this feedback into process-level supervision.Experiments across three benchmarks show improved task performance, favorable cost–performance trade-offs, and more effective training of small orchestrators.Source: §5 · p. 9

AI USE STATEMENT

  • The authors report using generative AI for literature research, coding, translation, editing, results interpretation, figure captions and labels, drafting, and formatting, with author review and responsibility.The statement assigns responsibility for the final paper, including its claims, analyses, and artifacts, to the authors.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper locates inference and training algorithms in Appendix A, benchmark descriptions and implementation settings in Appendix E, proofs in Appendix D, and prompts in Appendix F.An anonymous code repository is linked in the abstract.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 PORCHESTRA INFERENCE

  • A revision updates a paused worker’s configuration and records the changes in its local history, allowing that same worker to resume with accumulated context.The orchestrator delegates and responds to feedback, while the sub-agent executes local actions and proactively returns control when an adjustment is needed.Source: §A.1 · p. 14

A.2 OPOD TRAINING

OPOD combines outcome supervision with process-level feedback attributed to earlier orchestration decisions. Additional GAIA2 analyses examine the OPD weight, harness revisions, and communication protocol components.

A.2 OPOD TRAINING

  • OPOD pairs successful decisions from SFT-policy trajectories with attributed feedback hints, then combines RWR outcome supervision and reverse-KL OPD on current-student actions.Records are paired by decision context, and the OPD teacher is the frozen SFT policy conditioned on feedback hints.Source: §A.2 · p. 15

B.1 EFFECT OF OPD WEIGHT (RQ5)

  • λ = 1 achieves the highest success among tested OPD weights; success across nonzero settings spans 44.5%–49.2%, a 4.7-point spread.The authors describe sensitivity as moderate within the tested range and use λ = 1 as the default.Source: Figure 5 · p. 16, §B.1 · p. 16

HARNESS REVISION ABLATION

  • Removing any harness revision lowers success; task revision has the largest observed effect at a 10.2-point drop, versus 7.8, 7.1, and 6.3 points for context, tool, and budget updates.The losses persist with the other three capabilities retained, supporting complementary roles for task and resource revisions.Source: Figure 6 · p. 16, HARNESS REVISION ABLATION · p. 17

COMMUNICATION PROTOCOL ABLATION

B.3 COMMUNICATION COST ANALYSIS (RQ7)

The additional analyses report lower total inference cost for POrchestra on both benchmarks and compare distillation objectives on GAIA2. A case study traces proactive tool adaptation through a failed historical search.

B.3 COMMUNICATION COST ANALYSIS (RQ7)

  • POrchestra has the lowest total inference cost on GAIA and GAIA2, as savings in routine orchestration and sub-agent execution exceed proactive-communication overhead.The per-task comparison uses Gemini-3-Flash as orchestrator and DeepSeek-V3.2 as executor; GAIA communication costs are estimates.Source: Table 3 · p. 17, §B.3 · p. 17

B.4 DISTILLATION OBJECTIVE COMPARISON (RQ8)

  • Reverse KL exceeds forward KL and JSD by 6.3 and 3.9 percentage points, respectively, in GAIA2 success rates.The comparison supports reverse KL for the main joint-training configuration.Source: Table 4 · p. 17, §B.4 · pp. 17–18

2. Sub-agent: Search Environment interaction

  • After search and current-page extraction fail to find Greenland’s historical population, the worker requests historical-revision access and resumes with added tools and guidance to return 56,000.The retrieved estimate was 56,081; Figure 7 depicts the proactive tool-adaptation trace.Source: §2 · p. 19, Figure 7 · p. 19

C.1.2 TASK ADAPTATION

The GAIA2 case shows a worker requesting guidance as new notifications arrive, after which the orchestrator revises the active task and tools. Three OPOD cases illustrate supervision for context allocation, tool configuration, and revision policy.

5. Sub-agent: Second follow-up request Communication to Orchestrator

  • Through three revisions, the orchestrator extends the active worker’s task and tools, distinguishes unrelated replies from relevant confirmation, and directs verification before conditional follow-up actions.The same worker carries forward earlier event, email, and contact information while booking the cab and sending the group message.Source: §5 · pp. 20–21, §3 · p. 20, §4 · p. 20

C.2.1 CONTEXT ALLOCATION OPTIMIZATION

  • OPOD attributes omitted crime-rate context to the original delegation and generates guidance to include the available results in future delegation.Runtime revision supplies the missing data, so even a successful trajectory provides supervision for improving delegation.Source: Figure 9 · p. 22, §C.2.1 · p. 22

C.2.2 TOOL CONFIGURATION OPTIMIZATION

  • After archive searches fail, the worker requests archival API access; OPOD attributes the resource mismatch to delegation and generates guidance to include ExecuteCodeAction.The retrieval objective is retained while the assigned tool set is revised.Source: Figure 10 · p. 23, §C.2.2 · p. 23

TaskCraft OPOD case: revision policy optimization

  • When an unverified domain sends the worker to an irrelevant site, OPOD attributes the mismatch to the orchestrator’s revision and generates guidance to establish the website from article titles and keywords.The resulting supervision concerns how the orchestrator updates an active worker’s context.Source: Figure 11 · p. 24, TaskCraft OPOD case: revision policy… · p. 24

D.1 PROOF OF PROPOSITION 1

The proof characterizes lifecycle-level intervention delay and timely-intervention probability under an asynchronous mismatch assumption. It contrasts that structure with proactive communication, whose delay is not tied to the predefined lifecycle horizon.

D.1 PROOF OF PROPOSITION 1

  • With mismatch arrival uniform over a lifecycle of horizon H, lifecycle-level re-orchestration waits until the lifecycle boundary, tying adaptation delay and its dispersion to H.The proof assumes a fixed maximum horizon and that re-orchestration cannot modify the active configuration before the lifecycle ends.Source: §D.1 · p. 25
  • For a fixed recovery window δ, timely intervention becomes less probable as lifecycle horizon H increases; the result follows from the lifecycle boundary, not policy quality.The timely-intervention probability is non-increasing in H, and the proof states that preserving the required sub-agent horizon entails an intervention gap.Source: §D.1 · pp. 25–26
  • POrchestra lets a sub-agent request adaptation after a mismatch, so intervention delay is governed by mismatch detection and communication rather than residual lifecycle duration.This decouples the timescale of intervention from the horizon needed for coherent local execution.Source: §D.1 · p. 26

D.2 PROOF OF PROPOSITION 2

The proof compares terminal-reward and process-level feedback horizons and shows that, under stated assumptions, a positive OPD weight can reduce expected update error relative to RWR alone. The baseline details cover single-agent, offline-orchestration, and re-orchestration systems, along with optimization methods.

D.2 PROOF OF PROPOSITION 2

  • Under bounded attribution noise and bounded OPD bias, some positive OPD weight yields lower expected error than RWR alone.The proposition also states that terminal rewards propagate through an attribution horizon O(Horch) times longer than OPD feedback.Source: §D.2 · pp. 26, 28
  • Terminal-reward and process-supervision variance bounds scale as O(HorchHsub) and O(Hsub), respectively, with process supervision also introducing finite bias.The variance bound grows linearly with the attribution horizon.Source: §D.2 · pp. 27–28

E.1.1 BASELINE DETAILS

  • Single-agent baselines keep planning and execution within one agent, while Skill-MAS and AutoAgents generate teams and workflows before execution.ReAct interleaves reasoning and actions; Mini-SWE uses shell commands and a linear message history.Source: §E.1.1 · p. 29
  • MAS2 re-orchestrates on predefined triggers, Orchestra-o1 between sub-agent rounds, and AORCHESTRA at sub-agent return boundaries.On SWE-Bench, MAS2 generates, reviews, and selects patches; Orchestra-o1 workers generate repair candidates; AORCHESTRA retains delegation-and-return execution.Source: §E.1.1 · p. 29
  • SFT fine-tunes the orchestrator on offline expert trajectories by maximizing expert-action likelihood given decision contexts.The experiments use Qwen3.5-9B and collect decisions on TaskCraft for GAIA and training scenarios for GAIA2.Source: §E.1.1 · pp. 29–30
  • RWR uses successful trajectories as outcome-supervised targets, while DA-GRPO scores candidate decisions against expert references for process-level supervision.RWR is evaluated with Orchestra-o1, AORCHESTRA, and PORCHESTRA; DA-GRPO uses rubric scoring without executing each candidate in the environment.Source: §E.1.1 · p. 30

E.1.2 BENCHMARK DETAILS

The section specifies benchmark scope and evaluation procedures, then describes the data and model setup used for warm-start training and OPOD optimization. The benchmarks span real-world question answering, dynamic digital-assistant scenarios, and software issue resolution.

E.1.2 BENCHMARK DETAILS

  • GAIA evaluation uses all 165 tasks in its 2023 validation set, while SWE-Bench uses a fixed subset of 100 Verified test issues.GAIA success is weighted across task levels; SWE-Bench uses a containerized repository execution and patch-evaluation setup.Source: §E.1.2 · pp. 30–31
  • GAIA2 evaluates 128 mini scenarios across four capability categories, with 32 scenarios per category and 512 remaining scenarios used for training.Its stateful application environments deliver user messages and environmental events during execution; timeouts and missing outputs count as unsuccessful.Source: §E.1.2 · pp. 30–31

E.2.1 BASE MODEL AND WARM-START STAGE

  • The orchestrator is initialized from Qwen3.5-9B and separately SFT-trained for Orchestra-o1, AORCHESTRA, and PORCHESTRA on each benchmark.Expert trajectories use Gemini-3-Flash as orchestrator; sub-agent models remain fixed during subsequent optimization.Source: §E.2.1 · p. 31

E.2.2 TRAJECTORY COLLECTION AND OPTIMIZATION DATA

  • RWR uses successful trajectories, while OPOD also uses feedback from successful and unsuccessful trajectories, pairing their objectives by decision context.The collected contexts and hints remain fixed while the current student resamples actions for OPD.Source: §E.2.2 · pp. 31–32

E.2.3 OPTIMIZATION HYPERPARAMETERS

  • The section reports main SFT and joint OPOD settings, and specifies that DA-GRPO uses eight candidate actions per decision context.DA-GRPO uses full fine-tuning on GAIA and LoRA on GAIA2, with DeepSeek-V3.2 as the rubric reward model for both.Source: Table 7 · p. 32, §E.2.3 · p. 32

E.2.4 OPOD LOSS IMPLEMENTATION

  • OPOD uses a frozen base model to generate attribution flags and natural-language gradients, and computes its loss only for flagged decisions.A fixed SFT teacher scores the student's sampled action under a feedback-augmented context; token-level reverse KL updates only the student, while RWR applies cross-entropy to successful-trajectory decisions.Source: §E.2.4 · p. 32

E.3 INFERENCE AND EVALUATION CONFIGURATION

Inference configuration specifies model settings, execution budgets, and evaluation repetition for the main results. The prompt appendices describe how training hints and communication signals enter the agent prompts.

E.3 INFERENCE AND EVALUATION CONFIGURATION

  • Inference uses the listed model configurations, sets per-benchmark execution budgets, and evaluates the two best-performing methods in each task category twice.The total budgets are 300 steps for GAIA and GAIA2 and 500 for SWE-Bench, with up to 10 sub-agents on GAIA and GAIA2.Source: §E.3 · pp. 32–33

F PROMPTS AND COMMUNICATION PROTOCOL

  • The GAIA and GAIA2 main-agent prompts insert Ht for the training-time self-teacher at a highlighted prompt position.The prompt templates retain core instructions, decision rules, and output formats while using placeholders for long runtime inputs and repeated examples.Source: F PROMPTS AND COMMUNICATION PROTOCOL · p. 34

F.2 SUB-AGENT PROMPTS

  • Sub-agent prompts encode status, request, and evidence in state, action, and evidence fields, with action also specifying the communication type.The implementation action finish corresponds to the sub-agent submit action.Source: §F.2 · p. 37

F.3 GENERATIVE REWARD MODEL PROMPT

  • The generative reward model prompt's needs repair and hint fields correspond to FLAGt and Ht, and the hint enters the GAIA and GAIA2 main-agent templates.Source: §F.3 · p. 40

G LIMITATIONS AND FUTURE WORK

The limitations include fixed sub-agent parameters and communication rules, and a focus on single-active-agent runtime adjustments despite potential efficiency gains from concurrency. The ethics statement notes the absence of human-subject studies and calls for further deployment evaluation and oversight.

G LIMITATIONS AND FUTURE WORK

  • Only the orchestrator is optimized; sub-agent parameters and communication rules remain fixed, while learning transferable proactive communication is future work.The experiments nevertheless show the design works across multiple execution models.Source: G LIMITATIONS AND FUTURE WORK · p. 41
  • The study uses one active sub-agent at a time because non-concurrent baselines outperform the concurrent baseline on the complex interaction tasks studied.The authors note that concurrent execution may improve efficiency for independent subtasks, but coordinating feedback across concurrent agents remains future work.Source: G LIMITATIONS AND FUTURE WORK · p. 41

H ETHICS STATEMENT

  • The work uses existing benchmarks and includes no human-subject studies; deployment safety, permission handling, and privacy protection need further evaluation.The authors recommend explicit authorization boundaries and human oversight for consequential actions.Source: H ETHICS STATEMENT · p. 41
Loading 2610.13485v1…