PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning

Yutong Wang, Siyuan Xiong, Xuebo Liu, Wenkang Zhou, Liang Ding, Miao Zhang, Min Zhang

arXiv:2602.23258v2cs.AIcs.CL

TL;DR

Multi-agent systems can propagate individual agents’ errors, while static approaches cannot dynamically salvage correctable outputs at inference. AgentDropoutV2 intercepts outputs for rectify-or-reject pruning and achieves the highest overall average accuracies reported for Fixed-MAS and Dynamic-MAS.

  • Problem

    Static approaches lack active test-time correction, limiting their ability to salvage correctable agent-output errors before they propagate through multi-agent systems.

    Source: §1 · pp. 1–2

  • Method

    AgentDropoutV2 intercepts agent outputs, uses indicators mined from historical failure patterns to iteratively rectify errors, and prunes outputs when errors persist.

    Source: §1 · p. 2

  • Results

    ADv2 achieved the highest overall average accuracies on both vanilla Fixed-MAS and Dynamic-MAS: 53.50% and 53.69%, respectively.

    Source: §3.2 · p. 5

  • Takeaways & Limitations

    The reported adaptivity across task complexities and scenarios supports ADv2 as a plug-and-play intervention solution.

    Source: §1 · p. 2

    ADv2 incurs higher inference token consumption due to its iterative rectify-or-reject mechanism.

    Source: Limitations · p. 9

Abstract

from arXiv · show

While Multi-Agent Systems (MAS) excel in complex reasoning, they suffer from the cascading impact of erroneous information from individual agents. Current solutions often resort to rigid structural engineering or expensive fine-tuning, limiting their adaptability. We propose AgentDropoutV2 (ADv2), a test-time rectify-or-reject pruning framework that dynamically optimizes MAS information flow. Acting as an active firewall, ADv2 intercepts agent outputs and employs a retrieval-augmented rectifier to iteratively correct errors. This rectification is guided by an indicator pool, which is constructed offline by distilling error patterns from historical MAS failure trajectories. Irreparable outputs are subsequently pruned to prevent error propagation. Empirical results demonstrate that ADv2 significantly boosts performance on both fixed and dynamic MAS frameworks, achieving average accuracy gains of 6.39 and 2.28 percentage points on extensive math and code benchmarks, respectively. Furthermore, ADv2 exhibits remarkable adaptivity, dynamically modulating rectification efforts based on task difficulty to resolve a wide spectrum of error patterns. Our code is released at https://github.com/TonySY2/AgentDropoutV2.

1 Introduction

ADv2 intercepts MAS outputs at test time, tries to rectify them using indicators drawn from historical failure patterns, and prunes persistent errors before they propagate. The framework is designed to adapt across task complexities and integrate with fixed and dynamic MAS environments.

Introduction

  • MAS can suffer cascading error propagation from individual agents, making timely identification and pruning of incorrect information necessary to protect downstream agents and task outcomes.The introduction situates this risk in MAS applications including software development, long-context handling, and scientific discovery.Source: §1 · p. 1
  • Structural optimization and parameter internalization rely on offline changes, limiting test-time adaptivity and the ability to salvage correctable errors during inference.The introduction describes static communication graphs and frozen weights as preventing dynamic correction.Source: §1 · pp. 1–2
  • ADv2 iteratively rectifies intercepted agent outputs before broadcasting them, then prunes outputs whose errors persist to block error propagation.The framework applies this test-time rectify-or-reject process during execution.Source: §1 · p. 2
  • ADv2 distills error patterns from failed MAS trajectories into an off-the-shelf indicator pool for identifying reasoning pitfalls.A dedicated rectifier retrieves adversarial indicators from the pool to provide targeted feedback on detected errors.Source: §1 · p. 2
  • ADv2 demonstrates adaptivity across task complexities and scenarios, dynamically retrieving context-aware indicators and resolving distinct error patterns.The introduction also reports a correlation between pruning rates and reasoning difficulty.Source: §1 · p. 2
  • ADv2 is framework-unaware, enabling dynamic information-flow optimization in both fixed and dynamic MAS environments.It is presented as transcending ADv1’s static topology constraints.Source: §1 · p. 2

2 Methodology

AgentDropoutV2 intercepts messages in a sequential multi-agent workflow, audits them against retrieved error indicators, and accepts, retries, or rejects them. It builds the indicator pool offline from failed trajectories and removes redundant patterns.

2.1 Preliminary

  • The MAS is modeled as an ordered sequence of agents whose outputs update downstream knowledge bases, with the terminal agent aggregating prior outputs into the final answer.The workflow trajectory includes the task, activated agents, intermediate responses, and final outcome.Source: §2.1 · p. 2

2.2 Test-Time Rectify-or-Reject Pruning

  • Each adversarial indicator names an error, defines the abnormal behavior, and specifies its trigger context; embeddings retrieve candidate rules, which a rectifier filters using the agent’s input, role, and output.The rectifier selects a relevant active subset from the top-Kcand candidates, with at most Kact indicators used for auditing.Source: §2.2 · p. 3
  • The rectifier audits outputs against active indicators and uses their pass rate to accept outputs meeting τpass, retry sub-threshold outputs before Tmax, or reject persistent failures.If pruning leaves fewer than γ messages, the system resets and reruns the MAS without rectify-or-reject pruning to avoid fragmented reasoning.Source: §2.2 · pp. 3–4

2.3 Failure-Driven Indicator Pool Construction

  • Offline, a teacher examines full MAS rollouts whose solutions differ from ground truth and synthesizes indicators from agent deviations in the task and role context.New indicators are compared semantically with similar pool entries, then appended only when a deduplication model judges them a novel error pattern.Source: §2.3 · p. 4

3 Experiment

ADv2 improves average accuracy in fixed and dynamic mathematical MAS, transfers indicators from Qwen3-8B to Qwen3-14B, and raises code-generation performance.

3.1 Experimental Setup

  • Experiments compare static DAG Fixed-MAS and AutoGen SelectorGroupChat Dynamic-MAS against self-refinement, PRM-guided search, Multi-TAG, and fixed-only ADv1 across nine math and four code benchmarks.The setup uses Qwen3-8B and Qwen3-4B for reasoning agents and rectifiers, with separate training datasets for constructing math- and code-domain indicator pools.Source: §3.1 · pp. 4–5

3.2 Main Results

  • 53.50% in Fixed-MAS and 53.69% in Dynamic-MAS are ADv2’s highest overall average accuracies on mathematical benchmarks, with especially strong gains on hard Olympiad-level tasks.ADv2 is described as framework-unaware and plug-and-play, enabling integration into Dynamic-MAS despite ADv1’s restriction to static topologies.Source: Table 1 · p. 5, §3.2 · p. 5
  • 56.29% versus 52.56% average accuracy results when a Qwen3-8B-mined indicator pool is applied to Qwen3-14B, demonstrating cross-model portability.This supports using a lightweight model to build reusable indicators for a more capable target model without redundant mining.Source: Table 2 · p. 6, §3.2 · p. 6
  • 48.37% overall accuracy with ADv2 exceeds vanilla MAS’s 46.63% and the Single Agent baseline’s 46.09%, with 68.48% on MBPP and 32.00% on LiveCodeBench.The passage attributes the gains to intercepting and mitigating complex logical and syntactical errors in programming workflows.Source: Table 3 · p. 6, §3.2 · p. 6

4 Analysis

ADv2’s analysis shows that carefully tuned rectification and retrieval improve accuracy, while intervention depth adapts to task difficulty and retrieved indicators reflect task-specific failure patterns. These benefits come with increased token use, which the reported accuracy gains justify on math benchmarks.

4.1 Ablation Study

  • Ablations favor Tmax=3, Kact=5, and τpass=60%: fewer or excessive retries, indicator counts of 3 or 7, and thresholds of 40% or 100% degrade accuracy.The findings attribute these effects to incomplete correction or reasoning drift, information overload, error propagation, or over-rejection.Source: §4.1 · p. 6, Table 4 · p. 7
  • Omitting dual-stage deduplication lowers average accuracy to 52.32%, while replacing retrieved indicators with random constraints lowers it to 51.65%.Redundant indicators crowd out distinct patterns, while random constraints add noise, showing the value of pool diversity and semantically relevant retrieval.Source: §4.1 · p. 6, Table 4 · p. 7
  • ADv2 remains functional and beneficial without prior failure-pattern mining by using a single universally applicable indicator that checks overall logical correctness.This supports operation in zero-shot scenarios where a domain-specific indicator pool is unavailable.Source: §4.1 · pp. 6–7

4.2 Iteration Dynamics and Adaptability

  • Rectification depth tracks task difficulty: GSM8K has 94.2% immediate acceptance, while OlymMATH Hard has rejections exceeding 6.3%.ADv2 conserves resources on simple queries and sustains effort on complex ones; rectification depth and rejection rate can serve as task-complexity proxies.Source: §4.2 · p. 7, Figure 3 · p. 7

4.3 Distribution of Retrieved Indicators

  • Retrieved-indicator overlap tracks reasoning similarity: GSM8K/AQuA score 0.43 Jaccard similarity, versus 0.11 for GSM8K/OlymMATH Hard, supporting task-dependent failure patterns.OlymMATH Easy and AIME24 also show high overlap, while disparate difficulty levels show sharply lower overlap.Source: §4.3 · p. 7, Figure 4 · p. 8

4.4 Trade-off Between Performance and Cost

  • On the reported benchmarks, ADv2 increases accuracy from 50.86% to 53.69% versus vanilla AutoGen while consuming more tokens; Self-Refine and PRM use comparable budgets but perform significantly worse.Source: §4.4 · p. 7

5 Related Work

Prior work addresses MAS error propagation through robust architectures, error monitoring, and inference-trajectory utilization. AgentDropoutV2 combines these paradigms in a model-agnostic, plug-and-play framework that rectifies errors and uses historical failures for online supervision.

Robust MAS Architectures

  • Architecture research optimizes MAS graphs, reduces communication noise, or routes tasks to specialized cooperative teams to suppress errors.Graph structures are optimized through learning or search; sparse communication reduces noise, while orchestration and routing limit errors from underperforming agents.Source: §5 · p. 8

Error Monitoring Mechanisms

  • Monitoring methods detect anomalous information flow, rectify system messages at test time, or attribute hallucinations to agents after failures.Graph-based anomaly detectors use information flow and topology; other methods use intercept-detect-correct processes or root-cause analysis.Source: §5 · p. 8

Utilization of Inference Trajectories

  • Trajectory-based methods train key components from execution data, verify intermediate steps, or mine failures and exploration as hard negatives.These approaches use preference or contrastive data, process-aware supervision, and hard negatives to guard against misleading intermediate states.Source: §5 · p. 8

Our framework

  • AgentDropoutV2 makes structural robustness plug-and-play, turns passive monitoring into feedback-driven rectification, and distills historical failures into an adversarial indicator pool for online supervision.The framework is model-agnostic and adaptable to diverse MAS frameworks.Source: §5 · p. 8

Conclusion

AgentDropoutV2 uses indicators mined from historical MAS failures to rectify or reject agent outputs at test time, improving information flow and system accuracy. Its retrieval and rectification adapt to task difficulty and transfer across domains and backbone models.

Conclusion

  • AgentDropoutV2 significantly enhances system accuracy by cleansing MAS information flow before erroneous outputs propagate.The conclusion attributes this improvement to test-time interception and iterative refinement of agent outputs.Source: Conclusion · p. 8
  • The framework mines historical failure trajectories to build an indicator pool, then retrieves relevant indicators to iteratively refine agent outputs at test time.The indicators encapsulate domain-specific error patterns and are used to resolve latent errors before they propagate.Source: Conclusion · p. 8
  • Indicator retrieval and rectification adapt to varying task difficulties and transfer across domains and backbone models.Source: Conclusion · p. 8

Limitations

ADv2 trades higher inference-token use for accuracy without requiring model fine-tuning, and it can add overhead on simpler tasks. Its advantages are more apparent as task complexity exceeds the backbone model’s capabilities.

Limitations

  • ADv2 incurs higher inference-token consumption from iterative rectification, trading inference compute for accuracy while avoiding fine-tuning costs.The paper says plug-and-play indicator pools eliminate the massive implicit computational costs of model fine-tuning.Source: Limitations · p. 9
  • On simpler tasks such as GSM8K, ADv2 occasionally trails CoT or Self-Refine because interventions add overhead where the backbone already reasons efficiently.The paper attributes this to the backbone model’s high intrinsic capability on foundational datasets.Source: Limitations · p. 9
  • As task complexity exceeds the backbone’s intrinsic limits, simple methods falter; ADv2 excels and still consistently improves over unaugmented vanilla MAS on simpler tasks.The paper frames this complexity-dependent performance as the setting where its robust framework excels.Source: Limitations · p. 9

A Appendix · A.1 Pseudo Codes

The appendix pseudo-codes describe runtime rectify-or-reject pruning and offline construction of a deduplicated indicator pool from failed MAS executions.

A.1 Pseudo Codes

  • During MAS execution, each agent output is intercepted, similar candidate indicators are retrieved, and a rectifier selects the active indicators.The retrieved indicators provide reference error patterns for rectification.Source: §A.1 · p. 12
  • The rectifier checks outputs against active indicators and generates diagnostic rationales as feedback when constraints are violated.Source: §A.1 · p. 12
  • A tri-state gate ends iteration when all checks pass or the iteration budget is exhausted, and propagates verified outputs to successor agents.Source: §A.1 · p. 12
  • If information flow becomes critically sparse, a global fallback resets the system and re-initializes execution from scratch.Source: §A.1 · p. 12
  • The indicator-pool procedure collects failed MAS execution trajectories, then uses a teacher model to synthesize indicators capturing individual agents’ error patterns.Source: §A.1 · p. 12
  • To prevent repository bloating, candidate indicators are encoded to retrieve similar constraints, and a deduplication model admits only distinct, previously unrecorded error types.Source: §A.1 · p. 12

A.2 Indicator & Prompt Design

The section describes indicators for checking square-root precision and a general-purpose fallback, alongside rectifier and teacher-model prompt templates.

Indicator Design

  • Figure 6 shows an indicator tailored to verify square-root calculation precision, while Figure 7 illustrates a general-purpose indicator for cases without a predefined pool.A detailed application case for the square-root indicator is provided in Appendix A.3.Source: §A.2 · p. 12

Prompt Design

  • Figures 9 and 10 present math and code rectifier prompt templates, while Figure 11 shows the teacher prompt for generating indicators from failed MAS execution trajectories.Source: §A.2 · p. 12

A.3 Case Study · A.4 Dataset Statistics · 24 else

The case study shows iterative audits correcting a persistent angle error, while the accompanying statistics and algorithm excerpts describe how indicator pools are built and how outputs are gated or reset.

A.3 Case Study

  • After two audits exposed an internal-versus-external angle error, the solver changed its answer from 18 + 7π to 18 + 2π and passed all five selected indicators.The first retry repeated the wrong answer; the second audit directed the solver to use the nonagon’s external angle, 2π/9.Source: §A.3 · p. 12, Figure 16 · p. 23, Figure 17 · p. 24

A.4 Dataset Statistics

  • Table 5 lists dataset and indicator-pool statistics; math indicators come from failed MATH and AQuA training trajectories, while code indicators use MBPP, KodCode, and CodeContests trajectories.The supplied passages identify Table 5’s scope but do not provide its numerical cell values.Source: §A.4 · p. 12, Table 5 · p. 19
  • The algorithm’s tri-state gate calculates the proportion of passed indicators and accepts an output when it passes.The excerpt also specifies continued processing when the output does not pass and iterations remain.Source: §A.4 · p. 13

24 else

  • The algorithm’s fallback triggers a system-wide reset when needed, discarding outputs and reinitializing with fresh agents.The excerpts show this as Phase 2, a global fallback check.Source: §24 · p. 13
  • Indicator-pool construction collects MAS failure trajectories and uses a teacher model to generate candidate indicators from each agent’s trajectory and output.The algorithm excerpt identifies a source dataset containing questions and gold answers, then iterates over instances and agents.Source: §24 · p. 14
  • The construction algorithm removes redundant candidates by retrieving similar indicators and retaining a candidate only when it is novel.The excerpt specifies top-K similarity retrieval before the novelty check.Source: §24 · p. 14

Related papers

Loading 2602.23258v2…