Source-linked AI summary
AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning
Yutong Wang, Siyuan Xiong, Xuebo Liu, Wenkang Zhou, Liang Ding, Miao Zhang, Min Zhang
TL;DR
MAS can suffer cascading errors, while structural optimization and parameter internalization limit dynamic correction. ADv2 uses test-time rectify-or-reject pruning with a failure-derived indicator pool, improving fixed and dynamic MAS performance while adapting to task complexity.
Problem
MAS performance can be compromised by individual-agent errors propagating downstream, while existing structural and parameter-based approaches limit dynamic correction.
Method
ADv2 intercepts agent outputs for iterative rectification guided by an indicator pool distilled from historical MAS failure trajectories, pruning outputs whose errors persist.
Results
ADv2 improves performance across fixed and dynamic MAS frameworks and demonstrates adaptivity across task complexities and error patterns.
Takeaways & Limitations
ADv2 provides a framework-unaware, plug-and-play intervention for dynamically optimizing MAS information flow across diverse environments.
Takeaways & Limitations
ADv2 consumes more inference tokens and can trail lightweight baselines such as CoT or Self-Refine on simpler tasks like GSM8K.
Abstract
from arXiv · showhide
While Multi-Agent Systems (MAS) excel in complex reasoning, they suffer from the cascading impact of erroneous information from individual agents. Current solutions often resort to rigid structural engineering or expensive fine-tuning, limiting their adaptability. We propose AgentDropoutV2 (ADv2), a test-time rectify-or-reject pruning framework that dynamically optimizes MAS information flow. Acting as an active firewall, ADv2 intercepts agent outputs and employs a retrieval-augmented rectifier to iteratively correct errors. This rectification is guided by an indicator pool, which is constructed offline by distilling error patterns from historical MAS failure trajectories. Irreparable outputs are subsequently pruned to prevent error propagation. Empirical results demonstrate that ADv2 significantly boosts performance on both fixed and dynamic MAS frameworks, achieving average accuracy gains of 6.39 and 2.28 percentage points on extensive math and code benchmarks, respectively. Furthermore, ADv2 exhibits remarkable adaptivity, dynamically modulating rectification efforts based on task difficulty to resolve a wide spectrum of error patterns. Our code is released at https://github.com/TonySY2/AgentDropoutV2.
1 Introduction
Multi-agent systems improve complex-task performance but remain vulnerable to cascading errors and static intervention limits. ADv2 addresses this gap with framework-unaware, test-time rectify-or-reject pruning and historical error indicators for adaptive information-flow optimization.
- MAS harnesses collective intelligence for complex tasks but is vulnerable to erroneous outputs propagating through downstream agents.
- Existing approaches constrain error pathways structurally or improve agents through fine-tuning, but static designs cannot dynamically salvage correctable inference errors.
- ADv2 intercepts agent outputs, iteratively rectifies detected errors, and prunes irreparable outputs before they reach downstream agents.
- Its indicator pool distills error patterns from failed MAS trajectories into targeted inspection criteria for identifying reasoning pitfalls.
- ADv2 demonstrates robust adaptivity across diverse task complexities and scenarios as a plug-and-play intervention solution.
- ADv2 is framework-unaware and integrates across both fixed and dynamic MAS environments, overcoming ADv1’s static topology constraints.
2 Methodology
ADv2 models MAS execution as sequential information flow and intercepts each agent output for indicator-guided rectification or rejection before downstream transmission. Its indicator pool is mined offline from failed trajectories and deduplicated to support targeted auditing.
- 2.1 Preliminary: ADv2 represents the MAS as an ordered sequence of agents whose outputs update downstream knowledge bases and form an inference trajectory.The terminal agent aggregates preceding outputs to produce the final answer.
- 2.2 Test-Time Rectify-or-Reject Pruning: The framework intercepts intermediate outputs, iteratively refines them with a dedicated rectifier, and discards outputs that remain flawed before propagation.This protects downstream successors from unreliable information during real-time execution.
- 2.2 Test-Time Rectify-or-Reject Pruning: Adversarial indicators encode predefined error patterns that provide explicit inspection criteria for auditing agent outputs.Each indicator identifies an error type and its associated constraint, replacing open-ended self-correction with targeted scrutiny.
- 2.2 Test-Time Rectify-or-Reject Pruning: Two-stage retrieval first selects semantically similar candidate indicators, then filters them into a compact active subset for final auditing.The process uses task and action information from the reasoning context to guide coarse-to-fine retrieval.
- 2.2 Test-Time Rectify-or-Reject Pruning: The pass rate governs a tri-state policy: accept outputs meeting τpass, retry sub-threshold outputs before Tmax, and reject persistent failures.Rejected outputs are represented as empty messages to prevent error propagation; a safety threshold can instead trigger system-wide re-execution without pruning.
- 2.3 Failure-Driven Indicator Pool Construction: The indicator pool is built by mining failed MAS trajectories, having a teacher synthesize indicators, and retaining only strictly novel patterns through two-stage deduplication.Failure cases are identified when the MAS solution diverges from ground truth, while semantic retrieval and an LLM remove redundancy.
3 Experiment
Experiments evaluate ADv2 across fixed and dynamic MAS infrastructures, mathematical and code benchmarks, cross-model transfer, and representative test-time verification baselines. ADv2 improves both reasoning and code-generation performance, with especially strong results on harder mathematical tasks and evidence of indicator portability.
- Experimental Setup: The evaluation spans Fixed-MAS DAG workflows, Dynamic-MAS SelectorGroupChat, nine mathematical benchmarks, and four code-generation datasets.Baselines include Self-Refine Style, PRM-guided Search, and Multi-TAG Strategy.
- Mathematical Reasoning: ADv2 achieves the highest overall mathematical accuracies in both Fixed-MAS and Dynamic-MAS, reaching 53.50% and 53.69%, respectively.Table 1 covers relatively easy and harder mathematical benchmarks across both infrastructures.
- Mathematical Reasoning: ADv2 shows its most significant mathematical improvements on harder datasets, including OlymMATH Hard and AIME25.The method actively intercepts and rectifies complex logical flaws, unlike passive consensus and rigid step-wise scoring baselines.
- Indicator Portability Across Models: An indicator pool mined from a Qwen3-8B-based MAS transfers to Qwen3-14B, raising average accuracy from 52.56% to 56.29%.The experiment directly applies the pool to the larger-model system without redundant mining.
- Code Generation: ADv2 raises code-generation performance to 48.37%, compared with 46.63% for vanilla MAS and 46.09% for the Single Agent baseline.Gains are especially large on MBPP at 68.48% and LiveCodeBench at 32.00%.
4 Analysis
The ablations identify balanced rectification budgets, indicator retrieval, pass-rate thresholds, and pool deduplication as important to ADv2’s performance. Iteration depth also adapts to task difficulty, while indicator overlap reflects shared or distinct error patterns across benchmarks.
- Ablation Study: Tmax = 3 strikes the optimal balance between rectification thoroughness and system stability.Tmax = 2 fails to fully rectify reasoning flaws, while Tmax = 4 may induce over-correction or reasoning drift.
- Ablation Study: Kact = 5 outperforms both Kact = 3 and Kact = 7 because excessive indicators can overload reasoning with less relevant constraints.
- Ablation Study: A 60% pass-rate threshold best balances quality control with generative flexibility, avoiding propagation at 40% and over-rejection at 100%.
- Ablation Study: Omitting dual-stage deduplication reduces average accuracy to 52.32% by allowing redundant indicators to crowd out distinct error patterns.
- Ablation Study: Randomly sampled indicators reduce average accuracy to 51.65%, showing that semantically relevant retrieval is essential for locating specific error patterns.
- Iteration Dynamics and Adaptability: 94.2% of GSM8K outputs are accepted immediately, whereas OlymMATH Hard requires more multi-round rectification and exceeds 6.3% rejection.Rectification depth and rejection rate are proposed as proxies for task complexity.
- Retrieved Indicator Distribution: GSM8K and AQuA share 0.43 top-indicator similarity, while GSM8K and OlymMATH Hard share 0.11, indicating task-dependent error patterns.The analysis associates similar reasoning demands with higher indicator overlap.
- Performance–Cost Trade-off: ADv2 raises math accuracy from 50.86% to 53.69% while consuming more tokens than vanilla AutoGen, framing error mitigation as a compute–accuracy trade-off.
5 Related Work
Related work addresses MAS error propagation through architecture design, monitoring, and trajectory utilization. ADv2 combines these directions as a model-agnostic, plug-and-play module that actively rectifies errors at test time and uses historical failures for targeted supervision.
- Related Work: Prior MAS research uses robust architectures, error monitoring, and inference trajectories to mitigate cascading errors.
- Robust MAS Architectures: Robust architecture methods optimize graphs, reduce communication noise, or route specialized cooperative teams to suppress errors.
- Error Monitoring and Trajectory Utilization: Error monitoring methods detect workflow anomalies or rectify messages at test time, while trajectory-based methods construct training or supervision data.
- Positioning of ADv2: ADv2 is model-agnostic and plug-and-play, advancing monitoring from passive detection to active rectification through feedback-driven reflection.
Conclusion
AgentDropoutV2 optimizes MAS information flow through test-time rectify-or-reject pruning and an indicator pool distilled from historical failures. Its retrieval and rectification adapt to task difficulty and transfer across domains and backbone models.
- ADv2 intercepts agent outputs, retrieves domain-specific error indicators, iteratively refines them, and prunes unresolved errors before propagation.
- The framework shows adaptive retrieval and rectification across task difficulties, with transferability across domains and backbone models.
Limitations
ADv2 trades higher inference-token consumption for improved reasoning accuracy and can underperform lightweight baselines on simpler tasks. Its interventions remain advantageous relative to unaugmented vanilla MAS frameworks even in those settings.
- ADv2 incurs higher inference-token consumption because its rectify-or-reject mechanism iterates at test time.The authors frame this as a test-time scaling trade-off for enhanced accuracy.
- ADv2 occasionally trails CoT or Self-Refine on simpler tasks such as GSM8K, where straightforward reasoning is efficient for capable backbone models.The authors attribute this to overhead from aggressive intervention.
- Even on simpler tasks, integrating ADv2 consistently improves performance over unaugmented vanilla MAS frameworks.
A.1 Pseudo Codes
The framework uses two complementary procedures: test-time rectify-or-reject pruning during MAS execution and offline construction of an indicator pool from failure trajectories.
- Rectify-or-Reject Pruning: During MAS execution, each agent output is intercepted for rectification before information is propagated to successors.The process retrieves semantically similar indicators, selects active indicators, scrutinizes the output, and applies a gating decision.
- Rectify-or-Reject Pruning: The rectifier retrieves candidate indicators by semantic similarity and selects the final active indicators as references for detecting error patterns.The retrieved indicators guide output scrutiny and feedback generation.
- Indicator Pool Construction: The indicator pool is built by collecting failed MAS trajectories and having a teacher model synthesize indicators describing agents’ specific error patterns.The construction process also includes redundancy elimination to limit repository growth.
A.2 Indicator & Prompt Design
The indicator and prompt designs combine specialized checks with general-purpose fallbacks for mathematical and code rectification, while teacher prompts generate indicators from failed trajectories.
- Indicator Design: The constructed indicator pool includes domain-specific checks, such as verifying square-root calculation precision in mathematics.A general-purpose math indicator is used when no predefined pool is available.
- Indicator Design: The design supports both predefined domain indicators and general-purpose indicators when a specialized pool is unavailable.This combines targeted verification with a fallback design for mathematics.
- Prompt Design: Math and code rectifiers use separate prompt templates, while a teacher-model template generates indicators from failed MAS execution trajectories.The prompt designs assign distinct auditing and indicator-generation roles.
A.3 Case Study
The case study shows iterative auditing correcting a geometric misconception, while the broader design combines indicator retrieval, rectification, gating, and agent reset mechanisms.
- Case Study: The initial solver output and first retry reached the same wrong answer, 18 + 7π, because they used the nonagon’s internal angle for offset-boundary arcs.The first audit rejected the response but did not immediately resolve the error.
- Indicator Pool Construction: The indicator pool uses failed trajectories from MATH and AQuA for mathematics and MBPP, KodCode, and CodeContests for code.These datasets provide the source failures for offline indicator construction.
- Rectify-or-Reject Pruning: The rectification pipeline retrieves indicators, verifies outputs, and uses tri-state gating to pass, continue rectification, or trigger a system-wide reset.The algorithm includes a safety threshold and a maximum iteration count.
- Indicator Pool Construction: The pool-construction algorithm iterates over source instances, collects failures, generates candidate indicators with a teacher model, and removes redundant indicators.It outputs an optimized indicator pool for later retrieval.
- Prompt Design: The rectifier prompts require fatal, actionable errors with concrete mathematical or code corrections, while preserving outputs whose final answers remain correct.The math and code protocols use presumption of validity and impact tests.
- Case Study: The second solver revision identifies the external angle, rather than the internal angle, as determining each rounded corner of the nonagon’s offset boundary.This correction follows the second audit’s feedback.