DuQUAD: A Dual-View Framework with Quality-Aware Evidence Pruning for Multi-Document Question Answering

1 Introduction

• Lost in the Middle leaves evidence in central document positions underused, and answer-irrelevant or weakly relevant retrieved documents further disperse attention from answer-bearing evidence.The cited work describes attention concentrating at context boundaries despite localized spikes when gold documents appear in intermediate positions.Source: §1 · p. 1
• Existing approaches use document reordering or attention modification, but weakly calibrated relevance or attention signals remain vulnerable to noise around mid-context gold evidence.The described failure setting is gold evidence buried among weakly relevant distractors.Source: §1 · p. 2
• The paper formalizes W-shaped attention spikes by adapting the Laplacian edge-detection principle to distinguish local spikes from smooth positional bias.Source: §1 · p. 2
• Structural Fusion Score combines attention contrast, semantic relevance, and structural inference to identify and rerank documents for local evidence selection.The framework pairs that Local Agent view with a Global Agent reasoning over the full context, followed by DocScore-based filtering and Judge synthesis.Source: §1 · p. 2
• DuQUAD improves accuracy by 3.0% on average over the strongest competing baseline and raises gold evidence recall by 9.4%.It also achieves a 13.8% gain over the stronger multi-agent baseline on NQ-15.Source: §1 · p. 2

Positional Bias in Large Language Models

• Positional bias makes language models over-attend to input boundaries and underuse intermediate information, while the reviewed methods primarily target attention-level symptoms rather than evidence utilization in reasoning.Source: Positional Bias in Large Language Models · p. 2
• Training-based methods use fine-tuning or supervision to improve position-agnostic or mid-context evidence use, but require model adaptation and are unavailable for off-the-shelf inference.Source: Positional Bias in Large Language Models · p. 2

Training-free Approaches to LiTM Mitigation

• Inference-time reordering, reranking, and compression leave evidence vulnerable to suppression or discard, while reviewed multi-agent systems do not explicitly model positional bias.Source: Training-free Approaches to LiTM… · pp. 2–3

3.1 Structural Fusion Score (SFS)

• SFS combines semantic relevance, structural inference, and attention contrast, with weighted softmax fusion that gives structural inference greater weight.The weights are α = 0.5, β = 1.0, and γ = 0.5, selected by grid search on Natural Questions for average gold evidence recall across positions.Source: §3.1 · p. 3, §3.1.4 · p. 4

3.1.2 Structural Inference (Φppl)

• Structural inference scores query-token generation conditioned on a document; lower Φppl values indicate stronger contextual support for the query.The scoring uses the same backbone LLM as the main reasoning model, avoiding a separate scoring model.Source: §3.1.2 · p. 4

3.1.3 Attention Contrast (Φcont)

• Attention contrast uses second-order local differences in document attention to emphasize sharp spikes over smooth positional trends.At boundaries without symmetric neighbors, the method uses a one-sided finite difference scheme.Source: §3.1.3 · p. 4

3.2 Dual-View Agent Collaboration

• The Global Agent reasons over all retrieved documents while the Local Agent uses an SFS-ranked compact subset, allowing each view to compensate for information missed by the other.The Local Agent uses k = 6 documents based on a recall–accuracy trade-off analysis.Source: §3.2 · p. 4

3.3 Quality-Aware Evidence Pruning

• Each agent selects up to three important documents, and their selections form a candidate pool of up to six documents for recall-oriented filtering.The pool is intended to preserve answer-relevant evidence that positional bias or contextual noise might otherwise obscure.Source: §3.3 · p. 5
• DocScore estimates whether a candidate document alone can answer the question, and documents are retained only when their score meets the threshold.The fixed threshold is τ = 0.4, chosen to preserve recall while enabling pruning; lower thresholds admit more irrelevant documents, while higher thresholds risk discarding gold evidence early.Source: §3.3 · p. 5
• For each retained document, the method selects its top two sentences by word or keyword overlap with a document-specific provisional answer.This sentence-level compression reduces the Judge Agent’s computational burden while preserving information directly supporting final reasoning.Source: §3.3 · p. 5

3.4 Final Decision via Judge Agent

• The Judge Agent combines both agents’ reasoning histories, DocScores, and sentence-level evidence spans to produce a consolidated final answer.Aggregating complementary outputs and evidence signals avoids over-reliance on any single source.Source: §3.4 · p. 5

4.1 Experimental Setup

• The study uses Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct as backbones and applies them consistently across baselines for fair comparison.Main results are reported for both models; additional analyses use Llama-3.1-8B-Instruct by default.Source: §4.1 · p. 5
• Evaluation covers Natural Questions, MuSiQue, and HotpotQA, including controlled gold-document positions in NQ and multi-hop reasoning settings in MuSiQue and HotpotQA.NQ places one gold document among 19 distractors at positions 1, 5, 10, 15, or 20; MuSiQue requires 2–4-hop reasoning and HotpotQA uses two gold paragraphs among eight distractors.Source: §4.1 · pp. 5–6
• Accuracy measures whether the generated response contains the ground-truth answer string; multi-hop tasks also report Exact Match and F1.Exact Match measures strict identity, while F1 assesses word overlap and generation precision.Source: §4.1 · p. 6

4.2 Baselines

• Baselines span document reordering and reranking, multi-agent reasoning, and context optimization methods.The reordering and reranking group includes sparse, dense, and hybrid retrieval variants; the other groups include Tree of Agents, MAIN-RAG, Found in the Middle, and LongLLMLingua.Source: §4.2 · p. 6

5.1 Overall Performance

• DuQUAD achieves the best accuracy across all 14 accuracy settings, improving over the strongest competing baseline by 3.0 percentage points on average.The trend holds for both Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct.Source: §5.1 · p. 7
• On MuSiQue and HotpotQA, DuQUAD achieves the best accuracy under both backbones and the highest F1 on both multi-hop benchmarks.With Llama-3.1-8B-Instruct, it improves over the strongest competing baseline by 2.5 percentage points on MuSiQue and 2.9 percentage points on HotpotQA; HotpotQA EM is an exception.Source: §5.1 · p. 7

5.2 Gold Evidence Recall and DocScore Stability

• SFS reaches 90.8% gold evidence recall, a 9.4% gain over the best single-signal method and a 4.0% improvement over the strongest multi-signal baseline.DocScore gives gold documents higher scores than non-gold documents across NQ positions, with 90.4% cross-model agreement on gold evidence selection.Source: §5.2 · p. 8

5.3 Ablation Analysis

• Removing any component degrades performance; the Local Agent’s removal causes the largest NQ-15 drop, while Global Agent or pruning removal causes larger MuSiQue drops.Table 2 evaluates accuracy changes across three datasets when each component is omitted; HotpotQA shows smaller changes.Source: §5.3 · p. 8, Table 2 · p. 8

5.4 Cost Efficiency Analysis

• DuQUAD improves accuracy over MAIN-RAG by 13.8% and ToA by 25.1%, while reducing end-to-end latency by 3.3× and 3.9×, respectively.It incurs additional latency relative to lightweight retrieval and reranking methods; its Local and Global Agents can run in parallel to further reduce latency.Source: §5.4 · pp. 8–9, Figure 5 · p. 8

• DuQUAD pairs a Local Agent operating on prioritized evidence with a Global Agent reasoning over the full document set.The Local Agent uses SFS, which integrates semantic relevance, structural inference, and attention contrast to identify gold documents despite positional noise.Source: §6 · p. 9
• Quality-aware pruning removes low-confidence evidence before final reasoning, and the combined components raise gold evidence recall to 90.8%.Source: §6 · p. 9
• Across evaluated benchmarks, DuQUAD achieves the best accuracy among baselines while requiring substantially less computation than existing multi-agent frameworks.Source: §6 · p. 9

7 Limitations

• Pruning may reduce contextual coverage when relevant evidence is weakly expressed or distributed across documents, particularly for complex multi-hop questions.The authors frame this alongside the trade-off between contextual coverage, noise suppression, and reasoning stability.Source: §7 · p. 9
• If answer-bearing documents are absent from the initial retrieved set, internal reasoning alone cannot recover the missing evidence.The authors state that performance is bounded by retrieval quality, as in other retrieval-augmented frameworks.Source: §7 · p. 9
• Even with answer-bearing evidence correctly retrieved and scored, final synthesis may fail on complex multi-hop questions requiring close tracking of related intermediate entities.Source: §7 · p. 9
• Evaluation uses Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct; broader evaluation across model families, capabilities, and sizes is left for future assessment.Additional analyses use Llama-3.1-8B-Instruct by default.Source: §7 · p. 9

A Baseline Implementation Details

• Hybrid RRF fuses BM25 and Qwen3-Embedding rankings, then an optional Jina-Reranker further reorders the fused ranking.RRF sums reciprocal-rank scores; the described setup uses k = 60 and Qwen3-Embedding as the dense retriever.Source: A Baseline Implementation Details · pp. 11–12
• The baseline set includes hierarchical Tree of Agents and collaborative MAIN-RAG, plus FiM, a position-aware U-shaped reordering strategy.LongLLMLingua is also described as compressing prompts by discarding low-importance tokens estimated with Llama-2-7b-chat-hf.Source: A Baseline Implementation Details · p. 12

B Experimental Settings and Dataset Details

• NQ uses five positional splits, each with 1,000 examples, placing one gold document among 19 distractors in a 20-document sequence; all questions are single-hop.The five splits share document sets and differ only in the gold document’s position.Source: B Experimental Settings and Dataset… · p. 13
• MuSiQue evaluates answerable questions spanning 2–4 hops, with supporting and distractor paragraphs and no single paragraph sufficient to answer independently.The benchmark has approximately 25K questions across six reasoning graph types.Source: B Experimental Settings and Dataset… · p. 13
• HotpotQA uses the distractor setting with two gold paragraphs and eight distractors for two-hop bridge or comparison questions.Bridge questions require an intermediate entity to locate the answer; comparison questions contrast properties across paragraphs.Source: B Experimental Settings and Dataset… · p. 13
• Table 4 reports benchmark-level evaluation characteristics and context-length statistics, including evaluation-sample counts, document and gold-document counts, question type, and input-context lengths; experiment-specific settings appear in Table 3.Datasets and sample sizes for individual experiments vary according to experimental purpose.Source: B Experimental Settings and Dataset… · pp. 12–13

C DocScore Threshold Sensitivity Analysis

• 80.70% gold-document recall is unchanged from τ = 0.3 to τ = 0.4; τ = 0.4 is chosen as the highest threshold preserving recall while enabling pruning.Recall declines beyond τ = 0.4; the analysis uses 1,000 NQ examples.Source: C DocScore Threshold Sensitivity… · p. 13, Table 5 · p. 13

D Recall–Accuracy Trade-off for Local Document Selection

• On NQ-15, recall@k rises and begins to saturate around k = 6, while answer accuracy improves substantially up to k = 6 and gains little or fluctuates beyond it.The authors use k = 6 as the default budget, balancing high recall with controlled noise.Source: D Recall–Accuracy Trade-off for Local… · pp. 13–14, Figure 6 · p. 13

E Ablation on SFS Weight Selection

• 91.4% average gold evidence recall is achieved by SFS weights (0.5, 1.0, 0.5), the selected configuration that emphasizes structural inference signals.The grid search evaluates recall across answer positions on NQ; multiple configurations achieve comparable recall.Source: E Ablation on SFS Weight Selection · p. 14, Table 6 · p. 14

F Reproducibility

• The paper releases the exact prompts for reasoning agents, the Judge, and score estimation, and fixes decoding at temperature 0.0 unless otherwise specified.The same deterministic generation setting is used for baselines to support faithful replication and fair comparison.Source: F Reproducibility · p. 14

G.1 Statistical Significance Analysis

• 35 of 36 DuQUAD-versus-baseline comparisons are significant at p < 0.05, with positive mean improvements across all 36.The sole exception is FiM on MuSiQue; DuQUAD’s improvement there is +2.5%, the smallest MuSiQue gap.Source: §G.1 · p. 15, Table 7 · p. 15
• FiM is the only nonsignificant comparison, on MuSiQue, where its U-shaped reordering places relevant documents at salient positions suited to compositional evidence.The other MuSiQue gaps range from +6.4% to +22.1%; DuQUAD’s improvement direction remains consistent, with clearer statistical advantages in settings such as NQ.Source: §G.1 · p. 15

G.2 Robustness to Increased Document Count

• At 30 documents, DuQUAD reaches 75.6% accuracy, 7.8 points above Hybrid (RRF) + Rerank’s 67.8%.DuQUAD’s accuracy decreases by only 0.5 points as the NQ-15 context grows from 20 to 30 documents.Source: §G.2 · pp. 15–16, Table 8 · p. 16

H Cost Analysis

• Lightweight retrieval and reranking methods require one answer-generation LLM call, while DuQUAD costs more than those methods but less than existing multi-agent frameworks.The comparison measures LLM calls, token consumption, and end-to-end latency, with retrieval-side computation for retrieval and reranking methods excluded from LLM call and token counts.Source: H Cost Analysis · p. 16
• DuQUAD reaches 76.1% NQ-15 accuracy and, versus MAIN-RAG, uses 10× fewer LLM calls, 2.35× fewer tokens, and 3.29× less end-to-end latency.Its latency is 9.93× Vanilla; compared with ToA, it reduces calls by 5.65×, tokens by 2.81×, and latency by 3.88× while outperforming both on all three benchmarks.Source: H Cost Analysis · p. 16, Table 9 · p. 17

I Boundary Handling for Attention Contrast Computation

• At document boundaries, attention contrast uses the difference between a document and its nearest neighbor rather than padding or reflection.This one-sided finite-difference scheme is intended to reduce artificial smoothing and preserve relative attention salience at the edges.Source: I Boundary Handling for Attention… · p. 16

J Document-Level Attention Scoring

• Document-level attention scores average attention mass from the final query token over each document’s token span.The scoring requires access to the model’s self-attention tensors but does not modify model parameters or decoding behavior.Source: J Document-Level Attention Scoring · p. 17

K Analysis of Local–Global Complementarity

• The Judge resolves agent disagreements correctly on all three datasets, with 74.5% accuracy on NQ, 58.5% on HotpotQA, and 67.2% on MuSiQue.Disagreement rates range from 13.5% on HotpotQA to 24.3% on NQ; on MuSiQue, both agents are wrong on 52.3% of examples, often before final adjudication during multi-hop entity tracking.Source: K Analysis of Local–Global… · pp. 17–18

L DocScore Statistics and Cross-Model Evidence Stability

• Gold documents receive higher mean DocScores and selection rates than non-gold documents under both Llama and Qwen.When at least one backbone selects the gold document, both select it in 90.4% of cases, while additionally selected non-gold documents differ 90.1% of the time when both select gold.Source: L DocScore Statistics and Cross-Model… · p. 19

(1) Local selection via Structural Fusion Score (SFS)

• Local selection uses a Structural Fusion Score built from each document’s aggregated attention, semantic relevance, and structural inference scores.The Local Agent reasons over the selected local documents, while the Global Agent reasons over the full document set.Source: (1) Local selection via Structural… · p. 21

(3) DocScore-based Quality Gating

• The Score Agent assigns each important document a DocScore and answer span; documents with scores below threshold τ are pruned.Source: (3) DocScore-based Quality Gating · p. 21

(4) Evidence Snippet Extraction

• For each retained document, evidence snippets are selected by overlap between document sentences and the predicted answer span.Source: (4) Evidence Snippet Extraction · p. 21

(5) Final Decision

• The Judge Agent generates the final answer using the question, both agents’ outputs, selected snippets, and retained-document scores.Source: (5) Final Decision · p. 21