PaperlayerView exact revision on arXiv

Source-linked AI summary

CANOPY: Adaptive-Granularity Evidence Compression for Multimodal RAG

Hyojeong Yun, Jueun Kim, Wook-Shin Han

arXiv:2610.00923v1cs.IR

TL;DR

Multimodal RAG must determine not only which items to retrieve, but how much context to retain within each item; existing compression methods largely use modality-specific mechanisms. CANOPY applies hierarchical, parent-relative region selection and critic-guided follow-up retrieval, achieving higher average answer accuracy than evaluated baselines across five QA benchmarks while reducing reader-input evidence tokens in the stated setting.

  • Problem

    Retrieved items can contain irrelevant content, while uniform fine-grained selection can remove needed context, and existing compressors largely use modality-specific mechanisms rather than a shared region-by-region procedure.

    Source: §1 · pp. 1–2

  • Method

    CANOPY scores regions in hierarchies with a node encoder fine-tuned on gold evidence, applies parent-relative refinement, and couples compression with critic-guided follow-up retrieval.

    Source: §1 · p. 2

  • Results

    Across five QA benchmarks, CANOPY achieves higher average answer accuracy than evaluated retrieval baselines; in the unrouted Qwen3-VL-8B-Instruct setting, compression reduces evidence tokens by 14.2–27.7% with comparable accuracy.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Ablations indicate that additional retrieval drives the main accuracy gains on multi-hop QA, while compression reduces reader-input evidence tokens relative to the same iterative pipeline without compression.

    Source: §1 · p. 3

    The critic’s sufficiency judgment is an operational stopping criterion, not a guarantee of complete evidence; reaching the round limit also does not establish sufficiency.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Multimodal RAG retrieves text, tables, images, and videos, but choosing a retrieval granularity does not determine how much context to retain within each item. Coarse units include irrelevant content, while uniformly fine selection can remove context needed to interpret the evidence. Existing compressors address this trade-off with modality-specific mechanisms, leaving open a shared procedure for adapting the retained extent region by region across heterogeneous items. We introduce CANOPY (Canonical Projection over Hierarchy), a framework for adaptive-granularity post-retrieval evidence compression. CANOPY represents retrieved items as hierarchies and uses a node encoder fine-tuned on gold evidence to score regions against the query. Parent-relative refinement compares these scores to select multiple regions at different granularities without LLM calls for node-level pruning. Because compression cannot recover evidence that was never retrieved, a critic requests targeted follow-up retrieval when it judges the accumulated evidence insufficient; newly retrieved items are compressed before being added. Across five QA benchmarks over a 33M-item heterogeneous corpus, CANOPY achieves higher average answer accuracy than the evaluated retrieval baselines. Ablations indicate that additional retrieval drives the main accuracy gains on multi-hop QA. In the unrouted Qwen3-VL-8B-Instruct setting, compression reduces reader-input evidence tokens by 14.2-27.7% relative to the same iterative pipeline without compression, with comparable answer accuracy.

1 INTRODUCTION

CANOPY addresses the need to adapt how much context is retained within heterogeneous retrieved items, beyond choosing which items or retrieval granularity to use. It combines hierarchical region selection with critic-guided follow-up retrieval and evaluates the complete pipeline across five QA benchmarks.

1 INTRODUCTION

  • Existing compressors use modality-specific mechanisms, leaving open a shared, inexpensive procedure for adapting evidence granularity across heterogeneous items.Whole items may include distractions, while uniformly fine selection can remove context needed for interpretation; regions within one item can need different amounts of context.Source: §1 · pp. 1–2
  • CANOPY represents retrieved items as hierarchies and uses gold-evidence-fine-tuned query–region scores to retain multiple regions at different granularities without LLM calls for node-level pruning.Parent-relative refinement explores children scoring at least as high as their parent and retains the parent if no child qualifies.Source: §1 · p. 2
  • Across five QA benchmarks, CANOPY achieves higher average answer accuracy than evaluated retrieval baselines; compression cuts evidence tokens by 14.2–27.7% with comparable accuracy.The token reduction is relative to the same iterative pipeline without compression in the unrouted Qwen3-VL-8B-Instruct setting.Source: §1 · p. 3
  • Ablations indicate that additional retrieval drives the main accuracy gains on multi-hop QA.The critic requests targeted follow-up retrieval when it judges accumulated evidence insufficient, and newly retrieved items are compressed before addition.Source: §1 · pp. 2–3

2 RELATED WORK

Related work addresses multimodal retrieval, evidence compression, granularity selection, and iterative retrieval through different approaches. CANOPY focuses on a shared, within-item procedure for selecting original regions across heterogeneous retrieved items.

Multimodal retrieval-augmented generation

  • MuRAG, VisRAG, and UniversalRAG address source representation or item retrieval, while CANOPY complements corpus selection and query routing with a within-item decision.UniversalRAG routes queries to modality- and granularity-specific corpora.Source: §2 · p. 3

Evidence compression and selection granularity

  • Prior compression methods select or summarize evidence with mechanisms tailored to text, tables, or video, while CANOPY focuses on a shared procedure across heterogeneous items.Examples include sentence extraction or summarization, schema and cell retrieval, and informative-frame selection.Source: §2 · p. 3

Related approaches also vary the granularity of retrieved evidence

  • Mix-of-Granularity selects retrieval granularity by query, and RAPTOR retrieves over recursively clustered text summaries; CANOPY instead hierarchizes original regions within each retrieved item.Gold-derived parent–child preferences align node scores with fixed parent-relative refinement, allowing branches to stop independently without a retained-unit count per item.Source: §2 · p. 3

Iterative retrieval and evidence sufficiency

  • Iterative retrieval methods address missing information revealed by intermediate findings; CANOPY combines critic-guided additional retrieval with its distinct learned-score compression mechanism.The same refinement procedure is applied to items retrieved in subsequent rounds.Source: §2 · p. 4

3 METHOD

The method formulates post-retrieval compression as selecting original regions within each heterogeneous item, reducing reader input while preserving useful information and context. CANOPY combines hierarchical refinement, critic-guided follow-up retrieval, and gold-evidence-based encoder training.

3.1 PROBLEM FORMULATION

  • Post-retrieval compression selects regions within retrieved items to reduce reader input while retaining information and interpretive context useful for answering.The method treats evidence sufficiency separately because compression cannot supply information absent from retrieved items.Source: §3.1 · p. 4

3.2 CANONICAL PROJECTION OVER HIERARCHY

  • CANOPY builds hierarchies of original regions and uses parent-relative child scores to retain evidence at different depths as a forest.Refinement proceeds into qualifying children; otherwise, it retains the parent and its covered content.Source: §3.2 · pp. 4–5
  • Text leaves are sentences, table leaves are rows, video leaves are 30-second segments, and images remain whole; aggregate questions retain tables whole.Node-level pruning uses similarity comparisons without generative LLM calls, though node encoding and other LLM calls still incur costs.Source: §3.2 · p. 5

3.3 CRITIC-GUIDED ADDITIONAL RETRIEVAL

  • A critic judges accumulated compressed evidence against the original question and, when insufficient, formulates a targeted follow-up query for newly retrieved items.The loop stops when the critic judges evidence sufficient or the round limit is reached; neither condition guarantees completeness.Source: §3.3 · p. 5

3.4 ENCODER FINE-TUNING FOR CANOPY

  • Gold-evidence annotations define preferences that guide the encoder toward refining into gold-overlapping children and retaining fully gold nodes.Items without located gold spans are excluded from preference construction, and the resulting targets do not guarantee that all answer context is annotated or preserved.Source: §3.4 · p. 6
  • CANOPY trains these preferences with a pairwise ranking loss, updating only the node encoder’s LoRA parameters while keeping the query encoder frozen.The preferences train parent-relative comparisons used during refinement.Source: §3.4 · p. 6

4 EXPERIMENTS

CANOPY is evaluated on five QA benchmarks over a heterogeneous corpus, with component and selection ablations examining accuracy, evidence volume, and encoder effects. Results show stronger average accuracy than evaluated retrieval baselines, while compression reduces reader-input tokens at comparable accuracy in the unrouted setting.

4.1 EXPERIMENTAL SETUP

  • Experiments cover NQ, HotpotQA, OTT-QA, MMQA, and LVBench over about 33M heterogeneous items, using EM/F1 or LVBench accuracy.Retrieval baselines include Vanilla RAG, UniversalRAG, and IRCoT; additional compressors are compared on selected benchmarks.Source: §4.1 · pp. 6–7

4.2 EXPERIMENTAL RESULTS

  • CANOPY achieves higher average answer accuracy than evaluated retrieval baselines across five benchmarks; Table 1 reports overall performance and efficiency.With Qwen3-VL-8B-Instruct, CANOPY is best in EM and F1 on the four non-video benchmarks; its HotpotQA F1 gain over IRCoT is not significant.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 ABLATION STUDY AND ANALYSIS

  • Without routing, compression reduces evidence tokens by 14.2–27.7% versus w/o Projection, with accuracy changes whose 95% CIs include zero.Removing iterative retrieval lowers HotpotQA and OTT-QA EM by 10.1 and 14.9 points, respectively.Source: §4.3 · p. 8, Table 2 · p. 8
  • With gold items given directly, CANOPY exceeds both flat selectors on all five benchmarks while retaining more tokens.Against Gold Item, CANOPY saves 40.0% of LVBench tokens at comparable accuracy (56.4 versus 56.2); Gold Span reaches 53.0 LVBench accuracy.Source: §4.3 · p. 8, Figure 5 · p. 8
  • Larger branching factors do not consistently improve quality or reduce evidence volume, while retrieving more items generally increases tokens with modest or non-monotonic quality changes.Source: §4.3 · p. 9, Table 3 · p. 9
  • Fine-tuning raises correct parent–child refinement decisions from 28.3–43.0% to 34.3–71.9% across benchmarks.It also increases the share of pruned items from 30.4–39.6% to 85.2–95.6%, reducing reader-input tokens by 6.8–19.1% while changing F1/accuracy by +0.3 to +0.9 points, with all reported 95% CIs including zero.Source: §4.3 · p. 9, Figure 6 · p. 9, Table 4 · p. 9

5 CONCLUSION

The conclusion reports that CANOPY combines adaptive hierarchical compression with critic-guided additional retrieval, achieving higher average answer accuracy across five QA benchmarks. The accompanying benchmark details describe the in-domain tasks and their question splits.

5 CONCLUSION

  • Across five QA benchmarks over approximately 33M heterogeneous items, CANOPY achieves higher average accuracy than evaluated retrieval baselines.Ablations indicate that additional retrieval drives the main multi-hop QA accuracy gains, while compression reduces evidence volume at comparable accuracy in the unrouted Qwen3-VL-8B-Instruct setting.Source: §5 · p. 9

A.1 IN-DOMAIN BENCHMARKS

  • The in-domain evaluation spans single-hop text, multi-hop text, table-and-text, text/table/image, and video QA benchmarks, with disjoint training and test portions.The train portions are used only for node-encoder fine-tuning; NQ keeps 3,563 questions, HotpotQA 7,404, OTT-QA 2,214, MMQA 2,423, and LVBench 662.Source: A DATASET AND CORPUS DETAILS · p. 13, §A.1 · p. 13

A.2 OUT-OF-DOMAIN BENCHMARKS

The out-of-domain appendix describes three benchmark collections and their question/evidence conventions, then details the separate corpus pool and gold-span construction. These collections are kept separate from the in-domain unified corpus.

A.2 OUT-OF-DOMAIN BENCHMARKS

  • The out-of-domain benchmarks cover two-hop Wikipedia QA, financial-report table-and-paragraph QA, and video questions tied to moments in YouTube segments.The 2WikiMultiHopQA set uses 2,000 sampled development questions; TAT-QA excludes arithmetic questions; LongVideoBench keeps 733 questions with usable frame indices.Source: §A.2 · pp. 13–14

A.3 CORPUS CONSTRUCTION AND GOLD SPANS

  • The separate out-of-domain pool contains 45,020 items: 37,918 paragraphs, 5,613 tables, and 1,489 video segments.Its collections are merged into their own pool, which every out-of-domain query searches in full.Source: §A.3 · p. 15
  • Gold spans are answer-supporting fragments, sourced from benchmark annotations where available and derived from answers otherwise.Items without a located span are treated as whole-item gold and provide no fine-tuning instances.Source: §A.3 · p. 15

B IMPLEMENTATION DETAILS

Implementation details specify the shared fine-tuning setup, inference constraints, and evaluation conventions. The appendix algorithm refines each retrieved item by comparing child and parent query similarities.

B IMPLEMENTATION DETAILS

  • A single LoRA adapter is trained across five benchmarks from 4,162 questions and 18,575 gold-span-derived instances, with the vision tower frozen.Training uses only benchmark train portions disjoint from test questions.Source: B IMPLEMENTATION DETAILS · p. 15
  • Reader-input caps are 16k tokens on four benchmarks and 64k or 40k on LVBench, depending on the reader; repeated follow-up items are not compressed again.Of follow-up returns, 63.3% for CANOPY and 62.8% for CANOPY† were already in the accumulated evidence.Source: B IMPLEMENTATION DETAILS · p. 15
  • Latency is amortized batch wall-clock time for query encoding, routing, projection, and LLM calls, excluding vector search and node encoding.Evidence-token counts use the reader’s tokenizer and image preprocessing; paired bootstrap intervals are used where method differences are small.Source: B IMPLEMENTATION DETAILS · p. 16

C ALGORITHM OF CANOPY

  • The algorithm starts with each item’s root, replaces a parent with qualifying children when any child score meets or exceeds the parent score, and discards other child subtrees.Otherwise, it retains the parent without further refinement; the returned set contains selected evidence nodes.Source: C ALGORITHM OF CANOPY · p. 16

D COMPRESSION METHODS AT OTHER EVIDENCE BUDGETS

The appendix evaluates compression methods at larger evidence budgets and varies the frame budget for AKS on LVBench using gold videos.

D COMPRESSION METHODS AT OTHER EVIDENCE BUDGETS

  • Table 6 compares AKS at different frame budgets on LVBench, with both methods receiving the gold video instead of retrieved items.M is frames selected per video; its default is 16.Source: Table 6 · p. 16
  • AKS uses a default frame budget of M = 16 per video in the LVBench comparison.Both methods receive the gold video rather than retrieved items.Source: Table 6 · p. 16
  • Table 7 compares compression methods at larger evidence budgets using each benchmark’s own retrieval corpus.RECOMP-extractive defaults to N = 1 retained unit, and LongLLMLingua defaults to a target compression rate of ρ = 0.55.Source: Table 7 · p. 17

E ADDITIONAL COMPONENT ABLATIONS WITH MODALITY ROUTING

The modality-routed component ablation finds that compression reduces evidence tokens on all five benchmarks, while answer-quality effects vary by dataset. Removing iterative retrieval lowers HotpotQA and OTT-QA exact match substantially.

E ADDITIONAL COMPONENT ABLATIONS WITH MODALITY ROUTING

F SENSITIVITY TO BRANCHING FACTOR AND RETRIEVAL SIZE

Branching-factor and retrieval-size sweeps show that larger settings do not consistently improve answer quality, while increasing retrieval size can sharply raise token use. CANOPY with k = 5 also exceeds Vanilla RAG with k = 20 on three benchmarks without routing, using fewer evidence tokens.

F SENSITIVITY TO BRANCHING FACTOR AND RETRIEVAL SIZE

  • Within each routing configuration, varying branching factor B from 2 to 4 changes EM by at most 1.0 point on NQ, HotpotQA, and MMQA.The largest changes are 1.9 points in unrouted OTT-QA EM and 4.0 points in routed LVBench accuracy; B = 2 is the default.Source: F SENSITIVITY TO BRANCHING FACTOR AND… · p. 17, Table 9 · p. 18
  • Without routing, increasing k from 5 to 20 raises CANOPY evidence tokens 3.6–3.9×, while EM changes by at most 2.8 points and LVBench accuracy stops improving after k = 10.Average retrieval rounds remain between 2.0 and 2.1; the study uses k = 10 as default.Source: F SENSITIVITY TO BRANCHING FACTOR AND… · p. 17, Table 10 · p. 18
  • The pooled Qwen3-VL-8B-Instruct plot averages answer quality across five benchmarks and accumulates evidence tokens over retrieval rounds.Answer quality uses EM on the first four benchmarks and accuracy on LVBench.Source: F SENSITIVITY TO BRANCHING FACTOR AND… · p. 17
  • Without routing, CANOPY with k = 5 exceeds Vanilla RAG with k = 20 on HotpotQA, OTT-QA, and MMQA while using 31–35% as many evidence tokens.This compares the complete CANOPY pipeline, including additional retrieval, rather than isolating compression; on NQ, CANOPY at k = 10 approaches Vanilla RAG at k = 20 (39.7 versus 39.8 EM) at about half the tokens, while Vanilla RAG is stronger on LVBench.Source: F SENSITIVITY TO BRANCHING FACTOR AND… · p. 18, Table 10 · p. 18

G FLAT VERSUS HIERARCHICAL SELECTION WITH THE FINE-TUNED ENCODER

This comparison gives flat selection CANOPY’s fine-tuned encoder to isolate the selection rule from the encoder. CANOPY’s parent-relative traversal selects retained extent per item, whereas flat selection requires a reader-swept global count.

G FLAT VERSUS HIERARCHICAL SELECTION WITH THE FINE-TUNED ENCODER

  • Flat selection ranks leaves scored by the fine-tuned encoder and retains the top m, while CANOPY selects extent per item through parent-relative branch selection and stopping.The experiment sweeps m ∈ {1, 5, 10, 20} without modality routing, holding the rest of the pipeline unchanged.Source: G FLAT VERSUS HIERARCHICAL SELECTION… · pp. 18–19
  • Retaining a parent can preserve a lower-scoring leaf while discarding a higher-scoring leaf elsewhere, a selection no prefix of the leaf ranking reproduces.Retained parents account for 25–35% of refinement decisions in Appendix I.Source: G FLAT VERSUS HIERARCHICAL SELECTION… · p. 19
  • CANOPY lies at the knee of flat selection’s pooled quality–token curve, with token volume and quality comparable to the smallest m on the plateau.The comparison does not establish a quality–token advantage over the flat selector’s well-performing settings.Source: Figure 7 · p. 19, G FLAT VERSUS HIERARCHICAL SELECTION… · p. 19

H ACCURACY OF PARENT–CHILD REFINEMENT DECISIONS

This section evaluates fine-tuned parent–child refinement decisions and reports how refinement behaves across modalities and gold-item status.

H ACCURACY OF PARENT–CHILD REFINEMENT DECISIONS

I HOW REFINEMENT PROCEEDS

  • Text decisions keep 25.5% of nodes and drop 35.1%, while tables and video keep roughly a third and drop 28–30%.The authors attribute the difference to text questions often needing one or two sentences, while table rows and video segments more often carry answers together.Source: I HOW REFINEMENT PROCEEDS · p. 20, Figure 9 · p. 21

J REFINEMENT ON NON-GOLD ITEMS

K OUT-OF-DOMAIN EVALUATION

The out-of-domain evaluation finds CANOPY has the best average among five methods, with benchmark-specific results and case studies illustrating follow-up retrieval and adaptive retention. The prompts specify modality and aggregate-versus-lookup classification for retrieval.

K OUT-OF-DOMAIN EVALUATION

  • CANOPY has the best five-method average, 1.7 above Vanilla RAG, with the in-domain encoder and settings carried over unchanged.The evaluation uses one pooled corpus for three benchmarks not seen during in-domain evaluation.Source: K OUT-OF-DOMAIN EVALUATION · pp. 21–22, Table 12 · p. 22
  • CANOPY scores 72.1 vs. 72.4 F1 on TAT-QA and 48.2 vs. 49.5 accuracy on LongVideoBench.It passes 1,220 vs. 1,273 evidence tokens on TAT-QA and 57,195 vs. 57,736 on LongVideoBench.Source: Table 12 · p. 22, K OUT-OF-DOMAIN EVALUATION · p. 22
  • On 2WikiMultiHopQA, CANOPY is 10.0 F1 above Vanilla RAG; on compositional questions, scores are 47.0 vs. 25.7 F1.The initial top-10 contains both hops for only 47% of questions; the second hop is retrieved in a later round.Source: K OUT-OF-DOMAIN EVALUATION · p. 22

L CASE STUDY

  • In the Douillet case, CANOPY retains 3 of 13 UNESCO-table rows, then 2 of 8 Judo-passage sentences, retrieving the 1882 invention date in a follow-up.The retained table row links Douillet to judo, preserving an intermediate fact for the next reasoning step.Source: L CASE STUDY · p. 22, Table 13 · p. 23
  • In the Greenwich Village case, CANOPY retains the initial Center for Architecture passage in full and, from a follow-up Greenwich Village article, retains two of eight sentences to retrieve the name transformation.The retained sentences include the explanation that the Dutch name Groenwijck was Anglicized to Greenwich.Source: L CASE STUDY · p. 23

M PROMPTS

  • The prompts classify queries into one or more modalities and table questions as aggregate or lookup, where a wrong aggregate costs extra tokens but a wrong lookup can drop the answer row.For aggregate questions, retrieved tables are passed whole rather than pruned.Source: M PROMPTS · p. 25
Loading 2610.00923v1…