Beyond the Query: Do Retrieval Signals Improve Adaptive Multimodal RAG Routing?

• The study asks whether retrieval signals improve the next RUN/SKIP decision after the query is already known.Retrieval scores and evidence statistics are produced from the query, so they may correlate with step benefit without adding information beyond the query.Source: §1 · p. 1
• Matched query-only and query+retrieval routers hold optional actions, router family, training, and evaluation fixed.This comparison isolates the incremental routing value of the tested retrieval signals within the evaluated setup.Source: §1 · p. 1
• Adding retrieval information improves routing score by +0.01621 on development data, but the gain is +0.00288 on the 720-query final set, with a 95% confidence interval including zero.The development advantage does not hold clearly on the held-out final evaluation.Source: §1 · p. 1
• Document experiments span weakly predictable DOC CLIP benefit, Tile D2 context loss, and rare, hard-to-predict Whole-page D2 benefit.Audio and Video show positive correlation between predicted and observed step benefit without establishing a query+retrieval routing advantage.Source: §1 · p. 2
• On the held-out final evaluation, tested retrieval inputs do not yield a reliable routing advantage over query features.The authors distinguish this result from a claim that routing is useless or retrieval state can never help.Source: §1 · p. 2
• A retrieval signal can predict future step benefit without improving the final routing decision once query information is available.The authors therefore call for a matched query-only control when attributing routing value to retrieval state.Source: §1 · p. 2
• The contributions include paired before/after evaluation of optional document, audio, and video steps, a controlled query-versus-query-plus-retrieval ablation, and step-level diagnostics.The diagnostics examine context loss, rare benefit, weak pre-action signals, RUN/SKIP decisions, and serving-time leverage.Source: §1 · p. 2

• Prior adaptive-RAG methods decide when or what to retrieve, how much retrieval to use, or which resource level to select.Examples include systems using post-retrieval features or lightweight probes to choose retrieval depth or strategies.Source: §2 · p. 2
• Adaptive Re-Ranking and budget-aware Active-RAG separately examine learned routing, retrieval benefit, harmful retrieval, budgets, and trigger-side costs.The paper says these analyses are not individually new; its comparison asks whether query-conditioned retrieval signals add routing value.Source: §2 · p. 2
• Prior work distinguishes relevant evidence from evidence that helps answer correctly, motivating downstream answer change rather than retrieval relevance as the target.The paper also notes that a modality can be relevant even when using it is unnecessary for the final answer.Source: §2 · pp. 2–3
• The paper uses multimodal RAG as a test bed and does not propose a new multimodal retriever.Optional steps can alter representation, visual detail, or surrounding context, making their value difficult to predict beforehand.Source: §2 · p. 3
• The paper does not claim the general distinction between prediction and decisions as new; it tests routing against a matched query-only baseline.Its specific question is whether retrieval information improves routing beyond query information.Source: §2 · p. 3

3 What We Test

• Observed quality change is measured by running each optional step and comparing answer quality before and after; the router cannot see the future outcome when deciding.Positive change means help, negative means harm, and zero means no change.Source: §3 · p. 3

3.1 Primary Test: Does Query-Conditioned Retrieval State Add Value?

• The primary test compares matched query-only and query+retrieval routers while holding actions, router family, training, and evaluation fixed.The outcome measures incremental routing value, not prediction accuracy, serving time, or conditional mutual information.Source: §3.1 · pp. 3–4

3.2 Supporting Checks: Why Might the Incremental Gain Be Small?

• Supporting checks assess whether a step has value to capture, whether available inputs predict benefit, and what the step changes or costs.These checks help interpret the matched comparison; they are not a causal or additive decomposition, and positive benefit correlation alone does not establish added value beyond the query.Source: §3.2 · p. 4

4 Experimental Setup

• Document evaluation uses DocVQA with an ANLS-style score; Audio uses Clotho-AQA and Video uses NExT-QA, with answer quality reported for the latter two.Paired bootstrap intervals quantify routing-score differences, while Spearman rank correlation measures predicted versus observed benefit.Source: §4 · pp. 4–5
• The full before/after evaluation covers 3,574 queries and 10,722 evaluated system states; the held-out final set has 720 queries, 240 per modality.The final queries were excluded from router training and model selection, but source uniqueness across splits is not established.Source: §4 · p. 4
• Both main routers use histogram gradient boosting and predict whether an optional step will raise or lower answer quality; one adds pre-step retrieval and evidence statistics to query features.Actions, model family, and training procedure are otherwise matched.Source: §4 · pp. 4–5

5.1 RQ1: Do Retrieval Signals Add Routing Value Beyond the Query?

• +0.00288 is the query+retrieval router’s routing-score improvement over query-only on the 720-query final evaluation, with a 95% CI of [−0.00726, 0.01107] that includes zero.The development improvement was +0.01621; the final result does not establish a reliable routing improvement.Source: Figure 2 · p. 5, §5.1 · p. 5

5.2 RQ2: Is the Router Simply Too Weak?

• The strongest tested router design improves the development-time robust-regret score by only about 0.57% relative to baseline.Larger features and decision-weighted training change the result only slightly; this does not rule out stronger routers beyond those tested.Source: Table 1 · p. 5, §5.2 · p. 5

5.3 RQ3: What Do the Document Steps Reveal About the Available Signal?

• DOC CLIP improves 19 of 200 queries, leaves 179 unchanged, and harms 2; its benefit is only weakly predictable from pre-action features (ρ = 0.1436).Its median added latency is 21.76 ms against an 89.26 ms required base stage, limiting the maximum serving-time saving from skipping it.Source: Results for the three document steps · p. 6, Table 2 · p. 7
• Tile D2 improves only 4% of queries and harms 7%, with negative average quality change; selected tiles can replace useful evidence and remove broader page context.The action’s potential to damage evidence constrains what router improvement alone can address.Source: Results for the three document steps · p. 6, The three document steps · p. 6
• Whole-page D2 preserves full-page context but helps only 4.67% of queries, and query+retrieval benefit prediction has almost no rank association (ρ = 0.0100).It adds 42.86 ms median latency versus a 1030.32 ms base stage; the safer action still has rare, hard-to-identify useful cases.Source: Results for the three document steps · p. 6, The three document steps · p. 6, Table 2 · p. 7

5.4 RQ4: Can a Signal Exist Without Adding Value Beyond the Query?

• Audio and Video show benefit predictability without established routing gain: Audio’s query+retrieval routing score is 0.0835 lower, while Video’s difference is essentially zero (−0.0006).The diagnostic correlations are positive for both modalities, but the evidence does not establish a meaningful predictive advantage for Video.Source: §5.4 · pp. 6–7, Table 3 · p. 7

6 Discussion

• On the held-out final evaluation, adding tested retrieval signals does not reliably improve routing over the query-only baseline.The matched baseline matters because query difficulty can affect both retrieval behavior and the usefulness of later computation.Source: §6 · p. 7
• Audio and Video show positively correlated predicted and observed step benefit without query+retrieval routing outperforming query-only routing in the diagnostics.Predictive association therefore does not establish incremental routing value.Source: §6 · p. 7
• The result concerns the tested retrieval features, representations, router families, and decision points, not routing or retrieval state in general.Different state representations, observable signals, or decision points could perform better.Source: §6 · p. 7
• Document outcomes vary by action: Tile D2 can remove useful context, Whole-page D2 preserves it but useful cases remain rare, and DOC CLIP has limited serving-time leverage.These diagnostics do not causally decompose the main query-only versus query+retrieval result; routing difficulty depends on both available information and the action being predicted.Source: §6 · p. 7
• Table 4 presents supporting diagnostics whose rows are neither mutually exclusive failure classes nor an additive decomposition; Audio/Video serving-time savings remain unresolved.The discussion distinguishes best-case action gain, prediction correlation, and final routing quality as different questions that cannot alone attribute value to retrieval state.Source: Table 4 · p. 8, §6 · p. 8
• The contribution is a boundary-condition and measurement result: retrieval-state predictability need not become incremental routing value, guiding what future routing work should test.Possible next steps include a larger router, better state representation, different action, or different decision point.Source: §6 · p. 8

7 Limitations

• The held-out result is small and statistically inconclusive, and many other models and state representations remain possible.The authors therefore do not claim a better router or general ineffectiveness of retrieval-state features.Source: §7 · p. 8
• Audio and Video incremental retrieval time remains unresolved because stored zero timing values were placeholders, not valid warm-serving measurements.The final 720-query set was held out from router training and model selection, but artifacts do not establish complete source-level separation across every split.Source: §7 · p. 8
• The study lacks a completed positive-control experiment showing retrieval features clearly improving routing over a query-only control.The planned NFCorpus control stopped before scientific outcomes because a required source-separated split was not feasible.Source: §7 · p. 8
• The comparison covers limited benchmarks, one main multimodal generator, and finite actions and retrieval features; it is an operational ablation, not a conditional-mutual-information measurement or proof of feature redundancy.Decisions are within the current modality rather than jointly selecting across document, audio, and video.Source: §7 · p. 8

8 Conclusion

• Across the tested setting, retrieval signals do not provide a reliable held-out routing advantage over query features alone.Audio and Video also show that predicted benefit can track observed benefit without improving the final RUN/SKIP decision.Source: §8 · p. 8
• Adaptive-RAG studies should credit retrieval-state features with routing value only when they improve over a matched query-only control.The conclusion does not imply that routing is unnecessary or that other representations, actions, models, or decision points cannot work better.Source: §8 · p. 8

A Evidence Checks and Experiment History

• Final scientific results use the multimodal pipeline internally named S1 V2, excluding older branches that lacked real multimodal generation or used text-only or non-native substitutes.An older L0 proxy leaked query information, and ActivityNet was not used as the main video action family because authentic media and action conditions were unmet.Source: A Evidence Checks and Experiment History · p. 10
• Historical Audio and Video Spearman calculations mishandled ties with ordinal ranks; the final analysis uses average ranks.The earlier handling differed from the standard treatment stated in the passage.Source: A Evidence Checks and Experiment History · p. 10
• Stored zero-valued Audio and Video timing fields were placeholders, not valid latency measurements.S1 V2.1 was a valid development-set ablation but was not selected as the frozen baseline.Source: A Evidence Checks and Experiment History · p. 10

B Paired Before/After Data and Final Evaluation

• The corrected paired dataset physically evaluates each optional RAG step before and after, while routers use only information available before the step.It contains 3,574 queries and 10,722 evaluated states; the held-out final set has 720 queries, 240 per modality.Source: B Paired Before/After Data and Final… · pp. 10–11
• 0.00288 is Gmacro, the equal-weight average of final routing-score differences between query+retrieval and query-only routers.The table’s answer-quality trajectories are descriptive and do not imply that every optional step improves quality monotonically.Source: Table 5 · p. 11
• The final 720-query set was not used to train routers or choose router designs, but source separation across every split and benchmark is not established.No source overlap is reported only where directly verified, such as the fresh DOC CLIP evaluation.Source: B Paired Before/After Data and Final… · p. 11

C Additional Router Variants

• Development tested five router designs, including RUN/STOP classification, staged gain prediction, signed quality-change prediction, a 139-feature representation, and cost-weighted training.The optional RAG steps were held fixed across these designs.Source: C Additional Router Variants · p. 11
• Directly predicting signed quality change yields the best robust-regret result, improving over baseline by only about 0.57%; robust regret is a fixed development-only score across held-out folds and relative-cost settings.RUN cases make up 8.128% of examples and STOP cases 91.872%; exactly zero accounts for 81.464% of observed quality changes.Source: C Additional Router Variants · p. 11

D Correcting Spearman Correlation for Ties

• The final Spearman calculation uses standard average ranks for ties; earlier ordinal-rank values are excluded from final tables and claims.Source: D Correcting Spearman Correlation for… · p. 11

E Relative Evaluation Cost and Real Serving Time

• The routing comparison uses the same fixed relative policy cost for both routers, while production value depends on avoidable warm online latency.The evaluation cost and measured serving time are treated separately.Source: E Relative Evaluation Cost and Real… · p. 11
• Audio and Video retrieval timing fields are 0.0 placeholders, not measurements, so real incremental retrieval time remains unresolved for A0/A1/A2 and V0/V1/V2.The placeholders are not replaced with generation latency or values inferred from model complexity.Source: E Relative Evaluation Cost and Real… · p. 11

F Planned Positive Control That Could Not Be Completed

• The positive-control split was stopped after only 12 source-separated confirmation candidates remained, far short of the required 150.Retrieval, reranking, quality, ΔQ, best-case-selection, and routing analyses were not run; the failed split provides neither positive nor negative evidence about routing performance.Source: F Planned Positive Control That Could… · p. 12