PaperlayerView exact revision on arXiv

Source-linked AI summary

GRIP: Grounded Reasoning via Information-Restricted Premises

Lirui Teng

arXiv:2608.16776v1cs.AI

TL;DR

RAG systems can underuse retrieved evidence because query information dominates the latent state. GRIP restricts evidence through a low-dimensional noisy bottleneck while preserving a high-capacity query pathway, reducing query–bottleneck mutual information by roughly 30× and hallucination by 73% across five benchmarks.

  • Problem

    RAG systems can underuse retrieved evidence when high-capacity query pathways make latent states highly predictable from queries and weakly responsive to evidence.

  • Method

    GRIP routes retrieved evidence through a low-dimensional noisy bottleneck while retaining full-dimensional query access, restricting evidence representations to information unavailable from the query.

  • Results

    GRIP improves accuracy over strong baselines across five benchmarks, reduces estimated query–bottleneck mutual information by roughly 30×, and decreases hallucination by 73%.

  • Takeaways & Limitations

    The results support combining restricted evidence flow with high-capacity query access as GRIP’s operative design principle for controlling query dominance.

  • Takeaways & Limitations

    GRIP does not prove that its bottleneck enforces conditional residualisation, and its conclusions remain untested beyond the single Llama-3-8B backbone.

Abstract

from arXiv · show

High-capacity encoders in retrieval-augmented generation (RAG) can let the query dominate the latent state, leaving retrieved evidence functionally irrelevant. We call this failure mode query dominance. To address it, we introduce \textbf{GRIP} (Grounded Reasoning via Information-Restricted Premises), which imposes capacity asymmetry: the decoder keeps full-dimensional access to the query, while retrieved evidence passes through a severe stochastic bottleneck. This forces the evidence channel to encode only the residual information unavailable from the query. Across five reasoning benchmarks, GRIP outperforms strong iterative baselines, cuts a query--latent mutual-information diagnostic by roughly 30$\times$ (14.8 $\to$ 0.47 bits), and reduces hallucination by 73\%. Residual-alignment analysis further shows that the bottleneck output occupies subspaces less aligned with the query than baseline representations.

1 Introduction

The introduction identifies query dominance as a representational failure in RAG, where high-capacity query pathways cause models to under-use retrieved evidence. GRIP addresses this through capacity asymmetry, preserving full-dimensional query access while restricting evidence to a low-capacity bottleneck that transmits residual information.

  • Problem: RAG models often under-use retrieved text and fall back on parametric knowledge, even when it conflicts with the evidence.The introduction attributes this failure to query access through a high-capacity path while evidence shares the latent space.
  • Related work: Existing decoding and supervision interventions leave query–evidence latent geometry largely unchanged, allowing query-aligned features and parametric shortcuts to dominate.Examples include Self-RAG, context-aware decoding, and RAFT-style training.
  • GRIP: GRIP routes the query through a full-dimensional bypass and forces retrieved evidence through an aggressively low-dimensional, stochastic bottleneck.The bottleneck is described as having dz ≈4, while the decoder retains high-bandwidth access to the query.
  • GRIP: Under the tight capacity budget, GRIP pressures the evidence channel to transmit only information residual to the query.Query-correlated bits are redundant because the decoder already has high-bandwidth access to Q.
  • Diagnostic: Query–Latent Dependence measures I(Q; zk) between the query and evidence representation as a model-agnostic diagnostic of query dominance.Elevated QL dependence indicates that the evidence representation has collapsed into a compressed copy of the query.

2 Query Dominance in Latent States

Query dominance describes RAG systems whose decoder states remain strongly determined by the query while responding weakly to retrieved evidence. The section formalizes this failure behaviorally and representationally, while noting that query–latent dependence must be paired with behavioral tests because low mutual information alone is inconclusive.

  • Failure mode: Query dominance occurs when latent states are highly predictable from the query and only weakly responsive to retrieved-evidence variation.This representational failure makes generation resemble P(Y | Q) rather than the intended P(Y | Q, E).
  • Failure mode: Pretrained-transformer anisotropy lets query features occupy dominant directions, causing relevant evidence to act as a suppressed perturbation rather than part of a joint representation.The passage characterizes this as shortcut learning driven by broad semantic and frequency information.
  • Formalization: Under local Lipschitz and bounded-representation assumptions, weak separation between contrastive evidence conditions in latent space limits the decoder’s output separation.For fixed Q, evidence affects outputs through the internal evidence representation z.
  • Formalization: Behavioral query dominance requires low contrastive evidence sensitivity, SE(q) ≤τE, alongside high query-swap sensitivity, SQ(q) ≥τQ, with τE ≪τQ.The query-swap condition rules out trivial constant-output collapse by requiring sensitivity to query changes.
  • Diagnostics: High Query–Latent dependence indicates that evidence-channel state zk is predictable from the query, but low mutual information alone does not establish evidence use.The diagnostic is therefore interpreted through relative comparisons and paired with behavioral sensitivity tests, including zk randomization.

3 Design Principle: Capacity-Asymmetric Evidence

GRIP gives the query and reasoning context full-dimensional decoder access while forcing retrieved evidence through a restricted, noisy channel. This capacity asymmetry limits redundant query information and motivates predictions about dependence, channel reliance, and residual alignment.

  • Capacity-Asymmetric Principle: GRIP routes evidence through a deliberately restricted channel while preserving full-dimensional decoder access for the query and reasoning context.The goal is to prevent evidence from cheaply duplicating information already available from the query.
  • Capacity-Asymmetric Principle: Low dimensionality, premise-level compression, and additive noise limit transmitted features and discourage evidence representations from becoming second query representations.The evidence compressor maps extracted, verified premises to a low-dimensional noisy state.
  • Capacity-Asymmetric Principle: Under a tight capacity budget, query-predictable information in the evidence state has an opportunity cost unless it helps predict Y given Q.GRIP creates pressure to preserve evidence information predictive of Y while discouraging redundant query information, without explicitly optimizing the stated objective.
  • Empirical Predictions: The hypothesis predicts lower query–latent dependence, larger performance drops after evidence randomization, and weaker alignment between bottleneck states and query-dominant subspaces.Section 5 tests these diagnostics across five benchmarks, including an architecture-matched Llama-3 Iterative control on HotpotQA.

4 Architecture

GRIP implements capacity asymmetry as a four-stage pipeline that retrieves and verifies minimal predictive spans before noisy compression. The decoder receives the full-dimensional query and reasoning context alongside a severely capacity-limited evidence latent.

  • Pipeline: GRIP uses four stages: iterative retrieval, predictive span extraction, stochastic compression, and asymmetric decoding, with only the bottleneck and decoder trained.The retriever, extractor, and verifier remain frozen while gradients flow through the bottleneck and decoder.
  • Evidence processing: Retrieved candidates are entropy-ranked, reduced to minimal predictive spans, and filtered by an NLI verifier requiring entailment probability above 0.75.Entropy is computed under teacher-forced decoding, while the frozen extractor KL-matches span and full-passage next-step distributions with a length-sparsity penalty.
  • Training: Training uses a two-phase curriculum that first stabilises the bottleneck and decoder with dense retrieval, then enables entropy-guided selection while upstream modules stay frozen.Phase 1 covers epochs 1–5; Phase 2 covers epochs 6–20.
  • Stochastic bottleneck: With dz = 4 and σ2 = 1.0, noisy projection limits the evidence channel to approximately 2–4 bits per reasoning step after normalisation.The budget follows the Gaussian channel-capacity bound with average projected pre-noise power P ≈1.
  • Asymmetric decoding: The decoder conditions on the full-dimensional query and context plus zk as a single special token, creating a capacity gap of roughly three orders of magnitude.The context retains prior reasoning steps, while the evidence representation is constrained to four dimensions with additive noise.
  • Asymmetric decoding: The context update retains each verified span as text, preserving a full-dimensional semantic pathway alongside the bottleneck.This design is deliberate: ablating the raw-span pathway costs 8.2 accuracy points, while randomising zk costs 35.3 points.

5 Experiments

Experiments across five reasoning benchmarks show that GRIP’s capacity-asymmetric evidence processing improves performance and reduces query dominance. Ablations and diagnostics identify restricted, stochastic evidence encoding as central to this behavior.

  • Experimental Setup: GRIP is evaluated on five reasoning benchmarks spanning distractor, implicit, explicit, symbolic, and extractive reasoning regimes.The benchmarks are HotpotQA, StrategyQA, 2WikiMultihopQA, ProofWriter, and SQuAD 2.0.
  • Experimental Setup: All systems share the same frozen DPR-Wiki index, tokenization, and compute budget, while GRIP uses K = 2 steps with m = 10 passages per step.Evidence differs after re-ranking, extraction, and NLI filtering, isolating evidence-processing choices.
  • Task Performance: GRIP outperforms the architecture-matched Llama-3 Iterative control on all five benchmarks, including +7.2 EM on HotpotQA and +4.1 accuracy points on StrategyQA.Paired bootstrap comparisons are significant at p < 0.01 on HotpotQA (+7.2) and SQuAD 2.0 (+3.7).
  • Ablations: Removing the bottleneck raises QL dependence from 0.47 to 14.20 bits and reduces accuracy by 5.3 points, making it the most load-bearing component.Removing extraction or NLI verification instead increases hallucination through verbose or unsupported evidence entering the bottleneck.
  • Mechanism Diagnostics: GRIP reduces QL dependence by 20×–37× across all five benchmarks, while its capacity sweep shows that an intermediate bottleneck balances residual evidence transmission against query redundancy.Low QL dependence alone does not prove evidence use; the reported randomization test and converging diagnostics address this concern.
  • Ablations: Neither dimensional restriction nor stochastic corruption alone suffices: removing stochasticity raises QL dependence from 0.47 to 2.38 bits, while full-dimensional stochastic encoding raises it to 10.85 bits.The combination of restricted capacity and stochastic encoding is critical to GRIP’s information-control behavior.

6 Limitations

GRIP’s mechanism, evaluation scope, and measured information-control comparisons remain subject to important limitations. These include uncertainty about conditional residualisation, estimator bias, dataset dependence, architectural assumptions, and losses from severe compression.

  • Mechanism ambiguity: GRIP does not formally prove that its bottleneck enforces conditional residualisation.Deterministic and noise-only controls rule out dimensional restriction or stochasticity alone, while residual-alignment evidence shows weak query alignment.
  • MI estimator: CLUB is a loose, dimension-dependent upper bound, so the analysis assumes approximately consistent bias across models for meaningful relative comparisons.Variational mutual-information estimators may also violate basic self-consistency properties.
  • Dataset dependence: Hallucination reductions on SQuAD 2.0 and StrategyQA may reflect elicited parametric knowledge rather than improved evidence use.HotpotQA and 2WikiMultihopQA more cleanly probe low-coverage evidence dependence and therefore carry greater diagnostic weight.
  • Scope and failure modes: GRIP assumes explicit query–evidence pathway separation, which earlier-fusion architectures may not support.At dz = 4, severe compression can lose rare entities with strong parametric priors and cause substitution by frequent near-neighbours.

7 Conclusion

GRIP combines a low-dimensional, noisy evidence bottleneck with a high-capacity query bypass. Across five benchmarks, it lowers query–bottleneck mutual information, reduces hallucination, improves accuracy over Standard RAG, and produces bottleneck representations weakly aligned with the query.

  • Conclusion: GRIP routes evidence through a low-dimensional, noisy bottleneck while retaining a high-capacity query bypass.This capacity asymmetry restricts the evidence channel while preserving full query access.
  • Conclusion: 30× reduction in estimated query–bottleneck mutual information is observed across five benchmarks.The diagnostic decreases by roughly 30×.
  • Conclusion: 73% reduction in hallucination is observed across five benchmarks.GRIP also improves accuracy by 8.0 points on average over Standard RAG.
  • Conclusion: Residual-alignment analysis shows that the bottleneck output occupies subspaces weakly aligned with the query.This finding characterizes the separation between bottleneck representations and query representations.
Loading 2608.16776v1…