PaperlayerView exact revision on arXiv

Source-linked AI summary

CAGE: Coherence-Aware Graph Encoding for Retrieval-Augmented Generation

Tong Qi, Jingyu Wu, Youbing Yin, Spencer Hong, Daben Liu, Erin Babinsky

arXiv:2609.04647v1cs.CLcs.IR

TL;DR

RAG systems can retrieve passages that are individually relevant but collectively contradictory or disconnected, limiting the evidence available to generation. CAGE reranks passages using graph-based between-chunk coherence across four dimensions, and it improves downstream Exact Match while matching or exceeding retrieval baselines on key multi-hop benchmarks.

  • Problem

    Existing RAG retrieval and reranking score passages independently, leaving the coherence of the assembled evidence set under-modeled.

  • Method

    CAGE converts retrieved passages into directed heterogeneous entity graphs, reweights min-out-degree factual anchors, encodes graphs with R-GCN, and fuses coherence with query relevance.

  • Results

    CAGE achieves the highest Recall@5 on bridge-dominated benchmarks and consistently improves Exact Match, including when retrieval recall decreases on MuSiQue.

  • Takeaways & Limitations

    Structurally coherent context is a complementary signal to per-passage relevance and can yield more precise answers even with lower retrieval recall.

  • Takeaways & Limitations

    The evaluation uses multi-hop supporting-fact annotations as a proxy because benchmarks lack fine-grained labels for the four coherence dimensions.

Abstract

from arXiv · show

Traditional Retrieval-Augmented Generation (RAG) systems score each passage independently against the query, assembling context sets that may be individually relevant yet collectively incoherent. We introduce Coherence-Aware Graph Encoding (CAGE), a reranking framework that models "between-chunk coherence" across four dimensions: Intra-Domain Relevance, Noise Resistance, Informational Bonding, and Factual Consistency. Our pipeline transforms retrieved passages into directed heterogeneous entity graphs, amplifies factual anchors via min-out-degree reweighting, encodes structural patterns through a Relational Graph Convolutional Network, and fuses inter-chunk coherence with query relevance for final ranking. Evaluated across four multi-hop benchmarks, CAGE matches or outperforms strong baselines including monoT5 in Recall@5 on bridge-dominated datasets and consistently improves downstream Exact Match, demonstrating that structurally coherent context yields more precise answers even when retrieval recall is comparable or lower.

1 Introduction

RAG systems commonly rank passages independently, so individually relevant chunks can form contradictory or disconnected context. CAGE makes between-chunk coherence an explicit reranking signal alongside query relevance.

  • Motivation: Independent passage scoring can assemble a top-k context that is collectively incoherent despite strong individual relevance.Such contexts may contain outdated, off-target, or contradictory evidence.
  • Contribution: Between-chunk coherence comprises Intra-domain Relevance, Noise Resistance, Informational Bonding, and Factual Consistency.These dimensions cover referential alignment, distractor suppression, reasoning-chain connectivity, and logical compatibility.
  • Contribution: CAGE converts retrieved chunks into directed heterogeneous entity graphs, encodes them with R-GCNs, and combines query relevance with structural coherence.Min-out-degree reweighting emphasizes specific entities that anchor factual claims.
  • Results: Across four multi-hop benchmarks, CAGE improves Recall@5 against strong retrieval baselines and consistently achieves higher downstream Exact Match.The framework evaluates coherence before answer generation rather than only judging generated responses.

2 Related Work

Prior coherence and graph-based RAG work does not directly validate whether a retrieved passage set is structurally coherent. CAGE addresses this gap by optimizing between-chunk coherence before generation.

  • Coherence Research: Traditional text-coherence models study relational structure within continuous, single-author documents, unlike independently sourced retrieval passages.Their assumptions do not directly cover a retrieved set assembled without a shared discourse plan.
  • Evaluation Gap: RAGAS and TruLens evaluate generated-answer properties, while NLI detectors focus on claim-level contradictions rather than higher-order reasoning structure.These approaches do not assess whether retrieved chunks form connected reasoning chains or jointly support an answer.
  • RAG Baseline: Vanilla RAG retrieves top-k passages by embedding cosine similarity and concatenates them without an inter-passage signal.CAGE applies graph reranking to this same retrieved set to isolate coherence-aware scoring.
  • Graph-Based RAG: GraphRAG, HippoRAG, GNN-RAG, and HyperGraphRAG use graphs for navigation or retrieval reach rather than set-level evidence validation.Their designs do not explicitly prevent contradictions, missing inferential bridges, or uninformative distractors in the final context.
  • CAGE’s Position: CAGE introduces between-chunk coherence as an explicit optimization objective for post-retrieval scoring before answer generation.It repurposes graph topology from a navigation aid into a mechanism for validating the integrity of the evidence set.

3 Background Definition and Theory

The paper defines between-chunk coherence as a four-dimensional property of retrieved passage sets and argues that relational graph encoders can distinguish structurally coherent from incoherent evidence.

  • Definition: Between-chunk coherence is decomposed into intra-domain relevance, noise resistance, informational bonding, and factual consistency.The dimensions respectively address referential alignment, cross-domain distractors, bridge-based connectivity, and logical compatibility.
  • Definition: Intra-domain relevance requires shared referential grounding, while noise resistance suppresses keyword-matching distractors from unrelated domains.Examples include different McDonald’s branches and the food-versus-financial meanings of “Big Mac.”
  • Definition: Informational bonding captures bridge entities that connect disjoint facts into a traversable reasoning chain.Factual consistency instead tests whether relational assertions are logically compatible rather than mutually exclusive.
  • Theory: The R-GCN update is presented as injective over relational multisets and as matching the discriminative power of relational Weisfeiler–Lehman methods.This supports distinct embeddings for graphs differing in entity relations, connectivity, or directionality.
  • Theory: The proposed coherence-separability argument links shared entities and compatible relations with higher embedding similarity, while contradictory patterns produce divergent embeddings.The argument treats the graph embedding manifold as a structural validator for retrieval coherence.
  • Implication: CAGE therefore treats retrieval as validation of whether evidence is topologically and logically integrable, not merely topically related.This interpretation connects relational graph encoding to downstream coherence scoring.

4 CAGE

CAGE reranks initially retrieved passages through graph construction, factual-anchor reweighting, R-GCN encoding, and fused relevance–coherence scoring. The resulting passage scores combine query similarity with structural alignment across the retrieved set.

  • Graph Construction: CAGE constructs a directed heterogeneous entity graph for each retrieved passage before reranking.Nodes represent lemmatized content words with resolved pronouns, while edges encode linguistically derived relation types.
  • Graph Reweighting: Min-out-degree reweighting amplifies nodes with the smallest out-degree because they often represent specific entities, values, or factual endpoints.Their feature vectors and adjacent edge weights are scaled by β > 1.
  • R-GCN Encoding: A single R-GCN layer with ReLU and global sum pooling produces a graph-level embedding Zi ∈ R128.A diversity objective separates structurally distinct passage graphs in the embedding space.
  • Coherence Scoring: The coherence score for each passage aggregates its structural alignment with every other retrieved passage.High-scoring passages share entities, compatible relations, or connected paths; low-scoring passages are structural outliers.
  • Reranking: The final ranking fuses query–passage cosine similarity with normalized coherence using α to control their trade-off.Reranked top-k passages are passed to the generator.

5 Experimental Setup

CAGE is evaluated on four multi-hop benchmarks using shared candidate pools, gold supporting-fact chains as a coherence proxy, and Recall@5 comparisons across methods.

  • Datasets: The evaluation covers HotpotQA, 2WikiMultihopQA, MuSiQue, and RAMDocs, using multi-hop gold supporting-fact chains as a proxy for coherent evidence.These chains represent evidence subsets that must collectively provide structural connectivity and logical harmony for valid reasoning.
  • Dataset Characteristics: The datasets stress different coherence dimensions: bridge-heavy HotpotQA and 2WikiMultihopQA emphasize informational bonding, while MuSiQue adds long chains and dense distractors.HotpotQA contains 81% bridge questions, 2WikiMultihopQA 48% bridge questions, and MuSiQue uses 3–4 hop chains with high distractor density.
  • Results: CAGE achieves the highest Recall@5 on 2WikiMultihopQA and HotpotQA while remaining competitive on MuSiQue and RAMDocs.Table 1 reports cross-benchmark Recall@5 comparisons among the evaluated methods.
  • Evaluation Protocol: All methods retrieve from the same benchmark passage pool, including gold supporting facts and distractors, for comparable evaluation.The shared pool contains the full set of passages provided by each benchmark.

6 Results

CAGE improves retrieval and answer generation by adding graph-based coherence to query relevance, especially for bridge-heavy multi-hop benchmarks. Its gains are strongest when structural connectivity identifies essential passages, while adversarially contradictory passages remain a limitation.

  • Evaluation setup: CAGE evaluates fused query relevance and graph coherence against retrieval baselines including monoT5, mxbai-rerank, ColBERTv2, and Vanilla RAG.The evaluation reports Recall@5 and downstream EM/F1 using the top-five retrieved chunks as generator context.
  • Retrieval performance: 85.4 Recall@5 on 2WikiMultihopQA and 84.2 on HotpotQA show CAGE’s strongest retrieval gains on bridge-dominated datasets.CAGE outperforms Vanilla RAG by 5 points on 2WikiMultihopQA and matches monoT5 on HotpotQA.
  • Retrieval performance: 64.0 Recall@5 on MuSiQue narrowly exceeds Vanilla RAG at 63.6 and monoT5 at 62.8, whereas RAMDocs methods cluster within 1 point at 86.8–87.6.The narrower MuSiQue margin reflects high distractor density, while RAMDocs exposes the difficulty of contradictory passages sharing entity structure.
  • Answer generation: CAGE achieves the highest Exact Match on 5 of 8 dataset–model combinations, with more modest F1 improvements.The reported asymmetry reflects EM’s requirement for a complete reasoning chain versus F1’s tolerance of partial token overlap and noisy context.
  • Ablation: Replacing the standard GCN with an R-GCN produces the largest retrieval improvement, including a +10.5-point Recall@5 jump on MuSiQue.Min-out-degree reweighting adds further gains across datasets, including +3.9 points on 2Wiki and +1.8 on HotpotQA.
  • Coherence ablation: +4.4 EM on HotpotQA is the largest coherence-signal gain, alongside +1.4 on MuSiQue and +1.7 on 2Wiki.The corresponding F1 changes are +0.9, +0.9, and -0.3, respectively.
  • Limitations: Structural connectivity alone cannot resolve RAMDocs conflicts because contradictory passages preserve surrounding entity structure while changing the target fact.The resulting near-isomorphic graphs receive similarly high structural coherence scores, motivating complementary claim-level NLI verification.

7 Conclusion

CAGE reranks retrieved passages using between-chunk structural coherence rather than individual query relevance alone. Its four-dimensional coherence formulation and R-GCN-based encoding improve retrieval and downstream answer precision.

  • 7 Conclusion: CAGE reranks passages by structural coherence rather than individual query relevance alone.Its central contribution is a four-dimensional decomposition of between-chunk coherence.
  • 7 Conclusion: Table 3 evaluates Recall@5 across all benchmarks to isolate the contributions of R-GCN architecture and min-out-degree node reweighting.
  • 7 Conclusion: The framework operationalizes intra-domain relevance, noise resistance, informational bonding, and factual consistency through directed heterogeneous entity graphs encoded via R-GCN.
  • 7 Conclusion: CAGE achieves the highest Recall@5 on bridge-dominated benchmarks and consistently improves Exact Match in downstream generation.The reported benchmarks are 2WikiMultihopQA and HotpotQA.
  • 7 Conclusion: Even when retrieval recall decreases on MuSiQue, generation quality improves, indicating that coherence complements rather than replaces per-chunk relevance.

Limitations

The framework is constrained by parser-dependent graph construction, proxy-based coherence evaluation, and manually tuned fusion weighting. These limitations motivate better relation extraction, dedicated coherence annotations, and adaptive weighting.

  • Limitations: Rule-based dependency parsing limits relation extraction quality, with entity-recognition or dependency-resolution errors propagating into graph topology.
  • Limitations: The fusion weight α is set via grid search, while adaptive query-dependent weighting could improve robustness on adversarial benchmarks such as RAMDocs.
  • Limitations: The evaluation uses multi-hop supporting-fact annotations as a proxy because existing RAG benchmarks lack direct per-dimension coherence labels.
  • Limitations: Constructing a benchmark with explicit per-dimension coherence labels is identified as an important direction for future research.

Ethical considerations

The study uses publicly available, non-personal multi-hop benchmarks and locally run open-weight models without collecting new human-subject data. The authors caution that improved retrieval coherence does not guarantee factual answers in high-stakes domains.

  • Ethical considerations: The evaluation uses publicly available HotpotQA, 2WikiMultihopQA, and MuSiQue benchmarks derived from Wikipedia without personally identifying or offensive content.
  • Ethical considerations: No new human-subject data was collected, and the study involved no crowdsourcing or annotation.
  • Ethical considerations: Open-weight Gemma and LLaMA models run locally without transmitting data to external APIs.
  • Ethical considerations: Improved retrieval coherence does not guarantee factual correctness, so outputs require caution and verification in healthcare or legal applications.

A Dataset summary

The evaluation datasets summarize different linguistic demands, while the theoretical framework uses relational graph encodings to model passage coherence. Its analysis covers relevance, noise attenuation, bridge connectivity, and the limits of normalized aggregation.

  • Table 4 summarizes evaluation datasets by sample size and linguistic characteristics across four coherence dimensions.
  • R-GCN graph representations use relation-specific transformations and a global graph encoding function to map relational structure into a latent embedding space.The framework characterizes coherence through relevance, noise, bonding, and consistency sub-dimensions.
  • The implementation uses degree-normalized aggregation, which is not injective in general but becomes injective when compared neighborhoods have constant degree.Normalization is adopted for gradient stability, while diversity training discourages embedding collapse.
  • Graph encoding distinguishes connected factual backbones from isolated noise islands, while shared entities support informational bonding across passage graphs.The framework describes isolated nodes as receiving limited relational enrichment and connected nodes as contributing stronger structural signal.
  • Shared bridge entities increase cross-graph similarity because their independently enriched relational embeddings contribute aligned terms to both passage representations.This effect does not require a cross-document edge or additional shared vocabulary.

B.5 Proof of Lemma 4

Lemma 4 argues that relation-sensitive, non-commutative operators penalize reversed or incompatible relational paths. Its stated boundary is single-entity substitution within an otherwise unchanged relational structure.

  • Lemma 4 states that contradictions changing relation type or ordering along a shared reasoning path are penalized by non-commutative operators.
  • For paths ordered as (r1, r2) versus (r2, r1), the R-GCN applies different matrix products, producing divergent node embeddings.The argument relies on matrix multiplication being non-commutative.
  • The similarity score is maximized when retrieved relational logic aligns with the query’s expected logical flow, under a well-trained model.The contrastive objective is identified as driving relation-discriminative matrices.
  • Lemma 4 does not cover contradictions formed by substituting one entity while preserving the surrounding relation types and edge directions.
  • For such substitutions, embedding divergence depends on the changed leaf representation and may remain small when the entities are lexically or semantically similar.This boundary is linked to the RAMDocs failure analysis and motivates complementary claim-level verification.

F Table for Ablation Study

The paper documents dataset characteristics, implementation resources, qualitative coherence cases, and an ablation table for downstream generation. The case studies illustrate entity disambiguation, noise filtering, bridge reasoning, and factual conflict.

  • The case studies associate entity disambiguation with intra-domain relevance and shared professional vocabulary distractors with noise resistance.
  • HotpotQA, 2WikiMultihopQA, MuSiQue, and RAMDocs are used under their respective research licenses.
  • The implementation processes approximately 10,000 passages in roughly 6 hours on CPU for coreference-aware graph construction, while each per-query R-GCN instance completes in under 0.5 seconds.
  • A HotpotQA bridge requires identifying Sherlock Holmes before linking him to Sir Arthur Conan Doyle, while the factual-consistency case contrasts Aguadilla with San Juan.
  • Table 5 evaluates Exact Match and F1 across four multi-hop datasets and two generators to isolate component contributions.
Loading 2609.04647v1…