Hybrid Retrieval-Augmented Generation with Knowledge Graph Expansion, RRF Fusion, and Per-Chunk Grounded Evaluation for Enterprise Document Search

Harish Saragadam; Sudhanshu Sharma; Meghana Pujari

Source-linked AI summary

Hybrid Retrieval-Augmented Generation with Knowledge Graph Expansion, RRF Fusion, and Per-Chunk Grounded Evaluation for Enterprise Document Search

Harish Saragadam, Sudhanshu Sharma, Meghana Pujari

arXiv:2609.01617v1Jun 30, 2026cs.IRcs.AIcs.LG

TL;DR

Enterprise telecom document search must handle specialized terminology, fragmented chunks, and relevant information distributed across non-adjacent sections. DocuSearch addresses these challenges with hybrid retrieval and per-chunk agentic groundedness checks. On its telecom corpus, it reports Precision@10 of 0.69, Recall@10 of 0.79, and a grounding rate of 89.6%.

Problem
Enterprise document search needs to retrieve and answer from evidence despite incomplete signals, fragmented chunks, redundant results, and unverified grounding.
Method
DocuSearch combines vector, BM25, and knowledge-graph retrieval with weighted RRF, reranking, MMR, and per-chunk context, sufficiency, generation, and groundedness checks.
Results
Precision@10 of 0.69, Recall@10 of 0.79, and a grounding rate of 89.6% are reported for DocuSearch.
Takeaways & Limitations
The system provides a local hybrid document-intelligence pipeline that returns grounded per-chunk answers or uses a grounded fallback merge.
Takeaways & Limitations
Projected future gains from offline Q-Learning remain unvalidated, with shadow testing planned before live deployment.

Abstract

from arXiv · show

Getting accurate, grounded answers out of large enterprise document repositories is a difficult problem. Dense vector retrieval alone frequently performs poorly on queries that mix technical terminology, vendor-specific acronyms, or require reasoning across several non-adjacent sections. DocuSearch was built to address exactly this gap - an offline, multi-agent document intelligence system developed and evaluated in a production telecom network operations environment. Rather than relying on a single retrieval signal, DocuSearch pulls together three complementary sources of evidence: semantic search over a Qdrant vector store using BGE-Large embeddings, BM25 full text search over an SQLite FTS5 index, and Knowledge Graph neighbour expansion from a structured edge table. These three ranked lists are merged through Reciprocal Rank Fusion with signal weights of 0.50 for vector search, 0.35 for BM25, and 0.15 for the knowledge graph, using a smoothing constant of 60 to stabilize scores. A cross-encoder then reranks the fused list, and Maximal Marginal Relevance with a balance factor of 0.65 prunes results for relevance and diversity. What makes DocuSearch distinctive is a per-chunk evaluation loop treating each chunk as its own mini-retrieval problem: an LLM decides whether the chunk needs more context, whether it fully answers the query, and whether the answer is grounded in retrieved text. Ungrounded answers are not returned; the system falls back to a multi-chunk merge instead. On a telecom corpus, DocuSearch reaches Precision@10 of 0.69, Recall@10 of 0.79, and a grounding rate of 89.6% - gains of 15, 16, and 18.4 percentage points over a dense-only RAG baseline. Index Terms: retrieval-augmented generation, knowledge graph, reciprocal rank fusion, enterprise document search, agentic evaluation, BM25, cross-encoder reranking, on-premise deployment, LangGraph, telecom AI.

I. INTRODUCTION

DocuSearch targets enterprise telecom search failures caused by specialized vocabulary, fragmented chunks, and information spread across non-adjacent sections. It combines hybrid retrieval with per-chunk grounded evaluation in a local, production-oriented system.

Telecom engineers search large multi-vendor repositories for specific procedures under time pressure.
Standard keyword search and single-signal RAG struggle with technical terminology, fragmented chunks, and non-adjacent relevant information.
DocuSearch fuses dense, lexical, and structural retrieval signals through weighted RRF with wv=0.50, wb=0.35, and wkg=0.15.
A per-chunk agentic loop detects context needs, scores sufficiency, and verifies groundedness before committing an answer.
Vendor-aware retrieval scopes searches to relevant vendor namespaces in multi-vendor document corpora.
The system is designed as a production-ready LangGraph pipeline with a Dash interface and fully local LLM inference.

III. PROPOSED SOLUTION

DocuSearch uses a two-phase pipeline: hybrid retrieval produces a diverse evidence set, then per-chunk evaluation selects a sufficiently grounded answer or invokes a fallback merge.

Pre-MMR Phase: Template selection, keyword extraction, and vendor detection prepare the query before parallel retrieval begins.
Pre-MMR Phase: Vector search, BM25 retrieval, and KG expansion run in parallel before weighted RRF fusion, cross-encoder reranking, and MMR selection.
Post-MMR Phase: Each evidence chunk passes context detection, sufficiency scoring, answer generation, and groundedness verification.
Post-MMR Phase: The first chunk clearing all four checks produces the answer; otherwise expanded contexts are merged and checked once more.

IV. HIGH-LEVEL ARCHITECTURE

The architecture spans ingestion, multi-signal retrieval, ranking, agentic evaluation, and user interaction layers. Its post-MMR loop evaluates candidate chunks individually and falls back to consolidated context when necessary.

System Architecture: Document ingestion creates overlapping 900-character segments with 140-character overlap and stores them in Qdrant and SQLite FTS5.
System Architecture: The retrieval layer runs vector, BM25, and KG agents, while ranking produces an evidence set of |E|=10.
Agentic Evaluation: The agentic evaluation layer applies four sequential LLM checks to each candidate chunk.
Agentic Evaluation: If a chunk needs context, neighbours are fetched; grounded answers terminate the loop, while failed candidates lead to fallback merging.
User Interface: Users access conversational search, pipeline visualization, knowledge resources, and ingestion panels through a Plotly Dash interface.

A. Query Analysis

Query analysis classifies intent, extracts domain terms, detects vendors, and launches complementary retrieval methods suited to semantic, lexical, and structural relevance.

Query Analysis: Template selection maps queries to five answer types, using a deterministic heuristic when the LLM is uncertain.
Query Analysis: BM25 keyword extraction combines a curated telecom vocabulary with query-specific expansions.
Query Analysis: Vendor detection combines LLM classification and regex matching to scope retrieval to a detected vendor’s documents.
Hybrid Retrieval: BGE-Large-EN-v1.5 encodes queries for cosine-similarity search over Qdrant, returning the top kv=40 chunks.
Hybrid Retrieval: BM25 retrieves the top kb=40 chunks and is useful for acronyms and model numbers lacking meaningful semantic neighbourhoods.
Hybrid Retrieval: KG expansion traverses SQLite edges to retrieve top kkg=15 structurally connected chunks that may be lexically and semantically distant.

C. Reciprocal Rank Fusion

DocuSearch fuses three ranked retrieval lists using weighted Reciprocal Rank Fusion. The smoothing constant prevents highly ranked items from dominating the combined score.

Chunks appearing in multiple lists receive additive contributions in the fused ranking.
The smoothing constant k = 60 prevents very high-ranked items from dominating the fused score entirely.

D. Cross-Encoder Reranking

DocuSearch reranks the top fused chunks with a cross-encoder and then applies MMR to select a relevance-and-diversity-balanced evidence set of ten chunks.

The top kr = 12 fused chunks are reranked with a cross-encoder that jointly encodes the query and each chunk.
The cross-encoder score is weighted higher because it captures fine-grained query-chunk interactions that bi-encoder cosine similarity cannot.
MMR iteratively filters the reranked list until the selected evidence set contains |E| = 10 chunks.
With λ=0.65, MMR favours relevance slightly over diversity and eliminates clusters of near-duplicate chunks.

F. Per-Chunk Agentic Evaluation Loop

DocuSearch evaluates each retrieved chunk through sequential context, sufficiency, generation, and groundedness checks. It expands fragmented chunks dynamically and uses a grounded multi-chunk fallback when no individual chunk passes.

Per-Chunk Agentic Evaluation Loop: Each selected chunk passes four sequential LLM-driven checks: context need detection, sufficiency scoring, answer generation, and groundedness verification.
Per-Chunk Agentic Evaluation Loop: Chunks require a sufficiency score σ(˜c, q) ≥τ = 7 before answer generation is attempted.
Dynamic Neighbour Context Expansion: Fragmented chunks can receive up to L = 2 iterations of neighbouring-context expansion before evaluation continues.
Per-Chunk Agentic Evaluation Loop: If no individual chunk clears all checks, expanded contexts are merged and subjected to a final groundedness check before return.
Implementation: The implementation uses 900-character chunks with 140-character overlap after smaller chunks hurt sufficiency scoring and larger ones degraded retrieval precision.

VIII. RESULTS AND ANALYSIS

DocuSearch was evaluated on a diverse enterprise telecom corpus containing documentation from multiple major equipment vendors. The reported evaluation materials compare retrieval quality and answer grounding and hallucination rates.

The evaluation corpus spans configuration manuals, SOPs, RCA reports, and audit records from multiple major equipment vendors.
Table III compares retrieval quality for the evaluated retrieval systems.
Table IV reports answer grounding and hallucination rates.
The corpus diversity provides a reasonable stress test for a retrieval system.

A. Retrieval Quality

DocuSearch improves retrieval coverage through multi-signal evidence and maintains grounded answers, while its per-chunk evaluation introduces measurable latency.

The full three-signal system outperforms dense-only and BM25 baselines across Precision@k and Recall@k on 120 manually annotated queries.KG expansion particularly recovers structurally related troubleshooting chunks that other retrieval signals miss.
89.6% grounding rate represents an 18.4 percentage point improvement over single-pass RAG, while hallucinations fall from 21.4% to 6.8%.
The fallback path merges all expanded contexts for the remaining 3.6% of queries and still produces grounded answers.
Mean end-to-end latency is 12.6 seconds across 200 queries, with per-chunk evaluation accounting for 8.4 seconds.Retrieval itself finishes in under one second; reducing evaluation latency is identified as future work.

D. Ablation Study

Ablation results show that groundedness verification and sufficiency scoring contribute most to grounding performance, while smaller components remain collectively meaningful.

Groundedness verification has the largest individual impact on grounding rate at 15.3 percentage points, followed by sufficiency scoring at 11.5 points.
Vendor scoping contributes 3.5 points and KG expansion 4.4 points when removed individually, and their combined contribution remains meaningful.
No single component is redundant, although groundedness verification and sufficiency scoring drive the largest improvements.

A. Limitations and Future Work

The paper identifies latency, domain-specific tuning, and limited knowledge-graph structure as constraints, while proposing adaptive retrieval and multimodal expansion as future directions. Projected reinforcement-learning gains are presented as conservative estimates under the current system assumptions.

Limitations: 8.4 seconds is the current average latency for sequential LLM calls during per-chunk evaluation.Parallelising chunk evaluation across inference threads is proposed as a first optimisation step.
Limitations: The RRF signal weights were tuned empirically on telecom documentation and may not transfer reliably to other domains.The paper proposes learning them from query-relevance feedback through a differentiable ranking objective.
Future Work: Typed entity-relation triples are proposed to strengthen knowledge-graph support for multi-hop queries.Named entity recognition is identified as a potential way to populate the graph with these triples.
Future Work: Multimodal retrieval covering embedded diagrams and specification tables is proposed for vendor documentation.The paper identifies these document elements as a direction for extending DocuSearch.
Adaptive Retrieval: Approximately 6.2 percentage points of overall improvement are projected, lifting composite accuracy from ≈80% to ≈86%.The projection assumes an offline-trained Q-Learning policy optimized on logged production trajectories and shadow-tested before deployment.