StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization

• Existing RAG struggles when useful information is dispersed across documents because retrieving all relevant chunks and integrating them for reasoning is difficult.Source: §1 · pp. 1–2
• The framework draws on cognitive theories of structuring scattered information and matching knowledge formats to task requirements.Source: §1 · p. 2
• StructRAG selects a suitable structure type, converts raw documents into that format, and uses question decomposition and precise knowledge extraction to infer answers.Source: §1 · p. 2, Figure 1 · p. 2
• A synthesizing, simulating, and judging pipeline creates preference data for DPO training of the structure router, addressing scarce real-world training data.Source: §1 · p. 2
• StructRAG achieves state-of-the-art performance across knowledge-intensive reasoning tasks, with larger improvements as task complexity increases and faster average operation than recent Graph RAG methods.Source: §1 · p. 3

2.1 RETRIEVAL-AUGMENTED GENERATION

• Chunk-based RAG struggles with knowledge-intensive tasks because chunks contain textual noise and fail to capture information interconnections.Source: §2.1 · p. 3

2.2 GRAPH RETRIEVAL-AUGMENTED GENERATION

• Graph RAG methods use pre-built or document-extracted graphs, but their focus on triplets limits applicability across diverse domains and reasoning tasks.Source: §2.2 · p. 3

• StructRAG first selects a task-suitable structure, restructures raw documents into that format, then decomposes complex questions to support reasoning over the structured knowledge.Source: §3 · p. 3
• Knowledge-intensive tasks may involve more than 20 documents and up to 200K tokens, with useful information dispersed across documents and requiring complex comparison.Source: §3 · pp. 3–4
• The router uses a question and document core content to choose among table, graph, algorithm, catalogue, and chunk structures suited to different task types.The listed matches are table for statistical tasks, graph for long-chain tasks, algorithm for planning, catalogue for summarizing, and chunk for simple single-hop tasks.Source: §3 · p. 4
• An LLM-based structurizer reconstructs knowledge scattered across raw documents into the selected type and generates a description of the structured knowledge.Source: §3 · p. 4
• The utilizer decomposes the question into sub-questions, extracts precise knowledge for each, and integrates the results to infer an answer.Source: §3 · p. 4
• Router preference data are constructed by synthesizing tasks, simulating solutions with different structure types, and judging them to form candidate-type preference pairs.Source: Figure 2 · p. 5

Data Constructing

• Synthetic preference pairs are built by generating tasks from seed tasks, simulating solutions using different structure types, and having an LLM judge the solutions.Each entry includes the question, document core contents, chosen structure type, and rejected structure type.Source: §4 · p. 5
• The synthetic preference data include both English and Chinese examples to improve universality.Source: §4 · p. 5

Preference Training

• DPO trains the router to select a structure type from a question and document core contents, without requiring an additional reward model.The authors state that preference training helps the router distinguish knowledge types and their suitability for tasks.Source: §4 · p. 5

Evaluation Datasets

• Loong covers four reasoning tasks across four document-length settings, with LLM-judged scores and exact-match rate as metrics.The paper also evaluates Podcast Transcripts, a query-focused summarization task, using LLM-judged win rates across comprehensiveness, diversity, empowerment, and directness.Source: §5.1 · p. 6

Implementation Details

• StructRAG uses Qwen2-7B-Instruct for its router and Qwen2-72B-Instruct for the structurizer and utilizer.The router is trained on 900 preference examples for three epochs at a learning rate of 1e-5.Source: §5.1 · p. 6

Selected Baselines

• The baselines are Long-Context, RAG, RQ-RAG, and GraphRAG, representing extended context, retrieval, question refinement, and graph-based augmentation.For fair comparison, GraphRAG also uses Qwen-72B-Instruct as its base model.Source: §5.1 · p. 6

5.2 OVERALL RESULTS

• StructRAG exceeds all baselines on Loong overall LLM score and exact-match rate, and achieves the best average performance on Podcast Transcripts.The Loong results are reported in Table 1 and the Podcast Transcripts results in Table 2.Source: §5.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
• Approximately 9, 15, 22, and 23 points separate StructRAG from Long-Context on Sets 1–4, respectively.The sets represent increasing document lengths and increasingly dispersed information.Source: §5.2 · pp. 6–7
• Against RAG, StructRAG improves by around 15, 15, and 22 points on Sets 2–4, respectively.The authors report that improvements over baselines become more significant with longer documents and more scattered information.Source: §5.2 · p. 7

5.3 ABLATION RESULTS OF MODULES

• The ablations compare random routing, chunk-only input, and direct concatenation of structured knowledge with the original question.“W/o router” means random routing, “w/o structurizer” uses only chunks, and “w/o utilizer” directly concatenates structured knowledge with the question.Source: §5.3 · p. 7
• Structure selection and construction matter more than complex utilization; question-refining alone provides limited improvement for knowledge-intensive reasoning.The router removal causes a larger decline than utilizer removal, from 60.38 to 45.33 versus 60.38 to 55.94.Source: §5.3 · pp. 7–8
• 60.38 falls to 45.33 without the router, 53.92 without the structurizer, and 55.94 without the utilizer, showing each module contributes.These are the overall performance values reported for the full framework and the three module removals.Source: §5.3 · p. 7, Table 3 · p. 8

5.4.1 EFFECT OF THE ROUTER

• The trained Qwen2-7B-Instruct router significantly outperforms 72B few-shot routing, and router accuracy positively correlates with StructRAG performance.The authors report that selecting task-suitable knowledge types is challenging for raw LLMs without special training.Source: §5.4.1 · p. 8

5.4.4 EFFICIENCY REPORT

• StructRAG has slightly higher average latency than RQ-RAG but is obviously faster than GraphRAG; latency includes construction and reading.Construction includes determining the optimal knowledge type and building its structure; reading is generating answers from augmented knowledge.Source: Figure 4 · p. 9, §5.4.4 · p. 9

5.4.2 DRAWBACK OF A FIXED TYPE OF KNOWLEDGE

• 60.38 to 53.92: fixed chunk knowledge has the smallest reported performance reduction, but fixed types underperform task-matched selection on scores and exact match.The other fixed structure types show more significant performance declines.Source: §5.4.2 · pp. 8–9

5.4.3 CASE STUDY ABOUT EM METRIC

• StructRAG’s general score exceeds baselines but exact-match rate trails in seven sub-situations, mainly where probabilistic structurization alters textual formatting.The paper gives a formatting change from “$ 1,308,463” to “138463” as an example.Source: §5.4.3 · p. 9

• The framework routes each task to a suitable structure type, converts documents into that structure, and uses it to decompose questions and infer answers.Source: §6 · p. 9
• StructRAG achieves state-of-the-art performance on extensive knowledge-intensive reasoning tasks and large improvements in badly information-scattered scenarios.Source: §6 · p. 10
• Router training data is built through synthesizing, simulating, and judging, followed by preference training with DPO.Source: §6 · pp. 9–10

A.1 PROMPTS IN DATA CONSTRUCTING

• Because real-world data for choosing task-appropriate structures is lacking and difficult to collect, the appendix describes a synthesis, simulation, and judgment pipeline for DPO training data.Source: §A.1 · p. 16

A.2 PROMPTS OF SCATTERED KNOWLEDGE STRUCTURIZER

• Because structuring large amounts of scattered information is complex, prompts guide LLMs in constructing varied structured knowledge.Source: §A.2 · p. 16
• The prompt design draws on the stated capacity of LLMs to integrate structured knowledge in prior work.Source: §A.2 · p. 16

A.3 PROMPTS OF STRUCTURED KNOWLEDGE UTILIZER

• Because knowledge-intensive questions can be combinatorial, prompts guide question decomposition and precise knowledge extraction.Source: §A.3 · p. 16
• Figure 7 presents prompts and examples for synthesizing tasks, simulating solutions, and judging differing solutions in router-data construction.Source: Figure 7 · p. 17
• Conversion prompts differ by structured-knowledge type, while chunks are created by direct splitting without a prompt.Source: Figure 8 · p. 18