Source-linked AI summary
Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph
Ibrahim Siddig, Sviatoslav Tugeev, Munir Georges
TL;DR
Legal RAG depends on retrieving the right sources and citing them accurately, yet retrieval across large legal sources can miss critical context. This paper uses a structure-driven KG to compare chunking and one-hop retrieval strategies for German statutes. Nummer chunking leads at Code and § depths, while Satz leads at deeper evaluation depths.
Problem
Legal RAG must retrieve necessary sources, align answers with case facts and retrieved documents, and cite sources correctly; the paper investigates how structure-driven retrieval affects these outcomes.
Source: §1 · pp. 1–2Method
The paper builds a hierarchy-based KG of German statutes and compares KG-informed chunking and one-hop expansion strategies using depth-wise citation matching.
Source: §3.1 · p. 3, §3.2.3 · p. 5Results
Nummer leads at Code (ToER 14.23) and § (3.25), while Satz leads at deeper evaluation depths; both recover about three times more citations per 1,000 words than Naive-100w.
Source: §4 · p. 5Takeaways & Limitations
Aligning retrieval with the hierarchical structure of legal codes is essential for maximizing citation recall and token efficiency.
Source: §5 · p. 7Even the strongest chunking granularity’s ToER decays with evaluation depth, reflecting the domain gap between factual case descriptions and abstract legal norms.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Any automation within reliability-critical contexts such as medicine or law has to be transparent and verifiable. Despite their advantages, current LLM-based systems do not meet these requirements due to their inherent black-box nature. Retrieval Augmented Generation (RAG) attempts to ground the generative process in the relevant interpretable documents, thereby reducing the risk of AI 'hallucinations' and potentially enabling verification of the initial generation context. However, as the number and size of potential sources increases, naive retrieval approaches tend to perform worse, ignoring critical context elements. In this paper we aim to tackle this problem in the context of German Legal Code. Using structure-driven parsing to create a Knowledge Graph (KG) for the source document navigation and retrieval, we compare several KG-informed chunking and retrieval strategies and evaluate their effect on the accuracy of retrieved citations.
1. Introduction
Legal RAG answers need accurate, verifiable citations: plausible but incorrect citations can make answers harmful. The paper studies structure-driven KG retrieval for German law and evaluates its retrieval accuracy and token efficiency.
1. Introduction
- RAG retrieval must find necessary legal sources, align generated answers with case facts and retrieved documents, and cite those sources correctly.Incorrect citations can make a linguistically sound answer appear trustworthy while remaining factually wrong or incomplete.Source: §1 · p. 1
- The study asks how a structure-driven KG affects retrieval relevance and citation precision, which chunk granularity works best, and how one-hop expansion affects retrieval accuracy.The questions concern retrieval from German legal documents using case facts as the query.Source: §1 · pp. 1–2
1.1. Key contributions
- The paper builds a German federal statute KG and a token-normalized citation-faithfulness metric, then compares chunking and one-hop retrieval strategies on 12,037 held-out BGH cases.The KG covers 90.3% of ground-truth citations in the evaluation set; the metric matches hierarchical citation paths at different depths.Source: §1.1 · p. 2
2. Related Work
Prior work covers legal citation parsing, graph-based RAG, and hierarchical retrieval, but differs in its data, scope, or evaluation. This paper positions its statute-only retrieval and depth-aware citation evaluation against those approaches.
2. Related Work
- Bundesrecht parses citation strings and supports depth-aware lookup over a flat representation, while this work builds a graph with explicit cross-reference edges.The two approaches are complementary rather than competing.Source: §2 · p. 2
- Earlier legal graph-RAG work either lacks evaluation on real court decisions, conflates statute and case retrieval, or benchmarks legal documents with a different scope.This work restricts its retrieval knowledge base to statute law, holds case law out as evaluation gold, and measures citation faithfulness at courts’ citation depth.Source: §2 · p. 2
- Flat or abstract citation matching misses distinctions between hierarchical subdivisions, while long-context degradation and token costs motivate structure-aware, token-efficient retrieval.Prior work cited here includes multi-level segmentation and a graph retrieval approach reporting 3%–11% of GraphRAG token costs while maintaining retrieval fidelity [12].Source: §2 · p. 2
3. Methodology
The methodology builds a hierarchical graph of German statutes and uses its structure to chunk and optionally expand retrieval. Retrieved citations are matched to case references by hierarchical depth and evaluated relative to chunk length.
3.1. Knowledge graph construction
- The KG parses 58 federal codes into hierarchical nodes, linking text segments through containment and within-code citation edges; the code subtrees share a root.The graph hierarchy includes 168,469 nodes and 240,507 edges.Source: §3.1 · p. 3, Figure 1 · p. 3
3.2. Experiment setup
- The pipeline embeds statutory chunks and each case’s Tatbestand, retrieves top-k chunks, optionally expands them one hop through the KG, and compares their references with the case’s structured references.The setup uses KaLM-Embedding-Gemma3-12B-2511 for the text embeddings.Source: §3.2 · p. 3, Figure 2 · p. 4
3.2.1. Chunking strategies
- KG-based chunking selects text at §, Absatz, Satz, Nummer, or Buchstabe granularity, while the structure-agnostic baseline uses consecutive 100-word chunks.The chunking levels are compared because chunk size trades off the chance of including relevant information against context-window use.Source: §3.2.1 · p. 4
3.2.2. One-hop retrieval strategies
- One-hop expansion adds parent-node text, directly cited nodes, or both, using contains_* and cites_* edges; results are compared with the unexpanded top-k baseline.Source: §3.2.2 · p. 4
3.2.3. Citation Matching and Evaluation Metric
- Evaluation decodes retrieved KG nodes and case references into hierarchical paths, credits matching specified prefixes, and reports depth-stratified ToER normalized by mean chunk length.Unspecified deeper citation levels count as implicit hits, and results are micro-averaged across 12,037 cases.Source: §3.2.3 · pp. 4–5
4. Results & Discussion
Hierarchical chunking granularity drives token-efficient citation retrieval. One-hop expansion can raise ToER when uncharged, but accounting for its token cost reverses the gains.
4. Results & Discussion
- § chunking scores ToER 3.06 at Code and 0.85 at §, below Naive-100w; Absatz reaches 7.78 and 1.93, respectively.The results distinguish chunking granularity from structural awareness alone.Source: §4 · p. 5
- At Absatz evaluation depth, cited expansion raises ToER 2.05→2.17 at Buchstabe, 2.27→2.40 at Nummer, and 2.28→2.40 at Satz.The evaluation depth is where ground-truth citations most often terminate; without charging expansion tokens, parent strategy raises Absatz from 1.36 to 2.13.Source: §4 · pp. 5–6
- ToER peaks at Nummer for Code (14.23) and § (3.25), while Satz leads at deeper depths: Absatz 2.28, Satz 1.53, and Nummer 2.79.The cited configurations recover about three times more citations per 1,000 words than Naive-100w at Code, §, and Nummer evaluation depths.Source: §4 · p. 5, Table 1 · p. 6
- Hybrid expansion raises Absatz ToER to 2.18, yet remains below unexpanded Nummer (2.27) and Satz (2.28); expanding either smaller chunking level yields 2.40.The initial chunking choice remains consequential even with generous context expansion.Source: §4 · p. 6
- When expansion tokens count toward W_m, every strategy falls below its unexpanded baseline; Absatz plus parent drops ToER 1.36→0.36 as context grows from 70.6 to 415.0 words.The charged configuration measures expansion efficiency using the base text plus appended context.Source: §4 · p. 6
5. Conclusion
The paper introduces a structure-driven Knowledge Graph approach for retrieval in German statutory law and argues that aligning retrieval with legal-code hierarchy supports citation recall and token efficiency. It also reports steep ToER decay with evaluation depth and identifies downstream validation as future work.
5. Conclusion
- The paper introduces a structure-driven Knowledge Graph approach for German statutory law and concludes that hierarchy-aligned retrieval is important for citation recall and token efficiency.The study evaluates chunking and one-hop context expansion strategies.Source: §5 · p. 7
- Even the strongest chunking granularities decline from ToER ≈14.2 at Code depth to ≈3.2 at § and ≈1.5 at Satz.The authors attribute the difficulty to the gap between factual case descriptions and abstract legal norms, which dense retrieval embeddings struggle to bridge without explicit domain adaptation.Source: §5 · p. 7
- The study isolates retrieval performance, leaving effects on downstream generation quality and hallucination mitigation to be validated.The authors also propose comparing embedders, adding entity-driven KG links, and evaluating retrieved items with legal specialists.Source: §5 · p. 7
- The knowledge graph, six chunk corpora, evaluation set, results, and retrieval/evaluation implementation are archived or made available through the stated repositories.The paper provides a Zenodo archive and a GitHub reference implementation.Source: §5 · p. 7
Declaration on Generative AI
The authors disclose using Claude, Gemini, and Quillbot for language editing and image generation, then reviewing the material and taking responsibility for the publication’s content.
Declaration on Generative AI
- Claude, Gemini, and Quillbot were used for grammar, paraphrasing, and image generation; the authors reviewed the work and take responsibility for its content.Source: Declaration on Generative AI · p. 7