Source-linked AI summary
NS-ST-GraphRAG: Neuro-Symbolic Spatio-Temporal GraphRAG for Literary Knowledge Processing
Zheng Kui Lin
TL;DR
Long-form literary narratives challenge retrieval because evidence spans chapters while relations evolve across time and space. NS-ST-GraphRAG combines ontology-guided neuro-symbolic checking, spatio-temporal graph states, and dynamic sub-graph retrieval, and evaluates them with Red-Chamber-QA. On a 120-question held-out split, it achieves 0.733 mechanical answer reproduction versus 0.675 for the frozen window baseline and 0.083 for the closed-book floor, while semantic-judge rates are 0.866 versus 0.850.
Problem
Long-form narratives require retrieval systems to preserve distributed evidence and relations that vary across narrative time, space, and relational context.
Method
NS-ST-GraphRAG uses ontology-guided constraint checking, dual temporal coordinates, spatial scene attributes, dynamic sub-graph retrieval, and traceable evidence grounding.
Results
0.733 mechanical answer reproduction was achieved on the 120-question held-out split versus 0.675 for the frozen window baseline and 0.083 for the closed-book floor; semantic-judge rates were 0.866 versus 0.850.
Takeaways & Limitations
The framework integrates temporal graph representation, constrained extraction, and auditable evaluation for verifiable knowledge processing over long-form narrative.
Takeaways & Limitations
The rejection component’s value on the gold error classes remains undemonstrated, and entity-merge error measurement is outside the present measured installment.
Abstract
from arXiv · showhide
Long-form literary narratives pose a distinctive information-processing challenge for retrieval-augmented generation: relevant evidence is distributed across chapters, relations evolve over narrative time, and correct answers may depend jointly on temporal, spatial, and relational constraints. We propose NS-ST-GraphRAG, a neuro-symbolic spatio-temporal GraphRAG framework that integrates ontology-guided extraction, deterministic constraint checking, dual temporal coordinates, spatial scene attributes, and dynamic sub-graph retrieval. Instead of retrieving from a single corpus-level graph, the framework selects the graph state valid for the temporal and spatial scope of a query and grounds generated answers in traceable evidence. We further introduce Red-Chamber-QA, to our knowledge the first open multi-hop question-answering benchmark for classical Chinese literature, with time-, space-, and general-question categories, per-part evidence spans, and deterministic shortcut controls. On a 120-question held-out split, NS-ST-GraphRAG achieves mechanical answer reproduction of 0.733 versus 0.675 for the frozen window baseline and 0.083 for a closed-book model (McNemar exact p = 0.092, directionally favorable but not significant); semantic-judge accuracy is 0.866 versus 0.850. The pre-specified constrained-category condition of H2 is not supported by the delivered comparison. These results show how temporal graph representation, constrained extraction, and auditable evaluation integrate into a unified framework for verifiable knowledge processing over long-form narrative.
1. Introduction
Long-form narratives challenge retrieval systems because evidence is distributed across chapters and relations change over narrative time. NS-ST-GraphRAG addresses this with constrained extraction, spatio-temporal graph representation, dynamic retrieval, and the Red-Chamber-QA benchmark.
- Motivation: Long-form literary questions combine relational, temporal, and spatial constraints that standard retrieval pipelines are poorly equipped to preserve.Evidence may span chapters, involve changing character relations, and depend on who was present at a particular scene.
- Research gaps: Static graphs omit narrative-time validity intervals, preventing retrieval of the graph state corresponding to a specific story event.The framework also identifies spatial structure as information that conventional graph construction discards.
- Framework: NS-ST-GraphRAG combines ontology-based neuro-symbolic checking, dual temporal coordinates with spatial scene attributes, and dynamic sub-graph retrieval.Its components are designed to reject or repair inconsistent relations, version the graph over the story, and retrieve the relevant narrative slice.
- Auditability: Every rejected relation, retrieved slice, and generated answer is linked to an auditable rule, coordinate, or evidence set.The design targets heterogeneous hallucination-mitigation evaluation by making extraction and answer provenance traceable.
- Benchmark: Red-Chamber-QA is introduced as a multi-hop benchmark for classical Chinese literature with time- and space-constrained categories and provenance annotations.The contribution includes a 104-question preliminary set with mechanical and expert verification.
2. Related Works
The paper builds on retrieval, literary knowledge-graph, neuro-symbolic, hallucination-mitigation, and temporal-graph research. It differentiates itself from the nearest prior work through ontology constraints, dynamic spatio-temporal representation, and a dedicated QA benchmark.
- Structured retrieval: Graph-based retrieval systems organize corpus evidence through entity-relation graphs, thematic retrieval, path ranking, or community summaries.The related systems include GraphRAG, LightRAG, HippoRAG, and other retrieval-augmentation designs.
- Literary knowledge graphs: Literary knowledge-graph research addresses character networks, story representations, entity resolution, and classical Chinese text construction.Prior work includes dialogue-derived social networks and human–AI collaborative extraction for the same novel.
- Technical lineages: Neuro-symbolic checking, hallucination-mitigation research, and temporal-graph methods motivate deterministic verification, provenance grounding, and time-aware representations.The paper identifies extraction-level evaluation and unified spatio-temporal modeling as underdeveloped areas.
- Positioning: The nearest prior work shares the corpus and LLM extraction setting but lacks ontology constraints, temporal and spatial coordinates, and a QA benchmark.Its prompt-iteration loop improves extraction yield, whereas this paper targets constraint-based control of relation validity.
3. Methodology
NS-ST-GraphRAG constructs an auditable knowledge-processing pipeline for long-form literary narratives, combining constrained extraction, spatio-temporal graph construction, and dynamic retrieval. The method represents evolving relations with temporal and spatial coordinates, retrieves query-matching graph slices, and grounds answers in traceable evidence.
- Problem Formulation: Long-form literary narratives require graph representations that preserve evolving relations, temporal intervals, spatial context, and indirect references.Static triple sets lose narrative-time information, while metaphor-rich prose makes unrestricted extraction prone to internally inconsistent relations.
- Ontology-Guided Extraction: The pipeline processes chapter prose into ontology-constrained candidate triples, repairs hard-rule violations through bounded feedback, and records unrepaired violations as uncertainty.Hard rules cover genealogical, master-servant, and title or gender consistency; soft constraints penalize atypical but textually supported relations.
- Spatio-Temporal Graph Construction: The graph stores entities and relations with coarse chapter coordinates, fine narrative-time validity intervals, and spatial scene attributes.Entity resolution precedes assembly, while event-timeline extraction anchors relation evidence to story events rather than calendar dates.
- Dynamic Sub-Graph Retrieval: Dynamic Sub-Graph Retrieval parses temporal and spatial constraints, selects the matching versioned graph slice through structured indexes, and passes evidence-bearing triples to generation.The design avoids a learned text-to-query stage so retrieval effectiveness is not confounded with an additional uncontrolled component.
- Grounded Generation and Auditability: Grounded generation requires answers to map to supporting triples whose chapter, narrative-time, and scene coordinates make provenance mechanically checkable.The pipeline treats construction cost as an explicit design object, including extraction, event-timeline, and bounded repair-pass costs.
4. Experimental Results
The preliminary and development evaluations characterize extraction quality, benchmark construction, retrieval behavior, and citation-faithfulness failures before the held-out campaign. Deterministic repairs corrected coordinate errors and reduced residual hallucination in the adapted sample, while retrieval and prompt mechanisms improved evidence grounding but left paraphrase as a major failure mode.
- Extraction and validation: 0% post-adaptation strict hallucination rate remained after six evidence or predicate fixes and seven deletions among 26 deduplicated relations.The adaptation cost was estimated at 25–35 minutes, with 19 validated items retained after correction and filtering.
- Benchmark and protocol: 104-question benchmark construction achieved a 0% shortcut rate, with verbatim premise fragments, two-part answers, per-part evidence spans, and expert rewriting of remaining shortcut-admitting questions.The benchmark spans 60 chapters and applies mechanical rejection checks before expert review.
- Failure analysis: Few-shot demonstrations increased CF from 0.109 to 0.510 for long-context and from 0.065 to 0.500 for naive-RAG in an exploratory 46-question run.This comparison is prompt-design context rather than a second test-split evaluation.
- Retrieval and assembly: Question-anchored retrieval raised questions with both evidence spans in-window from 11 to 49 of 104 and increased CF from 0.240 to 0.538.The controlled deterministic repair added 0.029 CF and 0.096 DPC-mech, while residual failures in the final configuration were partially rewritten evidence.
- Extraction and validation: 1,099 copied Chapter 1 labels were deterministically corrected by locating evidence spans in source chapters, with zero LLM repair rounds.The same pass identified 64 spans without verbatim chapter matches as candidates for expert annotation; 142 rule violations remained after repair.
- Temporal construction: The event-timeline pass extracted 1,989 events and reduced unresolved participant mentions from 156 to 58, or 1.3% of 4,577 mentions.Fourteen anchor-event candidates were narrowed to twelve confirmed events spanning Chapters 13 to 120.
- Retrieval and assembly: Retrieval width fell from 942 to 71 supporting triples without losing the answer in the multi-hop smoke question.The diagnostic quantity is intended to measure retrieval efficiency at scale in the full campaign.
5. Discussion and Analysis
The discussion separates measured preliminary failure mechanisms from hypotheses and evaluations that remain incomplete or untested. It also frames distant reading as a fixed protocol for examining Wang Xifeng’s network and spatial activity across a narrative turning point.
- Hypotheses and evaluation scope: H1 receives a split verdict: the constraint layer eliminates coordinate errors, but current ontology rules flag none of ten gold violations.The rejection component’s value on those gold error classes remains undemonstrated, while active repair is deferred.
- Measured failure mechanisms: The preliminary evaluation identifies MIXED evidence, ungrounded parameter memory, and divergence between semantic correctness and exact reproduction as distinct failure mechanisms.Long-context and naive-RAG systems produced 30 and 28 MIXED answers; the long-context baseline scored 0.915 semantically but 0.231 on exact reproduction.
- Measured failure mechanisms: Held-out failure analysis distinguishes partial evidence, verbatim wrong passages, and retrieval misses through deterministic checks of evidence spans and chapter ranges.Citation misalignment can remain mechanically correct while lacking either golden evidence span, motivating provenance tracing beyond verbatimness.
- Planned safeguards: The full campaign’s planned safeguards address ontology incompleteness through relation-type coverage, expert spot-checking, disjoint annotations, and out-of-distribution rule-transfer testing.These measures bound claims about hallucination rates when novel violation patterns fall outside the current rules.
- Distant-reading analysis: The distant-reading demonstration fixes before-and-after windows around the Grand View Garden investigation to track Wang Xifeng’s network centrality and spatial activity.The protocol is fixed in advance to avoid selecting favorable narrative windows after inspecting results.
6. Conclusion
NS-ST-GraphRAG combines constrained extraction, spatio-temporal graph modeling, and dynamic retrieval with the Red-Chamber-QA benchmark. The held-out campaign favored the system mechanically, but the constrained-category condition of H2 was not supported.
- NS-ST-GraphRAG combines ontology-guided extraction, dual temporal coordinates, spatial attributes, and dynamic sub-graph retrieval for traceable answers.The framework rejects or repairs logically impossible relations and retrieves from the narrative slice matching question constraints.
- Red-Chamber-QA provides an open multi-hop benchmark for classical Chinese literature with expert verification and a 0% shortcut rate in its 104-question preliminary set.The benchmark includes a 16-question multi-hop subset and was constructed under a mechanical rule set.
- 0.733 mechanical reproduction exceeded 0.675 for the frozen window baseline and 0.083 for the closed-book floor on the 120-question held-out campaign.The McNemar exact p-value was 0.092, so the comparison was not significant.
- 0.866/0.850/0.274 were the held-out judge rates for the system, frozen window baseline, and closed-book floor, respectively.Mechanical reproduction fell to 0.444 on nine hard questions and 0.250 on four multi-hop questions.
- The pre-specified constrained-category condition of H2 was unmet, while comparisons with four graph-augmented baselines and several construction and generalization measures remained pending.The reported campaign therefore does not establish the planned constrained-category hypothesis.