PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces

Shreyas Rajesh, Pavan Holur, Chenda Duan, David Chong, Vwani Roychowdhury

arXiv:2511.07587v2cs.AIcs.CL

TL;DR

Long-context LLMs face finite input windows and performance degradation, while structured RAG is oriented toward fact retrieval rather than evolving narrative events. GSW builds an actor-centric workspace from incoming context and reports strong EpBench results with lower query-time token use.

  • Problem

    Structured RAG models cross-chunk relationships, but existing methods have primarily been evaluated on fact-rich texts rather than narratives tracking actors, roles, and events across space and time.

    Source: §1 · p. 1

  • Method

    GSW maintains an interpretable, actor-centric model of evolving situations, with the Operator mapping new context to an intermediate state and the Reconciler updating a persistent workspace.

    Source: §1 · p. 1, §2.2 · p. 4

  • Results

    GSW achieves 0.850 overall F1-Score on EpBench-200 and 0.773 overall F1-score on EpBench-2000; its token use is 51% lower than GraphRAG.

    Source: §4 · pp. 5–6

  • Takeaways & Limitations

    The workspace offers LLM systems a persistent, interpretable representation for reasoning over evolving situations.

    Source: §6 · p. 7

    EpBench does not thoroughly probe complex evolution of actor roles and states within extended narratives.

    Source: §6 · p. 7

Abstract

from arXiv · show

Large Language Models (LLMs) face fundamental challenges in long-context reasoning: many documents exceed their finite context windows, while performance on texts that do fit degrades with sequence length, necessitating their augmentation with external memory frameworks. Current solutions, which have evolved from retrieval using semantic embeddings to more sophisticated structured knowledge graphs representations for improved sense-making and associativity, are tailored for fact-based retrieval and fail to build the space-time-anchored narrative representations required for tracking entities through episodic events. To bridge this gap, we propose the \textbf{Generative Semantic Workspace} (GSW), a neuro-inspired generative memory framework that builds structured, interpretable representations of evolving situations, enabling LLMs to reason over evolving roles, actions, and spatiotemporal contexts. Our framework comprises an \textit{Operator}, which maps incoming observations to intermediate semantic structures, and a \textit{Reconciler}, which integrates these into a persistent workspace that enforces temporal, spatial, and logical coherence. On the Episodic Memory Benchmark (EpBench) \cite{huet_episodic_2025} comprising corpora ranging from 100k to 1M tokens in length, GSW outperforms existing RAG based baselines by up to \textbf{20\%}. Furthermore, GSW is highly efficient, reducing query-time context tokens by \textbf{51\%} compared to the next most token-efficient baseline, reducing inference time costs considerably. More broadly, GSW offers a concrete blueprint for endowing LLMs with human-like episodic memory, paving the way for more capable agents that can reason over long horizons. Code is available at https://github.com/roychowdhuryresearch/gsw-memory.

1 Introduction

GSW targets narrative reasoning that requires tracking actors, roles, events, and their spatiotemporal context, rather than only retrieving facts. It builds structured episodic memories and reports strong results on EpBench.

1 Introduction

  • Existing structured RAG methods model relationships across chunks, but have primarily been evaluated on fact-rich texts rather than narratives of actors changing roles and states across space and time.Such narratives include crime reports, political briefings, corporate filings, war dispatches, and multi-day news coverage.Source: §1 · p. 1
  • GSW represents evolving situations as interpretable, actor-centric, predictive semantics that can serve as an episodic-memory module and bridge language with latent world models.Its world model tracks who is involved, what was done, where and when events occur, how roles change, and what consequences follow.Source: §1 · pp. 1–2
  • For a query about Carter Stewart’s conference presentations, GSW links an indirectly described narrator’s location and date to Stewart, then retrieves targeted memory for relevant entities.The example contrasts this with GraphRAG missing a location and hallucinating two, and HippoRAG2 retrieving many documents and hallucinating three locations.Source: §1 · p. 2
  • On EpBench-200, GSW achieves 0.85 F1-score, improves recall by up to 20% on queries spanning as many as 17 documents, and reduces context tokens by 51% versus the most token-efficient baseline.On EpBench-2000, a 10x larger dataset, GSW achieves 0.773 F1-score and exceeds the best baseline by more than 15% on overall recall.Source: §1 · p. 2

2 The Generative Semantic Workspace (GSW) Framework

GSW models episodic memory as a structured, evolving semantic workspace inspired by complementary neocortical and hippocampal functions. The Operator extracts semantic structure from new context, and the Reconciler updates the persistent workspace.

2 The Generative Semantic Workspace (GSW) Framework

  • GSW draws on the neocortex–Operator and hippocampus–Reconciler analogy to motivate structured representations that track actors, evolving roles and states, spatiotemporal grounding, and updates as information arrives.The framework also draws on the hippocampal-neocortical account of sequence modeling, memory consolidation, and predictive modeling.Source: §2 · pp. 2–3, Figure 1 · p. 3
  • LLMs make bidirectional mapping tractable: they can infer concise semantic identifiers from text and generate coherent narratives from those identifiers.The paper contrasts this dynamic, probabilistic process with earlier symbolic frameworks relying on fixed ontologies.Source: §2 · p. 3

2.1 A Probabilistic Model for Semantic Memory: The Operator Framework

  • The Operator’s semantic map encodes actors, roles that shape possible actions, states that constrain those actions, verbs that signal role or state changes, and temporal and spatial continuity.The framework also encodes potential future outcomes and actions as questions Q_n.Source: §2.1 · pp. 3–4

2.2 Enabling Recursive Updates: A State Space Approach (The Reconciler Framework)

  • The Reconciler uses a Markovian transition model to combine the prior workspace with the Operator’s current-context representation, recursively updating M_n−1 to M_n.The stated factorization assumes conditional independence between the previous workspace and the current intermediate representation given the context sequence.Source: §2.2 · p. 4

3 Question Answering with GSW

GSW answers questions by retrieving and reranking summaries from its workspace for an LLM. The evaluation uses EpBench’s episodic questions, LLM-judged answer metrics, several RAG baselines, and GPT-4o under standardized settings.

3 Question Answering with GSW

  • For question answering, GSW matches query entities to its workspace, generates contextual summaries for those entities, reranks them for relevance, and passes the top summaries to an LLM to synthesize an answer.The implementation uses string matching to locate entities and semantic similarity to rerank their summaries.Source: §3 · p. 5, §3.4 · p. 5

3.1 EpBench: An Episodic Memory Benchmark

  • EpBench tests episodic recall and reasoning over long narratives, including recall of events in unique spatiotemporal contexts and distinction between recurring events involving the same actors.Its synthetic chapters reuse time, location, and actor cues; the evaluation uses 200-chapter and 2000-chapter versions.Source: §3.1 · p. 5

3.2 Evaluation Metrics

  • The evaluation follows EpBench’s answer-processing and extraction procedure, using an LLM judge to extract answers before calculating Precision, Recall, and F1 scores.The extraction procedure accommodates responses that are longer or more elaborate than concise ground-truth answers.Source: §3.2 · p. 5

3.3 Baseline Methods

  • Baselines comprise Vanilla LLM, embedding-based RAG using Voyage-032, and the structured RAG methods GraphRAG, HippoRAG2, and LightRAG.Voyage-032 was selected for its strong performance on retrieval benchmarks.Source: §3.3 · p. 5

3.4 Implementation Details

  • The comparison standardizes GPT-4o answer generation and limits context to 17 chapters per query across methods.The GSW Operator and Reconciler are prompted with task-specific instructions at temperature 0; summaries for matched entities are reranked by semantic similarity.Source: §3.4 · p. 5

4 Results and Discussion

GSW performs strongly across episodic-recall cue levels, including the most demanding queries, and retains an advantage on a tenfold-larger corpus while using fewer query-context tokens.

QA Performance

  • GSW achieves the highest overall F1 (0.850), Precision (0.865), and Recall (0.894), and ranks first in 16 of 18 cue-level metric computations.For 6+ cues, it scores F1 0.834, Precision 0.891, and Recall 0.822, with approximately 20% higher recall than HippoRAG2.Source: §4 · p. 5
  • The Vanilla LLM is the weakest baseline, with an overall F1 score of 0.642.Source: §4 · p. 6

Scalability on EpBench-2000

  • On EpBench-2000, GSW reaches an overall F1-score of 0.773, 15% above embedding RAG and 22% above other structured RAG methods.The corpus is ten times larger than EpBench-200.Source: §4 · p. 6

Token Efficiency

  • GSW reduces query-time token usage by 51% versus GraphRAG and nearly 59% versus Embedding RAG and HippoRAG2.It generates entity-specific summaries instead of passing entire chapters or raw chunks to the LLM.Source: §4 · p. 6

Ablation Studies

  • Ablations in the full version examine identifier types, shortened EpBench, and retrieval strategies to assess component contributions and memory querying.The supplied passage does not report their results.Source: §4 · p. 6

5 Related Work

Related work addresses long-context limits through retrieval and structured representations, while episodic-memory approaches include model-specific designs; GSW is presented as compatible without specialized training.

Long-Context Processing

  • Long-context processing quality can degrade at extreme lengths, with performance notably dipping for information in the middle of contexts (Liu et al. 2023).Source: §5 · p. 6

Retrieval-Augmented Generation

  • Standard RAG retrieves relevant chunks but can miss information dispersed across chunks; graph-based methods model relationships to support multi-hop reasoning.The discussion names GraphRAG (Edge et al. 2025), LightRAG (Guo et al. 2025), and HippoRAG (Gutiérrez et al. 2025a,b).Source: §5 · p. 6

Episodic Memory

  • Larimar modifies LLM attention and EM-LLM integrates memory components with open-weight models, whereas GSW requires no specialized training or fine-tuning (Das et al. 2024; Fountas et al. 2024).Source: §5 · pp. 6–7

6 Concluding Remarks and Limitations

The conclusion presents GSW as persistent structured memory for evolving roles and interactions, reports benchmark advantages, and identifies evaluation scope and implementation dependencies as limitations.

6 Concluding Remarks and Limitations

  • The Operator interprets local semantics and the Reconciler integrates and updates representations into persistent structured memory mapping evolving roles, states, and interactions.On EpBench, the paper reports advantages in accuracy and token efficiency; evaluation scope, reliance on GPT-4o, and text-only modalities remain limitations or future-work boundaries.Source: §6 · p. 7

Appendix

  • The appendix lists materials on prompts, a GSW instance and QA example, qualitative analysis, implementation, ablations, related work, computational resources, and workspace models.Source: Appendix · p. 12

A Prompts to the LLM

Operator

  • The Operator prompt generates representations, with figures showing the prompt introduction and first task alongside summaries of remaining tasks.Source: Operator · p. 13

Reconciler

  • The Reconciler prompt addresses unanswered queries using incoming context.Source: Reconciler · p. 13

Question Answering

  • The QA prompts generate entity summaries for the answering agent and specify the answering agent’s prompt.Source: Question Answering · p. 13

B Example GSW instance

  • The example instance illustrates Operator representations for two chunks, their reconciliation, and a resulting workspace with reconciled entities and space-time coupling.It also includes answered forward-looking questions.Source: B Example GSW instance · p. 13

C GSW QA Example

  • The QA example section introduces a figure tracing a sample EpBench query through the GSW pipeline.Source: C GSW QA Example · p. 13

D Qualitative Analysis of GSW performance

The qualitative analysis examines queries requiring synthesis across two to seven contextual cues and compares GSW with baseline approaches. It describes how GraphRAG and HippoRAG2 can lose or overwhelm relevant context, while GSW reconciles scattered spatiotemporal information into targeted entity summaries.

D Qualitative Analysis of GSW performance

  • The selected queries vary in complexity, with answers requiring synthesis across two to seven distinct contextual cues.Their full text and ground-truth answers are provided in Table 5; the qualitative comparison is in Table 6.Source: D Qualitative Analysis of GSW… · p. 13, Table 5 · p. 15, Table 6 · p. 16
  • GraphRAG can lose information and supply irrelevant context, while its summaries may miss precise spatial and temporal details.The analysis notes these problems particularly for Q3 and Q4; Table 6 compares qualitative performance on selected queries.Source: D Qualitative Analysis of GSW… · p. 13, Table 6 · p. 16
  • HippoRAG2 retrieves full chapters after graph-based retrieval, an approach that can overwhelm the LLM when queries require many documents.The benchmark questions can require as many as 17 documents, and the analysis also reports cases where HippoRAG2 did not retrieve all pertinent documents.Source: D Qualitative Analysis of GSW… · p. 13
  • GSW’s structured representation and targeted summaries mitigate baseline issues by collating spatiotemporal information scattered across a document.The Carter Stewart summary combines information drawn from three different paragraphs, although he is initially referred to only as “He.”Source: D Qualitative Analysis of GSW… · p. 13

E Implementation Details

The implementation details describe how GSW representations are produced and reconciled, how entity summaries are used for answering, and how baseline configurations are aligned for comparison.

Operator

  • The Operator uses GPT-4o at temperature 0, after chapter-level coreference resolution and chunking into non-overlapping three-sentence chunks.Space-Time coupling is performed after Operator extraction, using GPT-4o with temperature 0 and a maximum of 1,000 generation tokens.Source: Operator · p. 17

Reconciler

  • Reconciliation combines consecutive Operator chunks into one chapter-level GSW and preserves timestamped entity roles, states, and spatiotemporal information.New space/time information updates the associated nodes, while earlier information is preserved; updates propagate to other entities coupled to the same node.Source: Reconciler · p. 17
  • Entity summaries incorporate roles, states, spatiotemporal information, and addressed questions; query answering matches named entities, retrieves summaries, reranks them by cosine similarity, and passes them to GPT-4o.The answering context is limited to summaries from at most 17 chapters; the described workflow also allows possible further token reductions by prioritizing chapters containing all queried entities.Source: Reconciler · pp. 17, 21, Figure 11 · p. 20

Baselines

  • The baseline configurations use each method’s default hyperparameters and prompts, with HippoRAG2’s answering model set to GPT-4o and top-k set to 17.Configuration details are listed for GraphRAG, HippoRAG2, and LightRAG in Tables 7–9.Source: Baselines · p. 21, Table 7 · p. 21, Table 8 · p. 21

Bootstrapping for Evaluation

Evaluation variability is estimated by bootstrap resampling test predictions 1,000 times for EpBench-200 and EpBench-2000.

Bootstrapping for Evaluation

  • 1,000 bootstrap resamples estimate score variability by resampling test predictions with replacement for EpBench-200 and EpBench-2000.The reported standard deviations reflect variability under different weightings of test examples; the LLM judge used temperature 0.Source: Bootstrapping for Evaluation · p. 21

F Ablation Studies

The section reports that ablation studies were performed on the GSW framework.

F Ablation Studies

  • The authors performed ablation studies on the GSW framework.The supplied passage does not specify the ablation variants or findings.Source: F Ablation Studies · p. 21

Evaluating the GSW on the Short Book Dataset

On the shorter 20-chapter EpBench variant, Vanilla LLM performs strongly because the context fits within its window, but GSW does better as matching-cue complexity increases. On the 200-chapter version, Vanilla LLM struggles beyond 100,000 tokens while GSW remains robust across both lengths.

Evaluating the GSW on the Short Book Dataset

  • In the 3-5 cue category, GSW achieves superior recall (0.910 vs 0.781) and F1-score (0.857 vs 0.777) on the shorter variant.The context is approximately 10,000 tokens, which fits within the model’s context window.Source: Evaluating the GSW on the Short Book… · p. 21
  • Vanilla LLM performs particularly well on the 20-chapter version because its approximately 10,000-token context fits within the model’s window.Despite this, its performance begins to lag GSW as matching cues increase.Source: Evaluating the GSW on the Short Book… · p. 21
  • Vanilla LLM struggles on the 200-chapter version when context exceeds 100,000 tokens, while GSW maintains robust performance across both short and long narratives.The authors describe this as evidence that Vanilla LLM performance deteriorates with increased context length.Source: Evaluating the GSW on the Short Book… · p. 21

Detailed Results on EpBench-2000

On EpBench-2000, GSW retains its lead across cue-complexity levels under a 17-chapter-per-query cap. The accompanying discussion relates episodic reasoning to limits of long-context processing, chunk retrieval, and static semantic structures.

Detailed Results on EpBench-2000

  • Queries are capped at 17 chapters although the dataset reaches 138, because that extreme is an outlier and would add computational overhead.The cap matches EpBench-200; the authors expect similar token usage under that configuration.Source: Detailed Results on EpBench-2000 · p. 21

G Related work on Memory Augmentation for LLMs

  • Episodic memory requires a dynamic, coherent account of events across space and time, including evolving entity states and roles.The paper frames this as a challenge for LLM reasoning over extended sequences.Source: G Related work on Memory Augmentation… · p. 22

Leveraging Long context LLMs

  • Long-context processing remains limited by finite windows, cost, and quality degradation; unstructured input also leaves state changes and temporal relations to attention alone.Irrelevant text may distract the model from details needed for a specific episodic query.Source: Leveraging Long context LLMs · p. 22

Detailed Results on EpBench-2000

Memory Augmentation for LLMs

  • Standard RAG can fragment evolving narratives through disconnected chunks, and arbitrary chunk boundaries or weak spatiotemporal encoding can lose context needed for episodic reasoning.The passages contrast this with standard RAG’s effectiveness on fact-based question answering.Source: Memory Augmentation for LLMs · p. 23

Structured Representations as Memory

  • Structured graph methods preserve relationships but generally lack explicit tracking of temporal state and role changes; GSW is presented as a plug-and-play module needing no specialized training or fine-tuning.The framework is described as compatible with any underlying LLM.Source: Structured Representations as Memory · pp. 23–24

H Computational Costs and Resources for Building the GSW

  • About $15 and roughly 1 hour: indexing 200 EpBench chapters with GPT-4o covers GSW construction, reconciliation, and entity-specific summaries.Parallel OpenAI API calls enable the stated time; the Batch API can reduce cost but takes hours.Source: H Computational Costs and Resources… · p. 24

I Related Computational Models of Semantics

  • PropBank and FrameNet rely on finite semantic-role lexicons, while event graph models use predefined schemas and may marginalize individual document contexts; GSW instead generates evolving actor-centric maps.PropBank’s Framesets are verb-specific shared syntactic patterns, with sentence arguments manually annotated and associated with verb-specific definitions.Source: I Related Computational Models of… · p. 25

Comparing Existing Semantic Models to the Operator

The Operator’s representations were preferred to existing semantic frameworks across five situations. The evaluation compared anonymized outputs for the same short context and asked annotators which better represented its semantics.

Annotator Guidelines

Comparing Existing Semantic Models to the Operator

Loading 2511.07587v2…