PaperlayerView exact revision on arXiv

Source-linked AI summary

Disentangling Long-Term Memory via Latent Neuro-Symbolic Reasoning

Cai Ke, Xinghao Chen, Xiaoyu Shen, Keyu Chen, Siyu An, Junnan Dong, Ruifeng Xu, Ruizhi Qiao, Xing Sun

arXiv:2609.18461v2cs.AIcs.CLcs.LG

TL;DR

Long-term personalization must infer explicit preferences and implicit behavioral evidence from history, yet flat retrieval misses distributed clues and static graphs cannot adapt to query-specific relations. LGM represents history as sparse concepts in a query-conditioned latent graph and uses that state to condition generation. Across benchmarks and backbones, it improves explicit and implicit preference performance, with the largest gains under long contexts.

  • Problem

    Flat retrieval misses distributed evidence and static graphs cannot adapt relations to a query, while entangled textual memories make fine-grained cross-session reasoning computationally costly.

    Source: §1 · pp. 1–2

  • Method

    LGM maps interactions into sparse latent concepts, constructs query-conditioned graphs, and turns graph states into compact prefix tokens for generation.

    Source: §1 · p. 2, §4.1 · p. 5, §4.3 · p. 6

  • Results

    Across three benchmarks and multiple backbones, LGM improves explicit and implicit preferences, with the largest gains under long contexts.

    Source: §6 · p. 11

  • Takeaways & Limitations

    The authors conclude that on-demand latent-space memory organization is a promising practical path, with LGM on the low-cost frontier for calls, tokens, and latency.

    Source: §6 · p. 11

Abstract

from arXiv · show

Personalized agents are required to reason over long-term history interactions to infer both explicit preferences and implicit behavioral evidence. While early flat retrieval methods score memory fragments independently and neglect the distributed information, current structured memory frameworks rely on query-agnostic static graphs that fail to capture the context-dependent relations. Crucially, raw textual memories are inherently entangled and noisy, making fine-grained personalization and cross-session reasoning computationally prohibitive. To this end, we present LGM, a novel neuro-symbolic framework that shifts long-term memory disentanglement into a continuous latent space. Specifically, (i) instead of persisting fixed graphs, we design a tailored latent graph construction with a sparse autoencoder. Subject to each query, it maps historical interactions into latent memory nodes and disentangles the memory traces into sparse concept activations, dynamically synthesizing query-aware relational edge weights. (ii) A graph encoder then treats the query embedding as a conditioning preference to direct non-linear message passing across the task-specific latent subgraph. This yields a highly expressive memory representation for effective activations. Extensive experiments on long-term personalization benchmarks demonstrate that LGM significantly outperforms state-of-the-art baselines in capturing both explicit and implicit preferences while enabling personalized responses.

1 Introduction

Long-term personalization requires linking explicit preferences and implicit behavioral evidence across interactions, but flat retrieval misses distributed clues and static graphs cannot adapt their relations to a query. LGM shifts memory disentanglement and relational reasoning into latent space, and experiments report improvements across long-term personalization benchmarks.

Motivation

  • Flat retrieval misses distributed, multi-hop evidence, while static structured graphs cannot capture relations that depend on the incoming task.Historical interactions can entangle topic, intent, sentiment, and context, only some of which matter to a given request.Source: §1 · pp. 1–2

LGM framework

  • LGM uses sparse concept decomposition and query-conditioned latent graphs to disentangle historical interactions and dynamically connect relevant memory cues.Relational message passing produces a compact state for generation; an evidence-reconstruction objective is intended to prevent information loss.Source: §1 · pp. 2–3

Contributions

  • The paper formulates long-term personalization as query-aware memory disentanglement and relational reasoning in latent space.Its stated contributions combine sparse concept decomposition, query-conditioned graph construction, relational message passing, and graph-conditioned generation.Source: §1 · p. 3
  • Experiments across long-term personalization benchmarks report consistent gains in explicit and implicit preferences, especially for long contexts, with lower textual-memory replay overhead.The abstract reports evaluation across multiple language-model backbones.Source: §1 · p. 3

2 Related Work

Prior memory approaches organize history as fixed structures, isolated textual records, or discrete read/write policies. LGM instead defers relational structure to inference and makes memory a differentiable latent representation.

Structured Retrieval-Augmented Generation

  • Tree- and graph-based retrieval methods commit to a global topology, whereas LGM lets each query induce its own latent structure at inference time.The related-work discussion names RAPTOR [24], MemTree [14], GraphRAG [25], LightRAG [26], HippoRAG [13], and Youtu-GraphRAG [16].Source: §2 · p. 3

Agentic Memory Management

  • Agentic memory systems store discrete textual records, while LGM composes scattered weak clues relationally in latent space.The passage contrasts profile compression and interconnected notes with evidence that may not appear in any single record.Source: §2 · p. 3

Reinforcement Learning for Memory

  • RL memory methods optimize discrete read/write policies over fixed textual memory; LGM makes memory itself a differentiable, learnable representation.The passage discusses MemAgent [36], Mem-α [35], MemCoE [37], and Memory-R1 [38].Source: §2 · p. 3

3 Problem Formulation

The task is to generate a personalized response from long-term interactions and a current query. The response should be grounded in relevant history while the memory state captures explicit preferences and implicit behavior scattered across contexts.

Problem Formulation

  • The formulation defines H as multi-session interaction history, q as the current query, fθ as the backbone model, and y as the target response.It also denotes the model embedding lookup as E(·) and hidden representations from layer l as H(l).Source: §3 · p. 4
  • The objective is a faithful, personalized response grounded in relevant historical evidence, conditioned on the current query and a query-dependent memory state.The backbone model autoregressively predicts the target response from the query embedding and constructed memory state.Source: §3 · p. 4
  • The memory state Mq must capture explicit user preferences and implicit behavior scattered across contexts in an arbitrarily long history.The input consists of historical interactions H and an incoming query q.Source: §3 · p. 4

4 Methodology

LGM encodes interactions and queries as sparse concepts, builds a graph whose relations adapt to each query, and uses graph reasoning to condition response generation. Evidence reconstruction anchors the latent channel to recoverable facts.

4.1 Latent Memory Disentanglement

  • Each interaction and query becomes a shared-space latent node, then sparse autoencoder codes align explicit statements with implicit behavioral factors through shared concepts.The sparse, non-negative codes are the interpretable units used for this alignment.Source: §4.1 · pp. 4–5

4.2 Query-Conditioned Latent Graph Construction

  • Concept overlap defines query-dependent edge weights, and bidirectional query-memory relations let message passing combine scattered weak clues into an inferred implicit preference.The same memory pool can induce a different graph for each query.Source: §4.1 · p. 5, §4.2 · pp. 5–6

4.3 Graph-Conditioned Reasoning

  • The query controls latent disentanglement, relations, and aggregation; a compact graph-state prefix conditions autoregressive generation, while evidence reconstruction anchors it and L_recon is the ablation’s most important term.Training generates reference-response text rather than selecting a discrete option.Source: §4.3 · p. 6

5 Experiments

The experiments evaluate LGM on long-term personalization benchmarks spanning explicit and implicit preferences. Results report stronger benchmark performance and lower memory-replay cost, with further analyses examining component contributions, latent-state structure, token budget, and implicit-preference cases.

5.1 Experimental Settings

  • Experiments cover PersonaMem, PrefEval, and PersonaMem-v2’s implicit split, comparing three open backbones with context, structured-RAG, agentic-memory, and RL-based baselines.The implementation uses 16 latent prefix tokens and encodes long histories in chunks, with segments truncated to 2048 tokens.Source: §5.1 · pp. 6–7

5.2 Main Results

  • LGM achieves the best average on both Qwen2.5-7B (72.28) and Gemma3-4B (66.16), improving over MemCoE by +11.20 and +11.70, respectively.Table 1 reports accuracy on PersonaMem and PrefEval; the prose identifies MemCoE as the strongest baseline.Source: §5.2 · p. 7, Table 1 · p. 8
  • Across four datasets, LGM occupies the bottom-left accuracy–cost frontier, with the fewest LLM calls, lowest token consumption, and lowest latency.The prose reports stable latency as context grows to 128K and says graph-construction and hierarchical-scheduling baselines incur higher overhead for weaker results.Source: §5.2 · pp. 7–8, Figure 3 · p. 8

5.3 Ablation Study

  • Removing any component hurts performance; −Reconstruct causes the largest drop, especially for implicit preferences, while −SAE and −GNN also degrade results.The ablation examines the sparse autoencoder, graph propagation, and evidence-reconstruction objective on PrefEval and PersonaMem.Source: Figure 4 · p. 9, §5.3 · p. 9

5.4 Analysis on Inferring Implicit Preferences

  • On PersonaMem-v2, LGM scores 62.9 at 32K and 51.9 at 128K, the best in both settings, outperforming proprietary models and same-backbone prompting, RL-tuned, and agentic-memory variants.The reported same-Qwen3-4B comparison exceeds agentic memory at 55.2 by +7.7.Source: Table 2 · p. 9, §5.4 · pp. 9–10

5.5 Visualization of Graph States

  • On PersonaMem-32K with Gemma3-4B, graph states form scenario-aligned clusters, and semantically related scenarios lie closer together.The authors interpret the projection as evidence that the query-conditioned graph preserves user- and topic-level semantics rather than forming a noisy mixture.Source: Figure 5 · p. 9, §5.5 · p. 10

5.6 Effect of Latent Token Number

  • Accuracy rises quickly with a few latent prefix tokens, peaks around 16, then saturates or slightly declines as the count increases to 32.The trend holds across both backbones and all four settings; the authors use 16 tokens by default.Source: Figure 6 · p. 10, §5.6 · p. 10

5.7 Case Study: Explicit vs. Implicit Preferences

  • In two cases, LGM selects preference-consistent options from indirect behavioral cues, while text-retrieval baselines fall back to preference-violating distractors.The cases pair the same preference in explicit and implicit forms; LGM maps both forms to the same sparse concepts and links them to the query through the latent graph.Source: §5.7 · p. 11

6 Conclusion

LGM reframes long-term personalization as query-aware memory disentanglement and relational reasoning in latent space, and reports gains across benchmarks while remaining low-cost.

6 Conclusion

  • Across three benchmarks and multiple backbones, LGM improves explicit and implicit preferences, with the largest gains under long contexts.It remains on the low-cost frontier in calls, tokens, and latency.Source: §6 · p. 11
  • LGM encodes interactions as latent nodes and sparse concepts, induces a query-conditioned graph, and propagates the query into a compact generation-conditioning state.An evidence-reconstruction objective keeps the process faithful.Source: §6 · p. 11
  • The conclusion presents on-demand latent-space memory organization, rather than replaying raw history, as a promising practical path toward personalized agents.Source: §6 · p. 11

Appendix

The appendix supplies benchmark, baseline, algorithm, and implementation details, and its benchmark table summarizes the three evaluation datasets.

Appendix

  • Appendix A describes the personalization benchmarks, Appendix B the baselines and fair-comparison protocol, Appendix C the LGM algorithm, and Appendix D implementation and hyperparameters.Source: Appendix · p. 14
  • The appendix is presented as additional material omitted from the main paper because of space constraints.Source: Appendix · p. 14
  • Table 4 summarizes the three personalization benchmarks and identifies splits where target preferences are never stated verbatim.Source: Table 4 · p. 15

A Dataset Details

The evaluation uses three long-term personalization benchmarks that test explicit preference tracking and inference from behavioral cues, under a shared free-form response protocol.

A Dataset Details

  • The evaluation covers PersonaMem, PrefEval, and the implicit split of PersonaMem-v2, which probe explicit and implicit preference consistency across long, multi-session histories.Source: A Dataset Details · p. 15

PersonaMem

  • PersonaMem contains over 180 simulated histories with up to 60 sessions and tests persistence and preference updating at 32K- and 128K-token context scales.The 128K setting is harder because relevant evidence is diluted across longer histories.Source: A Dataset Details · p. 15

PrefEval

  • PrefEval contains 3,000 preferences across 20 topics and contrasts directly stated preferences with indirect behavioral cues that require inference.Accuracy is reported separately for explicit and implicit splits.Source: A Dataset Details · p. 15

PersonaMem-v2

  • PersonaMem-v2's implicit split covers 1,000 personas and over 20,000 preferences across 300+ scenarios, with target preferences reconstructed from scattered behaviors.It is evaluated at 32K and 128K history settings using the Qwen3-4B backbone.Source: A Dataset Details · pp. 15–16

Evaluation Protocol

  • All methods generate free-form responses and are scored for accuracy against reference answers under fixed backbones, data splits, and context scales.Source: A Dataset Details · p. 16

B Baseline Details

The baseline suite spans raw-context and retrieval methods, structured graphs, agentic memory systems, and reinforcement-learning approaches; LGM instead operates on query-specific latent memory.

Baseline Setup

  • Baselines use recommended public-code settings, shared backbones and data splits, and free-form responses for comparison with LGM.Source: B Baseline Details · p. 16

Context / Retrieval Basics

  • Long Context feeds the full history into the model, while Naive RAG retrieves top-K similar snippets without connecting distributed evidence.Long Context degradation at 128K illustrates the lost-in-the-middle problem [46].Source: B Baseline Details · p. 16

Structured Retrieval-Augmented Generation

  • Structured RAG baselines organize memory into explicit graph structures, but their largely query-agnostic topology motivates LGM's query-conditioned latent graph.Examples include GraphRAG, HippoRAG, LightRAG, and Youtu-GraphRAG.Source: B Baseline Details · pp. 16–17

Agentic Memory Management

  • Agentic systems maintain external memory through read/write operations, but retain textual records that are re-read as raw, entangled, noisy context.The family includes MemoryBank, Mem0, A-Mem, MemoryOS, and LightMem.Source: B Baseline Details · p. 17

Reinforcement Learning for Memory

  • Reinforcement-learning memory methods learn update policies over textual memories and incur repeated prompting or updates; MemCoE is the strongest baseline in the main results.Source: B Baseline Details · p. 17

LGM Algorithm

  • LGM encodes history and query into latent nodes, uses sparse concepts to weight query-memory links, propagates messages, reads out a compact state, and generates from projected prefix tokens.Source: B Baseline Details · p. 18

C Algorithm

LGM converts past interactions and the current query into latent memory nodes, builds a query-relevant graph, and uses its memory state to generate an answer. Training combines response generation, evidence reconstruction, and sparse-concept regularization.

C Algorithm

  • LGM turns each past interaction and the current query into compact latent memory nodes, then links the query to the most relevant memories and passes messages over the graph.The graph produces a single memory state that is supplied to the language model as prefix tokens for answer generation.Source: C Algorithm · p. 18
  • The resulting memory state is provided to the language model as prefix tokens to generate the answer.During training, the same prefix is also asked to reconstruct supporting evidence so the memory state carries information.Source: C Algorithm · p. 18
  • End-to-end training combines response-generation loss, supporting-evidence reconstruction loss, and an SAE loss that keeps concepts sparse.Evidence reconstruction requires the latent prefix to decode the supporting evidence.Source: C Algorithm · p. 18

D Implementation and Hyperparameter Details

LGM’s implementation uses jointly optimized backbone and graph modules, with a default full-fine-tuning setup and an optional LoRA variant. Long histories are encoded in chunks, and the paper lists the complete hyperparameter configuration.

D Implementation and Hyperparameter Details

Loading 2609.18461v2…