PaperlayerView exact revision on arXiv

Source-linked AI summary

ThinkFlow: Self-Evolving Probabilistic Latent Memory for Lifelong Conversational Agents

Cai Ke, Xin Liu, Han Zhang, Jiangyue Yan, Zike Yuan, Ling Deng, Yue Yu, Hui Wang, Ruifeng Xu

arXiv:2609.17010v1cs.AIcs.CL

TL;DR

Lifelong conversational agents need memory that preserves nuanced user states, but explicit textual pipelines lose behavioral detail and typically cannot adapt autonomously after deployment. ThinkFlow compresses conversational flows into disentangled probabilistic latent skills and refines them through teacher-guided alignment and self-supervised next-user-utterance prediction. Across long-term conversation benchmarks, it outperforms prevailing memory systems in personalization and contextual response quality while improving token and temporal efficiency.

  • Problem

    Explicit textual memory pipelines create an information bottleneck that loses subtle user preferences, emotional states, and behavioral patterns, while typically remaining static after deployment.

  • Method

    ThinkFlow uses disentangled probabilistic latent memory skills with teacher-guided latent alignment and self-supervised next-user-utterance prediction for continuous test-time evolution.

  • Results

    ThinkFlow significantly outperforms state-of-the-art explicit memory baselines in personalization and response quality while achieving token and temporal efficiency across long-term conversation benchmarks.

  • Takeaways & Limitations

    The framework supports continuous, label-free personalization by adapting to evolving user nuances through implicit predictive feedback.

  • Takeaways & Limitations

    The framework’s full cognitive potential remains untested across extreme-scale cross-modal lifelong streams and broader societal-scale multi-agent scenarios.

Abstract

from arXiv · show

Lifelong conversational agents rely on memory systems to maintain deep, context-aware interactions with users. However, existing explicit textual memory pipelines suffer from a severe information bottleneck, often losing subtle behavioral patterns and emotional shifts. Furthermore, being typically static post-deployment, they cannot autonomously adapt to personal habits and preferences without manual feedback. Cognitive science, however, suggests that humans maintain mental models purely in a latent space and continuously refine them through predictive coding. Inspired by this, we propose \textbf{ThinkFlow}, a novel end-to-end latent memory framework for lifelong conversational agents. ThinkFlow bypasses the text bottleneck by dynamically compressing conversational flows into probabilistic latent memory skills, autonomously consolidating complex user states into disentangled, continuous vectors without semantic interference. To break this barrier, we introduce a test-time evolution paradigm. By coupling teacher-guided latent alignment to bootstrap the initial state with a self-supervised next-user-utterance prediction task for continuous refinement, the framework successfully overcomes cold-start challenges and achieves label-free lifelong personalization. Extensive experiments on long-term conversation benchmarks demonstrate that ThinkFlow significantly outperforms prevailing memory systems, providing highly personalized and contextually accurate responses over extended multi-session interactions.

1 Introduction

ThinkFlow addresses information loss and static adaptation in explicit textual memory by representing conversational experience as evolving latent skills. It combines structured latent-memory components with test-time predictive refinement and outperforms existing memory systems in long-term personalization.

  • ThinkFlow targets explicit memory’s information bottleneck, which can discard implicit preferences, emotional shifts, and unspoken behavioral patterns while remaining static after deployment.
  • Probabilistic Latent Memory Skills compress conversational context into K disentangled vectors, while a Gated Latent Consolidator controls updates to filter noise and limit semantic dilution.
  • A Context-Aware Hyper-Aligner bridges historical latent skills and the LLM’s current semantic context.
  • Teacher-guided latent alignment bootstraps the system, after which next-user-utterance prediction uses implicit reactions to refine latent skills continuously without labels.
  • ThinkFlow outperforms prevailing memory systems by producing highly personalized and contextually accurate responses across extended multi-session interactions.

2 Related Work

Existing conversational memory predominantly uses explicit textual structures and pipelines, whereas latent-memory approaches encode experience in continuous representations. ThinkFlow follows the latter direction to avoid textual information loss and redundancy.

  • Explicit-memory systems organize past conversations through retrieval-augmented structures, condensed text, and architectures that read, write, and update textual memory blocks.
  • ThinkFlow differs from explicit textual pipelines by compressing and evolving conversational flows in continuous probabilistic latent space.
  • Latent memory is presented as a token-efficient, machine-native, end-to-end learnable substrate for encoding multi-session experiences.

3 Methodology

ThinkFlow stores conversational history as probabilistic latent memory skills and evolves them through gated consolidation, context-aware alignment, and self-supervised test-time learning.

  • Overall Architecture: ThinkFlow compresses conversational flows into multiple probabilistic latent memory skills that capture distinct user-profile aspects without semantic interference.The skills represent dimensions such as factual events, emotional states, and implicit preferences.
  • Probabilistic Latent Memory Skills: PLMS uses cross-attention to compress hidden-state thought streams into K skills, models their uncertainty with mean and log-variance outputs, and trains stochastic sampling with reparameterization and KL regularization.A distribution-preserving scaling factor prevents compressed-skill variance from collapsing into a deterministic point mass.
  • Gated Latent Consolidator: The Gated Latent Consolidator controls permanent memory updates by estimating information gain, filtering conversational noise, and updating skills sequentially with a GRU.This design limits semantic dilution while preserving temporal order and forgetting outdated elements.
  • Context-Aware Hyper-Aligner: The Context-Aware Hyper-Aligner generates a query-conditioned low-rank transformation, applies residual projection to historical skills, and prepends the aligned skills as soft prompts.The hypernetwork adapts latent memory to the current semantic context without adding massive parameters.
  • Self-Supervised Test-Time Evolution: Self-Supervised Test-Time Evolution first distills a full-history teacher into a latent-memory student, then refines skills through next-user-utterance prediction using implicit feedback without manual annotations.High prediction loss closes the GLC gate to protect the skill bank from bad updates, while a confidence prior discourages ambiguous semantic collapse.

4 Experiments

ThinkFlow is evaluated across long-term conversational generation, personalized memory, ablations, latent representations, scaling, efficiency, and user-evolution case studies. Across these analyses, it consistently improves response quality and personalization while reducing token and time overhead.

  • 4.2 Performance in Long-Term Generation: ThinkFlow consistently achieves the best overall performance across long-term conversational datasets, with especially large Mauve improvements over explicit memory baselines.The evaluation covers MSC, CC, GC, and PersonaMem, using generation metrics and PersonaMem accuracy.
  • 4.3 Performance in Personalized Memory: ThinkFlow achieves the highest average PersonaMem accuracy across contexts up to 1M tokens, particularly on dynamic user-evolution tasks, while ThinkFlow-8B rivals or surpasses larger models.The comparison includes powerful closed-source and 405B open-source models.
  • 4.4 Ablation Study: Removing PLMS, GLC, CAHA, Phase1, or Phase2 causes performance declines, showing that disentangled skills, gated consolidation, hyper-alignment, and both training phases are important.The ablation evidence is reported on CC and extended to MSC and GC in supplementary results.
  • 4.5 Framework Analysis: PersonaMem t-SNE representations separate cognitive tasks, with Shared Facts forming a localized cluster while Track Evolution occupies a broad continuous manifold.This structure is presented as evidence that static facts and shifting user states are represented differently in latent memory.
  • 4.5 Framework Analysis: Increasing latent skills from K=1 to K=10 steadily raises PersonaMem accuracy, with particularly strong gains on Revisit Reasons and Track Evolution.The 1M-context benchmark contains seven task categories, so K=10 tests capacity beyond task complexity.
  • 4.5 Framework Analysis: ThinkFlow uses 2,857 tokens and 12.1 seconds, whereas MemTree and A-Mem generate tens of thousands of hidden tokens and take nearly eight times longer.The comparison is presented as evidence of lower token and temporal overhead for latent memory.
  • 4.5 Framework Analysis: In a relationship-crisis case study, ThinkFlow tracks the user’s emotional shift and produces an empathetic response instead of retrieving outdated dressing-up interests.The case contrasts latent emotional-state tracking with MemTree’s surface-fact retrieval.

5 Conclusions

ThinkFlow proposes latent memory skills and self-supervised test-time evolution to address information bottlenecks and static deployment. Experiments show stronger personalization and response quality than explicit baselines, together with token and temporal efficiency.

  • 5 Conclusions: ThinkFlow compresses conversational flows into disentangled probabilistic memory skills and continuously adapts to evolving user nuances through implicit predictive feedback.The framework is designed for lifelong conversational agents and avoids manual annotations during adaptation.
  • 5 Conclusions: Across long-term conversation datasets and PersonaMem, ThinkFlow outperforms explicit memory baselines in personalization and response quality while achieving token and temporal efficiency.The conclusion reports this result across multiple long-term open-domain conversation datasets and PersonaMem.

Limitations

ThinkFlow’s current scope is bounded by untested extreme-scale, multimodal lifelong contexts and by a relatively narrow predictive signal for test-time evolution.

  • Limitations: The framework has not established its upper breakdown limits for ultra-long multi-session histories because its latent skills process them efficiently on standard hardware.Extreme-scale lifelong contexts remain untested despite the framework’s token efficiency and low latency.
  • Limitations: Infinite multimodal streams involving lifelong visual and audio inputs remain outside the demonstrated scope, although the continuous latent space is suited to cross-modal fusion.The stated future direction extends beyond text-only histories to continuous visual and audio inputs.
  • Limitations: Current test-time evolution relies primarily on next-user-utterance prediction, leaving its disentangled latent manifolds underused for long-horizon behavioral or multi-agent modeling.The authors identify broader cross-modal and societal-scale scenarios as areas where ThinkFlow’s cognitive potential remains unexplored.

A Dataset Information

The evaluation uses human-centered long-term conversation datasets and PersonaMem to test conversational generation, personalized memory, and reasoning across increasingly large memory contexts.

  • Long-Term Open-Domain Conversation: The study evaluates prolonged turn-by-turn response generation on Conversation Chronicles, Multi-Session Chat, and GapChat.These are representative long-term open-domain conversation datasets.
  • Dataset Statistics: Tables 5 and 6 report dataset statistics for the long-term conversation datasets and PersonaMem’s memory-corpus scales.The tables provide the stated statistical summaries for the evaluation resources.
  • Long-Term Open-Domain Conversation: The CC, MSC, and GC datasets are built through extensive crowdsourcing with real participants to reflect natural dialogue, topic transitions, and relationship development.Their human-centric construction supports evaluation of natural, coherent, and engaging responses in open-domain social interactions.
  • Personalized Memory Question Answering: PersonaMem expands dialogue history from about 24,000 to over 910,000 tokens while keeping queries near 400 tokens, creating a short-query, ultra-long-memory retrieval challenge.The benchmark also tests seven specific skills, ranging from simple fact retrieval to more demanding memory reasoning.

B Compared Baselines

ThinkFlow is compared with explicit text-based memory systems and latent memory approaches, spanning structured retrieval, memory governance, textual summarization, and continuous representations.

  • Compared Baselines: The baseline suite is organized into Explicit Memory Paradigm and Latent Memory Paradigm categories following the paper’s related-work taxonomy.This comparison distinguishes discrete textual memory from continuous latent encoding of multi-session experiences.
  • Explicit Memory Paradigm: Structured retrieval baselines organize conversational logs as graphs or trees, including GraphRAG’s entity-community summaries and MemTree’s dynamic hierarchical nodes.These methods support retrieval and reasoning over structured historical context.
  • Explicit Memory Paradigm: Memory-governance baselines actively read, write, and manage memory blocks, including MemGPT, Mem0, A-Mem, and MemoryOS.Their mechanisms range from virtual context paging and extraction-based updates to tagged notes and tiered storage.
  • Explicit Memory Paradigm: Textual-memory baselines build discrete factual memories or summaries through memorization cycles, hierarchical distillation, recursive updating, unified compression, modular personalization, record preservation, or tiered filtering.The listed systems are MemoChat, MemoryBank, Rsum, COMEDY, LD-Agent, THEANINE, and LightMem.
  • Latent Memory Paradigm: Latent-memory baselines encode experiences directly into continuous representations, using soft thought tokens, KV-cache embeddings, or dynamically woven latent token sequences.The cited approaches are SoftCoT, Co-processor, and MemGen.

C Implementation Details

Implementation uses LoRA and specified memory-module dimensions, AdamW optimization with gradient controls, and low-temperature nucleus sampling for inference.

  • Implementation Details: LoRA targets q_proj, k_proj, v_proj, and o_proj with rank r = 8 and α = 16.The PLMS cross-attention module uses 8 heads, while the GLC gate’s intermediate projection is D/2.
  • Implementation Details: The KL regularization and Phase 2 minimum-entropy confidence-prior weights are both 0.1, while inference uses top-p = 0.8, τ = 0.3, and a 1024-token maximum.Inference therefore combines nucleus sampling with low temperature and a fixed output-length cap.

D Ablation Study

ThinkFlow replaces retrieved textual summaries with continuous probabilistic latent memory skills and trains through a two-phase test-time evolution procedure. Ablations indicate that removing key memory components or either training phase reduces performance across most metrics.

  • D Ablation Study: Removing PLMS, GLC, CAHA, Phase1, or Phase2 causes performance drops across most metrics on MSC and GC.This extends the consistent ablation pattern observed on the CC dataset.
  • D Ablation Study: Phase 1 uses a teacher with full explicit context to align a student relying on CAHA-translated latent skills, while PLMS and GLC update the latent state.The student is aligned to the teacher's target distribution by minimizing KL divergence during cold-start training.
  • D Ablation Study: ThinkFlow injects historical user states as continuous probabilistic latent memory skills directly into the embedding space instead of appending retrieved text summaries.The approach maintains minimal textual context while dynamically translating historical states into latent representations.

F Parameter Analysis

The parameter analysis examines how latent memory capacity affects performance across context lengths. Larger capacity is especially important for ultra-long 1M-token interactions, where increasing the number of skills produces a sharp upward trend.

  • F Parameter Analysis: In the 1M context setting, accuracy rises sharply and robustly as latent memory skills K increases from 1 to 10.The analysis contrasts this with relatively high accuracy at shorter 32K contexts even with fewer skills.
  • F Parameter Analysis: As dialogue history grows, larger sets of disentangled latent vectors become essential for capturing expanding user nuances without semantic interference.The finding motivates scaling latent capacity for ultra-long interactions to prevent feature blending.
  • F Parameter Analysis: Figure 7 analyzes latent memory skill capacity across 32K, 128K, and 1M context lengths.The figure is the parameter-scaling analysis referenced in the accompanying discussion.
Loading 2609.17010v1…