Source-linked AI summary
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents
Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, Jie Tan
TL;DR
Long-horizon conversational agents must manage growing interaction histories despite finite context windows and limited temporal structure in existing memory systems. TiMem organizes memories with a Temporal Memory Tree, consolidates them progressively without fine-tuning, and performs complexity-aware recall. It achieves state-of-the-art accuracy on both evaluated benchmarks while reducing recalled context on LoCoMo.
Problem
Long-horizon agents need compact, temporally grounded memories because interaction histories grow beyond finite context windows, while existing systems provide limited temporal structure across hierarchical levels.
Method
TiMem uses a Temporal Memory Tree, instruction-guided hierarchical consolidation without fine-tuning, and complexity-aware recall with planning and gating.
Results
TiMem achieves state-of-the-art accuracy on LoCoMo and LongMemEval-S while reducing recalled context on LoCoMo.
Takeaways & Limitations
Temporal continuity serves as the organizing principle for coherent long-horizon memory representations and personalization in TiMem.
Takeaways & Limitations
The framework relies on general-purpose LLMs for middleware operations, and its high-level memories lack explicit categorical structures or knowledge graphs.
Abstract
from arXiv · showhide
Long-horizon conversational agents have to manage ever-growing interaction histories that quickly exceed the finite context windows of large language models (LLMs). Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, often leading to fragmented memories and unstable long-horizon personalization. We present TiMem, a temporal--hierarchical memory framework that organizes conversations through a Temporal Memory Tree (TMT), enabling systematic memory consolidation from raw conversational observations to progressively abstracted persona representations. TiMem is characterized by three core properties: (1) temporal--hierarchical organization through TMT; (2) semantic-guided consolidation that enables memory integration across hierarchical levels without fine-tuning; and (3) complexity-aware memory recall that balances precision and efficiency across queries of varying complexity. Under a consistent evaluation setup, TiMem achieves state-of-the-art accuracy on both benchmarks, reaching 75.30% on LoCoMo and 76.88% on LongMemEval-S. It outperforms all evaluated baselines while reducing the recalled memory length by 52.20% on LoCoMo. Manifold analysis indicates clear persona separation on LoCoMo and reduced dispersion on LongMemEval-S. Overall, TiMem treats temporal continuity as a first-class organizing principle for long-horizon memory in conversational agents. The code is available at https://github.com/TiMEM-AI/timem.
1 Introduction
TiMem addresses the challenge of converting unbounded, temporally evolving interactions into compact, stable memories for long-horizon personalization. It organizes and consolidates memories hierarchically, then recalls them according to query complexity without fine-tuning.
- Long-horizon agents must preserve temporal coherence and stable persona representations despite unbounded interaction histories and finite LLM context windows.
- Instruction-guided consolidation progressively transforms fine-grained episodic interactions into higher-level semantic patterns and persona representations without additional fine-tuning.
- Complexity-aware recall selects hierarchy levels through a planner and filters candidates with gating to balance factual detail and higher-level personalization.
- TiMem introduces the Temporal Memory Tree, which enforces explicit temporal containment and granularity across memory levels.
- 75.30% accuracy on LoCoMo and 76.88% on LongMemEval-S accompany 52.20% reduced recalled context on LoCoMo under a consistent evaluation setup.
2 Related Work
Existing approaches address context limits or persistent storage but generally do not treat temporal structure as a first-class organizing principle. TiMem instead combines temporal hierarchy with consolidation and adaptive recall.
- Parametric approaches expand or compress context but remain constrained by architectural limits and lack persistent cross-session memory.
- TiMem’s architecture uses a five-layer Temporal Memory Tree with separate consolidation and recall pipelines.
- Existing approaches often produce fragmented memory representations and unstable long-horizon behavior because temporal structure is not a first-class organizing principle.
3 Methodology
TiMem organizes long-horizon conversational memory in a Temporal Memory Tree that preserves temporal structure while progressively consolidating details into higher-level representations. Its planner, hierarchical recall, and gating modules adapt retrieved context to query complexity without fine-tuning.
- Temporal Memory Tree: The Temporal Memory Tree encodes temporal intervals, semantic memories, parent–child relationships, and abstraction levels for structured memory organization.Parent intervals contain child intervals, while higher levels contain fewer, progressively consolidated memories.
- Memory Consolidation: TiMem uses level-specific instruction prompts to consolidate child memories, historical same-level context, and dialog observations into factual summaries, patterns, and profile representations.Its reproducible hierarchy spans segment, session, day, week, and profile levels, with online consolidation at L1 and scheduled consolidation at L2–L5.
- Memory Consolidation: The hierarchy separates concrete facts, evolving patterns, and stable persona characteristics across progressively broader temporal levels.Lower levels preserve short-interval details, while higher levels summarize longer-term behavioral features and preferences.
- Recall Planning: The recall planner maps each query to a complexity label and keywords, then selects the TMT levels to search without dataset-specific training or labeled annotations.Simple queries access factual details and stable profiles, whereas complex queries traverse broader pattern and profile layers.
- Hierarchical Recall: Hierarchical recall first selects leaf memories using semantic and lexical matching, then propagates through selected ancestor levels to form a candidate set.The candidate set integrates leaves with ancestors, and the number of recalled memories per level depends on query complexity.
- Recall Gating: Recall gating filters candidate memories at inference time, retaining fewer memories for simple queries and broader context for complex queries before ranking by hierarchy and temporal proximity.This complexity-adaptive pipeline balances precision, efficiency, and temporal relevance while retaining query-relevant memories.
4 Experiments
TiMem is evaluated on two long-term conversational-memory benchmarks using common models, embeddings, recall budgets, and judgment protocols. It achieves the strongest reported accuracy while combining hierarchical recall, adaptive scope selection, and temporal consolidation to improve accuracy–efficiency trade-offs.
- Evaluation Setup: TiMem is evaluated on LoCoMo and LongMemEval-S against five representative memory baselines under a consistent LLM, embedding, and recall-budget setup.The setup uses gpt-4o-mini-2024-07-18, Qwen3-Embedding-0.6B, recall budget k = 20, and LLM-as-a-Judge accuracy.
- Benchmark Results: 75.30% LLJ accuracy on LoCoMo is the best overall result, exceeding MemOS at 69.24% ± 0.11%.TiMem also achieves the best LLJ score in each question type, with F1 54.40 and ROUGE-L 54.68.
- Benchmark Results: 76.88% ± 0.30% LLJ accuracy on LongMemEval-S is the best overall result with gpt-4o-mini-2024-07-18 as the answer model.With gpt-4o-2024-11-20 as the answer model, TiMem remains best overall at 78.96% ± 0.26%.
- Recall Ablations: The adaptive planner improves the accuracy–cost trade-off, reaching 75.30% with 511.25 tokens on LoCoMo and 76.88% with 1270.62 tokens on LongMemEval-S.Fixed-scope recall can under-recall simple queries or introduce noise for complex ones, while gating sharply reduces memory length but may lose accuracy when scope is too narrow.
- Hierarchy Ablations: Hierarchical recall propagation raises LongMemEval-S LLJ from 57.40% to 72.40% with L1-only memories, while the full hierarchy combines fine-grained grounding with higher-level context.L1-only memories remain weaker on LoCoMo for complex queries, and high-level layers alone cannot replace fine-grained evidence.
- Efficiency Analysis: 52.20% fewer recalled tokens on LoCoMo reduces context cost, with TiMem recalling 511.25 tokens per query versus Mem0’s 1,070.10.The full hierarchy increases internal LLM calls by 25%–30%, but this cost is amortized over interaction history and lowers external answering-model token cost and latency.
5 Conclusion
TiMem treats temporal continuity as the organizing principle for long-term conversational memory, combining temporal structure, instruction-guided consolidation, and complexity-aware recall. It achieves strong benchmark accuracy while reducing recalled context on LoCoMo.
- TiMem uses a Temporal Memory Tree to enforce temporal containment and order in long-horizon memory personalization.
- Instruction-guided consolidation without fine-tuning progressively transforms raw dialog into higher-level patterns and incrementally refined profiles.
- Complexity-aware recall plans scope, propagates evidence hierarchically from activated leaves, and gates memories to retain query-relevant information.
- 75.30% on LoCoMo and 76.88% on LongMemEval-S are achieved while recalled context is reduced by 52.20% on LoCoMo.
- Future work includes richer structured representations, storage-time forgetting, and adaptive temporal boundaries to improve efficiency and robustness.
6 Limitations
The limitations concern middleware efficiency, representation structure, forgetting, and temporal parameterization. The authors identify these as directions for improving efficiency, content modeling, factual integrity, and domain transferability.
- Consolidation and recall rely on general-purpose LLMs through instruction prompts, while specialized smaller models may improve efficiency.
- High-level memories lack explicit categorical structures or knowledge graphs, motivating hybrid temporal and typed-entity representations.
- The framework lacks a storage-time forgetting mechanism for selectively consolidating memories while maintaining critical facts and recurring patterns.
- TiMem uses realistic temporal boundaries for reproducibility, while adaptive boundary detection or interaction-density scheduling could improve domain transferability.
7 Ethics Statement
TiMem’s experiments use public benchmarks without human subjects or personally identifiable information, and its memory trees enforce single-user isolation. Deployed systems should add operational privacy safeguards.
- The experiments involve no human subjects or personally identifiable information and use publicly available benchmarks under appropriate licenses.
- Each memory tree is scoped to a single user, with no cross-user memory sharing or aggregation.
- Deployment should implement secure storage, explicit user consent, and data deletion mechanisms.
A Dataset Details
The evaluation uses LoCoMo and LongMemEval-S, which cover multi-session conversational memory under temporal and reasoning-oriented tasks. Their designs differ in source data and assessed capabilities.
- LoCoMo contains 10 user groups with multi-session conversations spanning over six months on average and explicit timestamps.
- LoCoMo evaluates 1,540 questions covering single-hop, multi-hop, open-domain, and temporal reasoning.
- LongMemEval-S is a synthetic benchmark with 500 conversations and 500 questions.
- LongMemEval-S assesses information extraction, multi-session reasoning, knowledge updates, temporal reasoning, and abstention.
B Implementation Details
TiMem’s implementation uses unified language-model and embedding settings, staged temporal memory construction, and complexity-aware recall. Additional experiments examine model configurations and L1 segment granularity, with larger segments reducing QA performance.
- Unified Settings: TiMem uses gpt-4o-mini-2024-07-18 for consolidation and Qwen3-Embedding-0.6B embeddings across comparisons.The embedding dimension is 1024.
- Memory Construction: L1 memories are created online from one-turn non-overlapping windows, while L2–L5 memories aggregate at session, daily, weekly, and monthly boundaries.These stages create progressively coarser temporal representations.
- Recall Configuration: TiMem’s recall uses a planner and gating call, with complexity prediction and keyword extraction setting level-specific budgets and search scope.The recall pipeline selects and filters candidate memories according to query complexity.
- Model Configuration: The model-configuration experiment varies internal consolidation and recall models and evaluates five external question-answering models with two judges.The internal configurations are GPT-4o-mini and Qwen3-32B.
- Segment Granularity: L1 segment granularity is evaluated at 1, 2, 4, and 8 dialogue turns to measure the trade-off between detail preservation and efficiency.The 1-turn setting is the finest and the 8-turn setting is the coarsest.
- Segment Granularity: As segment size increases, question-answering performance decreases, indicating a practical accuracy trade-off.The reported comparison concerns LoCoMo performance relative to the 1-turn configuration.
C.3 Sensitivity Analysis of λ
The λ sensitivity analysis evaluates the semantic–lexical balance on LoCoMo and finds consistently high performance, with a mild peak at λ = 0.9. The associated manifold tables describe progressive separation and convergence patterns across hierarchy levels.
- Sensitivity Analysis: λ balances semantic and lexical evidence in the LoCoMo sensitivity analysis.The experiment tests whether hybrid retrieval remains robust across λ values.
- Sensitivity Analysis: Performance remains consistently high across λ values, with a mild peak at λ = 0.9.The reported stability supports robustness across the tested configurations.
- Related Manifold Metrics: LoCoMo manifold metrics track progressive feature separation from L1 to L5 through increasing Silhouette Score and Separation Ratio.The table caption identifies these metrics as evidence of separation across hierarchy levels.
- Related Manifold Metrics: LongMemEval-S manifold metrics track convergence from L1 to L5 through reduced Spread and Radius95.The table caption characterizes this pattern as convergence toward unified persona templates.
D Memory Manifold Analysis
The manifold analysis shows that hierarchical consolidation changes memory geometry according to dataset characteristics. LoCoMo representations become more user-separated, while LongMemEval-S representations become less dispersed and more shared.
- Metrics: The analysis uses Intrinsic Dimensionality, Silhouette Score, Spread, and Trustworthiness to characterize hierarchical memory structure.These metrics assess dimensionality, clustering quality, dispersion, and neighborhood preservation.
- LoCoMo: LoCoMo dimensionality compresses 5.6-fold to 13 dimensions at L5, while Silhouette Score improves 6.2-fold to 0.574.Separation Ratio also rises from 0.30 to 2.14 across the hierarchy.
- LongMemEval-S: LongMemEval-S spread decreases 50% from 0.692 to 0.345, while effective radius shrinks from 0.789 to 0.444.Dimensionality remains 100 through L4 before dropping to 68 at L5.
- Interpretation: TiMem acts as a feature separator on LoCoMo and a noise filter on synthetic LongMemEval-S data.Both datasets undergo dimensionality reduction, but LoCoMo expands inter-user separation while LongMemEval-S reduces variance.
- Evaluation Prompt: The LoCoMo evaluation prompt emphasizes timestamps, direct memory evidence, recent information for contradictions, and conversion of relative times into dates.It instructs the assistant to answer only from the memories of both speakers.
- Evaluation Prompt: LongMemEval uses related memories with an explicit current date and question-answer format under its default non-CoT template.The paper notes task-specific prompts for several LongMemEval tasks and presents internal processing templates separately.
E.1 LoCoMo Benchmark
For LoCoMo evaluation, TiMem uses a Mem0-derived prompt that directs the memory assistant to retrieve accurate answers from timestamped conversation memories.
- Prompt: The LoCoMo evaluation adopts Mem0’s prompt template for question answering and evaluation.The prompt is used to guide retrieval from conversation memories.
- Prompt: The prompt frames the system as an intelligent memory assistant answering questions from memories involving two speakers.It emphasizes analyzing the provided conversational memories.
- Prompt: The prompt instructs the assistant to use timestamps and resolve relative time references into specific dates.It also prioritizes the most recent memory when memories contradict each other.