Source-linked AI summary
O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agents
Piaohong Wang, Motong Tian, Jiaxian Li, Yuan Liang, Yuqing Wang, Qianben Chen, Tiannan Wang, Zhicong Lu, Jiawei Ma, Yuchen Eleanor Jiang, Wangchunshu Zhou
TL;DR
Long-term LLM-agent interactions remain difficult because semantic memory retrieval can miss important user information and add retrieval noise. O-Mem uses active user profiling with hierarchical retrieval of persona and event context, achieving state-of-the-art personalized-memory results with lower token and latency costs. The paper reports limitations from stochastic evaluation conditions and the inability to repeat experiments for statistical summaries.
Problem
LLM agents struggle to maintain consistent context and dynamic personalization in long-term interactions, while semantic retrieval can overlook important user characteristics and situational context.
Method
O-Mem actively constructs and refines user profiles from ongoing interactions and uses hierarchical retrieval across persona and topic-related memory.
Results
O-Mem achieves state-of-the-art performance across three personalized benchmarks while reducing token consumption by 94% and inference latency by 80% versus its closest competitor.
Takeaways & Limitations
O-Mem provides an efficient framework for more personalized and context-aware long-term human-AI interactions.
Takeaways & Limitations
Evaluation conditions were stochastic and resource constraints prevented repeated experiments for statistical summaries such as mean and standard deviation.
Abstract
from arXiv · showhide
Recent advancements in LLM-powered agents have demonstrated significant potential in generating human-like responses; however, they continue to face challenges in maintaining long-term interactions within complex environments, primarily due to limitations in contextual consistency and dynamic personalization. Existing memory systems often depend on semantic grouping prior to retrieval, which can overlook semantically irrelevant yet critical user information and introduce retrieval noise. In this report, we propose the initial design of O-Mem, a novel memory framework based on active user profiling that dynamically extracts and updates user characteristics and event records from their proactive interactions with agents. O-Mem supports hierarchical retrieval of persona attributes and topic-related context, enabling more adaptive and coherent personalized responses. O-Mem achieves 51.67% on the public LoCoMo benchmark, a nearly 3% improvement upon LangMem,the previous state-of-the-art, and it achieves 62.99% on PERSONAMEM, a 3.5% improvement upon A-Mem,the previous state-of-the-art. O-Mem also boosts token and interaction response time efficiency compared to previous memory frameworks. Our work opens up promising directions for developing efficient and human-like personalized AI assistants in the future.
1 Introduction
LLM agents struggle to preserve consistent context and adapt personalization over long-term interactions. O-Mem addresses these limitations through active user profiling and hierarchical retrieval of persona and event information.
- Long-term interaction remains difficult because agents struggle to maintain contextual consistency and dynamically adapt responses to users’ changing situations.These limitations motivate memory systems that retain and retrieve past experiences for more personalized responses.
- Existing semantic-retrieval memory systems can miss semantically unrelated but important user characteristics and situational context needed for comprehensive interactions.The paper gives health conditions and recent schedules as examples of information relevant to planning activities.
- O-Mem actively extracts and updates user persona characteristics and event records from ongoing dialogs instead of only storing and grouping past interactions.The framework treats proactive user interactions as opportunities for iterative user modeling.
- O-Mem achieves 51.76% on LoCoMo and 62.99% on PERSONAMEM while improving performance across three persona-oriented tasks.The tasks are persona-based open question answering, persona-guided response selection, and persona-centric in-depth report generation.
2 Related Work
Related work presents external memory organization and retrieval as a plug-and-play route to extending LLM agents’ long-term capabilities. O-Mem is positioned as a user-centric alternative that combines characteristic identification, event recording, and topic-message indexing.
- Figure 2 contrasts O-Mem’s characteristic identification, event recording, and topic-message indexing with conventional semantic retrieval from message groupings.O-Mem also correlates virtual relationships among user interactions.
- External memory systems attract attention because they organize and retrieve information without the additional training costs required by parameter fine-tuning.The related-work discussion contrasts parameter fine-tuning with information organization and retrieval in external memory.
- O-Mem’s memory design combines episodic context retrieval, holistic persona memory, and working memory for more personalized and context-aware interactions.The described components connect topic cues, user profiles, and relevant current context.
- Personalized-agent research integrates personal data into LLM-based systems so responses can align with user-specific needs.This work motivates memory methods that help agents learn from and adapt to human preferences.
3 Method
O-Mem models user memory through persona, working, and episodic components, using active extraction, structured updates, and parallel retrieval. Its design combines long-term user profiling with topical and clue-triggered context to generate responses from aggregated memories.
- Memory components: O-Mem uses persona, working, and episodic memory to maintain long-term user knowledge, topical continuity, and clue-triggered recall.Persona memory stores abstracted user knowledge; topic interactions support working memory, while keyword-linked interactions support episodic memory.
- Memory construction: Persona memory curates attributes and events with LLM extraction and update decisions, whereas working and episodic memories use automated topic and clue indexing.The differing update mechanisms reflect persona memory’s curation needs and the efficiency of automated indexing for the other components.
- Memory construction: Each user interaction is processed by extracting a topic, user attribute, and past event, then updating interaction maps and persona facts.The event update uses an Add, Ignore, or Update operation against the existing persona fact list.
- Memory construction: O-Mem organizes recurring extracted attributes with an LLM-augmented nearest-neighbor graph and consolidates connected attribute groups into the final persona set.Attributes form graph vertices connected to nearest neighbors; connected components are analyzed by the language model to produce persona attributes.
- Memory retrieval: During retrieval, O-Mem queries relevant topics, clue-linked interactions, and persona information in parallel, concatenating the results before response generation.Working memory retrieves interactions associated with relevant topics, episodic memory retrieves interactions linked to a selected clue, and the language model generates the final response from aggregated memory.
4 Experiment
O-Mem is evaluated on three persona-oriented benchmarks against open-source and commercial memory frameworks, with comparisons spanning performance, alignment, and efficiency. It reports strongest results across the benchmarks, while evaluation conditions include model, resource, and reproducibility constraints.
- Datasets and Evaluation Metrics: LoCoMo evaluates extended dialogues across Single-hop, Multi-hop, Temporal, and Open-domain memory challenges using F1 and BLEU-1.O-Mem’s reported temporal-reasoning F1 scores are 57.48% with GPT-4.1 and 53.54% with GPT-4o-mini.
- Compared Baselines: The study compares O-Mem with open-source frameworks including A-Mem, MemoryOS, Mem0, and LangMem, plus commercial or proprietary systems including ZEP, Memos, and OpenAI.Commercial-framework results are reported from original publications, while Mem0 uses its open-source version.
- Implementation Details: Single-sample results limit statistical precision because experiments did not repeat trials, hardware conditions varied, and LLM outputs are stochastic.The authors frame the experiments primarily as evidence of feasibility and fundamental trends rather than highly precise performance benchmarks.
- Performance Comparison: 62.99% accuracy on PERSONAMEM exceeds A-Mem’s 59.42% by 3.57 percentage points, with Table 3 presenting the comparison.The benchmark covers personalized interaction scenarios across 15 diverse topics.
- Performance Comparison: 44.49% average alignment on Personalized Deep Research Bench exceeds Mem0’s 36.43%, with Table 4 reporting the benchmark comparison.The benchmark measures alignment with user characteristics and expectations in personalized deep-research scenarios.
5 Discussion
O-Mem balances performance with substantially lower computational and deployment overhead than direct raw-history retrieval and prior memory frameworks. Its efficiency is linked to parallel retrieval, compact persona representations, and attribute-based filtering, while memory-time scaling and ablations examine how its components support user understanding.
- Rethinking the Value of Memory Systems: 50.25 versus 51.67 F1 shows Direct RAG remains competitive, but its 2.6K-token cost exceeds O-Mem’s 1.5K-token cost.The comparison preserves raw interaction history for Direct RAG while highlighting O-Mem’s lower overhead.
- Rethinking the Value of Memory Systems: 30.6% lower peak memory overhead and 41.1% lower response delay make O-Mem more practical for large-scale personalized inference than Direct RAG.Peak memory falls from 33.16 MB to 22.99 MB, and latency from 4.01 seconds to 2.36 seconds.
- Efficiency Analysis: 51.67 F1 with 94% fewer tokens and 80% lower latency than LangMem establishes O-Mem’s strongest performance-efficiency comparison.O-Mem uses 1.5K versus 80K tokens and 2.4s versus 10.8s, while outperforming LangMem’s 48.72 F1.
- Efficiency Analysis: Parallel retrieval across three memory components and persona-based retrieval reduce noise, storage, LLM calls, latency, and token expenditure.O-Mem stores nearly 3 MB per user and generates each response with one LLM invocation.
- Memory-Time Scaling for User Understanding: Removing persona attributes causes performance to drop and retrieval length to increase, indicating their role in precise memory filtering.The attribute ablation is evaluated on the Personalized Deep Research Bench.
- Memory-Time Scaling for User Understanding: O-Mem’s user-understanding evaluation examines both persona-attribute extraction accuracy and the attributes’ utility for personalized responses as interactions accumulate.The analysis treats memory-time scaling as a two-part evaluation of extraction and downstream personalization.
6 Conclusion
O-Mem addresses long-term personalized interaction through dynamic user profiling and hierarchical memory retrieval. Across personalized benchmarks, it achieves state-of-the-art performance with substantially lower token consumption and inference latency.
- 6 Conclusion: O-Mem dynamically constructs and refines user profiles while using hierarchical memory retrieval to support long-term, consistent context.The framework differs from methods relying solely on semantic retrieval of past messages.
- 6 Conclusion: 94% lower token consumption and 80% lower inference latency accompany state-of-the-art performance across three personalized benchmarks.The conclusion presents these efficiency gains relative to O-Mem’s closest competitor.
7 Contributions
The contribution record assigns work across framework optimization, writing, benchmark construction, baseline experiments, and other main experiments and analyses. Two authors are identified as equal contributors.
- 7 Contributions: Piaohong Wang, Motong Tian, and Qianben Chen handled memory framework design and performance optimization.
- 7 Contributions: Piaohong Wang, Motong Tian, Tiannan Wang, Jiawei Ma, Zhicong Lu, and Qianben Chen contributed to paper writing.
- 7 Contributions: Yuan Liang, Jiaxian Li, Yuqing Wang, and Motong Tian constructed the DeepResearch benchmark and conducted its experiments.
- 7 Contributions: Motong Tian, Jiaxian Li, and Yuan Liang conducted the main baseline experiments, while other main experiments and results analysis involved Motong Tian and Piaohong Wang.
- 7 Contributions: Piaohong Wang and Motong Tian are marked as having contributed equally to the work.
8 Appendix
The appendix supplies prompts and criteria for evaluating personalization alignment in research articles. It covers goal, content, presentation, and actionability dimensions, with criteria tailored to the task and user persona.
- 8 Appendix: Goal Alignment criteria require analyzing explicit and implicit persona needs in relation to the task before evaluating the article.
- 8 Appendix: Content Alignment criteria assess whether research content matches the user’s interests, knowledge background, and preferences.
- 8 Appendix: The criteria-generation prompts require concrete explanations and weights summing exactly to 1.0.
- 8.1 Evaluation Prompt: The experiment used a temporary, unrefined scoring prompt whose additional personalized indicators were later discarded because of conceptual overlap.
- 8 Appendix: The appendix defines personalization alignment as evaluation across Goal Alignment, Content Alignment, Presentation Fit, and Actionability & Practicality.
- 8 Appendix: The scoring prompt instructs evaluators to analyze each criterion, identify strengths and weaknesses, and assign an integer score from 0 to 10.