ETAMem: Decoupling and Aligning Heterogeneous Memory for Long-Term Conversational Agents

• Information entanglement arises when conversational entries mix facts, preferences, and temporal contexts, making relevant evidence difficult to isolate and increasing retrieval noise.Source: §1 · pp. 1–2
• Organizational mismatch occurs when memory structures fail to align with information types, scattering related preferences across semantic branches and hindering collective affective retrieval.Source: §1 · p. 2
• ETAMem decomposes dialogue into atomic memory units and organizes them in Event, Affect, and Time hierarchies with view-specific representations.Event captures factual experiences, Affect organizes emotions, attitudes, and preferences, and Time indexes memories through temporal hierarchies.Source: §1 · p. 2
• ETAMem incrementally maintains Event and Affect hierarchies with adaptive thresholds, while entropy-based routing allocates query-specific budgets for hierarchical beam search across views.Source: §1 · p. 2
• ETAMem outperforms strong baselines on LoCoMo and LongMemEval-S, particularly on multi-hop and temporal reasoning questions, while maintaining competitive token costs.Source: §1 · p. 2

2.1 Memory Systems for LLM Agents

• Flat systems store facts or summaries as independent units, while MemGPT [19], MemoryOS [13], and LightMem [6] differ in paging, memory layers, and offline consolidation.Retrieval in these designs often relies on general semantic matching [2] [34].Source: §2.1 · p. 2
• Existing structured and multi-structure systems do not explicitly align memory hierarchies with different conversational information types.Some combine multiple structures without explicitly decoupling heterogeneous information; others distinguish memory types without specialized hierarchies.Source: §2.1 · p. 2
• Hierarchical and graph-based approaches include TiMem [14] and xMemory [10], and A-MEM [31], Zep [22], and HingeMem [35], respectively.They organize memories by temporal, semantic, or relational associations, without explicitly aligning hierarchies to distinct conversational information types.Source: §2.1 · p. 2
• Multi-structure approaches include MemWeaver [33], H-Mem [32], MAGMA [12], and MemGAS [30], while MemoType [28] distinguishes memory types without hierarchical organization.These approaches combine complementary structures or distinguish types, but do not both decouple heterogeneous information and organize it into specialized hierarchies.Source: §2.1 · p. 2

2.2 Affect-Aware Memory and Retrieval

• Recent work uses affective information for memory retrieval and personalization: Emotional RAG [11] combines emotional and semantic relevance; REG retrieves emotionally similar examples; PAMU [25] updates memories for evolving preferences; Sentipolis [7] models emotion-memory coupling.CloneMem [9] evaluates retention and retrieval of personal experiences, preferences, and opinions.Source: §2.2 · p. 2
• Existing affect-aware approaches primarily use affect to guide retrieval, memory updates, or generation, with limited attention to organizing memories through affective associations.This gap persists despite benchmarks emphasizing affective and personalized memory.Source: §2.2 · pp. 2–3

3.1 Problem Formulation

• ETAMem models dialogue as a timestamped sequence, incrementally updates memory after each turn, and retrieves an evidence set from the final memory to support inference.Source: §3.1 · p. 3

3.2 Heterogeneous Information Parsing

• Each dialogue turn is decomposed into information-preserving atomic memory units, then labeled negative, neutral, or positive to distinguish affective from factual content.The polarity labels are assigned using the highest-scoring class from an off-the-shelf sentiment classifier.Source: §3.2 · p. 3

3.3 View-Specific Memory Structuring

• The Event view groups neutral AMUs by textual semantics, the Affect view groups positive and negative AMUs by affective semantics, and the Time view organizes them as Session–Day–Week–Month.Event and Affect hierarchies capture recurring events, entities, and interactions, or persistent preferences and attitudes; Time stores AMUs at the Session level and summarizes higher levels.Source: §3.3 · p. 3

3.4.1 Memory Maintenance.

• Incoming AMUs are inserted online; Event and Affect trees undergo session-end split and merge maintenance, while Time uses deterministic timestamp-based insertion.Event and Affect insertion uses hierarchical beam search; splits address oversized or low-cohesion leaves, and merges reduce redundant small nodes.Source: §3.4.1 · pp. 3–4

3.4.2 Adaptive Calibration.

• ETAMem recalibrates per-view tree capacity and maintenance thresholds from observed memory statistics without rebuilding existing trees.It partitions observed similarities with Otsu’s method, estimates thresholds from specified low- and high-similarity percentiles, and smooths updates with an exponential moving average.Source: §3.4.2 · p. 4

3.5 Entropy-based Retrieval

• Entropy-based routing assigns larger retrieval budgets to views whose query-to-leaf similarity distributions are more concentrated, then hierarchical beam search combines semantic similarity with BM25 lexical relevance.Retrieved AMUs are merged and deduplicated across views to form the evidence set.Source: §3.5 · p. 5

4.2 Main Results

• 84.68% accuracy on LoCoMo and 83.00% on LongMemEval-S make ETAMem the overall leader, surpassing TiMem by 12.46% and 7.96%, respectively.On LoCoMo, memory construction tokens fall by 67.92% versus TiMem and 88.51% versus A-MEM. On LongMemEval-S, ETAMem uses 63.41k tokens per session, 88.75% fewer than TiMem and 67.70% fewer than MAGMA.Source: Table 2 · p. 6, Table 3 · p. 6, §4.2 · p. 6

4.3.1 Key Components.

• Removing atomic memory units lowers accuracy to 82.14% on LoCoMo and 80.60% on LongMemEval-S; removing hierarchy yields 82.86% and 80.60%, respectively.Disabling fallback causes a larger accuracy drop on LongMemEval-S than on LoCoMo; fallback can recover details from original conversations when retrieved memories are insufficient.Source: Table 4 · p. 6, §4.3.1 · pp. 6–7

4.3.2 Effect of View-Specific Memory Organization.

• Unified single-view strategies underperform ETAMem overall, and shared-representation variants reduce overall accuracy by 7.60 and 6.00 percentage points, respectively.Unified Semantic is the strongest unified strategy on LoCoMo, while Unified Affective leads on LongMemEval-S; Unified Affective matches ETAMem on SSP and Unified Temporal on TR. Shared Text uses textual embeddings for both Event and Affect views, while Shared Affect uses affective embeddings for both; the Time view remains unchanged.Source: Table 5 · p. 7, §4.3.2 · p. 7, Table 6 · p. 7

4.3.4 Effect of Memory View Configurations.

• With Qwen3-30B, Time-only is the strongest single-view configuration, while ETAMem reaches 80.84% on LoCoMo and 80.40% on LongMemEval-S, beating Mean Routing by 1.62 and 5.40 points.Time + Affect achieves 79.35% on LoCoMo, and Event + Time reaches 74.80% on LongMemEval-S; ETAMem's adaptive routing improves over Mean Routing's 79.22% and 75.00% on the respective benchmarks.Source: Table 7 · p. 7, §4.3.4 · p. 7, §4.3.5 · p. 7

4.4.1 Affect Analysis.

• On CloneMem, ETAMem achieves 92.59% choice accuracy, 0.9200 memory helpfulness, and 0.8031 recall, improving by 4.85 percentage points, 18.45%, and 23.71%, respectively.Routing weights are slightly higher for Affect on Emotion questions, favor Event for Experience, and favor Event and Time for Opinion, indicating complementary view roles.Source: Table 8 · p. 8, §4.4.1 · p. 8, Figure 3 · p. 8

4.4.2 Hyperparameter Sensitivity.

• Figure 4 shows best performance at β=0.1, K=20, and B=10, while adding lexical matching consistently improves performance.The experiments use Qwen3-30B as the backbone.Source: Figure 4 · p. 8, §4.4.2 · p. 8

4.4.3 Case Study.

• In a LongMemEval-S knowledge-update case, ETAMem combines Affect, Event, and Time evidence to identify the user's current BBQ sauce preference.Affect receives the highest routing weight because the answer depends on preference; Event retrieves related cooking experiences and sauce information, while Time locates the relevant update.Source: Table 9 · p. 8, §4.4.3 · p. 8

5 Conclusion

• 12.46% and 7.96% relative improvements over the strongest baselines were achieved on two long-term memory benchmarks, with strong multi-hop performance and competitive token efficiency.Further analysis highlights the complementary benefits of different memory views and the effectiveness of adaptive routing.Source: §5 · p. 8
• ETAMem combines heterogeneous information parsing, view-specific memory structuring, entropy-guided retrieval, dynamic maintenance, and adaptive calibration.Its multi-view framework decouples heterogeneous conversational information and aligns memory representations and organization with distinct characteristics.Source: §5 · p. 8
• Affective representations based on sentiment polarity may miss neutral expressions of preferences or attitudes; future work will add explicit affective relevance classification.This limitation qualifies the method's identification of affective information.Source: §5 · p. 8

LLM Calls

• Memory construction requires O(T + J + U/δ) LLM calls, with one call per turn for AMU extraction and none for leaf summaries built directly from AMU texts.Here J counts initial internal-node summary generations, and U counts pending-change counter increments; summaries are regenerated when a counter reaches δ.Source: A Complexity Analysis · p. 10

Memory Management

• Event and Affect insertion costs O(A D b_max(R + BH)) overall, while Time-view temporal indexing costs O(A).Event and Affect beam-search insertion costs O(D b_max(R + BH)) per AMU.Source: A Complexity Analysis · p. 10
• Event and Affect maintenance checks cost O(m^2D) per leaf for cohesion, add O(IkmD) when splitting, and cost O(N^2D) for worst-case pairwise merge similarities per maintenance step.The pre-split leaf size m may exceed M_max; split cost uses k clusters and I iterations.Source: A Complexity Analysis · p. 10

Retrieval

• Retrieval costs O(D[L + b_max(R + BH)]), combining O(LD) view routing with beam search and excluding BM25 scoring, encoder inference, and LLM computation.The analysis omits the constant factor for the three views.Source: A Complexity Analysis · p. 10

B.1 Dataset Details

• LoCoMo [16] contains 10 conversation groups and 1,540 questions spanning single-hop retrieval, multi-hop reasoning, open-domain queries, and temporal reasoning.The benchmark uses multi-session dialogues with temporal information.Source: §B.1 · p. 10
• LongMemEval-S [29] contains 500 conversations and 500 questions covering knowledge updates, multi-session reasoning, single-session interactions, temporal reasoning, and selective response generation.Source: §B.1 · p. 10
• CloneMem [9] uses diaries, social media posts, and emails spanning one to three years to evaluate Experience, Emotion, and Opinion.It is a long-term personal memory benchmark constructed from non-conversational digital traces.Source: §B.1 · p. 10

B.2 Evaluation Protocol

• Baseline results in Tables 2 and 3 come from TiMem [14], except LightMem and MAGMA, which are evaluated under the same settings.The paper states that these shared settings support fair comparison.Source: §B.2 · p. 10
• The protocol sets retrieval budget K=5, uses GPT-4o-mini as the backbone LLM, and uses textembedding-3-small as the embedding model.These settings follow the CloneMemBench evaluation protocol.Source: §B.2 · p. 10
• ETAMem is compared with A-MEM [31], Mem0 [2], and Flat Retrieval, using results from CloneMem [9].Source: §B.2 · p. 10
• Choice Accuracy measures the percentage of correctly answered multiple-choice questions.Source: §B.2 · p. 10
• Memory Helpfulness evaluates how useful retrieved evidence is for answering questions.Source: §B.2 · p. 10
• Memory Recall measures semantic coverage of ground-truth memory information, while Memory Helpfulness and Memory Recall are reported on a normalized 0-to-1 scale.Source: §B.2 · p. 10

C.1 Human Evaluation of Affect Classification

• Two annotators labeled 400 AMUs independently, achieving Cohen’s kappa of 0.651 before resolving disagreements into gold-standard labels.The AMUs were sampled from LoCoMo and LongMemEval-S, and annotators followed shared guidelines.Source: §C.1 · p. 10

C.2.1 Memory Structure Visualization.

• Event and Affect views form semantic trees with different branching patterns, whereas the Time view is a shallow forest with multiple roots.The structures are visualized using LoCoMo memory.Source: §C.2.1 · p. 10

C.2 Memory Visualization

• Across the visualizations, the memory views differ in both hierarchical organization and representation patterns.Event and Affect have distinct tree structures, and their LongMemEval representations show different clustering patterns; Time uses a shallow forest structure.Source: §C.2.1 · p. 10, §C.2.2 · pp. 10–11

C.2.2 Representation Visualization.

• The LongMemEval representation visualization uses t-SNE and excludes the temporal view because it is organized by temporal hierarchies rather than representation similarity.Figure 6 compares raw conversation, event, and affect representations.Source: §C.2.2 · p. 10, Figure 6 · p. 11

C.1 Human Evaluation of Affect Classification

• RoBERTa-Sentiment scores 80.53% F1 versus GPT-4o-mini’s 79.39% on the human-annotated affect benchmark.RoBERTa-Sentiment has higher precision (80.79% versus 73.60%) but lower recall (80.26% versus 86.18%); it avoids additional LLM inference during memory construction. GPT-4o-mini supplies binary affective/non-affective labels.Source: §C.1 · p. 10, Table 13 · p. 11

C.2.2 Representation Visualization.

• Affect representations cluster by polarity, while event-related memories have more concentrated distributions.The visualizations indicate that information mixed in dialogue representations becomes more coherently clustered within each memory view.Source: §C.2.2 · pp. 10–11

C.3 Error Analysis

• Table 14 reports the error distribution by evidence retrieval status.Source: Table 14 · p. 11

C.3.1 Representative Error Cases.

• In a partial retrieval case, ETAMem misses the May 25 dialogue containing the third fruit, causing an incorrect count.Source: §C.3.1 · p. 11
• In a retrieval miss case, related memories omit the recommendation Becoming Nicole, and fallback retrieval also fails to recover it.Source: §C.3.1 · p. 11

C.3.2 Error Distribution.

• 14.8% of all LongMemEval-S questions involve incorrect answers despite successful evidence retrieval, while 60.4% of LoCoMo errors involve missing or incomplete evidence.The passages associate the former with evidence aggregation and reasoning limitations, and the latter with retrieval coverage.Source: §C.3.2 · pp. 11–12

D Prompt Templates

• LoCoMo AMU extraction prompts require one atomic fact per line, third-person wording with names replacing pronouns, and context for states when inferable.They also attribute opinions and general statements to the speaker and exclude chit-chat and filler.Source: D Prompt Templates · p. 12
• LongMemEval-S AMU extraction prompts summarize hypothetical task, puzzle, riddle, or Q&A utterances in one line; otherwise, they extract personal atomic facts about the speaker.The prompts require third-person facts with subjects, attribute beliefs to the speaker, include inferable context, and skip filler.Source: D Prompt Templates · p. 12
• The memory summarization prompt requests a direct summary of the supplied contents in no more than 256 words.It provides numbered content placeholders and a Summary field.Source: D Prompt Templates · p. 12
• LoCoMo answer-generation prompts direct the assistant to reason over retrieved facts, use timestamps for temporal questions, prefer newer contradictions, and return fewer than 5–8 words.They also instruct the assistant to convert relative time references to dates, avoid confusing mentioned character names with users, and answer naturally rather than with timestamp formats.Source: D Prompt Templates · p. 12
• LongMemEval-S answer-generation prompts require answers grounded in relevant atomic facts and “no information” when the answer cannot be inferred.The prompt also supplies the question date as the current date.Source: D Prompt Templates · p. 12
• The answer-evaluation prompt labels generated answers CORRECT or WRONG based on whether they express the same meaning or topic as the gold answer.It requests JSON-only output containing the label.Source: D Prompt Templates · p. 12