On Memory Construction and Retrieval for Personalized Conversational Agents

• Turn-level retrieval can miss dispersed, request-relevant information, while session-level memories can include distracting material; all three common granularities have retrieval-accuracy and context-quality limitations.Summary-based memories can lose information during summarization.Source: §1 · pp. 2–3, Figure 1 · p. 2
• SECOM segments long conversations into topically coherent memory units and retrieves them directly as context, avoiding summarization-related information loss.Source: §1 · p. 3
• Above a 50% compression rate, LLMLingua-2 consistently improves retrieval recall across retrieval budgets K, while denoising increases similarity to relevant segments and decreases it for irrelevant segments.The reported evaluation uses BM25- and MPNet-based retrievers on Long-MT-Bench+.Source: Figure 3 · p. 3, §1 · p. 3
• SECOM outperforms baselines on LOCOMO and Long-MT-Bench+, with further analysis and ablations confirming contributions from segment-level memory and compression-based denoising.Source: §1 · p. 4

2.1 PRELIMINARY

• SeCom builds memory from conversation sessions as topical segments, retrieves N relevant units within context budget N, and passes them in chronological order to response generation.Conversation history consists of sessions of sequential user-agent turns.Source: §2.1 · p. 4

2.2 CONVERSATION SEGMENTATION

• The zero-shot segmenter assigns contiguous turn-index spans to topical segments; GPT-4 is used amid annotation scarcity, while Mistral-7B and RoBERTa-scale models can also perform segmentation.Turns are formatted with indices and user/agent role identifiers to clarify context.Source: §2.2 · p. 4
• With limited annotations, the LLM iteratively reflects on top-K sessions with the greatest WindowDiff errors against gold segmentations and updates segmentation guidance.The process selects hard examples by WindowDiff and uses the annotations to revise guidance, analogized to implicit SGD.Source: §2.2 · pp. 4–5

2.3 COMPRESSION BASED MEMORY DENOISING

• Before retrieval, LLMLingua-2 compresses memory units to remove natural-language redundancy treated as retrieval noise.The compression model is used as the denoising function fComp.Source: §2.3 · p. 5

Main Results

• SECOM outperforms all baselines; switching turn- and session-level methods from MPNet to BM25 improves GPT4Score by up to 11.98 points on LOCOMO and 7.89 on Long-MT-Bench+.Topical segments balance coherent relevant context against irrelevant content, making SECOM more robust to the retrieval system. Its advantage persists with Mistral-7B and competitive performance remains with RoBERTa-based segmentation.Source: Table 1 · p. 6, §3 · pp. 6–7, Figure 4 · p. 7

Ablation Study on Granularity of Memory Units

• Across context budgets, segment-level memory consistently outperforms turn- and session-level memory when compression-based denoising is applied to all granularities.This comparison isolates the end-to-end effect of memory granularity.Source: §3 · p. 7, Figure 5 · p. 7

Ablation Study on Compression based Memory Denoising

• Removing compression-based memory denoising causes a GPT4Score drop of up to 9.46 points on LOCOMO.The denoising mechanism improves the retrieval system and thereby enhances overall effectiveness; the ablation uses a 75% compression rate with the MPNet-based retriever.Source: §3 · pp. 7–8, Table 2 · p. 8

Mistral-7B Powered Response Generation

• With Mistral-7B-Instruct-v0.3 as response generator, SECOM significantly outperforms baselines, while Full History trails despite the model’s 32K context window.The comparison indicates that selecting relevant context and reducing noise can outperform using the entire conversation history without truncation.Source: Table 3 · p. 8, §3 · p. 8

Evaluation of Conversation Segmentation Model

• The segmentation model consistently beats baselines in the unsupervised setting and generalizes well in transfer learning despite learning its rubric from only 100 challenging source examples.In transfer learning, it surpasses the baseline trained on the full source training set and outperforms some supervised baselines; evaluation covers DialSeg711, TIAGE, and SuperDialSeg.Source: §3 · pp. 8–9, Table 4 · p. 9

4.1 MEMORY MANAGEMENT IN CONVERSATION

• Memory systems extract persona and intent from dialogue, summarize records (MPC, MemoryBank, COMEDY) or update them recursively (RecurSum, ConditionMem); RAG retrieval and LOCOMO findings expose limits of long-context and summary-only approaches.MSC uses DPR to select relevant summaries, while MemoChat uses an LLM for retrieval; LOCOMO experiments report hallucinations from long-context LLMs and information loss from summary-only memory.Source: §4.1 · p. 9

4.2 CHUNKING GRANULARITY IN RAG SYSTEM

• Chunking granularity affects RAG retrieval and response quality; semantic or LLM-based methods segment documents, but conversation systems commonly use utterance or turn boundaries, leaving conversation chunking overlooked.Semantic strategies use representation similarity to identify topic shifts, and LumberChunker uses Gemini to create semantically coherent narrative chunks.Source: §4.2 · p. 10

4.3 DENOISING IN RAG SYSTEM

• Dialogue noise can impair retrieval; LLM-summary denoising paired with retriever fine-tuning incurs latency or data costs, limits plug-and-play flexibility, and risks overfitting or catastrophic forgetting.COTED reports that redundant noise in dialogue rounds impairs conversational search; recent approaches denoise raw dialogue through prompted summaries and retriever embedding fine-tuning.Source: §4.3 · p. 10

5 CONCLUSION

• SECOM’s experiments show its effectiveness in handling long-term conversations, with ablations confirming contributions from segment-level memory units and compression-based denoising.The conclusion does not provide quantitative results or specific baseline comparisons.Source: §5 · p. 10
• SECOM constructs a segment-level memory bank and applies compression-based denoising to enhance retrieval performance.Source: §5 · p. 10
• The paper’s investigation finds limitations in turn-level and session-level memory granularities as well as summarization-based methods.Source: §5 · p. 10

A.1 DETAILS OF CONVERSATION SEGMENTATION MODEL

• The segmentation model uses GPT-4-0125 and generates all segmentation indices at once, avoiding LumberChunker’s iterative process and its unacceptable latency.Its output is specified in JSONL format for subsequent processing.Source: §A.1 · p. 15
• Segmentation guidance is built from the 100 training samples with the largest Window Diff scores, each identified as poorly segmented.The guidance combines a segmentation rubric with representative examples containing ground-truth segmentation, model predictions, and reflections on errors.Source: §A.1 · p. 15
• The model reflects on 10 mini-batches of the selected samples to generate 10 rubric items and concurrently selects one representative example per batch..Source: §A.1 · p. 15
• After learning the segmentation guidance, the authors use a few-shot segmentation prompt for subsequent segmentation.The prompt is shown in Figure 8.Source: §A.1 · p. 15
• For simplicity and fair comparison, the authors do not use a rubric for conversation segmentation in LOCOMO or Long-MT-Bench+.Source: §A.1 · p. 15

A.2 ADDITIONAL COST ANALYSIS

• Table 5 shows that the method improves performance over the baseline with only slightly higher computational overhead, and outperforms MemoChat in efficiency and effectiveness.The table compares overall costs for memory construction, memory retrieval, and response generation across methods.Source: §A.2 · p. 15

A.3 THE ANALOGY BETWEEN THE REFLECTION AUGMENTATION AND PREFIX-TUNING

• With annotated conversation data, the method iteratively optimizes a segmentation prompt as textual guidance G to transfer segmentation knowledge and align the model with human preferences.The approach is inspired by prefix-tuning and reflection mechanisms (Renze & Guven, 2024).Source: §A.3 · p. 15
• Like prefix-tuning, which prepends a learned prefix matrix P to Transformer activations without fine-tuning model parameters, the segmentation process begins with empty guidance G0 analogous to P0.The analogy concerns optimizing guidance rather than updating the language model's parameters.Source: §A.3 · pp. 16–17
• Starting from empty guidance, the model segments conversation batches zero-shot, reflects on mistakes against ground-truth segmentations, and updates guidance G through an LLM-estimated loss gradient.The update is described as analogous to SGD, with the gradient estimated implicitly by the LLM.Source: §A.3 · pp. 17–18

A.4 PROMPT FOR GPT-4 EVALUATION

• Evaluation uses MemoChat’s prompts for GPT4Score and pairwise comparison, repeating each pair with response order alternated to minimize position bias.The evaluation prompts are displayed in Figure 12.Source: §A.4 · p. 18

A.5 EVALUATION RESULTS ON THE OFFICIAL QA PAIRS OF LOCOMO

• SeCom outperformed baselines on LOCOMO’s official QA pairs, demonstrating superior effectiveness and robustness.The paper reports these results in Table 6.Source: §A.5 · p. 18

A.6 CASE STUDY

• Case studies show turn-level fragmentation, session-level irrelevant content, and summarization omissions can each prevent memory methods from retrieving information needed to answer user requests.The comparison includes RecurSum (Wang et al., 2023) and ConditionMem (Yuan et al., 2023), whose summaries can omit essential details.Source: §A.6 · pp. 18–19

A.7 DETAILS OF DATASET CONSTRUCTION

• The appendix constructs Long-MT-Bench+ from MT-Bench+ and generates LOCOMO question-answer pairs with GPT-4.MT-Bench+ (Lu et al., 2023) provides 54 human-written long-range questions; the reconstruction generates a question for each dialogue topic and merges five consecutive sessions. LOCOMO (Maharana et al., 2024) averages more than 9K tokens per sample.Source: §A.7 · p. 20

A.8 DETAILS OF RETRIEVAL PERFORMANCE MEASUREMENT

• Retrieval is measured with discounted cumulative gain, distributing relevance evenly across consecutive relevant turns and assigning zero relevance to irrelevant turns.This makes retrieval performance comparable across different granularities.Source: §A.8 · p. 20

A.9 ADDITIONAL EXPERIMENTS ON COQA AND PERSONA-CHAT

• SECOM outperforms baseline methods on CoQA and Persona-Chat, covering open-ended and multi-turn dialogue scenarios.CoQA samples were supplemented with text from 10 surrounding samples; Persona-Chat samples were aggregated with 5 surrounding samples, and subsets were used for experimentation.Source: §A.9 · pp. 20–21, Table 8 · p. 21, Table 9 · p. 21

A.10 HUMAN EVALUATION RESULTS

• Human evaluation rankings generally align with automated metrics across coherence, consistency, engagingness, humanness, and memorability.Ten annotators scored responses using the COMEDY (Chen et al., 2024) evaluation scheme.Source: §A.10 · p. 21, Table 10 · p. 22

A.11 PERFORMANCE USING SMALLER SEGMENTATION MODEL

• SECOM retains an advantage over baselines with Mistral-7B-Instruct-v0.3 segmentation and a substantial gap over granularity-based baselines with RoBERTa segmentation.The experiments replace GPT-4-Turbo with Mistral-7B-Instruct-v0.3 or a RoBERTa model fine-tuned on SuperDialseg.Source: §A.11 · p. 21, Table 11 · p. 22
• ConditionMem fails to answer satisfactorily because summarization loses information and discards useful turns; the response identifies Free Solo as the documentary that won two Academy Awards.The example presents Free Solo as a lesson in capturing authentic human stories and the technical challenges of filming in extreme environments.Source: §A.11 · p. 34, Figure 16 · p. 35