Source-linked AI summary
C3M: Cross-Session Multimodal Memory Maintenance for Long-Horizon Tasks
Xueshu Chen, Yan Wang, Zihao Xue, Jiefu Li, Zhenfang Liu, Jayden Chen, Zhen Bi, Jungang Lou
TL;DR
Long-horizon multimodal tasks require compact, query-blind memory that preserves distinctions and source evidence for future requests. C3M maintains a bounded active index over persistent text-image evidence, using relation-aware updates and budgeted routing to recover linked sources. In online session-by-session evaluation, it performs better overall than other memory compression methods under shared index and reader budgets.
Problem
Cross-session memory must preserve compactly the text-image evidence and distinctions that later requests may need, even when their relevance is not yet known.
Source: §1 · p. 1Method
C3M maintains a bounded active index over persistent source evidence, applies relation-aware updates, and routes queries to linked original text and images under a joint reader budget.
Source: §1 · p. 1, §2 · p. 2, §2.1 · p. 2Results
In online session-by-session evaluation across different cross-session histories, C3M performs better overall than other memory compression methods under shared index and reader budgets.
Source: §5 · p. 4Takeaways & Limitations
C3M provides a compact cross-session memory organization that preserves temporal distinctions and provenance-linked access to source evidence.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Long-horizon tasks require preserving and later recovering cross-session evidence under a bounded, query-blind memory budget. Existing compression can discard fine-grained visual cues or conflate semantically similar but incompatible observations. We present C3M, a cross-session multimodal memory organization that maintains a bounded active index over persistent source text-image evidence. Relation-aware updates consolidate safe redundancy while preserving complementary and incompatible records. At query time, budgeted routing selects useful index pages and expands their associated source evidence under a fixed reader budget. Together, these mechanisms establish a compact, provenance-preserving multimodal memory organization for cross-session long-horizon tasks, retaining temporal distinctions and source links required for reliable downstream reasoning. Code is available at https://github.com/HuzhouNLP/C3M.
1. INTRODUCTION
Cross-session multimodal memory must stay compact and query-blind while preserving distinctions and source evidence that may matter to later requests. C3M addresses this with relation-aware maintenance over a bounded index and provenance-linked evidence, alongside budgeted retrieval.
1. INTRODUCTION
- Long-horizon tasks require preserving earlier observations because later requests may depend on changed states or evidence whose relevance emerges retrospectively.Relevant evidence may be distributed across text and images, and fine-grained visual details can determine future answers.Source: §1 · p. 1
- Similar records may differ by entity, state, or time, while aggressive merging can erase distinctions and retaining near-duplicates consumes capacity and weakens routing.Provenance and source images are needed when compact summaries cannot preserve the distinctions future queries require.Source: §1 · p. 1, Figure 1 · p. 1
- Prior systems use visual-token compression, sparse memory, fixed-budget updates, gist distillation, and conflict-aware retention, but similarity alone does not make merging safe.Compression may discard visual cues, text-image bindings, source images, or temporal versions, while routing remains constrained by reader budgets.Source: §1 · p. 1
- C3M maintains a bounded active index over persistent source text-image evidence and expands evidence linked from selected pages under a fixed reader budget.The overview depicts relation-aware maintenance actions and query-time retrieval of linked Raw Nodes and original images.Source: §1 · p. 1, Figure 2 · p. 2
- C3M couples compact routing representations with persistent source evidence, aiming for more effective organization under a fixed memory budget while preserving distinctions and provenance-linked access to earlier images.The contribution describes the system as designed for cross-session long-horizon tasks.Source: §1 · p. 1
2. CROSS-SESSION MULTIMODAL MEMORY MAINTENANCE
C3M maintains a bounded active index over persistent source evidence, using relation-aware maintenance to preserve distinctions and budgeted routing to recover original evidence. Its two mechanisms reorganize access without overwriting the underlying observations.
2. CROSS-SESSION MULTIMODAL MEMORY MAINTENANCE
- C3M pairs a bounded active index with persistent source text-image evidence and applies RAMU as sessions arrive, before the final query is known.The index stores compact routing representations and pointers; the source evidence remains in the Cold Store.Source: §2 · p. 2, §2.1 · p. 2
2.1. Relation-Aware Multimodal Memory Update
- RAMU merges compatible descriptions of stable facts, updates later states of the same entity, and keeps related but non-interchangeable or uncertain evidence independently.At capacity, it consolidates only confirmed safe redundancy and otherwise uses a pointer-preserving fallback rather than a semantic merge.Source: §2.1 · p. 2
- RAMU preserves access to earlier and incoming evidence through source pointers, while merges consolidate stable-fact support and updates link successive states.Raw Nodes are never overwritten; ADD and KEEP retain pointers on independent entries.Source: §2.1 · p. 2
2.2. Budgeted Multimodal Evidence Routing
- BMER selects evidence using relevance and coverage under a joint read budget, charging accessed pages, nodes, images, and text tokens.Coverage rewards source sessions, recorded relations, and evidence clusters; selected entries resolve to associated Raw Nodes, text, and original images.Source: §2.2 · p. 3
3. EXPERIMENT
The experiments evaluate C3M through chronological replay with query-blind updates and budgeted retrieval, comparing memory organization under shared limits. Results favor C3M at shorter histories with both models, while longer histories and case studies expose the scope and behavior of those results.
3.1. Experimental Settings
- The evaluation uses chronological replay at 32K, 64K, and 128K histories, hiding future sessions, queries, answers, and question types during query-blind consolidation.The 195-question MemLens agent subset spans information extraction, multi-session reasoning, temporal reasoning, knowledge update, and answer refusal.Source: §3.1 · p. 3
3.2. Baselines and Fair Protocol
- Baselines share Raw Nodes and provenance, while active-store and reader limits are shared so maintenance and routing differ in how they organize and retrieve evidence.The protocol also accounts for CDRs graph bytes and pre-filter candidates within the same budgets.Source: §3.2 · p. 3
3.3. Main Results
- C3M leads Overall at 32K and 64K histories with both models, while CDRs leads Overall at 128K; C3M also leads GPT-5.6 Sol AR and ties on Qwen 3.8 Flash KU.Across task abilities, C3M leads or ties on different subsets for the two execution models; the case studies show correct recovery of fine-grained visual evidence where the baseline is incorrect or inconclusive.Source: §3.3 · pp. 3–4, Figure 4 · p. 4
- Fig. 3 contrasts history growth under fixed C1 limits with active-index scaling at fixed 128K history, showing higher utilization and lower outcomes as history grows and fewer overflow repairs but non-monotonic outcomes as capacity scales.Writer policy and reader budget are shared; Raw-Node pointers remain fixed in the scaling comparison.Source: Figure 3 · p. 4
- Fig. 4 illustrates three visual-routing cases: C3M identifies arched windows, water bottles on a cooler’s right, and McDonald’s above WEGO, unlike the incorrect or inconclusive baseline answers.Red boxes mark relevant evidence, with green and red text denoting C3M and baseline answers.Source: Figure 4 · p. 4
4. ABLATION STUDY
The ablations examine history pressure, active-index capacity, and multimodal evidence routing. They show that added capacity eases structural pressure but does not guarantee better task outcomes, while source-linked retrieval supports fine-grained visual answers.
4.1. Cross-Session Context Pressure
- At fixed C1 capacity, histories growing from 32K to 128K saturate index utilization as Route-Hit, Support Recall, and Answer Accuracy decline; overflow repairs rise from 0.00 to 21.44 per episode.Raw-Node pointers rise from 64.79 to 239.32 per episode, matching input Raw Nodes and indicating that index pressure limits routing organization rather than deleting source evidence.Source: §4.1 · p. 4
4.2. Active-Index Budget Scaling
- At fixed 128K history, scaling the active index cuts overflow repairs from 33.38 to 3.38 per episode, but Route-Hit, Support Recall, and Answer Accuracy peak at different scales.Route-Hit is highest at C1.75, while Overall accuracy peaks at C1.25 and C1.5; reader budget and persistent Raw-Node pointers remain fixed.Source: §4.2 · p. 4
4.3. Case Study: Multimodal Evidence Routing
- C3M retrieves source evidence to identify arched window frames, water bottles on a cooler’s right side, and McDonald’s above WEGO, while the baseline gives incorrect or inconclusive answers.Compact index entries and provenance pointers guide retrieval of Raw Nodes and original images to verify shape, location, and brand.Source: §4.3 · p. 4
5. CONCLUSION
C3M organizes cross-session memory with a bounded active index over persistent source evidence. Its relation-aware updates preserve distinctions, and budgeted routing retrieves original text and images; experiments report better overall performance than other compression methods under shared budgets.
5. CONCLUSION
- C3M maintains memory under a fixed active-index budget without access to future queries, linking compact routing entries to persistent source text-image evidence.Relation-aware maintenance preserves distinctions among redundant, evolving, and incompatible observations; budgeted routing retrieves original evidence through provenance-preserving pointers.Source: §5 · p. 4
- Under the online, session-by-session evaluation protocol across different cross-session histories, C3M performs better overall than other memory compression methods under shared index and reader budgets.Source: §5 · p. 4
- C3M integrates source evidence preservation, relation-aware updates, and budgeted retrieval for cross-session multimodal memory maintenance in long-horizon tasks.Source: §5 · p. 4
7. COMPLIANCE WITH ETHICAL STANDARDS
The study used publicly available data and did not collect new data from human participants or animals; ethical approval was not required.
7. COMPLIANCE WITH ETHICAL STANDARDS
- The study used publicly available data and did not collect new data from human participants or animals, so ethical approval was not required.Source: §7 · p. 5