Source-linked AI summary
MedCache: Efficient and Temporally Valid Memory for Longitudinal Clinical Agents
Hei Ting, Chan, Chenwei Wu, Xueshen Liu, Boyuan Zheng, Liyue Shen, Jiasi Chen, Z. Morley Mao
TL;DR
Longitudinal clinical agents need memory that remains valid as patient evidence changes across visits and specialties, but the best design is unclear. The paper introduces LEMMA-Bench, analyzes four memory dimensions, and proposes MedCache, which improves accuracy and memory efficiency through temporally valid, overlapping, routed memory with adaptive specialist reasoning. The benchmark findings indicate that temporal validity matters more than retaining more history, specialty views should preserve shared evidence, and multiple specialists help when distinct clinical constraints must be reconciled.
Problem
Longitudinal clinical agents must maintain an evolving patient state from distributed evidence, but effective memory design for this setting remains unclear.
Method
The paper introduces LEMMA-Bench and studies memory curation, organization, retrieval, and memory-augmented reasoning before proposing MedCache.
Results
MedCache achieves the highest accuracy among evaluated systems while maintaining high computational efficiency.
Takeaways & Limitations
Effective longitudinal memory requires a valid patient state, overlapping specialty memories, routed access, and specialist coordination only when needed.
Takeaways & Limitations
LEMMA-Bench uses 91 patients and 188 multiple-choice questions from a single retrospective MIMIC-IV cohort, so broader validation remains necessary.
Abstract
from arXiv · showhide
Longitudinal clinical agents must maintain an evolving patient state from evidence distributed across visits, time points, and specialties. However, how agent memory should be designed for this setting remains unclear. We introduce a benchmark of multi-visit, multi-specialty patient records that evaluates long-context evidence retrieval, cross-time evidence aggregation, and cross-specialty clinical reasoning. Using this benchmark, we systematically study four memory design choices: curation, organization, retrieval, and memory-augmented reasoning. We find that temporal validity is more important than simply retaining more history; specialty-factorized memory reduces context but can hide shared evidence; and multiple agents help when specialists must reason together, not merely when evidence comes from multiple memories. Guided by these findings, we propose \textit{MedCache}, a hybrid framework that constructs temporally valid patient memory, organizes evidence into overlapping specialty views, routes each query to relevant memories, and adaptively invokes one or multiple specialists. Experiments show that MedCache improves reasoning accuracy and memory efficiency over strong single-agent and multi-agent baselines, while generalizing across model backbones and external datasets.
Introduction
Longitudinal clinical agents must maintain an evolving patient state across visits, specialties, and care settings, but effective memory design remains unclear. The paper benchmarks memory designs and proposes MedCache, a hybrid framework guided by findings about temporal validity, specialty organization, and adaptive reasoning.
- Longitudinal clinical agents need reliable memory because outdated information or forgotten patient constraints can produce unsafe recommendations.
- Medical records create memory challenges because diagnoses, medications, laboratory values, and relevant evidence change over time and span departments.
- LEMMA-Bench evaluates long-context retrieval, cross-time evidence aggregation, and cross-specialty reasoning across multi-visit, multi-specialty records.
- More history does not guarantee better curation: valid patient-state memory must prevent outdated or superseded evidence from confusing agents.
- MedCache combines temporally valid memory, overlapping specialty views, routed retrieval, and adaptive single- or multi-specialist reasoning.
- Extensive experiments demonstrate improved reasoning accuracy, memory efficiency, and generalization for MedCache.
Related Work
Prior work studies long-context retrieval, persistent memory, and multi-agent medical reasoning, but MedCache specializes these ideas for longitudinal clinical care. Its novelty lies in combining temporally validated patient facts with specialty-factorized memory and routed access.
- Existing longitudinal clinical benchmarks cover extraction, temporal sorting, evidence across discharge summaries, interactive EHR actions, and diagnosis or treatment planning.
- Long-context and persistent-memory research addresses evidence selection, context tiers, salient-memory consolidation, and temporally aware knowledge graphs.
- MedCache adapts these memory ideas to longitudinal care through provenance-linked facts, temporal validation, and bounded clinical trajectories.
- Medical multi-agent systems use consultation, adaptive collaboration, semantic routing, evidence fusion, and search-based orchestration; MedCache places specialty factorization in persistent patient memory.
Medical Agent Memory Failures
LEMMA-Bench examines how memory curation, organization, retrieval, and reasoning affect longitudinal clinical-agent performance. Its analyses show that stale evidence, inaccessible cross-specialty history, and unreliable retrieval are central failure sources.
- Benchmark and research questions: LEMMA-Bench asks how SAS and MAS perform, which memory design choices explain failures, and whether those findings guide effective and efficient clinical memory.
- Benchmark and research questions: The benchmark tests long-context retrieval, cross-time aggregation, and cross-specialty reasoning using evidence distributed across longitudinal patient records.
- Existing systems: No evaluated method performs consistently best across all three capabilities; ClinicalAgents reaches .915 overall, while Med-SRAF leads cross-specialty accuracy at .967.
- Memory curation: Outdated evidence forms the largest error share for nearly every system, including 54.9% of Long-Context errors, 72.1% of Zep errors, and 79.7% of MDAgents errors.
- Memory organization and reasoning: Exclusive specialty partitions reduce retrieval and cross-time performance because relevant history may lie outside the active specialist’s memory, despite strong cross-specialty results.
- Memory retrieval: Lexical and dense retrieval recover required specialties unreliably, whereas the LLM selector and learned gate recover every required specialty and achieve .945 overall accuracy.
MedCache Design
MedCache builds a temporally valid patient state, organizes facts into overlapping specialty memories, and routes each query to relevant evidence and guideline modules. It then uses either a single reader or coordinated specialists, with answer verification for clinical decisions.
- Temporal curation and specialty organization: MedCache constructs a valid patient state from longitudinal records before organizing evidence into overlapping specialty memories.This preserves focused clinical views while retaining facts shared across domains.
- Temporal curation and specialty organization: Each clinical fact records temporal, evidential, and source metadata so the applicable state can be reconstructed at query time.Time-sensitive concepts retain bounded trajectories within clinical windows.
- Routing and execution: A learned owner gate selects relevant specialty memories, deduplicates shared facts, and ranks routed evidence while preserving complete clinical trajectories.Specialists receive reusable versioned guideline prefixes alongside dynamic patient-evidence suffixes.
- Routing and execution: Evidence-retrieval queries use a single reader, whereas cross-specialty integration queries activate patient-relevant specialists for coordinated reasoning.The framework separates memory access from the decision to invoke one or multiple specialists.
- Verification: MedCache verifies answers with deterministic guideline checks when available and model-based verification otherwise.Deterministic arithmetic or threshold checks take precedence over model-based verification.
Analysis of MedCache Results
MedCache improves the accuracy–context trade-off, remains robust to stale and off-specialty distractors, and benefits most from learned routing and validity curation. Its efficiency advantages grow as guideline coverage expands, while routing reduces context across backbones and external data.
- Overall accuracy and efficiency: 0.936 overall accuracy exceeds ClinicalAgents’ 0.915 while reducing input tokens from 31.5k to 8.7k, a 3.6× reduction.MedCache lies on the accuracy–context Pareto frontier.
- Robustness to distractors: MedCache maintains 0.708 accuracy from Clean through Heavy distractor severity, while RAG falls from 0.542 to 0.208.Its validity-window policy retains the gold fact on every question at every severity, with 1.000 retention matching the oracle.
- Component analysis: Removing the learned gate causes the largest overall performance drop, while validity curation is particularly important for cross-time questions.Specialty ownership provides smaller gains, and two cross-specialty ablation improvements cover only two questions.
- Serving efficiency: As guideline modules increase from 8 to 24, MedCache keeps prefix reuse above 0.926 and median TTFT near 0.165 s, while centralized TTFT rises to 0.716 s.Its TTFT advantage increases from 1.29× to 4.34× as coverage expands.
- Generalization: Owner-aware routing reduces context across Qwen and Mistral, but its accuracy gain depends on the backbone.On Qwen, accuracy rises from 0.904 to 0.936 while tokens fall from 11.0k to 8.7k; on Mistral, both systems reach 0.920.
- Generalization: On eICU-CRD 2.0, learned routing raises cross-owner accuracy from 0.583 to 0.858 while reading 27.5% of patient memory.Selecting every active specialty reaches similar 0.853 accuracy but reads 60.5% of memory.
Limitations
The evaluation is limited to a small, single-cohort retrospective benchmark, so broader validation remains necessary across institutions, decision formats, workflows, and clinician-reviewed outcomes.
- Scope boundaries: LEMMA-Bench contains 91 patients and 188 multiple-choice questions from a single retrospective MIMIC-IV cohort.Broader validation requires multi-institutional data, open-ended clinical decisions, prospective workflows, and clinician-reviewed outcomes.
Conclusion
The paper concludes that effective longitudinal clinical memory requires temporal validity, overlapping specialty organization, owner-aware retrieval, and selective specialist coordination. Guided by these findings, MedCache achieves the highest evaluated accuracy while maintaining high computational efficiency.
- Conclusion: Effective longitudinal memory combines a valid patient state, focused but overlapping specialty memories, appropriate-owner retrieval, and specialist coordination only when needed.These principles guide MedCache’s design.
- Conclusion: MedCache achieves the highest accuracy among evaluated systems while maintaining high computational efficiency.
Additional Results
On eICU transfer, learned owner routing preserved cross-owner accuracy while selecting fewer specialties and reading less memory. On patient-scoped note controls, full context remained more accurate, so routing’s advantage was not established after identity filtering.
- Owner-routing transfer: .856 overall and .858 on cross-owner questions, the learned gate outperformed requester-only selection at .433 and .063.It recovered every registered owner while matching the active-specialty union on cross-owner accuracy.
- Owner-routing transfer: The learned gate selected 1.53 specialties and read 27.5% of patient memory, versus 4.75 specialties and 60.5% for the active-specialty union.Central full memory reached .842 overall, below both selective configurations.
- Owner-routing transfer: BM25 reached .342 on cross-owner accuracy and recovered 70.3% of registered owners, trailing the learned gate.The comparison evaluates both routing accuracy and owner recovery.
- Patient-scoped controls: After patient-identity filtering, full-context accuracy changed from .760 to .740 on EHRNoteQA and from .825 to .820 on LongHealth when distractors were added.These controls use the same target questions and answers across clean and distractor conditions.
- Patient-scoped controls: Full context remained more accurate on these note benchmarks, so the results do not establish an accuracy advantage for specialty routing after patient-identity filtering.MedCache changed from .632 to .640 on EHRNoteQA and remained at .440 on LongHealth while using less context.
LEMMA-Bench Details
LEMMA-Bench evaluates temporally bounded longitudinal-memory questions with explicit evidence, specialty, and routing metadata. Its implementation compares shared baselines and adaptive memory systems under common outputs, guideline sources, and cost accounting.
- Cohort and evaluation: LEMMA-Bench contains 188 multiple-choice questions from 91 patients in MIMIC-IV v3.1, spanning 1,152 admissions and records averaging 5.8 years.Every patient has at least eight admissions; records range from 8 to 39 admissions and 0.8? No, only cite supplied figures.
- Cohort and evaluation: Each question is evaluated at a specified record cutoff, so only information available at or before that cutoff may be used.This defines the benchmark’s temporal-evaluation boundary.
- Question construction: Released items store patient, question, options, gold answer, cutoff, task label, required specialties, and supporting identifiers; support fields are withheld at test time.Options are shuffled after filtering and the gold index is recomputed from answer text.
- Question construction: Cross-time questions contain at least two values of one concept at different times, while cross-specialty questions span specialties and take precedence.The first two groups form 128 evidence questions, including 85 same-owner and 43 cross-owner cases.
- Memory organization: Specialty memory stores atoms for concepts owned by that specialty, preserving shared source identifiers so routed unions can deduplicate evidence without losing provenance.Updates recompute affected validity intervals, trajectories, partitions, and active-specialty metadata.
- Routing and assembly: The owner gate embeds each question with a frozen Qwen encoder and predicts a specialty; the read set unions requester and predicted-owner memories before ranking and token budgeting.The assembler preserves complete required trajectories before applying the token budget.
- Routing assumptions: Owner recall measures registry-consistent routing rather than independent clinical sufficiency, so it does not test whether the registry omitted or misassigned relevant evidence.Selecting the registered owner is operationally sufficient only if the registry is complete and correct.
- Adaptive execution: Cross-specialty questions activate relevant specialists concurrently, whereas retrieval and cross-time questions use one requesting specialist over the routed union.The coordinator receives structured specialist assessments containing decisions, evidence, uncertainty, conflicts, and source identifiers.