PaperlayerView exact revision on arXiv

Source-linked AI summary

Fortunate Recall: Ontology-Driven Memory Lifecycle Management for Persistent Coherence in LLMs

Ansuman Mullick, Eray Tüzün

arXiv:2609.10413v1cs.AI

TL;DR

Existing LLM memory systems need lifecycle management that distinguishes how personal facts persist, change, and expire. Fortunate Recall adds a behavioral ontology and deterministic lifecycle policies to LLM-extracted memory metadata. FR-Bank improves lifecycle and end-to-end outcomes across benchmarks, while ablations separate generic metadata’s correctness advantage from the ontology’s calibration benefit.

  • Problem

    Existing memory systems treat personal facts uniformly even though their appropriate persistence, replacement, and validity dynamics differ.

  • Method

    Fortunate Recall classifies facts into 10+1 behavioral categories and applies deterministic decay, supersession, event-time validity, and category-aware routing policies.

  • Results

    76.9% LifecycleBench pass rate and reduced downstream confabulation from Mem0’s 45.1% to 22.4% over answered queries were reported for FR-Bank.

  • Takeaways & Limitations

    Generic lifecycle metadata carries the correctness advantage, while the behavioral ontology enables per-category calibration and reduces confabulation.

  • Takeaways & Limitations

    Absolute rates are judge-relative because human validation is absent, and benchmark–method co-design risk is only partially bounded by BEAM transfer.

Abstract

from arXiv · show

Current LLM memory systems treat all personal facts identically, so stores grow without bound while retrieval precision degrades. The core challenge is lifecycle management: which memories should persist, which should be replaced, and at what rate, conditioned on the behavioral type of each fact. Fortunate Recall (FR) is a composable policy layer that classifies personal facts into a 10+1 behavioral ontology and applies category-specific lifecycle policies (differential temporal decay, slot-key supersession, event-time validity, and category-aware retrieval routing) as deterministic functions over LLM-extracted metadata. FR-Bank, our infrastructure-independent implementation, reaches a 76.9% pass rate on LifecycleBench, a new 516-question temporal-disambiguation benchmark, ahead of Mem0, A-MEM, Memory-R1, and MemoryOS (61% to 70.5%), and 75.2% on the full LongMemEval-S under the canonical Wu et al. judge protocol, so lifecycle policies impose no measurable cost on standard retrieval. A pre-registered ablation locates the gains: replacing the typed layer with three generic lifecycle primitives leaves correctness statistically unchanged (-1.7pp, 95% CI [-6.0, +2.7]), so the generic lifecycle metadata carries the correctness advantage, while the behavioral ontology carries calibration, halving downstream confabulation (12.0% vs 24.2%, p<0.001). End-to-end, FR-Bank cuts confabulation from Mem0's 45.1% to 22.4% over answered queries and from 32.2% to 13.0% over all queries while answering more of them correctly (31.2% vs 18.6%); the ranking replicates on the open-weight Kimi K2.5. The decomposition transfers to BEAM, an independently built benchmark: 46.8% correct vs Mem0's 32.9% over 280 questions, with the ontology's benefit concentrated in contradiction resolution and saturating near seven policy clusters. The ontology, benchmark, and code are released.

1 Introduction

Fortunate Recall frames persistent LLM memory as a lifecycle-management problem: personal facts require different persistence, replacement, and temporal-validity policies. It introduces a behavioral ontology and deterministic category-specific lifecycle layer to address that gap.

  • Existing memory systems treat personal facts uniformly despite differing persistence dynamics, making lifecycle management the central challenge.
  • Fortunate Recall classifies extracted facts into 10+1 behavioral categories that determine decay, supersession, validity, and retrieval routing.
  • FR applies lifecycle policies as deterministic functions, with every retrieval decision represented by a named term in one closed-form log-score.
  • The paper contributes LifecycleBench, FR-Bank, structural metadata results, and attribution studies separating generic lifecycle metadata from behavioral ontology effects.

2 Related Work and Gap Analysis

Prior memory systems provide infrastructure and organizational strategies but generally lack behavior-specific lifecycle semantics. Fortunate Recall positions LifecycleBench as a dense lifecycle stress test and evaluates transfer on an independently built benchmark.

  • Flat stores, dynamic organizers, cognitive typologies, and RL-based operations leave gaps in structured lifecycle handling, including expiry and retraction.
  • Existing long-horizon benchmarks do not consistently test whether systems distinguish current from outdated state, motivating a focused lifecycle evaluation.
  • LifecycleBench is presented as a dense stress test in which every question depends on lifecycle state, while BEAM provides an external transfer setting.
  • Fortunate Recall’s architecture combines asynchronous LLM-based metadata extraction with hybrid retrieval and deterministic lifecycle scoring.
  • The behavioral ontology differs from cognitive typologies by grouping facts according to temporal behavior rather than memory form.

3 Fortunate Recall

Fortunate Recall attaches behavioral lifecycle metadata to facts and applies category-conditioned policies for persistence, replacement, validity, and retrieval. Its design separates core lifecycle mechanisms from implementation engineering and uses LLMs for extraction while keeping lifecycle decisions deterministic.

  • 3.1 The Behavioral Ontology: Each fact receives a behavioral category whose lifecycle governs decay, supersession semantics, expiry logic, and retrieval routing.
  • 3.1 The Behavioral Ontology: The ontology uses soft category membership, with effective decay determined by a weighted harmonic mean of category-specific rates.
  • 3.2 Classify the Fact, Not the Entity: Classification targets relational facts rather than utterances or entities so each stored edge receives the policy appropriate to what it represents.
  • 3.3 Lifecycle Mechanisms: Lifecycle mechanisms include differential decay, slot-key supersession, event-time validity, anticipatory activation, and category-aware retrieval routing.
  • 3.3 Lifecycle Mechanisms: The implementation separates theoretically grounded core mechanisms from independently ablated engineering components.
  • LLMs perform extraction and candidate distillation, whereas decay, filtering, routing, and scoring execute as deterministic lifecycle functions.

4 Theoretical Foundations

The theoretical foundation identifies lifecycle metadata as necessary for lifecycle-aware ranking and characterizes deterministic scoring and calibration feasibility. The results distinguish representation-level guarantees from contingent empirical claims and do not establish that the ontology or pipeline is optimal.

  • FR represents each stored edge with behavioral category, slot key, lifecycle state, event-time anchor, and creation time.
  • The active-relevance model combines semantic similarity, category priors, survival, event-time validity, supersession survival, and query-conditioned state compatibility.
  • Taking logarithms yields a deterministic lifecycle score whose terms correspond to the model’s lifecycle factors.
  • Theorem 1 shows that scorers restricted to semantic similarity and age can incur binary error at least 1/2 when hidden lifecycle structure changes the correct ranking.
  • The metadata basis is sufficient, minimal, and individually necessary: removing any component conflates distinct lifecycle situations.
  • The feasible parameter region is convex, while a single global blending parameter is empirically infeasible across nearly all preservation–suppression pairs.
  • Stale facts materially affect outcomes, with empirical pass rates of 75.8% for clean contexts and 6.0% for stale contexts.
  • The theory does not establish that the 10+1 partition, decay rates, or pipeline are optimal, and cross-system numerical claims remain empirical and contingent.

5 LifecycleBench

LifecycleBench evaluates whether memory systems answer questions according to lifecycle state rather than merely retrieving stored facts. It covers 516 questions across 40 synthetic personas and nine temporal failure modes.

  • LifecycleBench contains 516 questions over 40 synthetic personas, each spanning approximately 35 multi-session conversations across 18 simulated months.Structured YAML ground truth records supersession events, expiry dates, retraction language, and ambiguity-resolution rules.
  • The benchmark tests nine attack vectors, including superseded preferences, expired logistics, buried identity, multi-version facts, contradictions, retractions, and numeric preservation.The attack vectors were defined from preliminary failures observed in Mem0 and Zep/Graphiti before Fortunate Recall mechanisms were designed.
  • Personas span 14 nationalities and ages 22–68, with conversations generated under rules enforcing natural dialogue and controlled temporal ground truth.Synthetic conversations permit evaluation because temporal state is otherwise unobservable without controlled design.

6 Experiments

Experiments show that Fortunate Recall improves lifecycle-sensitive retrieval and response safety, while its generic lifecycle metadata drives correctness and the behavioral ontology primarily improves calibration. Results transfer to an independently built benchmark, but robustness depends on metadata and judge quality.

  • LongMemEval-S: 75.2%±0.70pp pass@10 on full LongMemEval-S under the exact Wu et al. judge protocol, while the matched subset is aggregate-neutral at +2.2pp.The reported knowledge-update decrease is attributed to a judge criterion that rewards responses containing superseded facts alongside updated answers.
  • LifecycleBench Results: 76.9% pass rate on LifecycleBench is 16pp above Mem0, with 1.76× lower staleness.When the correct edge appears anywhere in the top-10, 94.5% of questions pass, indicating retrieval recall remains the main bottleneck.
  • End-to-End Response Quality: On AV7, FR-Bank achieves 37.5% end-to-end correctness despite only 5% retrieval pass, whereas MemoryOS reaches 57% retrieval pass but 7.5% correctness.Retracted facts are excluded by the lifecycle state mask, enabling the downstream model to abstain rather than confabulate.
  • End-to-End Response Quality: FR-Bank reaches 73.3% safe response rate versus Mem0’s 47.3% because lifecycle filtering supports correct abstention instead of confabulation.FR-Bank abstains at 42.1% versus Mem0’s 28.7%; safe response combines correct answers and abstentions.
  • Attribution: Correctness is statistically unchanged after replacing the typed layer with generic lifecycle metadata: Δ = −1.7pp, 95% CI [−6.0, +2.7].The typed layer instead reduces all-queries confabulation from 24.2% to 12.0%, while abstention rises 15.9pp and correct-over-total remains unchanged.
  • External Transfer: BEAM: On BEAM, FR-full is correct on 131 of 280 questions (46.8%) versus 92 (32.9%) for Mem0, with the ontology’s gain concentrated in contradiction resolution.Contradiction resolution is 29/70 for FR-full versus 19/70 for the collapsed configuration, with McNemar exact p≈0.006; the decomposition replicates across runs.
  • Robustness and Validation: Lifecycle robustness is bounded by LLM-generated metadata quality, with slot-key merge/split corruption causing Jaccard@10 to fall to 0.764/0.837 at 10% corruption.Retraction extraction is especially constrained: the detector fires on 19 of 40 scripted retractions, and only 2 of 40 are cleanly suppressed at metadata level.

7 Discussion and Conclusion

Fortunate Recall’s reported gains are paired with explicit limits: judge-relative evaluation, benchmark–method co-design, fairness constraints, and unresolved privacy and safety questions. The conclusion attributes correctness primarily to generic lifecycle metadata while assigning calibration and confabulation reduction to the behavioral ontology.

  • Limitations: Absolute performance rates remain judge-relative because all verdicts come from LLM judges, with human validation planned but not yet conducted.Two machine instruments agree on the direction of judge bias but not its magnitude.
  • Limitations: LifecycleBench’s benchmark–method co-design creates risk, although transfer to independently built BEAM bounds that risk without eliminating it.BEAM uses a different backbone, so its runs are not commensurable with the LifecycleBench tables.
  • Limitations: Privacy, consent, verifiable deletion, prompt-injection defenses, and cultural generalization of decay assumptions remain outside the work’s addressed scope.These concerns are discussed in Appendix AC.1.
  • Conclusion: The conclusion attributes the correctness advantage to generic lifecycle metadata, while the behavioral ontology enables per-category calibration and halves confabulation.The ontology also adds a localized correctness gain for contradiction resolution, with granularity benefits saturating near seven policy clusters.
  • Conclusion: Two implementations achieve indistinguishable LifecycleBench pass rates while reducing confabulation from Mem0’s 45.1% to 22.4% over answered queries and from 32.2% to 13.0% over all queries.The systems also answer more questions correctly in the process.

Supplementary Materials

Fortunate Recall’s formal and empirical analyses separate lifecycle metadata from ontology granularity: the metadata basis is structurally necessary, while behavioral categories mainly improve calibration and policy alignment.

  • Structural results: When temporal intent is opaque relative to the score margin, feature-restricted scorers produce identical rankings across intents and incur error probability at least 1/2.The deterministic result is stronger than the later stochastic entropy framing.
  • Regime analysis: Long-term activation collapse makes the scalar scoring family infeasible, whereas FR bypasses the regime through supersession filtering, event-time expiry, and category-forced retrieval.For Δt > 2000 hr, activations are empirically 3.3 × 10−10 at the median and semantic similarity alone cannot satisfy preservation and suppression constraints.
  • Structural results: The metadata basis (c, κ, ξ, h) is minimal: every strict subset permits a lifecycle task with deterministic error at least 1/2.A scorer using the full basis can attain error 0 under the stated model.
  • Partition analysis: The behavioral partition reduces within-cluster variance to 7.7% of the uniform baseline, compared with 44.6% for the cognitive partition.The measured relative gap is 36.9% with bootstrap 95% CI [36.0%, 37.6%].
  • Partition analysis: The cognitive partition’s weakness is structural: it merges Obligations and Logistical Context despite their maximally divergent lifecycle-policy floors.The optimal K=3 partition is 4.2× better than the cognitive K=3 partition.
  • Empirical decomposition: The full lifecycle stack contributes +12pp over uniform, while the behavioral ontology contributes a further 1.16–1.36pp of calibration precision.The ontology supplies the category-conditioned parameters through which lifecycle mechanisms are applied.

A.12 Staleness Decomposition: Derivation and Numerical Predictions

The staleness decomposition separates contamination effects from clean-context recall and formalizes why event-time validity requires more than age-based decay. These results support lifecycle-aware metadata as a necessary basis for representing supersession, retraction, and anticipatory activation.

  • Staleness decomposition: The staleness model expresses pass rate as a mixture of clean- and contaminated-context performance weighted by each system’s stale-context probability.The identity is P_S(pass) = (1 − π_S)p_0 + π_Sp_1.
  • Numerical prediction: A 13.3pp predicted pass advantage from contamination reduction closely matches the observed 12pp retrieval gap between FR-Graphiti and Mem0.The numerical example uses p_0 = 0.758, p_1 = 0.060, and contamination rates π_A = 0.08 versus π_B = 0.27.
  • Confabulation: The analogous confabulation decomposition links FR’s lower stale-context exposure to a 22.4% versus 45.1% confabulation gap against Mem0.The relationship is expressed through contaminated-context probability and conditional confabulation rates.
  • Model limitation: FR-Bank and Memory-R1 have nearly identical staleness rates but differ by 10.0pp in pass rate, falsifying staleness alone as a sufficient predictor.FR-Bank’s 76.9% pass rate reflects both low staleness and higher clean-context recall.
  • Event-time validity: No monotone nonincreasing function of edge age alone can represent anticipatory activation before an event.The contradiction arises because relevance must increase as an event approaches from the left, whereas age-only decay decreases with time.
  • Event-time validity: FR’s event-time kernel increases relevance before an event and decreases it after expiry, a shape age-only decay cannot reproduce.The kernel uses separate pre-event and post-event rates, with μ_c ≫ ν_c.
  • Theoretical interpretation: The theoretical results identify (c, κ, ξ, h) as a sufficient, minimal, and individually necessary metadata basis for lifecycle-aware scoring.The claims concern the representation basis, not optimality of the ontology, rates, or pipeline.

B.1.1 Classifier Agreement Audit

The classifier audit evaluates behavioral categorization under contextual, held-out judging and finds moderate overall agreement, with stronger reliability for categories whose lifecycle policies differ most. Perturbation results indicate that the main observed category confusions have limited retrieval impact, while context remains important for classification.

  • Audit setup: 70.7% majority-vote agreement with κ = +0.673 was obtained in the held-out classifier audit.The audit used 200 sampled facts and three independent judges with full session context.
  • Per-category reliability: The five categories with the sharpest lifecycle-policy differences all exceeded F1 = 0.65.Reported F1 values include Relational_Bonds 0.853, Preferences_Habits 0.791, Health_Wellbeing 0.750, Financial_Material 0.667, and Hobbies_Recreation 0.667.
  • Context sensitivity: Removing conversational context reduced agreement from κ = +0.67 to κ = +0.58, indicating that behavioral classification depends on session context.The comparison used a decontextualized fact-only audit.
  • Overall performance: Macro-averaged F1 was 0.595, weighted F1 was 0.647, and overall accuracy was 66.5% across the 11-category label set.The reported effective sample size was n_eff = 188 after majority-vote filtering.
  • Error interpretation: The audit identifies deliberate routing choices and possible Intellectual Interests over-assignment as distinct from ordinary classification errors.Obligations versus Logistical Context reflects event-time routing, while Projects & Endeavors versus Hobbies & Recreation reflects the deliverable-test override.
  • Downstream sensitivity: At 10% label perturbation, the original and perturbed top-10 retrieval sets had mean Jaccard overlap of 0.974 ± 0.002.The perturbation targeted Obligations–Logistical Context and Intellectual Interests–Projects & Endeavors across 516 LifecycleBench questions.
  • Operational implications: FR’s human-readable categories and policies make memory inspectable, auditable, and modifiable for users and developers.The system supports viewing, correcting, and adjusting retention by category.

C.6 Benchmark Independence Analysis

The independence analysis tests whether LifecycleBench favors FR through attack-vector coverage, cross-benchmark behavior, and ontology discrimination. The evidence includes cases where other systems lead and an explicit acknowledgment that co-design risk cannot be fully eliminated without independent replication.

  • Attack-vector coverage: LifecycleBench covers replacement, expiry, retraction, partial updates, buried facts, cross-edge aggregation, contradictions, and other temporal-state failure modes.The benchmark’s nine attack vectors are presented as an exhaustive set for persistent personal memory rather than FR-specific capabilities.
  • Competitive gaps: MemoryOS leads AV6 at 73% versus FR-Bank’s 64%, while FR-Graphiti leads AV2 and AV8.FR-Bank also achieves only 5% retrieval pass on AV7 and near-zero end-to-end correctness on AV5 and AV6.
  • Cross-benchmark validation: LongMemEval-S cross-validation shows lifecycle gains concentrate where temporal-state mechanisms apply, while aggregate effects are approximately neutral elsewhere.The matched subset reports a +2.2pp net contribution, with losses where LongMemEval still treats removed facts as retrievable.
  • Ontology discrimination: The behavioral ontology outperforms the cognitive partition on 7 of 9 attack vectors under the same pipeline.This comparison is used to assess ontology discrimination rather than system identity alone.
  • Scope of independence claim: The authors state that co-design risk cannot be fully eliminated without independent replication and release the benchmark artifacts for such evaluation.Released materials include personas, conversations, questions, ground truth, and evaluation code.

D.1 LongMemEval Detailed Results

FR-Bank’s primary LongMemEval-S evaluation reaches 75.2% under the canonical protocol, with negligible retrieval cost from lifecycle policies. Detailed analyses attribute gains to lifecycle-sensitive cases, while also exposing task categories and failure modes where lifecycle filtering or extraction remains limiting.

  • Primary result: 75.2% mean pass@10 was achieved on the 500-question LongMemEval-S benchmark across 10 identical reruns.All runs were at or above the 74.2% locked baseline, with a maximum of 76.6%.
  • Protocol comparison: 75.2% is reported as the peer-reviewed LongMemEval-S comparison for this work under a fixed canonical protocol.The authors distinguish these results from industry reports lacking sufficient protocol detail for direct comparison.
  • Retrieval cost: No decay engine achieved a significant advantage on the Graphiti-based evaluation, with all effect sizes negligible at |d| < 0.19.The result is interpreted as zero measurable retrieval cost for behavioral lifecycle policies.
  • Lifecycle decomposition: Lifecycle contributes +2.2pp net on the 317-question matched subset, increasing performance from 72.9% to 75.1%.The contribution concentrates on knowledge-update and multi-session questions requiring temporal-state management.
  • Ablation: FR-Graphiti’s lifecycle ablation improved the full system over baseline by +12.2pp, with a 95% CI of [8.3, 16.1] and McNemar p < 0.001.Removing routing or replacing behavioral decay also produced statistically significant losses.
  • Lifecycle-sensitive vectors: The largest FR-Graphiti gains occur on superseded preferences, expired logistics, multi-version facts, and selective forgetting.The reported improvements are +21pp on AV1, +20pp on AV2, +22pp on AV4, and +10pp on AV7.
  • Context efficiency: FR achieves 3× lower stale-token rate and 8× lower stale-fact-at-rank-1 rate than Mem0.These are context-efficiency metrics summarized in Table 27.
  • Runtime scaling: The deterministic lifecycle layer costs 47 μs at the median for production k = 20 and scales approximately linearly with candidate count.At k = 1000, the median cost is 48.5× the k = 20 median, versus a theoretical 50×.

I Rate Calibration: The Squatting Phenomenon

FR’s initial decay-rate spread caused Identity facts to dominate retrieval regardless of relevance. Calibrating category-specific rates and validating feasible intervals addressed this squatting phenomenon.

  • The Squatting Phenomenon: 64% of top-5 result slots were occupied by Identity facts under the original 800× rate spread.Identity therefore dominated ranking positions regardless of semantic relevance.
  • The Squatting Phenomenon: The cognitive ontology avoided squatting because its slowest rate was sufficiently high at multi-month timescales.FR reproduces this rate-floor effect through explicit calibration rather than accidental spacing.
  • Rate Calibration: Ten of eleven categories tolerated rate multipliers while preserving mean Jaccard@10 ≥0.95 in the feasibility analysis.The sweep varied each category independently while holding the others at default values.
  • Calibration Process: FR’s development phases expanded persona coverage, added normalization and filtering, and introduced extraction and expiry improvements.The progression included scaling from 8 to 40 personas and successive lifecycle mechanisms.

K Detailed Findings from LifecycleBench Evaluation

LifecycleBench findings isolate distinct failure modes addressed by FR’s filters and category-aware policies. The results show gains from expiry, retraction, decay, semantic floors, and improved extraction.

  • Temporal and Retraction Handling: Backward-looking query detection recovered 3 otherwise unreachable questions by bypassing the supersession filter.The detector targets phrases such as “before switching” and “used to.”
  • Temporal and Retraction Handling: Category-guarded date-aware expiry parsing handles ISO dates, month-day patterns, contextual months, and relative markers with a 14-day buffer.Expiry filtering is applied only to Logistical, Obligations, and Health categories.
  • Category-Specific Policies: Category-specific decay improved AV8 over uniform α=0.3, which produced 12.5% performance.The reported category rates were 0.05 for Financial and 0.40 for Logistical.
  • Category-Specific Policies: Semantic floors raised Hit@1 from 19% to 31% by preventing stable-category matches from being buried by zero activation.The reported floors were 0.97 for Financial and 0.00 for Logistical.
  • Failure Decomposition: AV2 showed pass-by-absence in both systems: FR had 50 cases from expiry filtering, while Mem0 had 29 from extraction failure.The same apparent absence arose through different mechanisms.
  • Failure Decomposition: AV3 failures decomposed into 77% extraction misses, 12% decay-killed cases, and 12% partial cases.Extraction misses were the largest reported component.

L Scalability Analysis

The scalability analysis links lifecycle quality to retrieval cleanliness and evaluates FR across larger workloads and end-to-end response behavior. FR reduces stale-context exposure while preserving composability and standard retrieval performance.

  • Scalability and Cost: FR’s stale token rate was 3.0% versus Mem0’s 9.0%, saving $25.12 per million queries in stale-token costs at GPT-4o pricing.The savings estimate covers stale token costs alone.
  • Composability: The AV3 gap narrowed from −16pp to −9pp through supplementary extraction while the lifecycle layer remained unchanged.The intervention added 388 supplementary edges, supporting FR’s composability thesis.
  • End-to-End Scaling: Clean retrieval contexts produced 18–30% confabulation, whereas stale contexts produced 72–78% across 1,032 end-to-end queries.The analysis partitions queries by whether any outdated fact appeared in the top-10 context.
  • End-to-End Scaling: Stale contexts increased GPT-5.4 reasoning tokens by only approximately 10%, separating confabulation differences from test-time compute effects.The reported token overhead was modest relative to the confabulation gap.
  • End-to-End Scaling: FR-Bank abstained on 42.1% of LifecycleBench queries versus Mem0’s 28.7%, with extra abstentions drawn almost entirely from would-be wrong or fabricated answers.FR-Graphiti abstained on 44.0% of queries.
  • End-to-End Scaling: FR removed outdated facts before generation in 14 cases where it answered correctly while Mem0 produced confabulatory wrong answers.Examples included retracted vaping, superseded vehicles, and abandoned business plans.

N.8 Extended Cross-System Analysis

Extended cross-system analysis finds that lifecycle mechanisms address architectural gaps that model upgrades cannot close. FR’s strongest advantages appear in expiry, retraction, selective forgetting, and cross-benchmark safety, with performance shaped by recall and generator behavior.

  • Architectural Gaps: Architectural absences cannot be closed by improving extraction recall, because extraction models cannot add missing lifecycle mechanisms.The analysis distinguishes how many facts are stored from how those facts are managed.
  • Architectural Gaps: Event-time expiry and retraction filtering are structural capabilities: AV2 reached 93% for FR-Graphiti and 88.0% for FR-Bank versus Mem0’s 65%.AV7 retraction handling reached 40% for FR-Graphiti versus Mem0’s 5%, while FR-Bank’s 5% retrieval pass reflects intended suppression.
  • Architectural Gaps: Memory-R1 achieved MRR 0.816 and AV3 94%, but lacked event-time expiry, producing AV2 results of 75% with 25% staleness.Its learned ADD, UPDATE, DELETE, and NOOP action space cannot represent the missing expiry operation.
  • Composability and Scaling: Supplementary extraction narrowed the AV3 gap from −16pp to −9pp, while 77% of failures remained attributable to extraction misses.The lifecycle layer was unchanged and 388 supplementary edges were added.
  • Composability and Scaling: Category-aware routing improved from zero differential at 8 personas to +6pp at 40, indicating that category density governs its benefit.The intermediate improvement was +3pp at 20 personas.
  • Selective Forgetting: FR-Bank improved selective-forgetting correctness to 37.5% versus Mem0’s 2.5%, after AV7 rose from 0% to 40% across development phases.Retraction-aware extraction and supplementary identity and health extraction drove the largest phase improvements.
  • Confabulation and Safety: Stale contexts yielded 72–78% confabulation compared with 18–30% for clean contexts, while FR controls the input distribution rather than the degradation function.The relationship held across systems.
  • Confabulation and Safety: FR-Bank reduced AV7 confabulation from Mem0’s 91.4% to 33.3% by filtering explicitly retracted plans before generation.Without retraction filtering, Mem0’s downstream model confidently discussed retracted plans on most AV7 queries.

O Mem0 Comparison Methodology

The comparison evaluates multiple memory systems under controlled extraction, backbone, embedding, storage, and judging configurations across 516 questions. It combines benchmark outcomes, lifecycle-specific diagnostics, ablations, and cross-system analyses to separate extraction quality from architectural and lifecycle effects.

  • Evaluation setup: Mem0 was evaluated in its default configuration with gpt-4.1-nano extraction, text-embedding-3-small embeddings, Qdrant storage, disk persistence, 1,400 sessions, and a shared Claude Sonnet judge.All 40 personas and 516 questions used the same evaluation pipeline.
  • Evaluation setup: Replacing Mem0’s extractor with gpt-4.1-mini raised AV-pass from 61.0% to 67.1% and reduced confabulation from 45.1% to 33.8%.Expired logistics remained at 61% versus FR-Bank’s 88.0%, indicating residual structural differences after improving extraction.
  • Compared systems: Memory-R1 was reimplemented with GPT-4.1-mini agents, while A-MEM and MemoryOS were evaluated with gpt-4.1-mini backbones and their paper-faithful configurations.The Memory-R1 implementation used the published ADD, UPDATE, DELETE, and NONE action vocabulary; MemoryOS incurred approximately 30× FR-Bank’s ingestion compute cost.
  • Compared systems: A-MEM achieved 65.3% pass and 87.6% Hit@5 retrieval recall, but had 30.4% staleness and 47.0% end-to-end confabulation.The result illustrates that high retrieval recall did not by itself provide lifecycle management.
  • Lifecycle diagnostics: FR showed 14.0% positional staleness exposure versus Mem0’s 37.2%, with weighted staleness risk of 0.51 versus 1.84.The evaluation also reports a full paired outcome matrix across all 516 questions and component-level ablations.
  • Lifecycle diagnostics: Fourteen AV2 questions isolated cases where FR’s expiry filter suppressed expired facts that Mem0 surfaced, producing opposite outcomes under the same judge and inputs.A separate replication found no significant positional effect at relevant retrieval depths on Claude Sonnet 2026 (r = −0.009, p = 0.65).

Z FR-Bank Architecture and Methodology

FR-Bank combines lifecycle-aware memory evaluation with cross-generator and end-to-end analyses, showing that designed lifecycle policies improve pass rates and reduce confabulation while exposing calibration and retrieval-quality trade-offs.

  • Cross-generator validation: The lifecycle-managed ranking replicates on Kimi K2.5, with a roughly 15-percentage-point tier-average gap between managed and unmanaged systems.The gap is 15.1pp on GPT-5.4 and 14.4pp on Kimi K2.5, although Kimi answers more readily and is less safe overall.
  • Failure modes: Soft supersession fails on Kimi AV9: FR-Bank reaches 10.0% correct versus Memory-R1’s 27.0% when both candidate values remain available and the generator commits.The paper identifies this as a limitation of soft supersession under high-commitment generators.
  • Substrate and staleness trade-offs: FR-Bank’s high-recall embedding substrate reaches 97% Hit@5 but 15.5% staleness, while FR-Graphiti reaches 75% Hit@5 and 8% staleness; both remain competitive overall.The comparison indicates a substrate trade-off between supersession precision and retrieval recall rather than a change to the policy layer.
  • Lifecycle-sensitive evaluation: FR-Bank’s 5% AV7 retrieval pass rate is intentional: retraction filtering excludes cancelled facts and yields the highest end-to-end AV7 correctness at 37.5%.MemoryOS has 57% retrieval pass but only 7.5% end-to-end correctness, illustrating the mismatch between presence-based retrieval metrics and retraction behavior.
  • Cross-system results: 76.9% pass rate makes FR-Bank the leading system in the cross-system comparison, ahead of MemoryOS at 70.5% and unmanaged or learned-operation baselines.The reported hierarchy is Mem0 < Memory-R1 < MemoryOS < FR-Bank.
Loading 2609.10413v1…