PaperlayerView exact revision on arXiv

Source-linked AI summary

From Lossy to Verified: A Provenance-Aware Tiered Memory for Agents

Qiming Zhu, Shunian Chen, Rui Yu, Zhehao Wu, Benyou Wang

arXiv:2602.17913v1cs.DBcs.AI

TL;DR

Long-horizon memory faces a write-before-query barrier: write-time compression can omit answer-critical evidence and undermine auditability. TierMem uses provenance-linked summaries, runtime sufficiency routing, and escalation to immutable raw logs when summaries are insufficient; router diagnostics show that false cache hits remain a key failure mode.

  • Problem

    Write-time compression can omit decisive constraints and answer-critical specificity before future queries are known, creating unverifiable omissions in long-horizon memory.

  • Method

    TierMem retrieves from a fast summary tier, uses a strict sufficiency policy to choose ANSWER or ESCALATE, and grounds insufficient cases in linked immutable raw pages.

  • Results

    72.0% of 378 Summary-Only errors were routed to ESCALATE, while 34.3% of all 431 error cases were false cache hits where summaries lacked gold evidence.

  • Takeaways & Limitations

    Effective memory access requires both cheap summary answering when evidence is sufficient and reliable escalation when summaries are underspecified or high-risk.

  • Takeaways & Limitations

    Remaining errors are dominated by raw-path retrieval, missing or generalized source evidence, and failures to extract or compose the correct answer.

Abstract

from arXiv · show

Long-horizon agents often compress interaction histories into write-time summaries. This creates a fundamental write-before-query barrier: compression decisions are made before the system knows what a future query will hinge on. As a result, summaries can cause unverifiable omissions -- decisive constraints (e.g., allergies) may be dropped, leaving the agent unable to justify an answer with traceable evidence. Retaining raw logs restores an authoritative source of truth, but grounding on raw logs by default is expensive: many queries are answerable from summaries, yet raw grounding still requires processing far longer contexts, inflating token consumption and latency. We propose TierMem, a provenance-linked framework that casts retrieval as an inference-time evidence allocation problem. TierMem uses a two-tier memory hierarchy to answer with the cheapest sufficient evidence: it queries a fast summary index by default, and a runtime sufficiency router Escalates to an immutable raw-log store only when summary evidence is insufficient. TierMem then writes back verified findings as new summary units linked to their raw sources. On LoCoMo, TierMem achieves 0.851 accuracy (vs.0.873 raw-only) while reducing input tokens by 54.1\% and latency by 60.7%.

1. Introduction

Long-horizon memory faces a write-before-query barrier: write-time compression can discard evidence later needed for faithful, auditable answers. TierMem addresses this by allocating evidence at inference time, combining fast summaries with selective raw-log escalation and verified write-back.

  • Write-time compression commits to retained information before future query requirements are known.
  • Summaries can erase decisive constraints, creating unverifiable omissions when later answers require precise evidence.A severe peanut allergy may be reduced to generic dietary preferences, preventing traceable support for a safety answer.
  • Raw-log grounding restores auditability but can impose substantial token and latency costs on every query.Most queries do not require raw-level evidence, motivating query-conditioned access.
  • TierMem treats retrieval as inference-time evidence allocation: choose the lowest-cost memory granularity that still supports faithful, auditable answering.
  • TierMem uses provenance-linked summaries, a runtime sufficiency router, selective raw escalation, and verified write-back.The system answers from summaries when sufficient and consults linked raw pages when they are not.
  • 0.851 accuracy versus 0.873 for always-raw on LoCoMo accompanied 54.1% fewer input tokens and 60.7% lower latency.TierMem improves the accuracy–efficiency trade-off on long-horizon benchmarks.

2. The TierMem Framework

TierMem organizes memory into provenance-linked summary and immutable raw-log tiers, then dynamically selects the minimal sufficient evidence for each query. Its router enables a fast summary path, targeted escalation, and provenance-preserving consolidation.

  • TierMem uses Tier-1 summaries and Tier-2 immutable raw pages connected by explicit provenance links.Tier-1 supports fast retrieval, while Tier-2 preserves stable, auditable source pages.
  • Each Tier-1 entry contains a concise summary, dense retrieval embedding, and links to supporting Tier-2 pages.
  • The tiers are built synchronously so every summary entry has a traceable raw source.
  • For each query, TierMem retrieves Tier-1 summaries and selects the minimal sufficient evidence granularity.
  • The router outputs ANSWER when summaries suffice and ESCALATE when precise constraints needed for faithful answers are missing.Relevant constraints include negations, exact values, and attribution.
  • On the default fast path, TierMem answers from top-k summaries; escalation reads linked raw pages and may perform bounded multi-hop retrieval.
  • After escalation, verified findings update Tier-1 while preserving links to the raw evidence used.

3. Router Optimization

TierMem trains its router in two stages to distinguish sufficient summaries from cases requiring raw grounding while managing the accuracy–efficiency trade-off. Supervised routing labels and cost-aware optimization shape this decision boundary.

  • The router is optimized to reduce inference costs while maintaining the accuracy of a raw-grounded upper bound.
  • Hindsight labeling compares Summary-Only and Raw-Grounded answers to identify whether escalation recovers necessary details.
  • ANSWER is assigned when the summary-only answer is correct and complete.
  • ESCALATE is assigned when the summary answer is vague or incorrect but raw grounding recovers the needed details.
  • The labeling process filters impossible queries where even raw logs fail and isolates compression-induced omissions.
  • TierMem first distills routing behavior from a stronger teacher, retaining examples whose decisions match oracle labels.
  • GRPO then explicitly optimizes the router's accuracy–efficiency trade-off.
  • The reward penalizes escalation cost and false alarms while rewarding answer correctness.This encourages escalation only when its expected faithfulness benefit outweighs retrieval cost.

4. Experiments

The experiments test whether inference-time evidence allocation improves the accuracy–efficiency trade-off, while evaluating routing, provenance, and online amortization. Across benchmarks, routed TierMem approaches raw-only accuracy with summary-like efficiency and recovers compression-related omissions through selective escalation.

  • Research questions: TierMem is evaluated against summary-only and raw-only policies to test whether query-conditioned evidence allocation improves the accuracy–efficiency trade-off.The evaluation also examines router control, provenance pointers, and online consolidation.
  • Evaluation protocol: TierMem variants isolate memory-access policy using the same generator, embedding model, and reranker, while the main benchmark disables online write-back.The comparison includes summary-only, always-raw, learned-router, and stronger-LLM-router variants.
  • Evaluation protocol: The study reports answer quality, compression loss, query-time efficiency, routing behavior, and token components to distinguish evidence loss from reasoning failure.Compression loss is measured with Unverifiable Omission Rate, while routed costs include QA and routing components.
  • Main results: Ours (router) approaches raw-only accuracy with summary-like efficiency by allocating evidence at inference time, whereas summary-only is efficient but suffers compression loss.This comparison is reported across both long-horizon conversational memory benchmarks.
  • LongMemEval: On LongMemEval, static summaries lose answerable evidence for nearly 30% of queries, while Ours (router) selectively escalates hard cases to bridge the summary–raw gap.Mem0 is reported with a 29.6% Unverifiable Omission Rate.
  • Routing mechanism: Routing uses a cheap ANSWER path for sufficient summaries and an expensive ESCALATE path for underspecified or high-risk cases, making high omission rates recoverable through provenance-guided escalation.The system aims to preserve accuracy on the ANSWER path while reliably triggering ESCALATE when summary evidence is insufficient.

5. Analysis and Ablations

TierMem’s analyses show that routing, provenance links, and verified consolidation jointly improve the accuracy–efficiency trade-off, while diagnostics expose failures in both sufficiency checking and downstream escalation. Across ablations and error analyses, selective escalation reduces raw-grounding cost but does not eliminate retrieval or generation failures.

  • Router Learning and Cost–Fidelity Trade-off: 71.7% of oracle-hard LoCoMo cases are recalled by the final SFT+GRPO router, with 678 tokens of routing overhead per query.The router escalates 39.0% of queries and accounts for 14.7% of total input tokens.
  • Router Learning and Cost–Fidelity Trade-off: SFT+GRPO matches GPT-4.1-mini’s 0.851 LoCoMo accuracy while substantially reducing latency versus raw-only grounding.The result supports using a lightweight router as a semantic cache controller that allocates raw evidence selectively.
  • Ablation: Effect of Provenance Pointers: 85.1% linked accuracy exceeds 83.6% without provenance pointers, with the +1.5-point gain concentrated on escalated queries.Acc.@R increases from 77.5% to 81.7%, while the ANSWER path remains unchanged.
  • Consolidation Over Replay Epochs: Verified write-back increases summary-path coverage and lowers later token and latency costs while maintaining stable overall accuracy across replay epochs.Under No-Recall write-back, S-Correct rises 32% from 819 to 1,083, while average tokens fall from 3,958 to 2,419 and latency from 5.14s to 3.39s.
  • Anatomy of Summary-Only Failures: 41.3% of Summary-Only errors arise from missing information, while 28.8% arise from over-generalization that loses answer-critical specificity.These error categories directly instantiate the write-before-query barrier.
  • Router Diagnostics: Among 378 Summary-Only errors, the router escalates 72.0%, yet downstream retrieval, source coverage, or generation failures can still leave the final answer wrong.The analysis identifies raw-page retrieval, missing source detail, and answer composition as remaining failure modes.

6. Related Work

Long-horizon agent memory must manage costly, degraded access to lengthy interaction histories while preserving reliable retrieval and grounding. Related work studies external memory, adaptive retrieval, and verifiable evidence use.

  • Long-horizon memory: Long-horizon agents must remain consistent over histories spanning days to months, but full raw-log processing is costly and can degrade with input length.Context-extension methods do not eliminate compute growth or position-sensitive degradation on long inputs.
  • External memory: Non-parametric memory systems use external read–write stores to support long-term personalization.Representative approaches include hierarchical management, controller-based memory operations, and write-time compression.
  • Adaptive retrieval: Retrieval is a control problem involving decisions about when and how much evidence to retrieve and when to stop.Related work examines iterative retrieval and reflection-based improvement.
  • Verifiable grounding: Verifiable grounding research emphasizes retrieve–revise pipelines and fine-grained citations.These approaches connect retrieval with evidence-supported answer revision.

7. Conclusion

TierMem addresses summary-centric memory failures by allocating evidence at inference time: it answers from summaries when sufficient and escalates to raw logs otherwise. Across two benchmarks, it approaches raw-grounded faithfulness while reducing query-time cost.

  • Conclusion: TierMem treats retrieval as inference-time evidence allocation, escalating from summaries to immutable raw logs only when necessary.Verified findings can then be written back with provenance links.
  • Conclusion: Across LoCoMo and LongMemEval, TierMem approaches raw-grounded faithfulness while substantially reducing query-time cost.The reported analyses also examine routing overhead, provenance-aware grounding, and verified write-back.

A. Implementation Details

The implementation builds TierMem around paged raw logs, indexed summaries, provenance pointers, two-path routing, iterative research, deterministic source linking, and optional consolidation. Benchmark evaluation fixes Tier-1 within epochs, while interactive write-back is described as a deployment extension.

  • Storage and provenance: Tier-1 embeds page summaries with provenance metadata that maps each summary hit to its originating raw page.Mem0 supplies summary indexing and metadata retrieval, while entity-graph components are disabled.
  • Storage and ingestion: Paged ingestion stores timestamped, speaker-tagged turns in persistent pages and summarizes pages when they reach the configured maximum length.The default page limit is 1000 tokens, with deferred summarization for incomplete or failed ingestions.
  • Two-path routing: The router selects summary-only answering when summaries explicitly contain the answer and research mode when they are ambiguous or incomplete.Research mode grounds generation in raw pages and records retrieval, routing, and answer-generation costs.
  • Research path: Research mode iteratively integrates evidence and plans further retrieval using summary and keyword search, deduplicating commands and stopping when no new pages are found.Evidence blocks combine summaries with full raw page content for verbatim grounding.
  • Provenance linking: Deterministic provenance linking matches normalized evidence quotes to raw pages and falls back to keyword overlap, reducing reliance on model-generated identifiers.The resulting source pages and snippets provide evidence links for extracted facts.
  • Consolidation and evaluation: Online consolidation compares verified facts with similar memory units and can skip, update, or add entries while inheriting provenance during updates.Main benchmark experiments instead use epoch-wise replay with Tier-1 fixed within each epoch; interactive write-back is deferred between epochs.
  • QA algorithm: The QA algorithm retrieves Tier-1 summaries first, escalates when the router predicts insufficient evidence, integrates bounded evidence, and optionally writes back the result.The router chooses ANSWER or ESCALATE using the query and initial summary evidence.

B.2. Data Construction: Summary Sufficiency Labels

The router dataset labels whether retrieved summaries are sufficient for exact, complete, non-guessing answers, then trains a policy to balance faithful escalation against retrieval cost. Controlled analyses further isolate provenance effects from routing frequency.

  • Label construction: Router training instances pair top-k Tier-1 summaries with linked raw memory states and execute fixed summary and research policies.LLM judges produce binary sufficiency indicators under deterministic, conservative prompting.
  • Sufficiency rubric: Summary sufficiency requires explicit, directly stated or clearly derivable answers without guessing, with completeness for list questions and exact details for dates, numbers, and names.Vague related information and causal inference from effects do not count as sufficient.
  • Conservative judging: The sufficiency prompt conservatively returns false when ambiguity or missing explicit support could make a factual answer unreliable.This targets false positives that would train the router to underescalate.
  • Filtering: Instances where both summary and research policies fail are removed because they provide no signal about whether escalation can recover missing evidence.The supervision label therefore focuses on cases where escalation can distinguish sufficient from insufficient summaries.
  • Router training: The router is trained with SFT from GPT-5 teacher decisions, followed by GRPO refinement under an explicit accuracy–cost trade-off.SFT regularizes behavior and formatting, while GRPO refines the decision boundary and discourages unnecessary escalation.
  • GRPO objective: The routing reward combines accuracy, cost, and waste terms, penalizing costly or unnecessary escalation while rewarding correct ANSWER and ESCALATE decisions.The policy uses the action field at inference time even though SFT targets the full teacher JSON.
  • Provenance ablation: Provenance pointers changed routing decisions on 15.2% of shared queries, while identical-route comparisons showed a net gain of +6 correct answers when both systems escalated.The controlled results attribute the main benefit to higher-quality raw evidence retrieval rather than more frequent escalation.

C.3. Cost Analysis by Route

Provenance linking changes the cost profile of escalation only modestly while preserving retrieval depth, yielding a favorable accuracy–efficiency trade-off. Linked and No-Linked routes cost nearly the same on summary-only queries, but Linked escalation uses slightly more tokens for more precise raw grounding.

  • Cost by route: Linked and No-Linked have nearly identical token costs on summary-only queries, while Linked uses slightly more tokens on escalated queries.The extra escalated-query cost reflects deeper or more precise raw evidence grounding.
  • Retrieval depth: Average research depth is 1.38 for Linked versus 1.36 for No-Linked, with both capped at a maximum depth of 3.The near-identical depths indicate that Linked’s gains do not come from additional retrieval iterations.
  • Trade-off: Provenance-aware escalation improves performance without relying on increased search depth, despite incurring slightly higher token cost.This produces a favorable accuracy–efficiency trade-off for provenance linking.

D. Additional Analysis: Why Higher Hard-Recall Can Tie on End-to-End Judge Score

Higher hard-recall does not guarantee higher end-to-end judge accuracy because routers select different escalated sets with different repair and regression profiles. Counterfactual S/R analysis makes these cost–fidelity differences explicit under identical stored memories and system components.

  • Metric distinction: Hard-recall measures escalation of oracle-hard cases, whereas judge accuracy depends on repairs of summary failures and regressions introduced by escalation.The analysis therefore evaluates both mechanism and outcome metrics.
  • Counterfactual diagnostics: RepairRate is Pr(R correct | S wrong), while RegressRate is Pr(R wrong | S correct) on the queries routed to R.These diagnostics quantify escalation’s repair benefit and regression exposure.
  • Measured trade-offs: GPT-4.1-mini achieves a 67.8% RepairRate and 12.1% RegressRate, while ours achieves 64.9% and 7.7%, respectively.GPT-4.1-mini repairs more summary failures but exposes more cases where escalation would be worse than the summary path.
  • Escalated-set composition: Different routers produce different escalated-set difficulty distributions even with identical writes and components.The hard-negative bucket contains 86 cases for GPT-4.1-mini and 92 for ours, with some still judged correct.
  • Reporting: S/R counterfactual matrices expose repair, regression, hard-negative, and agreement patterns underlying the gap between hard-recall and final judge accuracy.This reporting makes cost–fidelity behavior transparent under a fixed write state.

E. Router Error Analysis and Failure Modes

Router errors are asymmetric: false cache hits are rarer but threaten correctness, whereas false cache misses dominate and mainly increase cost. The analysis attributes these errors to summary incompleteness, precision demands, and conservative sufficiency judgments.

  • Error categories: Routers distinguish S→R false cache hits, where raw grounding is needed, from R→S false cache misses, where escalation is unnecessary.These two error types capture opposite routing failures.
  • False cache hits: S→R errors occur in 61 cases and directly affect correctness, mainly because summaries omit required information or fail precision and attribution requirements.A summary may be topically relevant yet underspecified for exact answering.
  • False cache misses: R→S errors occur in 371 cases and usually increase cost without degrading correctness because the router conservatively treats sufficient summaries as inadequate.The dominant issue is false information-gap detection.
  • Failure modes: TierMem’s routing policy prioritizes faithfulness, but conservative over-escalation remains a calibration and sufficiency-detection limitation.Future improvement targets better use of indirect but adequate evidence while avoiding false cache hits.
Loading 2602.17913v1…