PaperlayerView exact revision on arXiv

Source-linked AI summary

The Memory Trust Gap: Capability-Dependent Failures in Persistent-Memory Agents

Jundong Hu, Shekar Ramachandran

arXiv:2609.01852v1cs.AIcs.CL

TL;DR

Prior work documents failures to act on updated information but leaves unclear when harm emerges across capability levels and which memory features trigger it. This paper investigates those questions with a frozen two-suite benchmark and cross-size feature and mitigation analyses. Stale-value reliance remains ≈1.0 across model sizes, but net harm emerges only when models are capable enough for the stale value to cost them.

  • Problem

    Prior work leaves unclear when stale-memory harm begins as model capability changes and which observable memory features trigger it.

    Source: §1 · p. 1

  • Method

    The paper evaluates a frozen, two-suite, closed-set, action-scored benchmark across model sizes, then tests feature triggers and mitigations using a factorial design and direct cross-size contrasts.

    Source: §1 · p. 2

  • Results

    Stale-value reliance stays ≈1.0 across model sizes, while paired net harm (∆mem) is capability-gated, appearing only when models are accurate enough for stale values to cost them.

    Source: §1 · p. 2

  • Takeaways & Limitations

    For memory systems, keeping labels reduces stale-value reliance at every size, while smaller models still need conflicts pre-resolved and authority is a weak lever.

    Source: §8 · p. 9

    Closed-set, action-based scoring limits open-ended generality, and the primary model-size series covers only one model family.

    Source: §8 · p. 9

Abstract

from arXiv · show

Persistent memory supports personalized agents, but a stale stored fact can override current authoritative evidence without warning. We study when this harm begins as model capability changes. We evaluate a frozen, closed-set, action-scored benchmark with 2 suites that represent 2 different meanings of "no memory" (a Benefit suite, unsolvable without the stored fact, and a Safety suite, in which an authoritative tool always holds the correct value), on a same-family model-size series (Qwen3 0.6/1.7/4/8B). The Memory Trust Gap reflects over-trust rather than confusion. In the Benefit suite, models answer with the stale value 0.92-1.00 of the time at every scale. In the Safety suite, harm below the no-memory baseline under the trap conditions ($Δ_{\mathrm{mem}}$) is capability-gated, with the larger models collapsing most once a stale note is made to look current. In a $2\times2\times2\times2$ factorial, which feature triggers over-trust depends on both the feature and model scale. Removing a label amplifies over-trust at every size, and a recency feature (stale dated newer) fools the larger models harder. Source authority is weak and scale-flat, and position changes from positive to negative across the Qwen3 model-size series. We confirm these scale interactions with direct cross-size contrast tests rather than overlapping per-model intervals. Mitigation is likewise capability-dependent: exposing metadata improves accuracy for the capable models, but only pre-resolving the conflict restores accuracy for the 2 smaller checkpoints. The same pattern appears on the capable models in an independent Llama-Instruct model-size series and on 2 external datasets (RGB, MisBench). A framing control finds no consistent advantage for the memory label: at the 3 smaller scales, models trust a stale document more than a stale memory; at 8B, the difference is not significant.

1 Introduction

The study asks when stale persistent memory overrides current evidence as model capability changes, and which memory features trigger the failure. It introduces a benchmark and finds capability-dependent harm, triggers, and mitigations, with validation across model families and datasets.

Motivation

  • Stale stored facts can lead agents to act confidently on outdated information even when current authoritative context contains the correct answer.The introduction motivates the problem with personalized facts such as a usual airline, meeting room, or restaurant order.Source: §1 · p. 1

Harm

  • Stale-value reliance stays ≈1.0 across model sizes, but net harm (∆mem) appears only once models are accurate enough for the stale value to cost them.The Trust Gap reflects over-trust rather than confusion.Source: §1 · p. 2

Trigger

  • A factorial analysis with direct cross-size interaction tests finds that which memory feature triggers over-trust shifts with model scale.The introduction identifies this as a gap left open by prior work on failures to use updated information and context overriding parametric knowledge.Source: §1 · pp. 1–2

Mitigation

  • Metadata improves accuracy for capable models, while only pre-resolution restores accuracy for smaller models.The mitigation study also compares memory representation.Source: §1 · p. 2

Benchmark

  • The frozen, SHA-256-pinned benchmark separates a Benefit suite, where the stored fact is required, from a Safety suite, where an authoritative tool always has the correct value.The closed-set, action-scored design gives the Benefit suite a chance-level no-memory baseline and the Safety suite a ceiling-level baseline.Source: §1 · p. 2

Validation

  • The findings are validated on RGB and MisBench, in a Llama-Instruct size series, and with a control showing the effect is not memory-specific.The control finds that smaller models trust a stale document more than stale memory, while the difference is not significant at 8B.Source: §1 · p. 2

2 Related Work

Prior work establishes stale-memory harm, context-conflict susceptibility, and scale-related position effects. This study’s contribution is to test how stale-memory failures vary with model capability and conflict features, using controlled cross-size comparisons.

Stale memory and downstream failure

  • This study adds a controlled four-feature factorial and direct cross-size interaction tests to prior stale-memory work across a same-family Qwen3 0.6B–8B series.STALE benchmarks 400 scenarios, with a best-model score of 55.2%, while MemSyco-Bench compares verified evidence with conflicting user memory. Related work also examines self-consolidation drift, retrieval overload, and memory-update repair and scaling [Sun and He, 2026, Patel, 2026].Source: §2 · p. 2

Knowledge conflict and the authority feature

  • Related knowledge-conflict work establishes context-induced yielding and scale susceptibility, but lacks this study’s controlled factorial and cross-size interaction tests.ConflictBank [Su et al., 2024] spans four model families and size series and reports greater susceptibility in larger models, while AuthMem-Bench varies source authority only during write-time consolidation. This study treats the conflict as context versus parametric knowledge, not in-context learning [Longpre et al., 2021]; Xie et al. [2024] and Jeripity Venkata [2026] examine conflict yielding and regimes.Source: §2 · p. 2

Position effects across model scales

  • Prior work studies position sensitivity across scales, but this study identifies a scale-dependent sign reversal in stale-memory settings.Earlier work finds scale mainly reduces ordering variance rather than reversing it [Liu et al., 2024, Gabín et al., 2026, Byerly and Khashabi, 2026]. De Marez et al. [2026] find larger instruction-tuned models more robust to overt sycophantic flips, opposite to this study’s recency result.Source: §2 · p. 2

3 Experimental Setup

The setup distinguishes two meanings of “no memory” and uses a frozen benchmark with common conditions and metrics to compare experiments directly.

3.1 Benchmark and 2 Suites

  • The Benefit suite is unsolvable without stored facts, while the Safety suite provides an authoritative correct value and measures whether stale memory lowers accuracy below the no-memory baseline.The Benefit baseline is floored at chance; the Safety baseline is ceilinged. In Safety, an authoritative calendar always returns Room B in the illustrative room-booking scenario.Source: §3.1 · p. 3, Table 1 · p. 3
  • The frozen benchmark contains 300 manually reviewed base scenarios—150 per suite—across 33 template families, generated from a pilot and pinned by SHA-256 with a manifest.The review checked each scenario for an unambiguous, tool-consistent ground-truth action, correct suite and condition assignment, and absence of harmful or sensitive content.Source: §3.1 · p. 3

3.2 Conditions and Scoring

  • Each item yields a constrained action scored against construction-time ground truth by exact, regex, or canonical match, with random-sample validation against human reading and cyclic option-permutation averaging.The four conditions are no_memory, clean, stale, and explicit_conflict; each scenario is scored under every cyclic rotation of its three options and averaged.Source: §3.2 · p. 3

3.3 Metrics

  • Reliance measures acting on the stale value, whereas Δmem measures outcome against baseline; the Safety capability threshold is the first level where stale-memory use causes significant net harm.Δmem is paired per scenario, and 95% percentile bootstrap intervals use base scenarios as the statistical unit; an interval excluding zero is marked significant.Source: §3.3 · pp. 3–4

4 Question 1: Harm and the onset of net harm

Across the Qwen3 size series, stale-value reliance persists in the Benefit suite, while genuine net harm in the Safety suite emerges when stale notes become plausible despite an authoritative tool. The onset varies by model size, and greater apparent recency deepens harm for larger models.

4.1 Over-Trust Is Universal

  • With the correct value stored beside the stale one, explicit_conflict reliance on the stale value is 0.50 / 0.50 / 0.01 / 0.00 across 0.6/1.7/4/8B.The small models follow the stale value half the time despite the correct value being present, while the capable models resist; this is consistent with stored-value reliance rather than random choice.Source: §4.1 · p. 4
  • Benefit explicit_conflict does not provide a like-for-like net-harm measure because it supplies the current value that the no-memory condition lacks.The clean net-harm evidence is in the Safety suite.Source: §4.1 · p. 4

4.2 Net Harm in the Safety Suite: The Trap Sweep

  • At L3, 8B reliance is 1.00 and ∆mem is −1.00; at L2, 4B reliance is 0.83, as recency inflation increases harm for larger models.The direct cross-size test for this recency interaction is reported in §5.Source: §4.2 · pp. 4–5

4.1 Over-Trust Is Universal

  • Benefit stale-value reliance is 0.92 / 0.99 / 1.00 / 1.00 across 0.6/1.7/4/8B, with paired ∆mem = −0.33 / −0.35 / −0.35 / −0.37 and all CI upper bounds < 0.The no-memory baseline is 0.35–0.37, with a confidence interval containing chance (0.33), so the suite is floored rather than showing a formatting artifact.Source: §4.1 · p. 4, §4 · p. 4, Table 2 · p. 5

4.2 Net Harm in the Safety Suite: The Trap Sweep

  • ρ∗ = 0/1/1/1 across 0.6B/1.7B/4B/8B, with Safety no-memory accuracy ≥0.98 in every cell, so negative ∆mem reflects genuine net harm.The 0.6B model first crosses into significant net harm at L0; the other three first cross at L1, so the threshold does not order models monotonically by capability.Source: §4.2 · p. 4, Figure 2 · p. 5

5 Question 2: Which memory feature triggers over-trust, and does it scale?

In the Safety-suite factorial, over-trust responds to multiple observable memory features, with the dominant trigger depending on model scale. Direct cross-size tests confirm that recency and position interact with scale, while dose-response and intervention results show further capability-dependent effects.

5.1 The Memory Feature Factorial

  • Removing the label increases stale-value reliance at every size, newer dates have the greatest effect on larger models, authority is weak and scale-flat, and position reverses across sizes.The full 2×2×2×2 factorial independently toggles label, timestamp, source, and position; the findings describe behavioral effects, not an identified internal mechanism.Source: §5.1 · pp. 5–6, §5 · p. 5, Table 5 · p. 7

5.2 Direct Cross-Size Interaction Tests

  • Recency is significantly stronger on larger models (8B−0.6B = +.302∗; mean(large)−mean(small) = +.333∗), while position reverses significantly (8B−0.6B = −.326∗).Paired cross-size bootstrap contrasts establish these interactions directly; authority has no interaction and the label feature has a null group contrast.Source: §5.2 · p. 6, Table 6 · p. 7

5.3 Dose-Response Deep-Dives

  • Stale-value reliance jumps when the stale note is dated at least 1 day newer, then saturates; restoring the label removes this step.Larger models read timestamps correctly, and that higher accuracy is associated with greater reliance on the stale note.Source: §5.3 · p. 6
  • On neutral recency, conditioning reveals a smaller authority effect that grows with scale, while [CURRENT CONTEXT] reduces 8B reliance to 0 when an official-sounding note conflicts with the user's firsthand statement.The full-factorial authority effect is weak and roughly flat because it averages across recency conditions.Source: §5.3 · p. 6
  • At the +1d recency threshold, marking the authoritative item reduces reliance more for capable models (+.42∗ at 4B; +.33∗ at 8B) than smaller ones (+.06 at 0.6B; +.00 at 1.7B).The deterministic-adjudication intervention introduced by Reddy and Challaram [2026] helps capable models but has little effect on smaller ones; it differs from §6's intervention estimand.Source: §5.3 · pp. 6–7

6 Question 3: Mitigation is capability-dependent

Mitigation depends on model capability: larger checkpoints can use item metadata to resolve stale/current conflicts, while smaller checkpoints recover only when the conflict is pre-resolved.

6 Question 3: Mitigation is capability-dependent

  • Metadata improves accuracy for 4B and 8B models, while only pre-resolving the conflict restores accuracy for 0.6B and 1.7B checkpoints.The representation-oracle removes the stale item entirely, applying supersession before the model acts.Source: §6 · p. 7
  • Accuracy gains over raw were +.30/ + .28/ + .53/ + .54 for metadata and +.46/ + .57/ + .63/ + .59 for the representation-oracle.The passage reports gains in this sequence without assigning individual values to specific model sizes.Source: §6 · p. 7
  • The comparison tests raw unmarked conflicts, metadata frames with provenance and timestamps, and an oracle that removes the stale item.Figure 4 and Table 7 compare these three representations of the same conflict.Source: §6 · p. 7

7 Validation: External, Cross-Family, and Controls

External datasets and an independent Llama-Instruct size series extend the stale-evidence findings beyond the synthetic benchmark. Controls show the gap is not unique to memory framing, and style or thinking mode does not consistently remove it.

Style invariance

  • Across MisBench rhetorical styles, per-model over-trust spread is ≤0.19, and casual-blog framing is least trusted in all 8 hop-by-model cells (sign-test p ≈.004).News and wiki are typically highest, so the effect is not monotonic with authority.Source: §7 · p. 7

Cross-family replication

  • On capable Llama-Instruct sizes, Benefit-suite reliance rises from 0.46 to 0.96 to 1.00, while Δmem shifts from −0.10* to −0.31* to −0.36*.Label, recency, and authority feature signs replicate on capable sizes; position’s positive corner at Qwen3-0.6B does not, and unparsed responses make reported over-trust a conservative lower bound.Source: §7 · p. 8

Framing control

  • Paired reliance(memory)−reliance(document) is −.311*/−.058*/−.013*/+.016 (ns), showing no consistent memory-frame advantage and greater trust in stale documents at the three smaller scales.At 8B the difference is not significant, indicating the stale-evidence failure is not specific to persistent memory.Source: §7 · p. 8

Thinking mode

  • At 0.6B, thinking mode raises Benefit-suite reliance from 0.924 to 0.991 (paired Δ+ 0.067, CI [+.040, +.096]); at 8B the gap is saturated and unchanged.Parse rates were at least .999 at the two tested endpoints.Source: §7 · pp. 8–9

External datasets

  • On RGB, Δmem reaches −0.32* at 8B and −0.37* in the free-text arm, while stale-value reliance is 0.81–0.94 across scales on both external datasets.MisBench Δmem is near the floor because its facts are unknown to the models; reliance reaches 0.96 in its wikipedia·semantic cells and 0.95 in the RGB free-text arm.Source: §7 · p. 7, Table 8 · p. 9

8 Discussion and Conclusion

Persistent memory helps agents, yet stale stored values are over-trusted at every model size; net harm, triggering features, and effective mitigation vary with capability.

8 Discussion and Conclusion

  • Capable models exploit correctly read stale timestamps more but resist spurious provenance cues; this differs from robustness to overt sycophantic manipulation [De Marez et al. 2026].A plausible cue that a note is newer can be trusted even when a direct “you are wrong” challenge is resisted; the findings concern different features and are not contradictory.Source: §8 · p. 9
  • Keeping the memory label reduces stale-value reliance at every size, while authority is a weak lever and smaller models still need conflicts pre-resolved [Reddy and Challaram, 2026].The authority finding converges between the internal factorial and external style sweep; the implications extend to stale or conflicting retrieved evidence beyond persistent memory.Source: §8 · p. 9
  • Net-harm claims rely on the Safety suite, and the study’s measurement and model-family scope limit generality.The Benefit suite cannot express net harm on explicit_conflict because its baseline is at chance; closed-set action scoring is partly offset by RGB’s free-text arm. The primary size series is one family, with cross-family reproduction only on capable Llama sizes; ρ∗ is trap-sequence-defined, and failure is measured at memory consumption rather than across an end-to-end system.Source: §8 · p. 9
  • Stale stored values are over-trusted at every model size, while net harm, triggering features, and effective mitigation depend on capability.The pattern also appears across model families, external datasets, and non-memory stale evidence; the benchmark and harness are intended to measure the failure and evaluate mitigations.Source: §8 · p. 9

A Methods Reference · B Memory Feature Dose-Response Deep-Dives · C Cross-Family CI Ledger

This block defines the benchmark suites, scoring, and analysis procedures, then identifies the dose-response and cross-family result displays. The cross-family tables report paired-bootstrap confidence intervals, with parsed-only reliance reported as a conservative lower bound.

A Methods Reference

  • The Benefit suite is unsolvable without its stored fact, while the Safety suite always provides an authoritative tool; authors reviewed all 300 base scenarios for ground-truth and assignment validity.Conditions are no_memory, clean, stale, and explicit_conflict, where the stale and correct values are both stored in memory.Source: A Methods Reference · p. 11
  • Reliance measures stale-value answers, and Δmem is condition accuracy minus no_memory accuracy; ρ* is the first trap level whose stale Δmem CI upper bound is below 0.Options are scored under all three cyclic rotations; inference uses scenario-level 95% percentile bootstrap intervals, with template-family cluster bootstrap and leave-one-family-out robustness checks. The decomposition is exploratory, not pre-registered confirmatory testing.Source: A Methods Reference · p. 11
  • The benchmark uses greedy decoding with identical prompts on Qwen3 0.6/1.7/4/8B and Llama-Instruct 1B/3B/8B; each Qwen model fits on one A100 MIG slice.The frozen benchmark v1 has 300 base scenarios and is SHA-256 pinned; the four Qwen models ran concurrently across MIG slices using plain transformers, without vLLM.Source: A Methods Reference · p. 11

B Memory Feature Dose-Response Deep-Dives

  • Figure 6 collects §5.3 feature dose-response and probe panels plus the §7 MisBench rhetorical-style sweep; Figure 7 visualizes §7 external validation results tabulated in Table 8.Figure 5 plots the factorial main effects tabulated in Table 5.Source: B Memory Feature Dose-Response… · p. 11
  • Figure 5 presents Safety-suite factorial memory-feature main effects on stale-value reliance as trap-on minus trap-off with 95% CIs.It has one point per memory feature and model size, with values tabulated in Table 5.Source: B Memory Feature Dose-Response… · p. 11, Figure 5 · p. 12

C Cross-Family CI Ledger

  • Table 9 reports Llama Benefit-suite results and Table 10 factorial main effects with paired 95% CIs; Table 9’s parse-rate minimum is 0.851 (1B no_memory), and parsed-only reliance is a conservative lower bound.The Llama-Instruct analysis uses the same metrics and bootstrap procedure as the Qwen primary ledger, with a seeded (= 0) paired-bootstrap CI.Source: Table 9 · p. 12, Table 10 · p. 12, C Cross-Family CI Ledger · p. 12
  • Table 10 organizes Llama factorial main effects as one row per model size and one column per memory feature, marking effects whose paired 95% CI excludes 0.The cross-family reanalysis recomputes seeded paired-bootstrap intervals using the same procedure as the Qwen ledger.Source: Table 10 · p. 12, C Cross-Family CI Ledger · p. 12

D Control Experiments (with CIs)

The two control experiments use the same paired-contrast uncertainty standard as the body tables.

D Control Experiments (with CIs)

  • Each control reports a per-scenario seeded bootstrap 95% CI on the paired contrast across n = 150 scenarios.The passage identifies these as the two controls reported in §7.Source: D Control Experiments (with CIs) · p. 12

E Failure Transcripts

Failure transcripts show models taking stale values as actions despite conflicting authoritative evidence, including safety-override cases at Qwen3-8B and an explicit-conflict case at Qwen3-0.6B.

Failure Transcripts

  • In Qwen3-8B safety overrides, the model followed stale Room A over the authoritative calendar’s Room B and Gate A4 over gate B12.Both examples are trap L3 cases; the stale scheduling note was dated newer.Source: E Failure Transcripts · p. 12
  • Qwen3-0.6B chose United despite Delta also being stored, while Qwen3-8B chose Dallas over authoritative Austin.These transcripts show stale-value actions in an explicit-conflict travel case and a local safety-override case.Source: E Failure Transcripts · p. 13

Related papers

Loading 2609.01852v1…