Source-linked AI summary
Invalidation Contracts for Cross-Episode Agent Memory
Michael Wu, Arquimedes Canedo
TL;DR
LLM agents can save tokens by reusing API recovery suggestions, but server-side data drift makes cached fixes stale and re-deriving them forfeits the savings. The paper introduces invalidation contracts that attach cacheability hints, version stamps, and reload diffs so clients can evict stale suggestions, and evaluates their validity and compliance across seven models, three serving paths, two domains, and approximately 9,400 episodes. Row-level invalidation achieves precise validity and raises compliance by 0 to 66.7 percentage points across models, while model-specific input-schema conservatism limits compliance gains.
Problem
Cached recovery suggestions save tokens when reference data is stable but become wrong after server-side drift, while existing recovery interfaces provide no expiration or revalidation signal.
Method
The paper introduces invalidation contracts that attach cacheability hints, version stamps, and structured reload diffs to recovery suggestions and evaluates six protocol levels.
Results
0 to 66.7 percentage points: row-level invalidation raises compliance across seven models, while version-stamp validity is deterministic with zero contract failures and 1.00 row-level eviction precision.
Takeaways & Limitations
Validity is controlled by the protocol, but compliance remains model-dependent; precise invalidation preserves unaffected entries and exposes planner-specific limits.
Takeaways & Limitations
The evaluation covers two authored domains, so the domain count limits claims about generality and transfer to production APIs remains unknown.
Abstract
from arXiv · showhide
LLM agents that cache recovery suggestions from API errors can skip re-derivation in later episodes, spending fewer tokens and fewer model calls on constraints they have already learned. Server-side data drift turns those cached fixes into silent failures, and the usual remedy, re-deriving on every episode, gives the savings back. We introduce invalidation contracts, a protocol layer that attaches version stamps and cacheability hints to every recovery suggestion so the client can evict stale entries without trial and error, and keep the rest. The contract decomposes realized savings into two independent factors: validity, the fraction of cached suggestions that remain correct after a drift event, and compliance, the fraction the planner applies on the first attempt. Validity depends only on the protocol and is vendor-independent. Compliance depends on the planner model: identical wire bytes yield 100% first-try compliance on Claude Haiku 4.5 and 11% or below on Claude Sonnet 5, which exhibits input-schema conservatism, refusing fixes that add fields the original request did not contain. We evaluate across seven models, three serving paths, two domains, and approximately 9,400 episodes. Row-level invalidation raises compliance by 0 to 66.7 percentage points across the seven models, 55.6 to 66.7 on three, and recovers 29-33% of baseline token cost on four of seven models, while table-level invalidation destroys co-located entries and drops post-drift first-try rates to 0% on five of seven. Eviction precision is 1.00 at row granularity on every model under the row-level oracle of Section 4.1. The contract adds 15% to response payload. Version-stamp validity is deterministic by construction and produced identical results across every model and serving path, with zero contract failures in the entire evaluation.
1 Introduction
Cross-episode memory saves tokens but becomes unsafe when server-side reference data drifts, because recovery suggestions carry no expiration signal. The paper introduces invalidation contracts to let clients evict stale suggestions using protocol metadata and separates savings into validity and compliance.
- 5–28% token savings from naive memory disappear when table-level invalidation destroys co-located cached entries after drift.Post-drift first-try rates drop to 0% on five of seven models, making imprecise invalidation actively harmful.
- Recovery suggestions lack metadata indicating what they depend on, when they expire, or when to revalidate.Agents can otherwise discover staleness only by replaying an error and observing that the cached fix fails.
- Invalidation contracts attach cacheability hints, version stamps, and structured reload diffs so clients can evict stale knowledge without trial and error.The protocol is evaluated across seven models, three serving paths, two domains, and approximately 9,400 episodes.
- Validity measures cached suggestions that remain correct after drift, whereas compliance measures valid suggestions applied successfully on the first attempt.The contract controls validity, while compliance depends on the planner model and protocol level.
- 0 to 66.7 percentage points: row-level invalidation raises compliance across seven models, with gains of 55.6 to 66.7 points on three models.These changes occur without changing the model, separating protocol effects from model behavior.
- The paper contributes six protocol levels, large-scale evidence with zero contract failures, a validity–compliance savings decomposition, and a cost comparison for drift detection.It also identifies input-schema conservatism as a boundary on what protocols can achieve.
2 Related Work
Related work establishes persistent agent memory, external verification, and freshness metadata as adjacent foundations. The paper positions invalidation contracts as addressing the missing recovery-layer protocol for deciding what cached API fixes remain usable.
- Persistent stores let deployed agents reuse successful calls and repairs across later episodes, while standardized interfaces carry those calls across agent systems.Benchmarks evaluate these agents against API suites, web environments, and repository tasks.
- Cached entries become problematic because their validity is settled when written, while the interface does not state when that validity stops holding.
- External compilers, tests, and tool responses can support repair, whereas self-correction without an external signal does not reliably improve answers.The reported gain in externally evaluated repair belongs to the signal rather than introspection.
- HTTP and service-description systems provide freshness lifetimes, validators, constraints, schemas, and typed introspection for callers.API error standards define diagnostic envelopes, but the recovery layer does not specify what agents should cache or when to revalidate.
3 The Invalidation Contract
The invalidation contract adds machine-readable dependency and freshness metadata to recovery suggestions, while the evaluation separates server protocol levels from client consumption arms. Its central distinction is precise validity engineering versus model-dependent compliance.
- An invalidation contract travels with API responses and tells callers what they may cache, what dependencies matter, and when those dependencies change.
- 3.1 The Two APIs: Each API uses versioned reference tables, derivation rules, policies, and typed recovery suggestions whose metadata records dependent tables and rules.Acme billing and recipe conversion share this model, while their domains and rule structures differ.
- 3.2 Contract Fields: cache_hint distinguishes cacheable fixes derived from closed reference data from recompute fixes derivable from the request itself.The contract also supplies scalar or vector table dependencies and response-level version information.
- 3.4 The Consumption Levels: Level 2 instructs planners to merge structured suggestion parameters verbatim, converting recovery advice into a patch and improving compliance in calibration.
- 3.5 Two Ladders: Server Levels, Client Arms: Server levels change emitted information, whereas client arms determine how received metadata is consumed; L0–L3 target compliance and L4–L6 target validity.L1 straddles both factors, and the evaluation holds compliance fixed at L3 while varying validity.
- 3.5 Two Ladders: Server Levels, Client Arms: A2 evicts an entire affected table, while A2D uses row-level diffs to evict only changed rows and restamp unaffected entries.The A1-to-A2 comparison measures invalidation generally; A2-to-A2D measures the value of precision.
- 5 Results: Validity is model- and vendor-independent, with identical table-version results across seven models, three serving paths, and all random seeds.The version check produced identical results cell for cell across the tested policy columns.
- 5 Results: 100% compliance at A2D: Claude Haiku 4.5 reaches full aggregate compliance, while Claude Sonnet 5 reaches only 0–11%.Sonnet 5 applies some rewrite and conditional-rewrite hints but refuses add-a-field hints at 10%.
4 Experimental Setup
The experiment evaluates recovery and invalidation across seven models, three serving paths, two domains, and scheduled drift events, using first-try rates, evictions, retries, and within-model token comparisons. Measurement excludes the first governed episode after each rotation and tracks completed episodes to expose exhaustion-related precision artifacts.
- Domains and drift schedule: The primary payments domain uses a 36-episode stream with scheduled code-table and later drift events affecting distinct constraint classes.The governed class requires an added promotional-code field, while the funding class requires a conditional payment-token rewrite.
- Metrics: Compliance is the first-try rate over injection-eligible governed-class episodes, while eviction precision is the reciprocal of a run's eviction count.Retries are summed over the stream, and token counts are compared only within each model against its own A0.
- Measurement hygiene: The first governed episode after each rotation is excluded because every memory arm still injects the pre-rotation code before the server signals change.The exclusion uses a fixed episode-index list rather than inspection of results.
- Measurement hygiene: An exhausted run can report artificially high eviction precision because it performs fewer evictions by failing to reach later drift events.gemini-3.5-flash reports 1.00 at A2 despite completing fewer relevant episodes, so event counts are reported alongside completion counts.
5 Results
Across 250 arm-level runs and 9,432 episodes, row-level invalidation improves post-drift recovery while table-level invalidation can evict entries that never drifted. Gains vary by model and persist across domains, drift rates, and serving paths, but compliance—not protocol validity—determines realized savings.
- Evaluation scope: 250 arm-level runs over 9,432 episodes evaluate the invalidation ladder across seven models, three serving paths, and two domains.Table 3 reports three seeds per cell; the evaluation includes both drifted and control streams.
- Table versus row granularity: 0.0% post-drift funding first-try at A2 follows table-level eviction, versus 66.7% at A1 and 66.7% at A2D on four of seven models.gpt-5.4-mini falls to 22.2% at A2 and returns to 66.7% at A2D; eviction precision is 0.25 for tables and 1.00 for rows.
- Robustness across settings: 48.3 points is the A2D-over-A1 targeted-class gain for claude-haiku-4-5 in the second domain, compared with 35.0 for claude-sonnet-5 and 15.0 for claude-sonnet-4-6.Across one, two, and three rotations, A2D compliance remains stable while A1 varies; the policy-rule stream also produces identical retry vectors across three models within each arm.
- Token cost: 10.2 to 15.7 percentage points is the row-granularity token-saving increment on four compliant models, while naive memory alone recovers 10.3% to 28.2% of A0 token cost.The increment follows client compliance; claude-sonnet-5 adds 3.4 points and deepseek-v4-flash loses 1.0.
- Model-dependent compliance: 0–11% aggregate compliance on claude-sonnet-5 contrasts with 100% A2D compliance on claude-haiku-4-5, showing that identical protocol information can yield different client behavior.On claude-sonnet-5, add-a-field hints are applied only 10.0% of the time, whereas rewrite and conditional-rewrite hints reach 55.0–66.7%.
6 Discussion
The discussion separates protocol-controlled validity from model-controlled compliance, showing that precise invalidation can preserve correct memory while model conservatism still limits first-try use. It also identifies detection trade-offs, evaluation boundaries, and practical preflight guidance.
- Validity and compliance: 1.00 eviction precision at A2D holds across all seven models, while table-level invalidation over-evicts and can reduce post-drift first-try rates to 0%.Validity depends on server version stamps rather than model willingness to act.
- Validity and compliance: 55.6 to 66.7 percentage-point compliance gains occur on three models with row-level invalidation, but Sonnet 5 applies add-a-field hints at only 10%.The input-schema conservatism boundary persists across protocol levels.
- Practical recommendation: Tens of API calls with known-valid fixes provide a compliance preflight for the action types a deployment needs.The preflight reads the model factor directly before a memory system is trusted to save tokens.
- Eager versus lazy detection: Vector-only detection has precision 1.00 but is failure-gated because dependency vectors appear only on recovery feedback after failed requests.A cached family that stops producing errors can become unobservable to this policy.
- Eager versus lazy detection: Stamp-plus-vectors matches the lazy arm’s disposition, alerts one episode earlier, and changes token cost by +0.16%, +0.20%, and −0.11% across three models.The stamp supplies zero-lag detection, while vectors identify stale entries surgically.
- Limitations and future work: The evaluation covers two domains, seven models, and approximately 9,400 episodes, but the domain count limits claims about generality.Prompt sensitivity and production transfer remain untested, and dependency-vector compliance has not been isolated from graph propagation.
7 Conclusion
The conclusion presents invalidation contracts as a way to keep cross-episode memory correct under server-side drift, while separating reliable validity from variable model compliance. It recommends measuring compliance per model and action type before relying on memory savings.
- Protocol contribution: Two fields per suggestion plus a structured schema-reload diff let clients evict stale entries without trial and error.The fields are a cacheability hint and a version stamp.
- Validity: 1.00 row-level eviction precision is achieved on every model, while table-level invalidation over-evicts at 0.25 precision; the contract adds 15% response payload.Version-stamp checks were deterministic and identical across seven models, three serving paths, two domains, and approximately 9,400 episodes.
- Compliance: 0 to 66.7 percentage-point compliance gains occur across seven models, but two models decline to draw on cached suggestions and no protocol crosses the add-a-field conservatism boundary.Identical wire bytes can produce opposite outcomes because compliance remains model-dependent.
- Practical consequence: A compliance preflight using tens of API calls with known-valid fixes should be run per model and per action type before memory is trusted to save anything.The preflight directly measures the model factor without requiring the full benchmark.