PaperlayerView exact revision on arXiv

Source-linked AI summary

Amnesia by Design, Memory By Necessity: Persistent State for Document Intelligence

Souhail Bakkali, Ayoub Merimi

arXiv:2609.32041v1cs.CVcs.AIcs.CLcs.IRcs.LG

TL;DR

Document AI systems process documents as stateless functions, while persistent, document-native state remains absent from current mechanisms. This survey formalizes evidence-grounded state and proposes an audit and counterfactual benchmark harness; its audit finds cross-session state evolution largely untested. The framework and harness are specifications, and implementations remain an open empirical task.

  • Problem

    Document AI lacks mechanisms coupling persistent state with document-native structure, provenance, and temporal validity.

    Source: §I · p. 1, D1 P(y | D1) y1 RESET · p. 2, §II · p. 3

  • Method

    The survey formalizes a six-component persistent-state framework with lifecycle operations and specifies a longitudinal harness using five counterfactual metrics.

    Source: §VI · p. 16

  • Results

    Audited benchmarks evaluate within-episode tasks while leaving cross-session state evolution largely untested.

    Source: A. The Statefulness Audit · p. 11, §VI · p. 16

  • Takeaways & Limitations

    The framework, audit, and evaluation protocol are specifications; implementations satisfying them remain an open empirical task.

    Source: §VI · p. 16

    The review is scoped to dominant published corpora biased toward English-language, Latin-script, modern documents, and the proposed specifications are not validated implementations.

    Source: F. Limitations · p. 15

Abstract

from arXiv · show

Modern Document AI reads contracts, extracts fields, reasons over tables, and grounds answers to page regions, then forgets everything. Processing an amendment the next day begins from scratch: no schema retained, no contradiction detected, no experience carried forward. This is a structural choice, not a scale failure: current systems are stateless functions. We call this the statelessness bottleneck. This bottleneck lies beyond parameter scaling, context extension, and retrieval augmentation: storage provides persistence and retrieval provides access, but neither consolidates observations into knowledge that improves future processing. This survey formalizes persistent evidence-grounded document state as a unifying framework, specifying the operations and invariants required to convert multimodal evidence into durable, provenance-linked state. We introduce a statefulness audit showing that ten representative benchmarks, coded against eight statefulness criteria, leave cross-session state evolution untested, and derive a longitudinal benchmark harness with five counterfactual metrics: Experience Gain, Cost Efficiency, Memory Harm, Forgetting Fidelity, Coverage Retention, to characterize the benefit, cost, risk, and governability of persistent document state. Document AI lacks mechanisms coupling persistent state to document-native structure, provenance, and temporal validity. The next era of Document AI will be defined by what systems retain across documents, sessions, and time.

I. INTRODUCTION

The introduction identifies Document AI’s statelessness as a structural bottleneck: systems process documents, answer, and reset rather than carrying knowledge forward. The survey proposes persistent evidence-grounded state and an audit and evaluation framework for studying it.

D1 P(y | D1) y1 RESET

  • Current Document AI processes documents as isolated functions, discarding intermediate work so the next filing starts from nothing.The proposed paradigm carries validated evidence from one document into persistent state that conditions later answers.Source: §I · p. 1, D1 P(y | D1) y1 RESET · pp. 1–2, Figure 1 · p. 1
  • Scaling, longer context, and retrieval provide knowledge, workspace, or access, but not consolidation that changes future behavior.The survey distinguishes persistent storage and retrieval from learning through prior processing.Source: D1 P(y | D1) y1 RESET · p. 2
  • The survey frames the statelessness bottleneck as structural across task formulations, benchmarks, and training objectives, rather than a limitation of one architecture.It argues that existing systems lack document-native persistence coupled with spatial layout, span-level provenance, temporal validity, and governed deletion.Source: D1 P(y | D1) y1 RESET · p. 2
  • The survey formalizes a six-component state and lifecycle operations, identifies persistence-related failure modes, and derives five counterfactual evaluation metrics.The metrics cover Experience Gain, Cost Efficiency, Memory Harm, Forgetting Fidelity, and Coverage Retention.Source: D1 P(y | D1) y1 RESET · p. 2
  • The audit finds that ten representative benchmarks leave cross-session state evolution largely untested.The proposed harness evaluates benefit, cost, risk, and governability through counterfactual metrics.Source: D1 P(y | D1) y1 RESET · p. 2
  • Persistent document state is information retained, validated, consolidated, temporally versioned, and governed across sessions.The framework links stored conclusions to source-document evidence and tracks when claims remain authoritative.Source: D1 P(y | D1) y1 RESET · p. 2

II. CAPABILITY EVOLUTION AND THE ACCESS–CONSOLIDATION BOUNDARY

The section traces Document AI’s expanding scope while distinguishing access to information from its consolidation into persistent state. Across perception, representation, and downstream task families, standard evaluation and inference remain episode-local.

II. CAPABILITY EVOLUTION AND THE ACCESS–CONSOLIDATION BOUNDARY

  • Under the placement rubric, no published system reaches the Semantic or Governed persistence bands; those bands require document-native structure and reusable schemas with span-level provenance.Scope is the broadest document scope processed, while persistence is the highest regime retained across explicit session boundaries.Source: §II · pp. 2–3, Figure 2 · p. 3

A. Perception, Recognition, and Layout Analysis

  • Perception and recognition capabilities evolved through learned visual representations and increasingly diverse layout benchmarks, but standard inference remains episode-local without explicit adaptation or memory.Repeated inference alone does not expose persistent operational state to later episodes.Source: A. Perception, Recognition, and Layout… · p. 3

B. Cross-Modal and Layout-Aware Representation

  • Cross-modal, layout-aware, and OCR-free representations are typically transient, and their native-state columns remain empty under the survey’s definition of persistent document state.These representations are not, by themselves, knowledge that can be validated, consolidated, versioned, or governed across sessions.Source: B. Cross-Modal and Layout-Aware… · pp. 3–4

C. Downstream Tasks and the Single-Shot Convergence

  • Downstream tasks expanded from page-level questions to corpus-level synthesis, yet retained the single-shot assumption of fixed input, one function, and one evaluation.No benchmark in this task genealogy asks whether processing a document changes the system.Source: C. Downstream Tasks and the… · p. 4

D. Long-Context Scaling and Efficient Architectures

Longer context and retrieval improve access to document information but do not create state that learns across sessions. The survey distinguishes persistence from consolidation and argues that adjacent memory paradigms do not jointly meet document-native requirements.

D. Long-Context Scaling and Efficient Architectures

  • Context extensions enlarge working memory, but it clears after inference and long-context accuracy can fall with middle-position evidence or more distractors.The passage also notes that million-token contexts remain scratchpads rather than persistent state.Source: D. Long-Context Scaling and Efficient… · p. 4

E. Retrieval, Multimodal RAG, and State Integration

  • Retrieval finds relevant passages, but retrieval stores and ingestion-reorganized indices do not consolidate experience across queries or preserve temporal validity, provenance, and governed deletion.The survey distinguishes access to evidence from integration that changes future processing.Source: E. Retrieval, Multimodal RAG, and… · p. 5

F. Agents, Continual Learning, and Adjacent Persistent State

  • Agent scratchpads and workflows can retain state within an episode, but that state is discarded when the episode ends.The passage contrasts five tool invocations with knowledge tracked across hundreds of documents.Source: F. Agents, Continual Learning, and… · p. 5
  • Persistent agent and continual-learning approaches demonstrate cross-session persistence, yet lack the document-specific combination of multimodal evidence, layout, temporal validity, provenance-linked consolidation, and governed deletion.The survey treats persistence as insufficient without document-native structure and governance.Source: F. Agents, Continual Learning, and… · pp. 5–6

G. Commercial Memory and State Operators

  • Commercial memory features retain facts across sessions but discard their source evidence and cannot track superseded clauses or perform verifiable deletion.The survey distinguishes this persistent context from persistent memory.Source: G. Commercial Memory and State Operators · p. 6
  • Access reads stored content, adaptation changes model parameters, and consolidation transforms validated persistent state to alter future processing.The survey formalizes these as distinct operators that can be composed but not conflated.Source: G. Commercial Memory and State Operators · p. 6
  • No existing paradigm satisfies all six conjunctive requirements, and the gap is a shared document-native task formulation, representation, and evaluation protocol.The survey frames this as a missing intersection, not as the absence of every component individually.Source: G. Commercial Memory and State Operators · p. 6

H. Positioning Against Existing Surveys

The survey unifies prior Document AI strands by treating persistence as a first-class axis and auditing whether state survives an episode. It then specifies persistent document state and the roles its components and transitions must preserve.

H. Positioning Against Existing Surveys

  • The survey connects isolated research strands by treating persistence across the full stack as a first-class axis and auditing ten representative benchmarks for statefulness.It formalizes the state tuple, lifecycle operations, and operator set, alongside an evaluation protocol.Source: H. Positioning Against Existing Surveys · p. 6

III. FORMALIZATION OF PERSISTENT EVIDENCE-GROUNDED STATE

  • Persistent evidence-grounded document state has explicit content, relations, temporal validity, provenance, and authorization, unlike mechanisms that differ in persistence, inspectability, and governability.The definition distinguishes the document-state framework from hidden states, caches, scratchpads, parametric knowledge, and external datastores.Source: §III · pp. 6–7

A. Document Stream and State Specification

  • The six-component state includes episodic evidence, consolidated semantic knowledge, associative relations, temporal validity, provenance, and authorization policy.The provenance ledger links conclusions through consolidation history to source-document bounding boxes; the authorization state governs read, write, update, and delete permissions.Source: A. Document Stream and State… · p. 7
  • The state representation is implementation-agnostic: a knowledge graph, hybrid database, neural memory module, or combination may realize it if all six roles and joint transitions are preserved.The document stream is ordered and may include documents that revise or supersede earlier ones.Source: A. Document Stream and State… · p. 7

B. State Transitions and Lifecycle Operations

Persistent state changes through a gated, six-operation lifecycle. Its operators consolidate evidence, reconcile temporal conflicts, and remove requested information while preserving provenance.

B. State Transitions and Lifecycle Operations

  • A pre-write validation gate is required because incorrect outputs can become durable errors that contaminate later sessions.Observations below the uncertainty threshold are rejected, and failed validation leaves state unchanged.Source: B. State Transitions and Lifecycle… · pp. 7–8
  • Schema consolidation abstracts episodic traces, temporal supersession resolves conflicts into five outcomes, and unlearning removes targets while repairing dependent structures.The five outcomes are support, supersede, scope-coexist, retain-conflict, and escalate-for-review.Source: B. State Transitions and Lifecycle… · p. 8
  • The lifecycle acquires multimodal observations, validates them before writing, then consolidates, reconciles, or forgets state.Admitted evidence is appended with a timestamp and provenance.Source: B. State Transitions and Lifecycle… · p. 8
  • Consolidation selects abstractions by expected future-task utility against storage or retrieval cost, while provenance remains an invariant across operations.Episodic details may remain valuable for tail queries even when a semantic summary serves common ones.Source: B. State Transitions and Lifecycle… · p. 8

C. Architecture and Degenerate Regimes

The architecture combines six complementary state components rather than relying on any one store. Zero-state, append-only, and unstructured-summary systems each omit parts of the full specification.

C. Architecture and Degenerate Regimes

  • The six components divide roles across episodic and semantic stores, an associative index, temporal layer, provenance ledger, and authorization state.The episodic store favors fast, high-fidelity writes; the semantic store favors slower, compressed consolidation.Source: C. Architecture and Degenerate Regimes · p. 8
  • The components are complementary: fidelity, generalization, retrieval, temporal validity, traceability, and governance are not interchangeable.The specification derives value from their composition.Source: C. Architecture and Degenerate Regimes · p. 9
  • Zero-state, append-only retrieval, and unstructured summarization each omit required capabilities; no identified published system satisfies the full operational specification.The full specification requires all six state components, six lifecycle operations, and the provenance invariant.Source: C. Architecture and Degenerate Regimes · p. 9

D. Formal Guarantees: Memory Harm and Governed Forgetting

The formalization characterizes how erroneous observations can accumulate into Memory Harm and defines deletion in terms of both removal and retained-task quality. It also identifies unresolved conditions for guaranteeing schema repair.

D. Formal Guarantees: Memory Harm and Governed Forgetting

  • Under independent ingestion steps, the theorem relates erroneous consolidation to the observation error rate and the gate’s conditional false-accept rate, then derives expected Memory Harm under a binary reliance model.The binary model assigns harm when a contaminated state item is retrieved and relied upon.Source: D. Formal Guarantees: Memory Harm and… · p. 9
  • A useful validation gate has a false-accept rate far below its true-accept rate; the theorem quantifies residual risk from false accepts.No independence between error status and gate decision within one step is assumed.Source: D. Formal Guarantees: Memory Harm and… · p. 9
  • Forgetting Fidelity measures reduction in target leakage, while Coverage Retention bounds loss in quality on un-targeted evidence.Coverage Retention of 1 means deletion left retained knowledge untouched; values below 1 indicate retention loss.Source: D. Formal Guarantees: Memory Harm and… · p. 10
  • Deleting a source and its index pointer can leave information in a consolidated schema, so repair may need to account for dependencies in other state components.The stated schema-repair sufficiency condition is a conjecture requiring restrictive conditions on the schema, other components, and deterministic stable reconsolidation.Source: D. Formal Guarantees: Memory Harm and… · p. 10
  • The schema-repair sufficiency conditions do not hold for a general consolidated store, and verifying them for a specific implementation remains open.The deletion hierarchy distinguishes physical removal, representation invalidation, schema repair, parametric unlearning, and certified non-recoverability; one level does not imply the next.Source: D. Formal Guarantees: Memory Harm and… · p. 10

E. Substrate Composition

No single storage substrate supplies all properties required for persistent document state. The specification therefore leaves substrate choice open while defining requirements for evaluating combinations.

E. Substrate Composition

  • Vector stores, knowledge graphs, symbolic document stores, and parameter adaptation each trade off retrieval, relations, source fidelity, generalization, attribution, or deletion.The text notes that no single substrate provides all required properties.Source: E. Substrate Composition · p. 10
  • Hybrid combinations remain eligible if they preserve the six state roles, support six lifecycle operations, and maintain the provenance invariant.The specification does not privilege a particular substrate.Source: E. Substrate Composition · p. 11
  • The benchmark harness evaluates implementations against the specification without privileging a particular substrate.Source: E. Substrate Composition · p. 11

IV. THE STATEFULNESS AUDIT AND LONGITUDINAL BENCHMARK

The audit finds that existing Document AI benchmarks test within-episode capabilities but not state evolution across sessions. The proposed harness makes persistence measurable through required stream properties and a counterfactual comparison of warm, retrieval-only, and cold systems.

A. The Statefulness Audit

  • The audit codes benchmark suites against eight criteria, including sequential presentation, temporal revision, provenance persistence, session persistence, and answerability.The search corpus spans 2017 through 16 September 2026.Source: A. The Statefulness Audit · p. 11
  • The audit’s coding rules distinguish required capabilities from partial neighboring capabilities and treat provenance as persistence across episodes, not single-episode grounding.All audited benchmarks score no on provenance persistence under that definition.Source: A. The Statefulness Audit · p. 11
  • No audited benchmark tests cross-session dependency, temporal revision, consolidation benefit, or verifiable forgetting, despite covering major Document AI task families.The audit coded suites against eight operational criteria and found that all evaluate within-episode perception or reasoning.Source: A. The Statefulness Audit · p. 11

B. The Longitudinal Benchmark Harness

  • The harness is motivated by an evaluation equilibrium in which benchmarks do not reward persistence and therefore do not measure it.The proposed harness makes persistence the independent variable and future performance the dependent variable.Source: B. The Longitudinal Benchmark Harness · p. 12
  • The harness requires sequential sessions, cross-document evidence, evolving content, repeated families, forgetting, provenance queries, session breaks, and abstention.Explicit session boundaries clear context so that only persistent state survives.Source: B. The Longitudinal Benchmark Harness · p. 12
  • At the same future task, the three-arm experiment compares consolidated warm state, retrieval-only access, and a cold baseline to separate consolidation gain from access gain.Experience Gain is the warm-versus-retrieval-only contrast; the warm-versus-cold difference includes both gains.Source: B. The Longitudinal Benchmark Harness · p. 12

C. Counterfactual Evaluation Metrics

The counterfactual metrics assess persistent state across benefit, cost, risk, and governability. They are intended to be reported jointly because optimizing one dimension alone can conceal harms or loss of useful knowledge.

C. Counterfactual Evaluation Metrics

  • Experience Gain measures the consolidation gain, while Cost Efficiency measures compute reduction attributable to prior exposure.Experience Gain may grow with exposure before saturating; Cost Efficiency can matter even when accuracy saturates.Source: C. Counterfactual Evaluation Metrics · p. 12
  • Memory Harm measures performance degradation caused by contaminated or outdated state.The metric captures a risk that persistent state can amplify errors.Source: C. Counterfactual Evaluation Metrics · p. 12
  • Forgetting Fidelity and Coverage Retention measure whether deletion is genuine while unrelated knowledge remains intact, so the five metrics must be reported jointly.Maximizing Experience Gain while ignoring Memory Harm can amplify errors; perfect forgetting with destroyed Coverage Retention cannot support selective remembering.Source: C. Counterfactual Evaluation Metrics · pp. 12–13

D. Statistical Design and Contamination Control

The proposed statistical design accounts for document-family heterogeneity and serial dependence while estimating warm–cold quality differences. It also specifies controlled corruption and stream-level cost accounting, then defines the combined behavior required to demonstrate persistent state.

D. Statistical Design and Contamination Control

  • A mixed-effects model estimates the warm–cold quality contrast underlying Experience Gain while accounting for family-level variation and serial correlation within streams.For binary outcomes, the paper recommends a logistic mixed model; paired bootstrap intervals offer a distribution-free alternative when streams are few.Source: D. Statistical Design and… · p. 13
  • The model’s estimated Experience Gain is the warm-state effect plus an exposure interaction, with exposure modeled as log(1+k) to represent growth that asymptotically saturates.The fixed warm-state effect captures an initial advantage; its interaction with log-exposure captures the exposure-related gain.Source: D. Statistical Design and… · p. 13
  • Memory Harm is estimated by comparing clean and corrupted parallel streams that differ through controlled replacement of observations with plausible incorrect values.The harness minimizes pretraining overlap, records retrieval access, and prefers synthetic document families to isolate stream-induced state.Source: D. Statistical Design and… · p. 13
  • Cumulative Cost Efficiency compares compute across the full stream, including perception, retrieval, reasoning, and memory maintenance; it can be positive even when per-task efficiency is zero.Economic justification is defined for the target stream length when cumulative Cost Efficiency is greater than zero.Source: D. Statistical Design and… · p. 13
  • Persistent state meets the operational definition only when it shows positive Experience Gain and Cost Efficiency, bounded Memory Harm, and high Forgetting Fidelity with Coverage Retention under the eight required properties.The specification evaluates behavior rather than privileging a particular architecture.Source: D. Statistical Design and… · pp. 13–14

V. GOVERNANCE, FAILURE MODES, AND RESEARCH AGENDA

Persistent state can propagate errors, must preserve evidence provenance through abstraction, and must support deletion across multiple persistence mechanisms. The section links these risks to cost trade-offs and identifies schema repair as an unresolved part of verifiable forgetting.

A. Memory Contamination and the Experience-Cost Curve

  • A persistent system can propagate a single error across many future sessions, so Memory Harm evaluation requires controlled corruption and contamination-accumulation analysis.The design challenge is to make memory robust to its own fallibility.Source: A. Memory Contamination and the… · p. 14
  • Recurring document families can shift processing from repeated perception toward schema-conditioned work, but memory is valuable only when cumulative savings exceed state-maintenance costs.The relevant comparison is total stream compute for warm versus cold conditions, not per-document latency alone.Source: A. Memory Contamination and the… · p. 14

B. Recursive Provenance and Bitemporal Supersession

  • Recursive provenance must survive schema abstraction so a consolidated claim can be traced back to the source evidence needed for an audit.The chain links higher-level schemas through evidence to source documents and locations.Source: B. Recursive Provenance and Bitemporal… · p. 14

C. Verifiable Forgetting and the Unlearning Hierarchy

  • The five-level unlearning hierarchy covers raw-file deletion, representation removal, schema repair, parametric unlearning, and certified non-recoverability.Adapting unlearning methods to document-native Level 3 schema repair remains an open algorithmic problem.Source: C. Verifiable Forgetting and the… · p. 14
  • Current Document AI systems operate almost exclusively at Levels 1 and 2, and the formalization showed that skipping Level 3 leaves non-zero information leakage in the semantic store.Source: C. Verifiable Forgetting and the… · p. 15

D. Enterprise Deployment Constraints

Enterprise deployment requires persistent memory that remains efficient, consistent, secure, and governed across users and time. The survey sets out an implementation-neutral research agenda and evaluation protocol, while leaving implementations and empirical validation open.

D. Enterprise Deployment Constraints

  • Enterprise applications span legal, healthcare, financial auditing, and scientific synthesis, while deployment requires efficient, consistent, secure, policy-aware memory across users and time.Requirements include multi-tenant and document-level permissions, protection against corrupted memory, and retention and data-residency controls.Source: D. Enterprise Deployment Constraints · p. 15

E. An Implementation-Neutral Research Agenda

  • The six substrate-neutral research problems cover provenance-preserving consolidation, adaptive evidence acquisition, memory-conditioned parsing, bitemporal supersession, certified unlearning, and policy-aware memory.The agenda asks how to preserve links to source bounding boxes, stop inspection when information gain falls below compute cost, and resolve version conflicts without losing audit trails.Source: E. An Implementation-Neutral Research… · p. 15

F. Limitations

  • The review underrepresents English-, Latin-script, modern documents; its formal specifications are unvalidated, and novel-stream contamination is the harness’s strongest methodological challenge.The survey says credible longitudinal benchmarks should report contamination controls and include private or synthetically generated document families.Source: F. Limitations · p. 15

VI. CONCLUSION

  • The statelessness bottleneck persists through parameter scaling, context extension, and retrieval augmentation because these address working-memory capacity and access, not knowledge accumulation.The conclusion distinguishes storage, retrieval, and consolidation as separate functions.Source: §VI · p. 16
  • The survey formalizes persistent evidence-grounded state as a six-component tuple with lifecycle operations and invariants, derives a contamination-accumulation theorem, and specifies five falsifiable metrics.Unvalidated writes drive state corruption toward certainty over long horizons; a validation gate bounds the accumulation rate, while implementations remain open work.Source: §VI · p. 16
  • The decisive experiment holds the base model and future documents constant while varying valid prior histories, then evaluates five counterfactual metrics under eight mandatory harness properties.The required outcomes are positive Experience Gain and Cost Efficiency, bounded Memory Harm, high Forgetting Fidelity, and Coverage Retention.Source: §VI · p. 16
  • Document AI must move from episodic reading toward accumulating, reconciling, and governing knowledge across time; engineering, experimentation, and benchmarks remain to prove that transition.The paper presents its vocabulary, formalism, and evaluation protocol as groundwork for that work.Source: §VI · p. 16

Related papers

Loading 2609.32041v1…