PaperlayerView exact revision on arXiv

Source-linked AI summary

Lineage-Aware Memory Governance: A Derivation-Gated Framework for Privacy-Preserving Column-Level Access Control in Enterprise AI Agents

Venkata M Sangaraju, Sudhir Vissa

arXiv:2610.07258v1cs.CRcs.AIcs.CLcs.LG

TL;DR

Shared agent memory can leak results derived from columns a requester cannot access, while teams can compute same-named metrics using different logic. The paper adds lineage graphs and derivation checks to cached analytical results. In its completeness study, measured leakage reaches zero at schema-dependent lineage-completeness thresholds, with the guarantee limited by lineage completeness and the threat model.

Abstract

from arXiv · show

Enterprise AI agents that share a memory store face two unaddressed risks: sensitive data can leak through legitimately computed results the requester could not derive, and departments can silently compute a same-named key performance indicator (KPI) through conflicting logic. Existing agent-memory systems (e.g., MemGPT, Zep, A-MEM) gate retrieval by content, ownership, and role, not derivation, missing a cached insight that embeds a forbidden column. We introduce the Analytical Memory Unit (AMU), a memory schema that attaches a full derivation (lineage) graph to every cached result, gated by a retrieval policy that serves a hit only when the requester is authorised for every column touched. Provided lineage recording is complete, we prove by construction that the policy blocks retrieval of results derived from a sensitive column outside the requester's permissions, at O(n) worst case -- a conditional design guarantee, not an empirical claim, that excludes derived features encoding sensitive information without naming their source. Eliminating measured leakage required 75-90% recorded lineage completeness, so we treat 90% as a conservative deployment target. Across six experiments, lineage-gated retrieval removes the 18.8-25.5% cross-department leakage naive content-gated memory suffers, keeping 81.5-82.6% of memory reuse at 13.8 microsecond worst-case overhead. A real-agent proof-of-concept with LLM-generated SQL is consistent with the guarantee: zero leaks over 9 round-trips, two conflicts caught automatically -- though a feasibility demonstration, not evidence of production viability. This offers a practical governance layer for shared agent memory, complementing source-layer access control and supporting EU AI Act compliance.

I. INTRODUCTION

Shared agent memory can serve a result computed from columns the requester cannot access, because existing controls gate stored content rather than its derivation. The paper introduces derivation-aware memory governance and situates it against related memory, security, and metric-governance work.

I. INTRODUCTION

  • Finance’s customer segment can reach Marketing despite its derivation touching income, because the stored result has no sensitivity tag and content-gated safeguards do not expose the violation.The violation is visible in how the result was computed, not in its stored value.Source: §I · p. 1

A. MOTIVATION AND PRIOR GAP

  • Existing memory systems govern entries by content, ownership, or role rather than source-column derivation, leaving cached analytical results outside the reach of source-layer controls.Prior work also identifies same-named KPIs computed with different logic as a semantic-drift problem; the cited comparison concerns documented design features, not independently verified performance.Source: A. MOTIVATION AND PRIOR GAP · p. 2, §II · pp. 2–3

B. CONTRIBUTIONS

  • The contributions include an AMU schema with a derivation graph, lineage-gated retrieval, a conditional safety guarantee, a threat model, and six experiments.The guarantee depends on complete lineage recording and does not cover inference through undeclared derived features.Source: B. CONTRIBUTIONS · p. 2

II. RELATED WORK

  • Related work spans conversational memory, governed and multi-principal memory, column security and provenance, and multi-agent privacy, but does not provide AMU’s column-level derivation gate for cached analytical results.Collaborative Memory checks fragment-level permissions, while GateMem evaluates access-control utility without testing column-level lineage gating specifically.Source: §II · pp. 2–3

A. THE ANALYTICAL MEMORY UNIT

An Analytical Memory Unit (AMU) stores a computed metric together with the information needed to inspect how it was produced. Its lineage graph makes the derivation explicit for retrieval-time checks.

A. THE ANALYTICAL MEMORY UNIT

  • An AMU is a tuple containing a metric name, computed value, producing department, epoch, and lineage graph.Each lineage step records a table, the columns accessed, and a filter-logic string.Source: A. THE ANALYTICAL MEMORY UNIT · p. 3
  • Finance’s churn_rate example shows why derivation matters: joining three source tables touches income, making the cached result unsafe for departments without income access.The value 0.114 alone reveals nothing about that sensitive input.Source: Figure 1 · p. 4
  • The lineage graph makes a result’s derivation explicit and machine-checkable at retrieval, unlike the result value alone.The schema also derives and caches a sensitive-column set and a definition hash that fingerprints how the metric was computed.Source: A. THE ANALYTICAL MEMORY UNIT · pp. 3–4

B. LINEAGE-GATED RETRIEVAL AND CONFLICT DETECTION

Lineage-gated retrieval checks a requester’s permitted columns before returning a cached result, while a separate write operation flags cross-department definition conflicts. The retrieval scan favors the freshest safe candidate and has linear worst-case complexity in the number of candidates.

B. LINEAGE-GATED RETRIEVAL AND CONFLICT DETECTION

  • Retrieval scans a metric’s AMUs most-recent-first and returns the first candidate whose sensitive-column set is contained in the requester’s permissions.If no candidate is safe, it stores and returns the caller-supplied, already-in-scope fresh computation; the worst case is O(n) when the sensitive-column set is a small constant.Source: B. LINEAGE-GATED RETRIEVAL AND… · pp. 4–5
  • The write operation compares definition hashes across departments and surfaces a conflict when they differ, but still stores the new AMU.Hash comparison is O(1) per pair.Source: B. LINEAGE-GATED RETRIEVAL AND… · pp. 4–5
  • A rejected candidate is skipped rather than returned, and the algorithm continues searching for a safe stored value.The gate checks the cached sensitive-column set against the requester’s permitted-column set.Source: B. LINEAGE-GATED RETRIEVAL AND… · p. 4

C. IMPLEMENTATION CONSIDERATIONS

The framework couples a conditional column-level safety guarantee with implementation controls for lineage capture, fail-closed behavior, and policy changes.

C. IMPLEMENTATION CONSIDERATIONS

  • Cursor-level SQL interception can capture lineage without agent cooperation; extraction failures and unavailable permissions default to denial.The epoch field supports TTL eviction and lazy recomputation of cached lineage when the sensitive-column registry or department permissions change.Source: C. IMPLEMENTATION CONSIDERATIONS · p. 5

IV. FORMAL ANALYSIS

  • Under complete lineage recording, Algorithm 1 returns no result derived from a sensitive column outside the requesting department’s permissions.The proof covers both memory-served results and fresh fallback computations; unsafe candidates are skipped, and stale lineage after policy updates must be recomputed to restore the guarantee.Source: §IV · p. 5
  • Even if the store contains unsafe AMUs, the gate skips them regardless of their number or position.The theorem’s proof checks both possible return paths: a permitted memory hit or a caller-supplied fresh computation within the department’s permissions.Source: §IV · p. 5
  • A changed sensitive-column registry can leave cached lineage tags stale until they are recomputed against the new policy version.The guarantee is restored for subsequent retrievals once tags for AMUs predating the change are recomputed.Source: §IV · p. 5

A. COMPLEXITY AND STORAGE

Retrieval scans candidates most-recent-first to return the freshest safe value, trading a linear worst-case bound for a safety check and recency preference.

A. COMPLEXITY AND STORAGE

  • 13.82 µs is the worst-case retrieval latency at n=50; with n ≤5, the observed worst case is 1.74 µs.The expected-case latency is constant regardless of store size, and the realistic-store paths remain below 2 µs.Source: A. COMPLEXITY AND STORAGE · pp. 5–6, Table 3 · p. 6
  • O(n) worst-case retrieval preserves most-recent-first selection; a (metric, S(a)) index could reach O(1) but would add maintenance complexity and lose that preference.The authors recommend TTL eviction because a poorly bounded policy could allow candidate counts to grow.Source: A. COMPLEXITY AND STORAGE · p. 6
  • 18.8–25.5% structural leakage is eliminated at a 13–14 percentage-point reuse cost, while worst-case overhead is at most 13.82 µs.The comparison is against the O(1) dictionary lookup used by prior systems.Source: A. COMPLEXITY AND STORAGE · p. 6
  • An AMU adds lineage tuples, a filter string, and a 12-character hash, totaling roughly 4–8× a bare value entry.For a typical 2–4 table join, this is 200–600 bytes, smaller than the SQL query most systems already persist.Source: A. COMPLEXITY AND STORAGE · p. 6

V. THREAT MODEL

The threat model centers on sensitive-column leakage and conflicting KPI definitions, while identifying inference and multi-request risks that a single memory-layer gate cannot fully address.

V. THREAT MODEL

  • The gate directly addresses T1, retrieval of results derived from forbidden sensitive columns, while the definition-hash detector surfaces T2, conflicting same-named KPI definitions.The model assumes departments have legitimate business needs but may request metrics whose derivation paths exceed their column permissions.Source: §V · p. 6
  • A derived feature that encodes sensitive information without naming its source can evade the gate unless its dependency is recorded in lineage.The threat summary distinguishes direct mitigation from partial mitigation and threats requiring external controls.Source: §V · p. 6, Table 4 · p. 7
  • Single-retrieval gating does not prevent cross-metric triangulation or out-of-band sharing by departments with disjoint permissions.The paper recommends pairing AMU with organizational usage auditing and differential-privacy controls; other gaps require cooperation from feature engineering or infrastructure controls.Source: §V · pp. 6–7

VI. EXPERIMENTS

Across synthetic and TPC-H schemas, lineage-aware gating addresses cross-department leakage while reducing memory reuse; the reported zero-leak outcome is a formal guarantee, not an empirical discovery.

VI. EXPERIMENTS

  • The experiments used a synthetic five-table schema and an eight-table TPC-H schema, comparing No Memory, Naive Shared Memory, and Lineage-Aware systems over 30 seeds.The simulator isolates retrieval and conflict detection from LLM inference, network latency, and query-optimiser effects.Source: A. EXPERIMENTS 1–2: SYNTHETIC AND… · p. 7, §VI · p. 7

A. EXPERIMENTS 1–2: SYNTHETIC AND TPC-H SCHEMAS

  • The lineage-aware system’s 0% leak rate is a Theorem 1 policy guarantee, not an empirical discovery; the naive baseline’s leak rate was structurally non-zero.The authors report a 13–14 percentage-point reuse reduction, roughly 13 additional query executions per 100 requests, with cost appearing as compute rather than governance latency.Source: A. EXPERIMENTS 1–2: SYNTHETIC AND… · p. 7
  • Naive leak rates rose from 18.8% to 25.5% across schemas, while lineage-aware gating eliminated leakage in both configurations.The increase coincided with a larger sensitive-column registry and denser joins, though the authors note this does not prove a monotonic relationship.Source: A. EXPERIMENTS 1–2: SYNTHETIC AND… · p. 7, Table 6 · p. 7, Figure 3 · p. 8
  • A dedicated 2×2×2 sweep over step count, registry size, and sensitive-variant proportion is identified as the next test of the proposed scaling relationship.The authors describe the current scaling interpretation as consistent with a monotonic relationship, not proof of one.Source: A. EXPERIMENTS 1–2: SYNTHETIC AND… · pp. 7–8

B. EXPERIMENT 3: LINEAGE-COMPLETENESS DEGRADATION

The degradation study finds that lineage completeness determines measured leak protection, with complex TPC-H joins requiring a higher completeness level than the synthetic schema.

B. EXPERIMENT 3: LINEAGE-COMPLETENESS DEGRADATION

  • Completeness ρ is the fraction of each step’s columns correctly reported; a shadow store compares truncated lineage with ground-truth sensitivity to measure true leakage.The study treats under-reporting from bugs or opaque query plans as a practical concern.Source: B. EXPERIMENT 3: LINEAGE-COMPLETENESS… · p. 8
  • At ρ=0.50, measured leakage was 16.3 ± 3.3% for synthetic and 10.3 ± 4.7% for TPC-H; it reached 0% by ρ≥0.75 and ρ≥0.90, respectively.The authors recommend ρ≥0.9 as a minimum target.Source: B. EXPERIMENT 3: LINEAGE-COMPLETENESS… · p. 8
  • TPC-H requires higher completeness because sensitive columns are spread across more join steps, making join depth a sensitivity multiplier on the completeness requirement.The authors favor automatic lineage extraction over agent self-reporting because self-reporting is hard to audit and degrades unpredictably on complex multihop joins.Source: B. EXPERIMENT 3: LINEAGE-COMPLETENESS… · p. 8

C. EXPERIMENT 4: EXTENDED FUZZY CONFLICT-DETECTION STUDY

The extended conflict study exposes a blind spot in column-graph detection, while runtime measurements show linear worst-case retrieval scaling with low measured overhead on the tested platform.

C. EXPERIMENT 4: EXTENDED FUZZY CONFLICT-DETECTION STUDY

  • The extended evaluation tested D1, D2, and D3 on 43 hand-labelled lineage pairs spanning True Conflicts, Threshold Variants, Logic-Operator changes, and Column-Subset cases.The categories contained 15, 15, 8, and 5 pairs, respectively.Source: C. EXPERIMENT 4: EXTENDED FUZZY… · p. 8
  • D3 missed all 8 Logic-Operator pairs in the 43-pair evaluation, with F1 falling from 1.00 on the original 10 pairs to 0.833.D3 ignores filter-string variation; D1 caught all 8 Logic-Operator pairs but had precision 0.651, false-alerting on every threshold recalibration.Source: C. EXPERIMENT 4: EXTENDED FUZZY… · p. 8

D. EXPERIMENT 5: EMPIRICAL RUNTIME ANALYSIS

  • Runtime operations were measured over 2,000 iterations at five store sizes in a single-threaded containerised Python environment.The setup used a 4-core Apple Silicon virtual machine without GPU, SIMD intrinsics, or JIT acceleration.Source: D. EXPERIMENT 5: EMPIRICAL RUNTIME… · p. 8
  • Worst-case retrieval scaled from 0.66 µs at n=1 to 13.82 µs at n=50, while best-case retrieval and early-exit conflict detection stayed flat.Measurements used one platform; the absolute microsecond values are platform-dependent, and cross-platform confirmation remains a reproducibility item.Source: D. EXPERIMENT 5: EMPIRICAL RUNTIME… · p. 9

E. EXPERIMENT 6: REAL-AGENT INTEGRATION

The real-agent integration observed zero sensitive-column leaks and two correctly detected metric conflicts across nine cross-department round-trips. The broader experiments characterize a reuse trade-off and show that automatic SQL lineage extraction is feasible in the tested setting, with important completeness and scale limits.

E. EXPERIMENT 6: REAL-AGENT INTEGRATION

  • SQLGlot extracted executed-query tables and columns from SQL abstract syntax trees without agent-declared lineage, in a four-department Northwind proof-of-concept.The integration tested safe reuse, blocked conflicts, and reuse of a metric with no sensitive columns.Source: E. EXPERIMENT 6: REAL-AGENT INTEGRATION · p. 9
  • Four requests were served from cache and five ran fresh SQL; the reported cache-hit latency was 0.009 ms.The authors describe this as a nine-round-trip feasibility demonstration, not a large-scale trial.Source: E. EXPERIMENT 6: REAL-AGENT INTEGRATION · p. 9

A. ENTERPRISE VALUE AND WHAT THE EXPERIMENTS ESTABLISH

B. LIMITATIONS AND FUTURE WORK

  • The reported naive leak rates reflect the tested workload mixes, while inference attacks through derived features and the small fuzzy dataset remain limitations.Runtime constants were measured on one hardware and operating-system configuration.Source: B. LIMITATIONS AND FUTURE WORK · p. 9
  • Unexpanded views and opaque stored procedures can hide sensitive columns from static extraction; deeply nested aliases can also defeat resolution, with fail-closed behavior favoring over-blocking.The authors identify catalog-aware view and procedure expansion as future work.Source: B. LIMITATIONS AND FUTURE WORK · pp. 9–10
  • Future work prioritizes adversarial lineage tests, non-SQL and catalog-aware extraction, production-scale trials, and evaluation of D4 on a larger analytics corpus.Source: B. LIMITATIONS AND FUTURE WORK · p. 10

C. CONCLUSION

The conclusion presents derivation-aware governance as a complement to controls over memory ownership and access. Its guarantee is scoped to named, logged columns under reasonably complete lineage, and the implementation is not yet validated at production scale.

C. CONCLUSION

  • Within named, logged columns and reasonably complete lineage, AMU adds 4–8× storage and at most 13.8 µs retrieval latency, with a formal safety guarantee.The conclusion characterizes the real-agent integration as preliminary feasibility evidence, not production-scale viability.Source: C. CONCLUSION · p. 10
  • Derivation-gated governance asks whether a requester could have produced a result themselves, beyond asking who owns the memory.Source: C. CONCLUSION · p. 10

VIII. REPRODUCIBILITY, DATA, AND ETHICS

  • The reference implementation reproduces the paper’s tables and figures, and the authors report using generated, benchmark, and public fictional data rather than proprietary or personally identifiable data.They report no human subjects or data requiring ethical review.Source: §VIII · p. 10
Loading 2610.07258v1…