PaperlayerView exact revision on arXiv

Source-linked AI summary

MemLeak: Cross-User Semantic Leakage in Multi-Tenant AI Agent Memory

Priyanka Mudgal, Kai Zhao, Guilin Zhang, Andy Olsen, Ezekiel Miller, Xu Chu, Aletta Johanna Blanken

arXiv:2610.04195v1cs.AIcs.LG

TL;DR

The paper studies cross-user leakage when multi-tenant agent memories share a vector store, a case not examined by prior single-user memory-trustworthiness work. It formalizes admissibility failure and evaluates it across controlled experiments and retrieval settings, finding leakage and response contamination in pooled deployments.

  • Problem

    Prior memory-trustworthiness work focuses on a user’s own memories, while shared multi-tenant stores expose soft ownership metadata and can retrieve another user’s memories.

    Source: §1 · p. 1

  • Method

    The paper formalizes cross-user admissibility failure and evaluates leakage through six controlled experiments under sparse TF-IDF and dense MiniLM retrieval.

    Source: §1 · p. 2

  • Results

    70–100% incidental leakage under pooled same-team retrieval, active-exploitation score lifts of +0.416 to +0.511, and downstream contamination increases of +1.67 to +4.00 points.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Similarity ranking alone is a poor proxy for admissibility, and strict user partitioning is an effective baseline that eliminates leakage.

    Source: §1 · p. 2

    The controlled proof of concept uses four hand-authored synthetic user pairs and n = 10 queries per condition, so rates are not population-level prevalence estimates for real enterprise deployments.

    Source: §5 · p. 9

Abstract

from arXiv · show

Personal AI agents in enterprise multi-tenant deployments share a common vector store for long-term memory. Shared embedding spaces create a surface for cross-user memory leakage: a user's query can retrieve semantically adjacent memories belonging to another user through ordinary cosine-similarity retrieval, without any exploit. We formalize this as cross-user admissibility failure and evaluate it across six experiments, plus follow-up ablations, under both sparse (TF-IDF) and production-faithful (MiniLM-L6-v2) retrieval. Non-adversarial, incidental leakage reaches 70--100\% under pooled {same-team} retrieval; adversarially crafted memories achieve 90--100\% top-$k$ placement, exceeding weaker keyword-based attacker baselines, with score lifts of $+0.416$ to $+0.511$ under production-faithful dense retrieval (Config B); and end-to-end response contamination reaches 5.00/5 under a production retrieval path and 4.67/5 with Claude Sonnet~4.5, with contaminated responses often scoring as helpful or more helpful than clean ones, a gap validated against human judgment. Among three architectural mitigations, only hard post-retrieval ownership gating consistently restores the clean baseline (1.00/5) across {two generation models, at a measured latency overhead of roughly 1.4~ms per query.

1 Introduction

The paper identifies cross-user memory leakage in shared, multi-tenant agent stores as a threat distinct from prior single-user memory-trustworthiness work. It formalizes the failure and evaluates its severity across controlled experiments and retrieval settings.

1 Introduction

  • Inter-user semantic resonance describes embedding-space proximity pulling one user’s memories into another user’s session without adversarial changes to prompts, code, or model weights.The shared-store design exposes per-user ownership as soft metadata rather than a hard-enforced boundary.Source: §1 · p. 1
  • Prior memory-trustworthiness work considers whether a user’s own memories should influence that user’s task, but does not examine memories reaching another user’s session.The cited work includes Tan et al. [2025] and Wu et al. [2025].Source: §1 · p. 1
  • The study examines pooled stores with soft ownership metadata or intentionally shared collaboration scopes, rather than the trivial case of a fully absent access-control layer.The paper frames its contribution as quantifying residual risk in these common deployment patterns.Source: §1 · p. 2
  • The paper formalizes cross-user admissibility failure and defines leakage metrics, a privacy–utility frontier, an exploratory Resonance Index, and preliminary covert-channel capacity measures.It reports 95% Wilson confidence intervals for n = 10 queries per condition and presents the work as a controlled proof of concept, not a population-level prevalence estimate.Source: §1 · p. 2
  • 70–100% incidental leakage under pooled same-team retrieval, alongside active-exploitation score lifts of +0.416 to +0.511 and downstream contamination increases of +1.67 to +4.00 points.The experiments also report a negative covert-channel-capacity result, bounding the active-exploitation claim rather than establishing a functioning channel.Source: §1 · p. 2

2 Related Work

Prior work covers persistent agent memory, retrieval-augmented generation, and memory trustworthiness within single-user boundaries. The paper distinguishes cross-user semantic resonance from corpus poisoning and prompt injection because ordinary memories can affect another user’s context without malicious instructions.

2 Related Work

  • Prior memory-trustworthiness studies address failure modes within a single user boundary; none of the cited work examines the multi-user case.The passage names Pulipaka et al. [2026], Guo et al. [2026], Tan et al. [2025], and Wu et al. [2025].Source: §2 · p. 2
  • Inter-user semantic resonance needs neither corpus poisoning nor malicious instructions: harm arises from ordinary professional notes being present in another user’s context.PoisonedRAG, Phantom [Chaudhari et al., 2024], and Shafran et al. [2025] attack the retrieval corpus directly; indirect prompt injection embeds malicious instructions in retrieved content.Source: §2 · p. 2
  • The paper evaluates downstream contamination using LLM-as-judge following Zheng et al. [2023] and motivates ownership-aware embeddings with calibrated embedding noise proposed by Feyisetan et al. [2020].The related work also places covert channels in the tradition of Lampson [1973], Denning [1976], and Myers and Liskov [1997].Source: §2 · p. 3

3 Threat Model and Formalization

The threat model distinguishes retrieval from an authorized pool from whether each retrieved memory is admissible for the querying user. It covers pooled or widened retrieval settings and defines passive leakage and active exploitation separately.

3.1 System Model

  • A pooled deployment places users’ memories in one vector index, while a partitioned deployment gives each user a separate index; top-k memories by cosine similarity enter the LLM context.Queries and memories are embedded in the same vector space.Source: §3.1 · p. 3

3.2 Cross-User Admissibility Failure

  • A memory owned by another user is inadmissible regardless of semantic similarity, and a leakage event occurs when such a memory is retrieved for the query.The paper defines cross-user admissibility as requiring both ownership by the querying user and contextual appropriateness.Source: §3.2 · p. 3

3.3 Leakage Probability

  • Leakage probability for a user pair is defined, and the paper hypothesizes that it increases with the cosine similarity between their memory centroids.Each centroid is the mean of that user’s memory embeddings.Source: §3.3 · p. 3

3.4 Threat Scenarios

  • Passive leakage occurs organically through shared vocabulary, roles, or organizational context, without adversarial intent.Active exploitation instead uses crafted memories designed to resemble expected target queries.Source: §3.4 · p. 3

3.5 Deployment Patterns and the ACL Baseline

  • The threat model targets pooled or widened retrieval settings where cross-user memories remain reachable, even though strict user partitioning drives passive leakage to 0%.The settings include under-scoped metadata filters, team- or role-scoped retrieval, shared project namespaces, and enterprise workflows where cross-user content is in the pool but admissibility should depend on more than similarity.Source: §3.5 · p. 3
  • Similarity ranking alone does not restore admissibility within an authorized pool; Figure 1 depicts cross-user retrieval and marks three mitigation intercept points.The figure identifies M1 as metadata filtering, M2 as ownership-aware embeddings, and M3 as post-retrieval ownership gating.Source: §3.5 · p. 3, Figure 1 · p. 4

4 Experiments

Across controlled retrieval and generation experiments, pooled memory stores enable incidental and crafted cross-user leakage, and leaked memories can contaminate useful responses. Hard post-retrieval ownership gating restores the clean contamination baseline, while other tested mitigations are incomplete or costly in utility.

4.2 E1: Existence Proof

  • Under pooled same-team retrieval, leakage is 100% with TF-IDF and 70% with MiniLM, while partitioned retrieval records 0% in both configurations (n = 10).The experiments use hand-authored synthetic fixtures, two retrieval configurations, and 95% Wilson-score intervals; partitioned retrieval is the user-scoped ACL baseline.Source: §4.1 · pp. 4–5, Table 1 · p. 5, §4.2 · p. 5

4.3 E2: Leakage Rate vs. Organizational Distance

  • 10% leakage in the cross-company control contrasts with 70–100% across in-organization tiers, and centroid similarity does not reliably predict leakage risk.The tier comparison uses n = 10 queries; the finer ordering among T0–T2 is not statistically distinguishable, and the table reports leakage rate and centroid similarity by tier.Source: §4.3 · pp. 5–6, Table 3 · p. 6

4.8 E5: Cross-User Privacy–Utility Frontier

  • The cross-company control combines a 0.162 Resonance Index with 0.507 utility, while in-organization tiers show 70–90% XLR and moderate utility.The authors describe the Resonance Index as exploratory and uncalibrated, and treat finer rankings among T0–T2 as suggestive at n = 10 per tier.Source: Table 4 · p. 6, §4.8 · p. 7

4.4 E3: Active Exploitation; 4.5 E3b: Attacker Knowledge and Crafting Baselines

  • 90% crafted-bridging placement under Config B exceeds 10% organic placement and yields a +0.416 score lift over the organic baseline.Across both configurations, crafted memories reach 90–100% placement; the Config B attacker-knowledge ablation rises from 10% organic to 40% domain keywords, 80% per-query keywords, 90% crafted bridging, and 100% verbatim-copy oracle placement.Source: Table 2 · p. 5, §4.5 · p. 6, §4.4 · p. 6

4.6 E4: Downstream Response Contamination

  • 5.00/5 contamination with Gemini 2.5 Flash on the retrieval path is a +4.00 increase over clean; Claude Sonnet 4.5 reaches 4.67/5 on both paths.Leaked memories can provide useful context while violating privacy; human annotation found contamination judging weaker than helpfulness judging, with under-scoring that makes reported contamination a conservative lower bound.Source: Table 5 · p. 7, §4.6 · p. 7

4.7 E0: LLM-Judge / Human Agreement Pilot

  • Human review found mean absolute error of 0.53 for clean controls and helpfulness, versus 0.79 for contamination, driven largely by one under-scored Security response.The authors treat the under-scoring as directionally conservative for the E4/M3 contamination estimates.Source: §4.7 · p. 7

4.9 E6: Covert Channel Capacity (Preliminary, Negative Result)

  • Sequential decoding of an attempted 8-bit payload fails: per-bit errors rise from 0% for bits 1–2 to 100% for bits 5–8, so structured error-correcting encoding is needed.The paper reserves “covert channel” for E3 and E6, distinguishing those from incidental leakage without sender intent; none of its E1–E5 claims depends on E6.Source: §4.9 · pp. 7–8

4.10 Mitigation Evaluation

  • 1.00/5 mean E4 contamination under hard ownership gating matches the clean baseline across both models and memory paths; it also eliminates attacker placement at roughly 1.4 ms latency overhead.Metadata filtering succeeds only when the retrieval scope excludes the attacker; prefix injection reduces T0 leakage from 70% to 50%, while namespace offsets worsen leakage and utility. The suspicion gate still permits 30–40% attacker placement, and Sonnet helpfulness falls to 2.60–3.80/5 with gating.Source: Table 8 · p. 8, §4.10 · pp. 8–9

5 Limitations.

The evaluation is a controlled proof of concept using small, hand-authored samples, so its rates are not population-level prevalence estimates. Several model, evaluation, and infrastructure choices also limit how broadly the findings can be generalized.

5 Limitations.

  • Four hand-authored synthetic user pairs and n = 10 queries per condition make the results proof-of-concept evidence, not population-level prevalence estimates.The fixtures are fixed rather than sampled from production logs, and their representativeness has not been verified.Source: §5 · p. 9
  • Fine-grained condition rankings with overlapping 95% Wilson intervals are suggestive rather than established.The examples include T1/T2 versus T0 in E2 and Resonance Index ordering in E5.Source: §5 · p. 9
  • E4/M3 contamination and helpfulness are judged by the generator’s model family, while a nine-response human pilot found weaker agreement on contamination.The pilot’s single disagreement suggests reported contamination may be a conservative lower bound.Source: §5 · p. 9
  • The study covers embedding-similarity retrieval, two generation/judge models, and proof-of-concept mitigations rather than production deployments.A fuller discussion also addresses covert-channel capacity and infrastructure scope in Appendix A.Source: §5 · p. 9

6 Conclusion

MemLeak studies cross-user leakage in shared vector stores with soft, metadata-based ownership rather than hard partitioning. Its experiments find incidental and crafted-memory leakage, and show that hard post-retrieval ownership gating restores contamination to the clean baseline, with a measured latency overhead.

6 Conclusion

  • 70–100% incidental leakage occurs under pooled same-team retrieval, while crafted memories reach 90–100% top-k placement and score lifts of +0.416 to +0.511.Downstream contamination increases by +1.67 to +4.00 points while responses often remain highly helpful.Source: §6 · p. 9
  • Hard post-retrieval ownership gating restores contamination to the clean baseline of 1.00/5 for both generation models, with roughly 1.4 ms latency overhead per query.Metadata filtering alone is insufficient when retrieval scope includes an attacker; prefix injection only partially reduces leakage.Source: §6 · p. 9
  • The conclusion calls for evaluating retrieval architecture and generation-model selection jointly when designing trustworthy multi-tenant personalization.It also recommends layered defenses at the store, embedding, and post-retrieval layers.Source: §6 · p. 9

A Extended Limitations

The extended limitations constrain the strength and reach of the findings: the evaluation uses small, synthetic samples and narrow model, infrastructure, and retrieval settings. The study also leaves structured covert-channel measurement for future work.

A Extended Limitations

  • Four hand-built user pairs and three contamination scenarios support a controlled proof of concept, not population-level prevalence estimates.With n = 10 queries per condition, fine-grained rankings under overlapping 95% Wilson intervals are suggestive; strong claims are reserved for contrasts with non-overlapping intervals.Source: A Extended Limitations · p. 13
  • Hand-authored memories and queries have not been verified as representative of real enterprise data, vocabulary overlap, or query phrasing.They model plausible sensitive professional information rather than samples from users or production logs.Source: A Extended Limitations · p. 13
  • Self-judging may bias contamination and helpfulness scores; the n = 9 human pilot found weaker agreement on contamination and was too small to fully validate the judge.Cross-model judge–generator pairings remain future work.Source: A Extended Limitations · p. 13
  • The model and infrastructure evaluation is narrow, and the mitigation implementations are proof of concept rather than production deployments.Most experiments use in-memory cosine search; the production pgvector/HNSW latency benchmark uses one over-fetch factor, K = 15.Source: A Extended Limitations · p. 13
  • Sequential payload transmission reaches a 100% bit-error rate at bits 5–8, so channel-capacity measurement needs structured error-correcting encoding beyond positional placement.The paper does not claim a channel with controllable throughput; individual crafted-memory placement is reliable in E3.Source: A Extended Limitations · p. 13
  • The study evaluates embedding-similarity retrieval only, leaving LLM-based memory selection and other retrieval mechanisms empirically untested.Their leakage properties may differ, and the paper identifies larger, more diverse embedding-model evaluations as future work.Source: A Extended Limitations · p. 13

B Evaluation Metrics

This section defines the study’s metrics and deployment scope, then discusses risks, statistical uncertainty, alternative retrieval, and the synthetic evaluation fixtures.

B Evaluation Metrics

  • The Resonance Index is an exploratory heuristic composite, and the core claims instead rest on leakage-rate, top-k placement-rate, and contamination measurements.Its weights are illustrative and were not fit or validated against a ground-truth harm signal.Source: Table 9 · p. 14, B Evaluation Metrics · p. 14

C Broader impacts.

  • The paper frames crafted bridging as a dual-use technique that can steer cross-user retrieval using topical or role knowledge, without query-level or embedding-model access.The authors disclose it alongside hard post-retrieval ownership gating, which eliminates the attack in their evaluation at negligible latency cost.Source: C Broader impacts · p. 14

D Extended Discussion: Deployment Patterns, Statistical Notes, and Alternative Retrieval

  • The threat model covers pooled or scoped stores where cross-user memories remain reachable by design or common misconfiguration, not only deployments with no access control.Examples include under-scoped filters, team- or role-scoped retrieval, shared project namespaces, and scoped enterprise workflows.Source: D Extended Discussion: Deployment… · pp. 14–15
  • At n = 10, the T1/T2 versus T0 leakage-rate ordering and the tier ordering of Resonance Index are suggestive, not statistically established.The cross-company control T3 is clearly separated with low leakage and high utility, while T0–T2 are not reliably distinguishable at this sample size.Source: D Extended Discussion: Deployment… · p. 15
  • Budget Authority reaches 70% organic top-k placement in Config A, with broadly shared procurement vocabulary making attacker memories semantically proximate by coincidence.The passage also notes that the +0.484 Config A score lift understates the advantage relative to near-zero-baseline scenarios.Source: D Extended Discussion: Deployment… · p. 15
  • LLM-based memory selection could reduce reliance on embedding proximity, but the paper does not evaluate it and identifies store-size-dependent latency and token costs plus prompt-injection risk.The additional attack surface comes from stored memory content entering the selection step.Source: D Extended Discussion: Deployment… · p. 15

E Dataset Construction

  • The dataset uses hand-authored memories and queries across four domains, held fixed across organizational-distance tiers to isolate distance from content variation.The supplementary material includes the fixtures and LLM-judge meta-prompts; the rubric maps normalized scores to the paper’s 1–5 scale via paper = 4s + 1.Source: E Dataset Construction · pp. 15–16

F E3b: Full Attacker-Knowledge Ablation

The full attacker-knowledge breakdown is described qualitatively across three scenarios, while the reported Section 4.5 ablation focuses on Salary Negotiation.

F E3b: Full Attacker-Knowledge Ablation

  • Across Salary, Security, and Budget, crafted bridging closes most of the remaining gap to the verbatim-copy oracle, after organic and keyword baselines close part of it.Organic placement is highest for Budget Authority; the full scenario-by-baseline-by-configuration table is in the supplementary material.Source: F E3b: Full Attacker-Knowledge Ablation · p. 16
  • Domain-keyword and per-query-keyword baselines each close part of the gap between organic placement and crafted bridging across the three scenarios.Source: F E3b: Full Attacker-Knowledge Ablation · p. 16
  • Table 10 presents the E3b attacker-knowledge ablation for Salary Negotiation, reporting top-k placement rate with 95% Wilson confidence intervals and mean-score lift over the organic baseline under Config B.Source: Table 10 · p. 16

G E0: Full Judge/Human Agreement Protocol

The agreement pilot reports stronger judge alignment on clean controls and helpfulness than contamination, while the covert-channel breakdown finds no demonstrated controllable throughput.

G E0: Full Judge/Human Agreement Protocol

  • Two authors independently labeled nine synthetic E4 responses across Salary, Security, and PIP, covering clean contamination, leaked contamination, and leaked helpfulness conditions.They applied the same rubric used by the LLM judge.Source: G E0: Full Judge/Human Agreement… · p. 16, Table 11 · p. 16
  • Mean absolute error was 0.53 across conditions for helpfulness but 0.79 for contamination, with the latter driven almost entirely by one Security-scenario miss.The judge scored an unpatched-CVE detail as generic security advice (1.0), while both human annotators scored it as explicit disclosure (5.0).Source: G E0: Full Judge/Human Agreement… · p. 16, Table 11 · p. 16
  • The authors treat contamination scores as a lower bound on true leakage severity and identify further judge-prompt iteration as needed.They report the pilot as a limitation rather than a validation.Source: G E0: Full Judge/Human Agreement… · p. 17

H E6: Covert Channel Capacity, Full Bit-Position Breakdown

  • Later payload bits are displaced by legitimate same-user memories or rejected by the similarity cutoff, so crafted placement does not demonstrate controllable channel throughput.Sequential multi-bit transmission would require structured, error-correcting encoding beyond simple positional placement; Table 12’s caption describes the bit-position decode-failure breakdown.Source: H E6: Covert Channel Capacity, Full… · p. 17, Table 12 · p. 17

I M2: Full Ownership-Aware Embedding Results

The passage points to the full per-tier table for the ownership-aware embedding results but provides no result values.

I M2: Full Ownership-Aware Embedding Results

J Mitigation Latency Benchmark, Full Table

The latency benchmark measures retrieval over a 10,000-memory index, while the embedding-model spot-check finds that incidental leakage does not diminish with model scale or dimensionality across the models tested.

J Mitigation Latency Benchmark, Full Table

  • The latency and contamination benchmark uses 1,000 query cycles, a 10,000-memory pgvector HNSW index, and over-fetch K=15; M1’s safe-row result is for partitioned stores and does not eliminate active-failure contamination.M1’s latency figure is condition-independent.Source: Table 14 · p. 17, J Mitigation Latency Benchmark, Full… · p. 17

K Embedding-Model Scaling Spot-Check, Full Table

Loading 2610.04195v1…