PaperlayerView exact revision on arXiv

Source-linked AI summary

Observable Channels, Not Just Storage: Evaluating Privacy Leakage in LLM Agent Pipelines

Tao Huang, Chen Hou, Guosen Wu, Jiayang Meng

arXiv:2603.22751v2cs.CR

TL;DR

Privacy leakage in LLM agents is studied across separate storage and execution settings, limiting comparison of when hidden dependence becomes recoverable. CIPL provides a shared channel-oriented measurement interface and finds distinct risk patterns across memory, retrieval, and tool targets, with leakage shaped by channel conditions and not captured by exact-match extraction alone.

  • Problem

    Studies of memory, retrieval, and tool-mediated leakage are often isolated, making it difficult to compare when sensitive internal dependence becomes externally recoverable across agent pipelines.

    Source: §1 · pp. 1–2

  • Method

    CIPL represents targets through sensitive source, selection, assembly, execution, observation, and extraction stages and measures leakage under a shared protocol while preserving target-specific semantics.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Memory leakage is near-saturated, retrieval-mediated leakage is often frequent but partial, and tool-mediated leakage is shaped by observation surfaces and provider behavior; cleaned weak controls suppress leakage and semantic analysis reveals risk beyond exact-match extraction.

    Source: §8 · p. 14

  • Takeaways & Limitations

    Privacy evaluation should distinguish internal exposure, exactly recoverable leakage, and semantically recoverable leakage because attacker-useful disclosure can fall short of verbatim full-set extraction.

    Source: §5 · p. 12

    The controlled target set isolates channel realization rather than reproducing the full complexity of deployed retrieval and tool stacks, so it does not measure all real-world ecosystem privacy risk.

    Source: §6 · p. 13

Abstract

from arXiv · show

Privacy leakage in LLM agents is often studied through individual storage or execution components, such as memory modules, retrieval pipelines, or tool-mediated artifacts. However, these settings are typically analyzed in isolation, making it difficult to compare how private internal dependence becomes externally recoverable across heterogeneous agent pipelines. In this paper, we present CIPL (Channel Inversion for Privacy Leakage) as a unified channel-oriented measurement interface for evaluating privacy leakage in LLM agent pipelines. Rather than claiming a universally strongest attack recipe, CIPL provides a shared way to represent a target through its sensitive source, selection, assembly, execution, observation, and extraction stages, and to measure how internal exposure is transformed into attacker-recoverable leakage under a common protocol. Using memory-based, retrieval-mediated, and tool-mediated instantiations under this shared interface, we identify a distinct cross-target risk picture. Memory behaves as a near-saturated high-risk special case, while beyond-memory leakage exhibits a different regime: retrieval-mediated targets show frequent but often incomplete leakage, and tool-mediated targets are strongly shaped by the exposed observation surface and provider behavior. We further show that leakage is governed by channel conditions rather than by a universally dominant recipe: cleaned weak controls sharply suppress leakage, and semantic annotation reveals attacker-useful leakage beyond exact-match extraction. Together, these findings suggest that privacy risk in LLM agent pipelines is better understood through \emph{observable channels}, not just storage components. More broadly, our results motivate channel-oriented privacy evaluation as a necessary complement to component-local or exact-only analyses.

1 Introduction

Privacy risks in LLM agents span memory, retrieval, and tool workflows, but their separate evaluation makes it hard to compare when hidden dependence becomes recoverable. CIPL frames leakage around observable channels and finds distinct risk patterns across targets.

1 Introduction

  • Prior studies identify privacy risks in memory, retrieval-augmented generation, and tool workflows, but target-specific attacks, output surfaces, and reporting make their results difficult to compare.The broader question is when sensitive content used internally becomes externally recoverable.Source: §1 · pp. 1–2
  • CIPL represents targets through sensitive source, selection, assembly, execution, observation, and extraction, enabling shared-protocol comparison while preserving target-specific execution semantics.It is a measurement interface, not a claim that one attack recipe is universally strongest or that heterogeneous systems share one architecture.Source: §1 · p. 2
  • The paper frames leakage around visible channels: internal exposure may become partially, indirectly, or semantically recoverable rather than appearing as a verbatim dump.This separates sensitive content used in computation from what an attacker can recover.Source: §1 · p. 2
  • Memory is near-saturated, while retrieval-mediated leakage is often frequent but partial and tool-mediated leakage depends strongly on observation surfaces and provider behavior.Leakage also varies with prompt alignment and channel realization; cleaned weak controls sharply suppress it, and exact-match extraction can miss attacker-useful semantic leakage.Source: §1 · pp. 2–3
  • The paper’s central interpretation is that agent privacy risk should be understood through observable channels, not storage components alone.Memory is presented as a strong special case of a broader problem: externalizing hidden dependence through visible artifacts.Source: §1 · p. 2

2 Problem Formulation

The formulation asks when sensitive content selected and used inside an agent pipeline becomes observable to an attacker. It models the system as stages from sensitive source through observation, distinguishing internal exposure from externally recoverable leakage.

2 Problem Formulation

  • The pipeline selects sensitive content from a source, assembles it into context or an action-conditioned representation, executes computation, and exposes an attacker-visible artifact.Observation may be free-form text, structured evidence, tool-call arguments, tool outputs, or execution traces.Source: §2 · p. 3
  • The formulation applies beyond memory: memory extraction is a special case, while any internally consumed sensitive content that influences an attacker-visible channel fits the broader setting.Source: §2 · p. 3
  • Leakage is measured over target-specific normalized units, with canonicalization and observation-channel extraction distinguishing selected sensitive units from units recoverable to the attacker.The unit may be a memory record, retrieved-document identifier, snippet, or structured field.Source: §2 · p. 3
  • Internal exposure concerns sensitive units selected to influence computation, whereas external leakage concerns which units become recoverable from the observation channel.Selected content may remain hidden, be partially revealed, or become only a weak or indirect signal.Source: §2 · p. 3
  • The attacker seeks measurable exposure of private internal dependence through visible channels, rather than arbitrary model misbehavior, and interacts only through user-visible inputs.The formulation distinguishes knowledge levels, including possible limited inference about selection behavior, while keeping execution black-box.Source: §2 · pp. 3–4

3 Method

CIPL treats agent privacy leakage as a shared measurement problem: it asks when sensitive internal dependence becomes recoverable through observable channels. Its interface compares heterogeneous pipelines without claiming a universally strongest attack recipe.

3.1 CIPL as a Channel-Oriented Measurement Interface

  • CIPL measures whether sensitive content consumed inside an agent pipeline becomes externally recoverable through an observable channel.It makes heterogeneous leakage settings comparable while preserving target-specific execution semantics.Source: §3.1 · p. 4

3.2 A Shared Target Signature for Agentic Leakage

  • A shared target signature tracks the sensitive source, selection, assembly, execution, observation, and extraction stages.The signature provides a common analysis interface without assuming systems have identical structures or attack difficulty.Source: §3.2 · p. 4

3.3 Attack-Specification Dimensions under CIPL

  • CIPL represents attack choices through a locator, aligner, and diversification policy rather than a universally strongest prompt recipe.These dimensions respectively target sensitive content, match it to an observable channel, and distribute attacks to enlarge leaked-unit coverage.Source: §3.3 · p. 5

3.4 Interpreting Locator, Aligner, and Diversification

  • The locator targets the relevant internal object, the aligner matches its observation surface, and diversification seeks broader coverage through varied selections.Diversification can vary phrasing and task framing, or use knowledge of selection properties such as lexical sensitivity or semantic similarity.Source: §3.4 · p. 5

3.5 What CIPL Is and Is Not

  • CIPL enables common measurement across heterogeneous systems, but is neither a universally optimal attack construction nor a substitute for target-specific optimization.Its value is making cross-target patterns and boundary conditions explicit.Source: §3.5 · pp. 5–6

3.6 Recovering Memory Extraction as a Special Case

  • Classical memory extraction is a special case in which retrieved memory records are assembled into context and sought in answers or action-visible outputs.CIPL retains the shared evaluation abstraction while treating memory as one target-specific realization.Source: §3.6 · p. 6

3.7 Cross-Target Instantiation

  • The shared interface applies across memory, retrieval, and tool targets, whose observation channels include answers, evidence, arguments, and echoed returns.Their signature realizations differ while the measurement interface remains shared, enabling comparison across architectures.Source: §3.7 · p. 6

4 Experiments

The experiments apply a shared CIPL protocol to memory-, retrieval-, and tool-mediated targets and examine how channel conditions shape leakage. Memory is near-saturated, while beyond-memory leakage is often partial or dependent on observation surface and provider; controls and semantic annotation further qualify the results.

4.1 Evaluation Targets

  • The evaluation covers memory_ehr and memory_rap, controlled rag_ctrl retrieval, and tool_ctrl modes exposing tool-call arguments or echoed returns.The attack budget, metric vocabulary, and reporting format are shared across the four targets.Source: §4.1 · p. 7

4.2 Unified Experimental Setup and Metrics

  • The shared protocol uses 30 queries, one retry, five seeds, and reports internal exposure, external leakage, extraction rates, and execution errors.The common metrics enable comparison across target-specific leakage units without claiming equal difficulty or granularity.Source: §4.2 · pp. 7–8

4.3 Risk Regimes Across Observable Leakage Surfaces

  • Memory targets saturate, retrieval-mediated leakage is frequent but often incomplete, and tool-mediated leakage varies with observation surface and provider.For rag_ctrl, MiniMax-M2.5 has CER = 0.2133 and AER = 0.9533, while qwen3.5-plus, DeepSeek, and GPT-4o have CER = 0.0 and AER = 0.8000.Source: §4.3 · pp. 8–9, Figure 1 · p. 9

4.4 Leakage Realization Depends on Channel, Alignment, and Provider Behavior

  • Cleaned weak controls sharply suppress tool leakage, while increasing rag_ctrl retrieval depth does not monotonically increase leakage or complete extraction.At k = 4, MiniMax-M2.5 has AER = 1.0000 but CER = 0.0556; MiniMax-M2.7 has AER = 0.9889 and CER = 0.0222. Leakage strength also depends on prompt alignment, channel design, and provider behavior, not a universally dominant recipe.Source: §4.4 · pp. 9–11, Figure 2 · p. 11

4.5 Exact Extraction Underestimates Privacy Risk

  • Semantic annotation finds attacker-useful leakage beyond exact matching: 12 of 200 samples have no exact recovered unit but contain semantically recoverable sensitive information.The study reports semantic_AER = 0.5000 and semantic_CER = 0.4400, supporting the conclusion that exact-only evaluation can undercount risk.Source: §4.5 · pp. 11–12, Table 3 · p. 12

5 Discussion

The discussion recasts memory as a high-risk special case rather than the boundary of privacy leakage, with retrieval and tool channels occupying distinct regimes. It argues that leakage depends on channel conditions and that evaluation should consider both exact and semantic recoverability.

5 Discussion

  • Retrieval-mediated leakage is often frequent but incomplete, while tool-mediated leakage depends more strongly on the exposed observation surface and provider realization.These regimes should not be treated as memory extraction under different names.Source: §5 · p. 12
  • Exact-only extraction can understate attacker-useful risk when private information is disclosed partially, indirectly, or semantically.The discussion points to the gap between CER and AER, alongside semantic annotation results.Source: §5 · p. 12
  • No single prompt family or locator–aligner–diversification construction is uniformly dominant across prompts, targets, or providers.The authors recommend enumerating observable surfaces and measuring exact and semantic recoverability under adversarial alignment.Source: §5 · p. 12

6 Limitations

The study’s limitations primarily bound empirical coverage: its controlled targets do not represent the full complexity of deployed RAG and tool ecosystems. Semantic analysis, provider reproducibility, and defense coverage also remain constrained.

6 Limitations

  • The retrieval-mediated and tool-mediated targets isolate channel realization under a shared protocol rather than reproducing full deployed agent stacks.The authors therefore do not claim to have measured all privacy risk in real-world RAG or tool ecosystems.Source: §6 · p. 13
  • The annotation scale cannot yet support fine-grained target-wise or provider-wise semantic breakdowns.Closedweight API providers also leave residual variability from backend updates, nondeterminism, and formatting behavior.Source: §6 · p. 13
  • The paper does not benchmark a broader mitigation suite or evaluate richer surfaces such as logs, traces, and human-review interfaces.The study is measurement-first rather than defense-complete.Source: §6 · p. 13

7 Related Work

Related work documents privacy risks across memory, retrieval-augmented generation, and tool-integrated agents, but generally studies them with target-specific formulations. CIPL instead groups such risks under a shared channel-oriented measurement interface.

Memory in LLM agents and memory leakage

  • MEXTRA demonstrated black-box extraction of private user queries from agent memory using prompts that localize retrieved items and align them with workflow-dependent outputs Wang et al. [2025].CIPL treats memory leakage as one instance of a broader channel-inversion problem.Source: §7 · p. 13

Privacy leakage in retrieval-augmented generation and in-context retrieval pipelines

  • RAG studies have extracted private datastore content through adversarial prompting and automated attacks, but their formulations remain largely datastore- or RAG-specific Zeng et al. [2024a], Qi et al. [2024], Jiang et al. [2024], Maio et al. [2024].CIPL abstracts these attacks together with memory leakage under a shared target signature.Source: §7 · p. 13

Prompt injection, prompt leakage, and tool-integrated agent attacks

  • Prompt injection, prompt leakage, and tool-integrated attacks study control or exfiltration risks, while CIPL measures how defensive controls change leakage across heterogeneous channels Liu et al. [2023, 2024], Hui et al. [2024], Zhan et al. [2024], Wu et al. [2024a].CIPL does not propose a specific defense.Source: §7 · pp. 13–14

8 Conclusion

The conclusion presents CIPL as a shared interface for comparing how internal exposure becomes recoverable across memory, retrieval, and tool targets without collapsing them into one architecture. The appendix documents target definitions, probe conditions, and extraction rules for that evaluation.

8 Conclusion

  • Memory is a near-saturated high-risk special case; retrieval leakage is often frequent but partial, while tool leakage is channel-sensitive and provider-dependent.Cleaned weak controls indicate strong leakage depends on prompt–channel alignment, and semantic analysis shows exact-only extraction can underestimate attacker-useful risk.Source: §8 · p. 14
  • The conclusion frames the privacy question around which attacker-visible artifacts reveal hidden internal dependence and with what exact or semantic recoverability.It argues observable-channel risk should be treated as a core privacy problem, not merely a memory-only exception.Source: §8 · p. 14

A Appendix Roadmap

  • The appendix roadmap assigns target definitions and extraction rules to Appendix B, full main tables to Appendix C, channel-conditioning evidence to Appendices D–F, semantic protocol to Appendix G, and reproducibility details to Appendix I.Appendix H provides cross-table reading notes.Source: A Appendix Roadmap · p. 16

B Target Signatures and Extraction Rules

  • Appendix B aims to make the shared CIPL measurement interface precise and reproducible rather than introduce a new attack recipe.It records target-specific definitions, configurations, query construction, and leakage units.Source: B Target Signatures and Extraction Rules · p. 16

B.1 Target Definitions and Default Settings

  • The target-definition reference records each target’s sensitive source, selected unit, observable artifact, and extraction rule, deferring shared defaults and provider details to Appendix I..Source: §B.1 · p. 16

B.2 Target Definitions and Observable Surfaces

  • The four targets are memory_ehr, memory_rap, rag_ctrl, and tool_ctrl, which differ in sensitive-content origin and observation surface but share the CIPL signature.tool_ctrl includes args_exfil through tool-call arguments and return_echo through echoed tool results.Source: §B.2 · p. 17

B.3 Query Construction and Probe Conditions

  • Main evaluations standardize the budget to 30 queries, and weak-control prompts remove explicit extraction directives while preserving task structure.For tool_ctrl, deterministic probes test channel invertibility without generation uncertainty, while LLM-in-the-loop probes estimate leakage under ordinary generation behavior.Source: §B.3 · p. 17

B.4 Leakage Units and Canonicalization Rules

  • CIPL maps heterogeneous target-specific leakage units through canonicalization and extraction into shared metrics that distinguish internal exposure from externally recoverable leakage.Units may be memory records, document identifiers or snippets, or visible tool-artifact fields and returns.Source: §B.4 · pp. 17–18

C Full Main Results under the Unified Protocol

The unified protocol supports comparisons across tool-mediated conditions while preserving channel-specific behavior. Retrieval rules and prompt alignment influence how internal exposure becomes externally recoverable leakage.

C Full Main Results under the Unified Protocol

  • The unified-protocol tables report RN, EN, EE, CER, AER, and execution-error counts; the section directs interpretation to Sections 4.3–4.5.The tables provide archival completeness rather than a new interpretation.Source: C Full Main Results under the Unified… · p. 18

D Evidence for Channel-Conditioned Leakage

  • Cleaned weak controls distinguish attack alignment from ordinary task execution; strong prompts produce higher leakage through args_exfil and return_echo.The comparison separates channel invertibility, attack alignment, and realized provider behavior.Source: D Evidence for Channel-Conditioned… · p. 18

E Exposure and Selection Mechanisms

  • The selection analysis tests how upstream changes alter the mapping from internal exposure to externally recoverable leakage, rather than assuming more exposure always means more risk.Source: E Exposure and Selection Mechanisms · p. 18

E.1 Retrieval-Rule Ablation

  • Retrieval-rule changes preserve memory_rap’s saturated leakage, mildly shift rag_ctrl’s complete-versus-partial balance, and materially change tool_ctrl coverage and leakage for the MiniMax variants.DeepSeek and GPT-4o remain saturated under both tool_ctrl rules, while qwen3.5-plus remains highly leakage-prone.Source: §E.1 · p. 19

E.2 Source-Size Ablation

Source size and retrieval depth change exposure and leakage realization differently across targets and providers. Baseline comparisons and ablations also show that no attack recipe is uniformly strongest.

E.2 Source-Size Ablation

  • Memory targets remain fully saturated at CER = AER = 1.0 across all five providers as source size varies, while RN, EN, and EE change.For rag_ctrl, source-size effects are non-monotonic and provider-dependent; MiniMax variants retain high AER as CER declines with larger sources.Source: §E.2 · pp. 19–20, Table 8 · p. 21, Table 9 · p. 21

E.3 Per-Target Retrieval-Depth Tables

  • Memory targets stay saturated across tested retrieval depths, while non-memory targets show provider- and target-dependent changes, including preserved high AER alongside weaker complete recovery.On rag_ctrl, MiniMax CER declines sharply as depth increases; on tool_ctrl, deeper retrieval weakens complete extraction for some providers, while others remain saturated or high-AER.Source: §E.3 · pp. 20–21, Table 10 · p. 22, Table 11 · p. 22

F Boundary Conditions: No Universally Dominant Recipe

  • The full locator–aligner–diversification construction is not uniformly optimal; its component effects interact, with best-performing configurations varying by target and provider.For example, no_diversification achieves the strongest AER on rag_ctrl for two providers, while tool_return_echo’s best configuration differs by provider.Source: F Boundary Conditions: No Universally… · p. 21

F.1 Main-vs-Naive Baseline Comparison

  • The main prompts generally outperform naive baselines on memory- and tool-mediated targets, but rag_ctrl is a systematic counterexample where dump-style naive prompts perform better.The result is target-dependent, not evidence of a uniformly superior prompt family.Source: §F.1 · pp. 22–23, Table 13 · p. 23

F.2 Component Ablations and Interaction Effects

The ablations rule out uniform superiority for a single attack construction, while robustness checks preserve the cross-target patterns. Semantic annotation finds leakage that exact-match extraction misses.

F.2 Component Ablations and Interaction Effects

  • The component ablations provide no evidence that the full construction is uniformly best: rag_ctrl and tool_return_echo each favor different variants across providers.On rag_ctrl, no_diversification has the strongest AER for MiniMax-M2.7 and qwen; tool_return_echo’s best variant differs between MiniMax-M2.7 and qwen.Source: §F.2 · p. 23, Table 14 · p. 24

F.3 Robustness under Budget, Retries, and Seeds

  • Budget, retry, and seed checks preserve the qualitative cross-target findings, although exact values shift across configurations.Retry comparisons use corrected prompt-level metrics; ten-seed runs bound variance without changing regime assignments.Source: §F.3 · p. 24, Table 15 · p. 24

F.4 Defense-Prompt Suppression

  • Defense-style prompting sharply reduces leakage on tool channels, but this is a boundary check rather than a complete mitigation study.The check complements cleaned weak controls and does not claim a defense benchmark.Source: §F.4 · p. 24, Table 16 · p. 25

G.1 Annotation Protocol

  • The semantic protocol manually labels 200 sampled outputs on a three-level scale, distinguishing no recoverable content, partial useful disclosure, and semantically complete leakage.Complete leakage can remain operationally equivalent for an attacker without an exact canonical match.Source: §G.1 · p. 25

G.2 Summary Statistics

  • Among 200 annotated samples, semantic_AER = 0.5000 and semantic_CER = 0.4400; 12 samples show useful semantic leakage despite no exact recovery.Another 88 samples are exact=1 and semantic=2, while 100 are exact=0 and semantic=0.Source: §G.2 · p. 25

G.3 Representative Cases

The representative cases distinguish partial semantic disclosure, complete recovery, structured-secret recovery, and a true negative. The appendix also cautions against equating low complete extraction with low risk or greater exposure with stronger recovery.

G.3 Representative Cases

  • A rag_ctrl output can reveal one attacker-useful private token while failing to recover the full canonicalized selected set.Source: §G.3 · p. 26
  • A second rag_ctrl case exactly recovers a private sentence, which remains fully informative to the attacker.Source: §G.3 · p. 26
  • A third rag_ctrl case exactly recovers a structured supplier escalation code, showing that actionable leakage need not be a natural-language sentence.Source: §G.3 · p. 26
  • The tool_ctrl negative case acknowledges sensitive processing but reveals no semantically recoverable secret value.Source: §G.3 · p. 26

H.1 Partial Leakage without Complete Extraction

  • Low CER can still coincide with high AER and attacker-useful fragments, particularly in rag_ctrl’s partial-leakage regime.Exact recovery is informative but does not capture every useful disclosure.Source: §G.3 · p. 26, §H.1 · p. 27

H.2 Exposure-Induced Changes in Leakage Realization

  • Increasing retrieval depth can preserve any-leakage while weakening complete recovery; the response varies by target and provider rather than increasing monotonically with exposure.Source: §H.2 · p. 27

I Reproducibility Details

  • The reproducibility appendix documents a shared execution pipeline and default configurations, including a common five-provider setting and standardized 30-query prompt files for rag_ctrl and tool_ctrl.Main experiments use 30-query prompt files, one retry, and five seeds; auxiliary ablations generally use three seeds.Source: I Reproducibility Details · p. 27, §I.1 · p. 27, §I.2 · p. 27

I.4 Statistical Reporting and Retry Note

  • Reported values are seed means ± standard deviations, and retries are evaluated at the prompt level rather than the attempt level.Execution errors are reported separately from leakage failure; prompt-level metrics are authoritative.Source: §I.4 · p. 28

I.5 Supporting Evidence and Reproducibility Notes

The appendices provide explanatory evidence and reproducibility materials that complement the main cross-target results. They document target-specific protocol details, supporting analyses, semantic annotation, and rerun information.

I.5 Supporting Evidence and Reproducibility Notes

  • Appendix ablations and robustness checks clarify channel-conditioned leakage, how internal exposure changes leakage mode, and why no prompt family is universally dominant.These explanatory analyses are separated from the main cross-target results to keep the core risk picture legible.Source: §I.5 · p. 28
  • The appendices document target-specific signatures and extraction rules, provide complete main tables, and collect evidence on channel-conditioned leakage.The supporting evidence includes prompt-control comparisons, exposure and selection ablations, and boundary results.Source: §I.5 · p. 28
  • Appendix G records the semantic annotation protocol and representative cases, while Appendix I documents execution and reporting details for reruns.Source: §I.5 · p. 28

Related papers

Loading 2603.22751v2…