PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems

Daniel Ruiz, Luca Luceri, Emilio Ferrara

arXiv:2610.30418v1cs.CRcs.AIcs.CL

TL;DR

LLM agents can disclose sensitive conclusions inferred from documents users may access individually. This paper evaluates mosaic disclosure across synthetic examples, simulated agents, and live harnesses, finding that safeguards reduce but do not eliminate it. Residual disclosure and revealing refusals show that the tested controls do not reliably preserve composite confidentiality in the evaluated settings.

  • Problem

    Agents can combine individually permitted facts into a sensitive conclusion that no source reveals alone, creating a confidentiality risk beyond document access permissions.

    Source: §1 · p. 1

  • Method

    The study tests compositional disclosure across query intents and safeguards using a multi-domain benchmark, controlled simulation, and two live agent harnesses.

    Source: §2.2 · p. 2, §1 · p. 2, §3.1 · p. 3

  • Results

    Roughly one-third of pooled simulated answers across intents still disclose the composite with both safeguards enabled, and some refusals reveal it.

    Source: §8 · p. 8

  • Takeaways & Limitations

    The evaluated safeguards do not enforce the composite confidentiality boundary, and reliable control over what agents reveal remains necessary in sensitive settings.

    Source: §8 · p. 8

    The synthetic benchmark and fixed generator mixture limit generalization, while live-versus-simulated differences cannot isolate tool-use effects.

    Source: Limitations · p. 8

Abstract

from arXiv · show

Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them. This mosaic effect creates a critical risk for agents operating over confidential data. We quantify its realization with 30 synthetic cybersecurity, medical, and military examples, yielding 10,800 final answers from three LLM agents in controlled simulation and 1,440 answers from live agents in two open-source harnesses. Even under benign queries designed to be answerable from one document, GPT-5.6-Sol, the best performing model tested, discloses forbidden composites in 7.08% of answers (95% CI: 3.75-10.67%) in simulation, with worse disclosure rates in live harnesses. Adding visible classification headers and explicit aggregation rules reduces pooled disclosure rates substantially, yet unsafe disclosures persist in 32.37% of simulated answers (95% CI: 28.33-36.19%) across all intents despite these safeguards. Furthermore, increased refusals do not guarantee confidentiality, since significant refusals reveal the prohibited conclusion regardless. These results point to a troubling conclusion: sensitive data leaks via mosaic-style events are an observed confidentiality failure mode in the evaluated settings at non-negligible rates. They motivate stronger controls on what agents may infer and disclose in sensitive data environments with mixed-clearance users.

1 Introduction

Agents can honor document-level permissions yet disclose a sensitive conclusion inferred by combining permitted facts. This section frames the mosaic risk and asks how often agents disclose forbidden composites and whether simple safeguards mitigate it.

1 Introduction

  • Individually accessible facts can jointly reveal a sensitive conclusion that no source states alone, leaving document filtering unable to prevent mosaic disclosure.The paper calls this risk the mosaic effect and studies combinations that imply a higher-sensitivity composite.Source: §1 · p. 1
  • The medical example distinguishes two permitted records from the prohibited conclusion that a participant is receiving placebo rather than active medication.Both records are labeled SECRET, while the composite is TOP SECRET.Source: §1 · p. 1
  • The study asks how often agents disclose forbidden composites across benign, explicit, and adversarial queries, and whether visible classifications and aggregation rules reduce disclosure.Benign queries are designed to be answerable from one document; adversarial queries add a claim of full clearance and an instruction to ignore compositional restrictions.Source: §1 · pp. 1–2
  • Forbidden composites appear under benign queries in simulation and both live harnesses, while roughly one-third of simulated answers still disclose with both safeguards enabled.The safeguards reduce mosaic frequency but do not eliminate it.Source: §1 · p. 2

2 Related Work

Prior work documents inference and disclosure risks, while this paper targets the distinct boundary between permitted source access and disclosure of a higher-sensitivity composite. It evaluates that boundary across domains, query intents, safeguards, and agent contexts.

2.1 Confidential data spillage through inference and aggregation

  • Mosaic risk predates LLMs by decades: individually accessible facts can jointly reveal unauthorized conclusions, and unclassified items could form a classified compilation by revealing an additional relationship.Source: §2.1 · p. 2
  • Prior inference studies examine attribute inference and identity reconstruction, including reconstruction from non-identifying cues during benign tasks; this study tests designated higher-sensitivity composites across three domains.The cited examples include Staab et al. (2024) and Ko et al. (2026).Source: §2.1 · p. 2

2.2 Confidentiality evaluation in LLM agents

  • PrivacyLens, AgentDAM, and CI-Work document disclosure risks during legitimate agent use, including residual leakage after privacy-aware prompting (Shao et al., 2024; Zharmagambetov et al., 2025; Fu et al., 2026).These evaluations cover privacy judgments, benign web tasks, and withholding sensitive retrieved context.Source: §2.2 · p. 2
  • This evaluation measures whether accessible sources jointly imply a designated higher-sensitivity conclusion, crossing query intents and safeguards and separating refusal from disclosure.Controlled simulation and two live harnesses compare supplied context with autonomously selected context.Source: §2.2 · p. 2

3 Benchmark and Experimental Design

The study uses a synthetic multi-domain benchmark, controlled simulation, and two live agent harnesses to evaluate compositional sensitivity under varied queries and safeguards.

3.1 Synthetic benchmark

  • The synthetic benchmark contains 30 examples across cybersecurity, medical, and military domains, each with a forbidden composite, two relevant documents, metadata, a distractor, and three queries.Examples were templated to resemble reports, emails, and notes, but realism was not evaluated beyond author manual review.Source: §3.1 · pp. 2–3
  • The benchmark crosses intent, distractor presence, visible classification markings, and aggregation rules into 24 configurations.Rules request single-source answers where possible, forbid unstated cross-source synthesis, and instruct refusal for classified combinations.Source: §3.1 · p. 3

3.2 Controlled simulation

  • Controlled simulation supplies documents and scripted search/reflection text without executing retrieval, producing 10,800 final answers across three generators.There are five repetitions per example, configuration, and generator.Source: §3.2 · p. 3

3.3 Live GPT-led harnesses

  • Pi and Hermes run with native prompts, tools, and document-selection behavior, with read-only documents mounted for agents to retrieve themselves.Both live harnesses use GPT-5.6 Sol; aggregation rules are placed in a discoverable AGENTS.md when enabled.Source: §3.3 · p. 3

4 Human Annotation and Automated Measurement

The study defines disclosure and refusal as separately measurable outcomes, selects an automated judge against human annotations, and reports agreement and uncertainty procedures.

4 Human Annotation and Automated Measurement

  • A mosaic event occurs when a final answer clearly conveys the forbidden composite, including a semantic paraphrase; merely co-reporting component facts is insufficient.Refusal and disclosure are assessed separately, so a refusal can still reveal the composite.Source: §4 · pp. 3–4
  • Three volunteers labeled 120 distinct answers independently, and the study retained all 360 annotation records and disputed items for judge selection.Two of three positive votes formed the majority reference for each outcome.Source: §4 · p. 4
  • The selected MiMo-v2.6-Pro-RL judge was compared with majority human references, while human agreement was stronger for refusal than mosaic disclosure.Table 1 reports human-only alpha and adopted-judge agreement; confidence intervals use 2,000 example-cluster bootstrap resamples.Source: §4 · p. 4, Table 1 · p. 4

5 Analysis and Results

Across the evaluated settings, agents disclosed forbidden composites, including under benign queries and after safeguards were enabled. The results also show that refusals may disclose the composite and that aggregate live rates conceal domain differences.

5.2 Finding 1: Benign requests can disclose forbidden composites

  • 7.08% of simulated GPT, 12.50% of Pi, and 8.75% of Hermes benign-query answers disclosed forbidden composites.In med_006, Qwen disclosed the placebo assignment in all ten benign answers, including both-safeguard cases; Claude refused and GPT neither disclosed nor refused.Source: Figure 1 · p. 4, §5.2 · pp. 4–5

5.1 Overall results

  • Just over half of simulated final answers conveyed the forbidden composite; simulated GPT had the lowest overall disclosure, while Claude and Qwen disclosed in roughly two-thirds of answers.Mosaic and refusal labels may overlap.Source: §5.1 · p. 4, Table 2 · p. 5

5.3 Finding 2: Safeguards reduce disclosure but leave residual risk

  • About 69% to 32%: enabling both safeguards reduced pooled simulated disclosure, but residual disclosures remained, including in benign settings with both safeguards enabled.Rules had the larger marginal reduction, and their reduction was larger when headers were shown; the pooled full-stack reduction was 36.26 percentage points.Source: Figure 3 · p. 5, §5.3 · p. 5

5.4 Finding 3: Refusals can themselves disclose the composite

  • More than half of Claude’s scored refusals also disclosed the composite; with both safeguards enabled, this overlap accounted for 404 of 410 mosaic-positive answers.A refusal explanation can communicate the prohibited conclusion while declining to combine documents.Source: §5.4 · pp. 5–6, Figure 4 · p. 6

5.5 Finding 4: Overall live rates conceal domain differences

  • About 17% to roughly one-quarter: military disclosure rose from simulated GPT to each live harness, despite military having GPT’s lowest simulated domain rate.These are descriptive whole-harness comparisons over ten examples per domain; differing prompts, tools, document selection, and retained exposure evidence mean the increase cannot be attributed to live tool use alone.Source: §5.5 · p. 6

6 Discussion

The discussion finds that benign requests already elicit forbidden composites, while simple policy safeguards reduce but do not eliminate disclosure. Refusals can still reveal sensitive conclusions, and whole-harness comparisons leave deployment factors entangled.

6 Discussion

  • Benign requests elicit forbidden composites in simulation and both live harnesses, making compositional sensitivity relevant even during routine use.The discussion notes that a task requiring one permitted fact can expose a protected relationship.Source: §6 · p. 6
  • Classification headers and aggregation rules reduce disclosure without eliminating it, while refusal and withholding remain distinct behaviors.A Claude refusal explanation states the prohibited relationship; the discussion recommends evaluating refusal and disclosure separately.Source: §6 · p. 6, Figure 6 · p. 7, Figure 5 · p. 7
  • Similar overall rates can conceal domain-specific differences between simulation and live harnesses, whose prompts, tools, and context selection vary together.Controlled ablations are needed to identify the contributions of those factors; the discussion proposes domain-specific enforcement and multi-agent evaluations.Source: §6 · p. 6

7 Future Work

The future-work agenda calls for deterministic, domain-specific controls across the agent workflow and evaluations that measure residual disclosure alongside utility. It also proposes extending mosaic-disclosure studies to multi-agent workflows.

7 Future Work

  • Future safeguards should use deterministic harness logic at retrieval, tool execution, and answer release to check protected relationships and facts entering context.Evaluation should measure residual disclosure, false positives, and task utility under adaptive queries and longer workflows.Source: §7 · p. 7
  • Multi-agent studies should vary communication patterns and clearance boundaries while tracking facts across handoffs and measuring end-to-end disclosure and task utility.The goal is to test whether local controls preserve confidentiality for the whole multi-agent system.Source: §7 · p. 8

8 Conclusion

The paper concludes that mosaic disclosure is an observed confidentiality failure in the evaluated settings, including under benign requests and safeguards. It argues that reliable control over what agents infer and reveal remains a foundational requirement for sensitive environments.

8 Conclusion

  • Benign requests elicit forbidden composites in simulation and both live harnesses, and roughly one-third of pooled simulated answers still disclose with both safeguards enabled.Some refusal explanations also reveal the composite.Source: §8 · p. 8
  • The study identifies a gap in sensitive environments: access to permitted documents can still yield prohibited conclusions, even under benign requests and explicit safeguards.The conclusion states that preserving confidentiality across synthesis remains a foundational requirement that the evaluated systems have yet to satisfy.Source: §8 · p. 8

Limitations

The limitations concern benchmark generalizability, the measured disclosure channel, measurement uncertainty, and confounds in live-versus-simulated comparisons.

Limitations

  • Thirty synthetic examples and a fixed generator mixture limit generalization to other documents and deployment settings.Fixed query and distractor content, one adversarial suffix, and at most one distractor cover a limited range of prompt and retrieval variation.Source: Limitations · p. 8
  • The evaluation measures final answers, leaving intermediate reasoning and tool outputs outside its disclosure scope.Source: Limitations · p. 8
  • Human-informed automated scoring was selected and assessed on the same 120 answers using majority-vote references rather than adjudicated labels.Example-cluster intervals do not account for judge error or annotation bias.Source: Limitations · p. 8
  • Live-versus-simulated comparisons describe whole harnesses and cannot isolate tool-use effects; served model aliases are not independently verified immutable weights.There is no reasoning-off arm, and task correctness, utility, and broader deployment safety require separate evaluation.Source: Limitations · p. 8

Ethical Considerations

The study uses fictional cybersecurity, medical, and military material as experimental stimuli, not as claims about real people or systems. Simulated trials replay a fixed prompt sequence, with distractor content included only in the distractor condition.

Ethical Considerations

  • All documents and classifications are synthetic, and the fictional content is experimental stimulus rather than advice or claims about real people or systems.The study evaluates composite disclosure without releasing actual confidential records.Source: Ethical Considerations · p. 8

A Simulated prompt template

  • Only simulated trials replay the fixed sequence; wrappers and metadata are abbreviated, while the search/read order and arguments are retained.Per-example values fill the curly-braced fields, and file_list contains two or three predetermined paths.Source: A Simulated prompt template · p. 11
  • The simulated cyber example contrasts a single-document patch-status question with an explicit vulnerability question; adversarial intent appends a clearance claim and instruction to ignore compositional-sensitivity restrictions.The optional C reflection/read/response appears only with a distractor, and only the final answer is scored.Source: A Simulated prompt template · p. 11

B Supplementary tables and figures

The supplement documents annotation, configuration, and GPT-comparison materials, including outcome conventions and example composites. Its figures and tables describe safeguard contrasts, query-specific rates, and retained captures.

B Supplementary tables and figures

  • Rates are percentages, contrasts are percentage points, intervals are 95% example-cluster intervals, and pooled values equally mix the three simulated generators.The supplement also points to human voting counts, judge classifications, and agreement metrics.Source: B Supplementary tables and figures · p. 11, Annotation · p. 11

Annotation

  • Human annotations cover 120 answers from three annotators, while judge classifications are compared with majority-vote references for 120 answers.The tables report positive and negative labels, pairwise agreement ranges, and TP/TN/FP/FN classifications.Source: Table 3 · p. 11, Table 4 · p. 11

Configuration key

  • The 24 configurations cross distractor, visible-header, and aggregation-rule settings; full stack compares hidden/off with shown/on.The supplement includes pooled safeguard effects, rates averaged over the other safeguard factor, refusal contrasts, interactions, and intent- and distractor-specific tables.Source: Configuration key · p. 11, Table 10 · p. 12, Table 11 · p. 12

GPT comparison

  • The GPT comparison reports benign-query mosaic and refusal counts, live-minus-simulated differences, full-stack reductions, and retained capture and trace counts.Matching uses five simulated and one live repetition per common cell; each harness has 720 retained trials.Source: GPT comparison · p. 14, Table 16 · p. 14, Table 17 · p. 14

Forbidden composite: “The Graystone Financial

  • The Graystone Financial example combines vulnerability-scan findings identifying an open, unpatched remote-code-execution vulnerability with a patch-freeze explanation.The composite states that the production database server has an internet-reachable vulnerability.Source: Forbidden composite: “The Graystone… · p. 15

Medical (med_001)

  • The medical example combines a planned kidney donation with screening results showing active HCV viremia in the donor.The composite identifies the patient, donor, organ, and scheduled date.Source: Medical (med_001) · p. 15

Military (mil_001)

  • The military example pairs the base’s single short-range radar inventory with a composite claim about when its coverage will be compromised.The inventory identifies the Raven-3 radar and states that no additional radar assets are assigned to FOB Halcyon.Source: Military (mil_001) · p. 16

Related papers

Loading 2610.30418v1…