PaperlayerView exact revision on arXiv

Source-linked AI summary

CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents

Wenjie Fu, Xiaoting Qin, Jue Zhang, Qingwei Lin, Lukas Wutschitz, Robert Sim, Saravan Rajmohan, Dongmei Zhang

arXiv:2604.21308v1cs.CRcs.CL

TL;DR

Enterprise agents must retrieve and use internal context, creating a risk of sensitive information disclosure. CI-Work evaluates this risk across enterprise information flows and finds persistent privacy failures alongside a privacy–utility trade-off. Its results indicate that scaling model size or reasoning effort does not resolve the tension.

  • Problem

    Enterprise CI evaluations have not fully captured parallel information flows, dense retrieval contexts, and the contextual precision needed to separate essential from sensitive information.

    Source: §1 · p. 1

  • Method

    CI-Work is a CI-grounded benchmark that simulates five enterprise information-flow directions and tests agents on dense contexts containing essential and sensitive entries.

    Source: §1 · pp. 1–2

  • Results

    Frontier models show a privacy–utility trade-off; violation rates range from 15.80% (DeepSeek-R1) to 50.87% (Grok-3), and leakage reaches 26.66%.

    Source: §1 · p. 2, §5.1 · p. 5

  • Takeaways & Limitations

    The findings support evaluating enterprise privacy in dense, realistic contexts and developing context-centric mitigations rather than relying on model scaling alone.

    Source: §6 · p. 7

    CI-Work uses synthetic or manually authored scenarios rather than real organizational data, and enterprise privacy norms vary across organizations, jurisdictions, and cultures.

    Source: Ethical Considerations · p. 8

Abstract

from arXiv · show

Enterprise LLM agents can dramatically improve workplace productivity, but their core capability, retrieving and using internal context to act on a user's behalf, also creates new risks for sensitive information leakage. We introduce CI-Work, a Contextual Integrity (CI)-grounded benchmark that simulates enterprise workflows across five information-flow directions and evaluates whether agents can convey essential content while withholding sensitive context in dense retrieval settings. Our evaluation of frontier models reveals that privacy failures are prevalent (violation rates range from 15.8%-50.9%, with leakage reaching up to 26.7%) and uncovers a counterintuitive trade-off critical for industrial deployment: higher task utility often correlates with increased privacy violations. Moreover, the massive scale of enterprise data and potential user behavior further amplify this vulnerability. Simply increasing model size or reasoning depth fails to address the problem. We conclude that safeguarding enterprise workflows requires a paradigm shift, moving beyond model-centric scaling toward context-centric architectures.

1 Introduction

Enterprise agents’ access to internal data creates a risk of disclosing sensitive information while completing tasks. CI-Work evaluates this tension in dense, realistic workflows and finds persistent privacy failures that are not resolved by simply increasing model size or reasoning effort.

Motivation

  • Enterprise agents can leak sensitive information when they fail to distinguish essential content from information that violates privacy norms for a recipient and context.Such failures constitute violations of Contextual Integrity.Source: §1 · p. 1

Research gap

  • Prior CI evaluations often isolate information flows, use simplistic contexts, or measure task completion without assessing contextual precision in separating essential from sensitive data.Short attributes and simplified contexts also miss the scale and density of enterprise data.Source: §1 · p. 1

Benchmark

  • CI-Work evaluates enterprise agents across five organizational information-flow directions, with dense retrieval contexts partitioned into essential and sensitive information.Its construction combines human-reviewed seed generation with automated iterative refinement.Source: §1 · pp. 1–2

Findings

  • Frontier agents exhibit a persistent privacy–utility trade-off that worsens with enterprise data scale and density, and scaling model size or reasoning effort does not resolve it.Under potential user behavior, agents can leak more sensitive information while failing to convey essential data.Source: §1 · p. 2

2 Related Works

Prior work evaluates privacy reasoning in static and agentic settings, while enterprise benchmarks emphasize task performance. CI-Work addresses the comparatively underexamined risk of contextual privacy in organizational workflows.

Contextual Privacy Benchmarks

  • Earlier CI benchmarks assess tier-based disclosure, diverse-domain privacy norms, agent trajectories, and violations accumulating in persistent memory.Examples include ConfAide, CI-Bench, PrivacyLens, PrivacyLens-Live, and CIMemories.Source: §2 · p. 2
  • CI-Work targets enterprise privacy norms shaped by organizational hierarchies and proprietary workflows, unlike CI benchmarks focused mainly on daily-life settings.Those benchmarks include static disclosure tests, evolving agent trajectories, multiagent workflows, and persistent memory.Source: §2 · p. 2

Enterprise Agents Evaluation

  • Enterprise task benchmarks cover office work, software companies, customer service, and heterogeneous-data search, but primarily evaluate utility and task success.The related work identifies privacy leakage and sensitive-information exposure as comparatively overlooked.Source: §2 · p. 2

3 Risk Model of Enterprise LLM Agents

The risk model frames an enterprise agent as retrieving internal data for a task and sharing information with a recipient. Privacy risk arises when the response discloses sensitive entries while conveying essential ones.

Risk Model

  • The agent retrieves entries from a user’s private data store through tools in response to an instruction requiring information to be shared with a recipient.The model represents tool calls and their returned entries as an execution trajectory.Source: §3 · p. 2
  • Retrieved entries are divided into an Essential Set needed for the task and a Sensitive Set whose disclosure would violate privacy norms.The model treats retrieved contexts as semantically related rather than explicitly modeling irrelevant entries.Source: §3 · pp. 2–3
  • The modeled privacy risk occurs when the agent’s final response discloses sensitive entries while conveying essential entries to complete the task.Source: §3 · p. 3

4 CI-Work Benchmark

CI-Work constructs enterprise scenarios through seed generation, contextual-entry generation, case episodes, and simulated agent trajectories. Its design combines multiple information flows with separate essential and sensitive ground-truth entries and evaluates both disclosure and task conveyance.

4 CI-Work Benchmark

  • The benchmark uses five enterprise information-flow directions and a four-stage pipeline for constructing and evaluating cases.The stages are seed generation, contextual-entry generation, case-episode generation, and trajectory simulation and evaluation.Source: §4 · p. 3

4.1 Task-oriented Seed Generation

  • Seeds define a sender, recipient, and task, and represent downward, upward, lateral, diagonal, or external communication.Human experts refined generated examples across industries, and enterprise practitioners validated the finalized seeds against business practices.Source: §4.1 · p. 3

4.2 Contextual Entries Generation

  • An LLM generates atomic essential and sensitive entries for each seed, keeping the categories disjoint as evaluation ground truth.A blind classification-and-revision loop addresses label discrepancies; human verification found 82.5%–95.0% agreement with annotators’ privacy-norm judgments.Source: §4.2 · pp. 3–4

4.3 Case Episode Generation

  • Each case episode specifies the participants, environment, sender instruction, and entries retrievable through available tools.The episode provides a semantic blueprint for generating full textual observations during trajectory simulation.Source: §4.3 · p. 4

4.4 Trajectory Simulation and Evaluation

  • A tool-centric simulator generates observations and can mix sensitive entries with relevant non-sensitive content in concrete artifacts.Evaluation measures sensitive-entry leakage, whether any sensitive information is disclosed, and conveyance of essential entries.Source: §4.4 · pp. 4–5

4.5 Benchmark Instantiation and Statistics

  • CI-Work includes 125 seeds total: 25 curated task-oriented seeds spanning five information-flow directions, plus 100 additional seeds synthesized via interactions with Gemini-3-Pro.Unless otherwise specified, each seed is instantiated with 4 sensitive and 4 essential entries, resulting in 1,000 contextual entries overall.Source: §4.5 · p. 5

5 Evaluating Frontier LLMs on CI-Work

Across enterprise information flows, frontier LLMs show substantial privacy violations, and denser contexts, user pressure, and common scaling strategies expose further weaknesses. Defense prompts reduce leakage and violations, but at a cost to conveyance.

5.2 Impact of Contextual Entries

  • Leakage varies by data type and flow: personal data is vulnerable in lateral and upward interactions, and internal drafts can be disclosed to external recipients.Models show coarse privacy awareness but struggle with fine-grained role–information compatibility, including revealing a negotiation walk-away price.Source: §5.2 · pp. 5–6

5.1 Overall Performance

  • Table 1 reports violation rates from 15.80% (DeepSeek-R1) to 50.87% (Grok-3), with leakage reaching 26.66%.Across five enterprise information-flow directions, higher conveyance correlates positively with leakage and violations (Pearson r=0.40 and 0.39, p < 0.05).Source: §5.1 · p. 5, Table 1 · p. 6

5.2 Impact of Contextual Entries

  • Increasing contextual entries from 1 to 12 raises violations and lowers conveyance, even as the proportion of sensitive details leaked falls.The entries are maintained in equal proportions; the authors describe the declining leakage proportion as a dilution effect.Source: §5.2 · p. 6, Figure 4 · p. 6
  • Longer contextual entries improve conveyance but also increase leakage and violations, exposing a privacy–utility trade-off.The tested entries span five length tiers, from very short to very long.Source: §5.2 · p. 6, Figure 4 · p. 6

5.3 Impact of User Pressure

  • Under implicit and explicit user pressure, leakage and violations rise while conveyance declines; explicit pressure brings violations to nearly double baseline.Explicit pressure supplies concrete information handles, and the resulting outcome can sacrifice essential content while eliciting more sensitive information.Source: §5.3 · p. 6, Figure 5 · p. 6

5.4 Impact of Scaling and Prompting

  • Larger GPT-4.1 variants improve conveyance but increase leakage, while higher GPT-5 reasoning effort only marginally reduces leakage and violations.The reasoning-effort change leaves conveyance virtually unchanged.Source: §5.4 · p. 7
  • Prompt Defense and CI-CoT both reduce GPT-5 leakage and violations, but CI-CoT preserves substantially more conveyance; its violation rate remains above 20%.CI-CoT reduces conveyance by 9%, so prompt-level mitigation alone remains insufficient.Source: §5.4 · p. 7, Table 2 · p. 7

6 Conclusion

The conclusion presents CI-Work as a Contextual Integrity benchmark for evaluating enterprise agents’ privacy–utility tension. It argues that dense retrieval contexts and user behavior exacerbate vulnerabilities, while scaling model size or reasoning depth does not resolve the tension.

6 Conclusion

  • CI-Work evaluates the tension between utility and privacy in enterprise LLM agents through a benchmark grounded in Contextual Integrity theory.The authors report that frontier LLMs are fragile in adhering to privacy norms.Source: §6 · p. 7
  • Dense retrieval contexts and user behavior exacerbate privacy vulnerability, while increasing model size or reasoning depth does not resolve the tension.The authors aim for CI-Work to support rigorous evaluation and development of context-centric mitigation solutions.Source: §6 · p. 7

Ethical Considerations

CI-Work studies enterprise-agent privacy using synthetic or manually authored data, not real organizational records. Its coverage cannot capture every organization’s norms or legal obligations, and human judgments may differ.

Ethical Considerations

  • CI-Work uses synthetic or manually authored scenarios and contextual entries, not real employee or customer records, proprietary documents, or identifiable personal data.The human-annotation content also excludes harmful or unsafe material; contextual entries are work-related, including informal colleague conversations.Source: Ethical Considerations · p. 8
  • The benchmark covers diverse domains and information-flow directions but cannot represent every organization’s privacy norms or legal obligations.Users should not generalize its results to specific compliance regimes without additional organization-specific policy validation.Source: Ethical Considerations · p. 8
  • Human judgments of appropriateness may differ, so the benchmark does not establish a single universal privacy judgment.Source: Ethical Considerations · p. 8

A Details of the CI-Work Benchmark

CI-Work spans five organizational information-flow directions through a balanced set of task-oriented seeds and diverse enterprise domains and sensitive data types. Its scenarios include a range of proprietary, personal, and informal work-related content.

A Details of the CI-Work Benchmark

  • The benchmark contains 125 task-oriented seeds: 25 manually curated and 100 synthetically generated, evenly distributed across five information-flow directions.Upward, Downward, Lateral, Diagonal, and External each account for 20% of the seeds.Source: A Details of the CI-Work Benchmark · p. 11
  • Technology (13.6%), Supply Chain (10.4%), and Finance (8.8%) are the most frequent industry domains.Legal records (22.4%), HR (15.6%), and Financial records (13.8%) are the largest Sensitive-entry data types.Source: A Details of the CI-Work Benchmark · p. 11, Figure 7 · p. 11, Figure 8 · p. 11
  • Sensitive entries span nine artifact types, from legal, technical, financial, and draft materials to HR, executive, customer, casual, and off-work content.Examples include privileged attorney-client communications, security artifacts, employee records, identifiable customer data, and personal messages.Source: A Details of the CI-Work Benchmark · p. 11

B Experimental Setup

The experimental setup evaluates open- and closed-source agentic LLMs and compares CI-Work with prior privacy benchmarks. CI-Work is the enterprise benchmark in the comparison with dynamic trajectories and multiple context lengths.

B Experimental Setup

  • The evaluated open-source models are DeepSeek-V3, DeepSeek-R1, Kimi-K2, and Qwen-2.5 32B; the closed-source set includes GPT-4o, GPT-4.1, GPT-5, Grok-4, and o3.The open-source models are deployed via vLLMs across eight NVIDIA A100 80GB GPUs; the closed-source models are accessed through Azure AI Foundry.Source: B Experimental Setup · pp. 11–12
  • CI-Work is the enterprise benchmark in the comparison, with dynamic trajectories, context lengths of 2/4/8/12/16/24, and 125 cases.The table reports 1007 for its context-conveyance column.Source: B Experimental Setup · p. 12
  • The comparison includes CI-Bench, PrivacyLens, CIMemories, and PrivaCI-bench, spanning daily, legal, and enterprise domains.The table contrasts agentic tasks, trajectory types, context conveyance, context length, and case counts.Source: B Experimental Setup · p. 12

C Quality Control of CI-Work

CI-Work’s human evaluation assesses scenario quality, context relevance, and privacy-norm alignment using independent enterprise-employee annotators. Ratings and labels show substantial agreement, though the authors note that contextual judgments can vary.

C Quality Control of CI-Work

  • Three independent full-time enterprise employees assessed a random sample of 25 cases, producing 75 scenario evaluations and 600 fine-grained privacy judgments.Each case had eight context items—four Essential and four Sensitive—and annotators received CI theory training.Source: C Quality Control of CI-Work · p. 12
  • Annotators first judged global scenario quality, then assessed individual context items for plausibility, relevance, and privacy-norm alignment.The protocol masked the Essential/Sensitive labels and shuffled items; privacy-norm alignment was a binary Appropriate/Inappropriate judgment.Source: C Quality Control of CI-Work · p. 12
  • Generated scenarios received mean ratings of 4.88 for Contextual Plausibility and 4.51 for Context Relevance, with Within-1 Agreement of 96.0% and 88.0%, respectively.Within-1 Agreement is the percentage of cases where annotators’ maximum rating difference was at most one point.Source: C Quality Control of CI-Work · p. 12, Table 3 · p. 12
  • 82.5%–95.0% agreement between human annotators and LLM labels supports CI-Work’s Essential/Sensitive distinction as its benchmark ground truth.The reported agreement is for Privacy-Norm Alignment.Source: C Quality Control of CI-Work · p. 12, Table 4 · p. 13
  • Inter-annotator agreement reached Fleiss’ κ = 0.685 and pairwise Cohen’s κ up to 0.77, values the authors describe as substantial agreement.The authors also note that contextual integrity judgments vary because privacy norms follow a distribution rather than a single truth.Source: C Quality Control of CI-Work · p. 12
  • The authors conclude that consistency across the evaluation metrics supports CI-Work as a high-quality benchmark for enterprise-agent privacy evaluation.Source: C Quality Control of CI-Work · p. 12

D Validation of LLM-as-a-Judge

The LLM judge shows substantial agreement with human labels for Context Usage, while residual errors skew toward missed usage. The appendices document the human-rating workflow and provide instruction, prompt, and qualitative examples.

D Validation of LLM-as-a-Judge

  • 83.0%–91.0% judge–human agreement supports using the LLM-as-a-Judge as a scalable proxy for Context Usage evaluation.Human annotators had 84.0%–86.0% pairwise agreement (Fleiss’ κ=0.658); against pooled human consensus, the judge matched 88.0% of instances.Source: D Validation of LLM-as-a-Judge · p. 13
  • The judge may miss about 10% of subtle cases, so reported leakage rates are a lower bound.The residual disagreement pattern is precision-prioritized and mainly reflects Human=Used, Judge=Not Used cases.Source: D Validation of LLM-as-a-Judge · p. 13

F Human Annotation Interface

  • The human interface pins scenario details and guides annotators through plausibility, relevance, norm-alignment, and context-usage judgments.Navigation requires completing the labels, and progress indicators support careful review.Source: F Human Annotation Interface · p. 13

E Correlation between Conveyance and Privacy risk

  • Pearson r=0.40 for CR–LR and r=0.39 for CR–VR; both correlations are statistically significant, but high conveyance does not define a fixed privacy-risk frontier.Some high-conveyance model–direction settings have comparatively low leakage or violation rates, while others show sharp increases.Source: E Correlation between Conveyance and… · p. 13, Figure 9 · p. 14

G Instruction Examples

H Prompt Templates

  • The appendix presents templates for entry generation, self-correction, episode generation, and trajectory evaluation.The trajectory templates cover context extraction and essential-use and sensitive-leakage judgments.Source: H Prompt Templates · p. 14

I Qualitative Examples

Related papers

Loading 2604.21308v1…