Source-linked AI summary
Demystifying the Privacy-Utility Trade-off in LLM Interactions
Zhenhua Liu, Zhanxu Xie, Junjie Yu, Tong Zhu, Lijun Li, Wenliang Chen
TL;DR
LLM interactions require detailed personal context, creating privacy risks that static sanitization methods often address at substantial utility cost. This paper analyzes how intent, task requirements, and attribute dependencies determine sanitization decisions, then implements those findings in an intent-driven local framework. The framework reaches a low-leakage privacy point while preserving substantially higher utility than existing privacy-oriented baselines.
Problem
The paper addresses the limited understanding of how different sanitization choices affect LLM performance and the utility loss caused by context-agnostic privacy methods.
Method
The paper combines systematic analysis with a distilled lightweight model that drives dynamic extraction, strategic sanitization, and post-hoc context restoration.
Results
The framework reaches a low-leakage privacy point while preserving substantially stronger response utility than existing privacy-oriented baselines.
Takeaways & Limitations
Sanitization decisions should account for user intent, task reliance on factual or structural validity, and combinatorial dependencies among sensitive attributes.
Abstract
from arXiv · showhide
The integration of Large Language Models into daily tasks relies on context-rich instructions, inevitably exposing sensitive user information. Current privacy-preserving methods typically employ context-agnostic static rules, causing severe utility degradation. However, the specific mechanisms governing how sanitization impacts downstream performance remain largely underexplored. To address this, we conduct a systematic analysis to deconstruct the privacy-utility trade-off, uncovering three underlying mechanisms: (1) Context-Dependent Utility, which first establishes when to sanitize by revealing that data value shifts from critical constraints to dispensable noise based on user intent; (2) Strategic Adaptation, which subsequently determines how to sanitize by dictating that the choice between removal and replacement depends on the task's reliance on factual integrity versus structural coherence; and (3) Combinatorial Interplay, which finally extends the protection scope by demonstrating that attributes form a semantic web of synergistic dependencies or antagonistic redundancies. Guided by these insights, we introduce an intent-driven local protection framework. By distilling a lightweight model Veilmind-4B to drive a dynamic extraction-sanitization-restoration pipeline, our approach reaches a low-leakage privacy point while preserving substantially higher response utility than existing privacy-oriented baselines, advancing the privacy-utility trade-off toward the Pareto frontier.
1 Introduction
LLM assistance benefits from detailed personal context, but these disclosures can expose sensitive attributes and enable profiling or abuse. The paper analyzes this privacy-utility trade-off and proposes an intent-driven framework that adapts sanitization to task needs.
- 1 Introduction: User prompts require granular personal context for precise assistance, but those disclosures can expose sensitive attributes to profiling and targeted abuse.The paper illustrates this risk with an insurance appeal involving clinical and financial information.
- 1 Introduction: The paper identifies an underexplored privacy-utility trade-off: sanitization can reduce privacy risk while removing context needed for useful responses.Prior methods often apply static rules without evaluating each detail’s marginal contribution to the user’s goal.
- 1 Introduction: The analysis distinguishes when to sanitize, how to sanitize, and what scope to sanitize by linking protection decisions to intent, task validity, and attribute dependencies.Sensitive information may be a critical constraint, dispensable noise, a factual premise requiring removal, or a structural anchor favoring replacement.
- 1 Introduction: The proposed intent-driven framework uses a distilled lightweight model to perform extraction, strategic sanitization, and post-hoc restoration with adjustable Utility Priority and Privacy Priority modes.The framework is designed to accommodate varying user tolerances.
- 1 Introduction: The framework reaches a low-leakage privacy point while preserving substantially stronger utility than existing privacy-oriented baselines.The authors characterize this outcome as advancing the privacy-utility trade-off toward the Pareto frontier.
2 Related Work
Prior privacy defenses span training-time sanitization, differential privacy, unlearning, representation perturbation, and text-level filtering. Context-agnostic methods can protect against leakage but may remove task-relevant information and degrade utility, motivating more flexible approaches.
- 2 Related Work: Training-side defenses include sanitization, differential privacy, and machine unlearning, but they face false negatives, utility loss, or catastrophic forgetting.These methods address memorization or removal of sensitive information but can disrupt capabilities or general knowledge.
- 2 Related Work: Inference-phase protection uses representation perturbation, formal privacy mechanisms, and direct text-level sanitization to safeguard user inputs from third-party providers.The literature has increasingly shifted toward protecting the input during interaction rather than only modifying training data.
- 2 Related Work: Rule-based and named-entity methods are efficient but context-agnostic, often removing task-relevant details and causing utility collapse.Recent LLM-based approaches seek more flexible reformulation or dynamic sanitization strategies.
3 Deconstructing the Privacy-Utility Trade-off
The paper systematically measures how intent, sanitization strategy, and attribute interactions shape the privacy–utility trade-off. Across controlled analyses, sensitive information can be essential, distracting, structurally useful, synergistic, or redundant depending on context.
- Experimental Framework: The evaluation uses 384 fixed-density samples with five sensitive statements, classifies six intent and seven privacy types, and measures reward-score shifts.The controlled design aims to isolate effects of intent, sanitization strategy, and attribute interaction; the broader dataset contains 9,757 samples with a 2.5% manual-subset error rate.
- Context-Dependent Utility: The Interplay of User Intent and Sensitive Information: ∆R = −14.25 for removing Social and Relational Information in Task Execution, while removing Health and Wellness data yields ∆R = +0.19.The results show that intent determines whether sensitive context constrains the solution or acts as distractor noise.
- Strategic Adaptation: The Efficacy of Removal versus Replacement: D = +14.04 for Task Execution with Health Information, favoring removal, whereas replacement outperforms removal in Personalized Interaction by D = −1.64.Removal better preserves factual integrity in objective tasks, while replacement preserves structural coherence and empathy in interactive contexts.
- Combinatorial Interplay: The Synergy and Antagonism of Sensitive Information: I = −6.05 for Health and Wellness with Interests, Beliefs, and Opinions, indicating synergy when linked statements jointly support a coherent causal story.The interaction score compares joint removal with the sum of individual impacts; negative values denote synergy.
- Combinatorial Interplay: The Synergy and Antagonism of Sensitive Information: I = +4.14 for Behavioral Data with Social Information, indicating antagonism when overlapping attributes provide redundant signals.Other positive interactions reach I = +47.8 for Social Information with Interests and I = +28.2 for Professional Background when Health context is present.
4 Framework Implementation and Evaluation
The framework distills privacy insights into a local extraction-sanitization-restoration pipeline that adapts protection to user intent. On Uprise, its two operating modes improve the privacy-utility trade-off over baselines, while ablations show that extraction, sanitization, and restoration serve distinct roles.
- Architecture Design: The framework extracts sensitive statements, assigns each a Keep, Remove, or Replace action, sanitizes the prompt, and restores necessary context after remote inference.The local model uses user intent, privacy category, and interactions between statements to construct the protection plan.
- Model Distillation: Knowledge distillation produces Veilmind-4B, a compact privacy-specialized model trained to perform extraction, sanitization, and restoration.Deepseek-V4-Flash synthesizes training data for a fine-tuned Qwen3-4B student, including reasoning-enhanced extraction and configurable utility- or privacy-priority modes.
- Evaluation Setup: Evaluation uses Uprise with pairwise Utility Score and Privacy Score based on sensitive-statement retention, where lower privacy score means less leakage.The framework is also evaluated on Pupa-tnb, while baselines include Papillon and PUFT variants.
- Main Results: In Privacy Priority Mode, the framework achieves higher utility than Papillon Optimized with only slightly higher privacy score, while outperforming other baselines on both metrics.Utility Priority Mode preserves critical constraints for exceptional response quality.
- Impact of Model Distillation: Fine-tuning improves Uprise utility-mode utility from 75.5% to 86.8% and reduces privacy-mode leakage from 52.6% to 40.5%.Extraction coverage rises from 52.4% to 72.7%, while privacy-mode removal falls from 54.5% to 33.6%.
- Stage Ablation: Removing extraction raises leakage from 52.6% to 71.6%, removing sanitization guidance raises it to 60.7%, and removing restoration lowers utility to 62.0%.These ablations separate privacy coverage, concrete edit decisions, and post-sanitization utility recovery.
5 Conclusion
The paper finds that privacy-utility effects depend on user intent and interactions among privacy attributes, motivating adaptive local protection. Experiments show improved balance between privacy and response quality relative to static baselines, with future work targeting evolving boundaries in multi-turn interactions.
- 5 Conclusion: The paper concludes that sensitive information’s marginal utility depends on user intent and the combinatorial interplay of privacy attributes.It therefore treats sanitization as requiring adaptive strategies rather than fixed rules.
- 5 Conclusion: Experiments show that the proposed distill-and-deploy local framework balances privacy protection with response quality compared with static baselines.Future work will extend the intent-centric paradigm to multi-turn interactions.
A Data Collection Details
The appendix introduces the implementation details of the paper’s data collection and annotation pipeline.
- A Data Collection Details: The appendix provides implementation details for the data collection and annotation pipeline.
A.1 Real-world Data Filtering
Real-world prompts are normalized, filtered, deduplicated, and balanced across language and length categories before dataset construction. This process yields an initial corpus of 61,275 candidate prompts.
- A.1 Real-world Data Filtering: The pipeline normalizes prompts from three public dialogue corpora and filters them by language, token length, and duplication.Balanced collection targets English, Chinese, and Other prompts at a 60:20:20 ratio and uses four length buckets.
- A.1 Real-world Data Filtering: 61,275 candidate prompts remain after the filtering process.
A.2 Synthetic Data Generation Pipeline
The synthetic data pipeline expands prompts with retrieved personas, filters them for injection suitability, extracts and verifies sensitive statements, and organizes the resulting corpus by language, intent, and privacy type.
- A.2 Synthetic Data Generation Pipeline: The pipeline retrieves and reranks candidate personas to select the most contextually relevant persona for each prompt.Qwen3-Embedding-8B retrieves the top eight candidates, and Qwen3-Reranker-8B selects one.
- A.2 Synthetic Data Generation Pipeline: Only prompts receiving suitability scores of at least 3 on a five-point scale are processed for privacy injection.Deepseek-V4-Flash performs the suitability assessment using the template in Table 8.
- A.2 Synthetic Data Generation Pipeline: Selected personas are injected into prompts, after which sensitive statements are extracted from both real and synthetic prompts.The extraction follows criteria specified in Table 10.
- A.2 Synthetic Data Generation Pipeline: A double-check stage re-evaluates extracted statements, reserving 200 manually validated samples for final evaluation.The remaining samples support the empirical study.
- A.2 Synthetic Data Generation Pipeline: The analysis uses six user-intent types and seven privacy types as a shared taxonomy for distribution, interaction, and qualitative analyses.The final distillation corpus is also characterized by language, intent, and privacy-type composition.
C Experiment Details and Additional Results
The experiments analyze privacy interactions across user intents and report additional benchmark results alongside the distillation setup and detailed privacy-utility comparisons.
- C Experiment Details and Additional Results: Privacy interactions are task-dependent: the same attribute pair can produce different utility and leakage effects under different user intents.Figure 8 visualizes synergy and antagonism patterns across privacy-attribute combinations.
- C Experiment Details and Additional Results: The distillation experiments train Veilmind-4B with full-parameter supervised fine-tuning on Qwen3-4B using teacher annotations from Deepseek-V4-Flash.The student learns privacy extraction, sanitization, and restoration locally on a single NVIDIA H200 GPU.
- C Experiment Details and Additional Results: Table 6 reports detailed percentage results for the privacy-utility trade-off comparison, with Figure 7 providing additional Pupa-tnb results.The supplied passages identify the result formats and visual scope but do not provide cell values.
C.3 Additional Benchmark Results
The evaluation uses Uprise and Pupa-tnb to measure privacy-utility performance under real-world interactions and transformed privacy-unit prompts.
- C.3 Additional Benchmark Results: The evaluation spans Uprise for privacy-sensitive real-world requests and Pupa-tnb for privacy units and transformed prompts.Together, the benchmarks test the framework beyond the Uprise setting.
- C.3 Additional Benchmark Results: Overall numerical comparisons are reported in Table 6, while Figure 7 visualizes the Pupa-tnb results.The passage does not state the numerical outcomes.
D Case Study
The case studies show Veilmind-4B identifying sensitive statements, applying removal or replacement before remote inference, and restoring details needed for task utility.
- D Case Study: Across four task types, Veilmind-4B performs explicit removal or replacement before remote inference and restores task-relevant details afterward.The examples cover interests task execution, professional problem solving, behavioral interaction, and financial problem solving.
- D Case Study: The case-study materials include templates for suitability assessment, injection, extraction, double-checking, removal, replacement, ablation, and restoration.Additional templates define utility- and privacy-priority modes and shared sanitization guidelines.
- D Case Study: In scenic-route planning, exact origins and destinations are replaced with regional descriptions before inference, then restored in the final itinerary.The original task asks for a scenic route from Belgrade to Rome.
- D Case Study: For noise-cancellation advice, podcast context is removed and the air-conditioner detail is replaced with generic background noise before the task-specific source is restored.The case concerns selecting a readily available algorithm for podcast recording affected by air-conditioner noise.
- D Case Study: For behavioral advice, the first-person request is generalized and the avoidant-attachment disclosure is removed before remote inference, then restored for personalization.The example concerns developing a secure attachment style as an adult.
- D Case Study: For financial advice, a specific buy-now-pay-later business model and purchase categories are replaced with a generic lending scenario before actionable details are restored.The original case involves rising losses and customers not repaying loans.