PaperlayerView exact revision on arXiv

Source-linked AI summary

Personalized Privacy Control in LLMs via Attention Head Intervention

Junseok Kim, Nakyeong Yang, Kyomin Jung

arXiv:2608.21209v1cs.AIcs.CLcs.LG

TL;DR

Personalized privacy addresses the gap between contextually appropriate disclosure and user-specific preferences, which can differ within the same context. The paper introduces P3Bench and REPAIR, an inference-time attention-head intervention method, and reports improved adherence while reducing policy violations. The benchmark remains limited in its coverage of PII fields, scenarios, and unstructured disclosures.

  • Problem

    Contextual privacy does not account for user-specific differences in acceptable disclosure within the same context, motivating personalized privacy control.

  • Method

    The paper introduces personalized privacy and P3Bench, then proposes REPAIR, which intervenes on policy-relevant attention heads at inference time.

  • Results

    Prompt-based policies fail to reliably enforce personalized privacy, while REPAIR improves adherence to user-specific policies and reduces policy violations.

  • Takeaways & Limitations

    Personalized privacy control requires accounting for user-specific disclosure preferences rather than relying only on contextual relevance or prompting.

  • Takeaways & Limitations

    The benchmark has limited coverage of PII fields and scenarios and does not fully capture unstructured personal disclosures.

Abstract

from arXiv · show

The rise of agentic AI enables LLMs to access diverse user data, raising critical privacy concerns. Prior work on contextual privacy studies whether LLMs regulate information disclosure according to context-dependent norms. However, acceptable disclosure boundaries may vary across users even within the same context. To address this limitation, we introduce \textit{personalized privacy}, which incorporates user-specific disclosure preferences into privacy control. We further present P3Bench~(\textbf{P}ersonalized \textbf{P}rivacy \textbf{P}reservation \textbf{Bench}mark), a novel benchmark extending contextual privacy policies with personalized disclosure policies. Experiments show that prompt-based policies fail to reliably enforce personalized privacy policies, with Qwen2.5-7B and Gemma3-4B showing average policy ignorance ratios of 51.25\% and 74.28\%, respectively. Finally, to address this problem, we propose \textsc{Repair}, a robust inference-time attention head intervention method that adjusts disclosure behavior toward policy-consistent responses. Our method significantly improves adherence to user-specific privacy preferences by reducing cases where the model fails to follow the given policy.

1 Introduction

Agentic LLMs can access sensitive user data, but contextual privacy alone cannot capture disclosure preferences that differ across users. This work introduces personalized privacy and P3Bench, shows prompt-based policies are unreliable, and proposes REPAIR to improve policy adherence.

  • Agentic LLMs can access diverse user data, creating privacy risks when sensitive information is exposed during interactions.
  • Contextual privacy regulates disclosure under a given context, but acceptable disclosure can vary across users within the same context.
  • Personalized privacy extends contextual privacy by incorporating user-specific disclosure tolerance, and P3Bench evaluates these personalized policies.
  • 51.25% and 74.28% are the average policy ignorance ratios for Qwen2.5-7B and Gemma3-4B, respectively, under prompt-based user policies.
  • Qwen2.5-3B and Qwen2.5-7B tend to over-refuse, whereas Gemma3-4B tends to over-share sensitive information.
  • REPAIR uses policy-relevant attention heads and inference-time intervention to steer disclosure behavior toward policy-consistent responses without retraining.

2 Problem Definition

The paper formalizes personalized contextual privacy by distinguishing contextually relevant information that users permit from information that must remain restricted. It evaluates model behavior through disclosure states and observable answer-or-refuse actions.

  • For task τ, Fτ contains contextually relevant fields, while user policy p defines permitted information Ap and restricted information Dp.
  • The assistant should retrieve information from Ap ∩ Fτ while avoiding leakage from Dp.
  • Each query receives one of three disclosure states: Disclosure, Policy-Refusal, or Base-Refusal, determined by task context and user policy.
  • Disclosure maps to ANSWER, while Policy-Refusal and Base-Refusal both map to REFUSE as observable model actions.

3 Can Prompt-level Policies Enforce Personalized Disclosure Control?

The section evaluates whether direct prompting can enforce personalized disclosure policies and finds that it remains insufficient for reliable policy compliance. P3Bench operationalizes these policies across four disclosure settings, while PIR analysis shows that prompting often fails to override default model behavior.

  • Personal Policy Design: P3Bench extends the AirGapAgent-R dataset with four personalized privacy settings reflecting different disclosure preferences.The benchmark uses 3,536 test instances covering 17 user profiles.
  • Personal Policy Design: The four settings are Privacy-Max, Contact-Open, Health-Open, and Preference-Open, covering increasingly specific disclosure preferences.They respectively govern name-only disclosure, contact fields, health fields, and lifestyle or preference fields.
  • Policy Compliance Under Direct Prompting: Direct prompting produces significant over-refusal and over-sharing across models and policies, showing that prompt-level control is insufficient for reliable personalized privacy enforcement.Over-refusal counts REFUSE predictions where ANSWER is required, whereas over-sharing counts ANSWER predictions where REFUSE is required.
  • Behavior Change under Personal Policies: High PIR indicates that models retain their no-policy behavior even when a personalized policy requires a different output.PIR is computed on cases where the personalized policy requires suppressing an answer.
  • Behavior Change under Personal Policies: Gemma-3-4B maintains PIR above 70% across all four policies, with several health and preference fields showing especially strong resistance to prompting.These field-level patterns indicate default answer-or-refuse tendencies that are difficult to override through prompting alone.

4 Methods

REPAIR identifies policy-relevant attention heads, constructs state-specific intervention representations, and applies state-adaptive edits during generation to steer disclosure behavior without retraining.

  • Policy-Relevant Head Selection: The method uses head activations as sparse intervention units because each attention head provides a separate representation that can be probed and edited independently.The paper contrasts this design with residual-stream and MLP interventions under matched edited-dimensionality budgets.
  • Policy-Relevant Head Selection: REPAIR identifies policy-relevant heads by training head-level probes to predict policy-conditioned disclosure states from final input-token activations.Each head is scored by AUROC, and the top-k heads are selected for intervention.
  • State-Specific Intervention Vectors: REPAIR constructs refusal patching vectors from gold-conditioned activations for Policy-Refusal and Base-Refusal states.Both refusal states target REFUSE, but they arise from personalized-policy and task-context requirements, respectively.
  • State-Specific Intervention Vectors: For Disclosure states, REPAIR uses an L2-normalized steering direction that suppresses refusal-related components while preserving input-specific content.This asymmetric design avoids directly replacing head activations with a mean Disclosure representation.
  • State-Adaptive Head Intervention: At inference time, selected-head probes predict the disclosure state by majority vote, and the predicted state determines the intervention applied during generation.The edited activation includes a disclosure steering coefficient α controlling intervention strength.
  • Evaluation: Table 2 evaluates policy-conditioned control using OR, OS, and PED, with lower values indicating better performance across all metrics.The table compares methods under the same model and policy, with OR and OS changes marked relative to Direct Prompting.

5 Experiments

Experiments evaluate REPAIR across instruction-tuned models, personalized policies, error metrics, robustness settings, and head-selection variants. REPAIR generally improves policy compliance by reducing both over-refusal and over-sharing, while AUROC-based selection outperforms arbitrary head selection.

  • Experimental Setup: Experiments evaluate Qwen2.5-3B, Qwen2.5-7B, and Gemma3-4B across four personal privacy policies and compare REPAIR with inference-time baselines.The evaluation uses instruction-tuned models and tests alignment with different user-specific disclosure preferences.
  • Evaluation Metrics: PED jointly measures over-refusal and over-sharing as Euclidean distance from the ideal point (OR, OS) = (0, 0), so lower PED indicates better overall compliance.This metric discourages improvements that merely shift errors from one type to the other.
  • Main Experimental Results: 90.5%: under Privacy-Max, REPAIR reduces PED from 71.11 to 6.78 on Qwen2.5-3B relative to Direct Prompting.On Gemma3-4B, the corresponding reduction is 67.8%, from 35.38 to 11.40.
  • Main Experimental Results: REPAIR reduces both OR and OS, improving personalized policy adherence without simply shifting the model toward refusal or disclosure.Appendix analyses further report more consistent adherence to the given personal policy.
  • Robustness: REPAIR consistently lowers OR, OS, and PED compared with Direct Prompting across random field-level policies with 4, 8, and 16 accessible fields.Results are averaged over three randomly sampled policies for each number of accessible fields.
  • Head Selection: AUROC-based head selection achieves lower PED than random selection across all policies, whereas random intervention only marginally improves over Direct Prompting.The comparison uses the same number of heads and the same state-adaptive intervention procedure.
  • Head Analysis: Policy-relevant heads exhibit diverse balanced, disclose-specialist, refuse-specialist, and weak roles, with weak heads consistently rare.The diversity of roles suggests complementary head-level signals in personalized disclosure control.

6 Related Works

Related work frames contextual privacy as regulating information flow according to task- or context-dependent norms for LLM interactions involving sensitive user data.

  • Contextual Privacy: Contextual privacy regulates information disclosure according to context-appropriate information-flow norms.The paper situates this framework in Nissenbaum’s notion of Contextual Integrity and subsequent LLM studies.

7 Conclusion

The work introduces personalized privacy and P3Bench for evaluating disclosure control under diverse personal preferences. It finds prompt-based policies unreliable and proposes REPAIR to improve adherence while reducing policy violations.

  • Prompt-based policies fail to reliably enforce personalized privacy constraints, producing both over-refusal and over-sharing in LLMs.
  • REPAIR adaptively controls disclosure through inference-time intervention on policy-relevant attention heads.
  • REPAIR improves adherence to personalized privacy policies while reducing policy violations.

Limitations

The benchmark evaluates diverse personalized policies across multiple PII fields, but its coverage remains limited. It focuses on structured fields and does not fully represent unstructured personal disclosures.

  • The benchmark’s coverage of PII fields and interaction scenarios remains limited despite evaluating multiple fields.
  • The benchmark primarily represents structured PII fields rather than unstructured disclosures such as sensitive experiences or interpersonal situations.
  • Broader PII categories, more realistic interactions, and unstructured disclosure scenarios remain directions for future work.

B P3Bench Details

P3Bench models personalized privacy through synthetic profiles, task scenarios, queried fields, and policies governing field accessibility. Its evaluation uses constrained prompts, deterministic output labels, and agreement checks with an LLM judge.

  • Benchmark construction: Each benchmark instance contains a synthetic user profile, task scenario, queried personal field, and personal privacy policy specifying accessibility.
  • Benchmark construction: Profiles contain 26 personal fields across identity, contact, health, and lifestyle or preference categories, evaluated in eight scenarios across three domains.
  • Policy design: The benchmark uses semantically defined policies—Privacy-Max, Contact-Open, Health-Open, and Preference-Open—with field-level accessible or denied decisions.
  • Dataset statistics: Each policy has 624 training instances and 3,536 test instances, with 8,112 training and 45,968 test instances overall.
  • Prompting setup: The system prompt supplies the privacy policy and user profile, while the user prompt specifies the domain, scenario, directive, and question.
  • Evaluation: Responses are classified as ANSWER or REFUSE by deterministic string matching, which closely agrees with an LLM judge across models.

C.2 Evaluation Protocol

The evaluation compares prompting and activation-steering baselines with REPAIR under fixed decoding and calibrated intervention settings. Results show that intervention breadth and steering strength require calibration, while state-specific intervention supports the full design.

  • Evaluation protocol: All methods use greedy decoding, with 95% confidence intervals reported for OR, OS, and PED.
  • Baselines: Direct Prompting provides the policy in the system prompt and requires either the requested value or the exact refusal string.
  • Baselines: Zero-shot CoT adds step-by-step reasoning to Direct Prompting without modifying model representations, while CAST uses calibrated activation differences as a training-free steering baseline.
  • REPAIR configuration: REPAIR selects the number of intervened heads k and steering coefficient α on calibration data, then fixes them during testing.
  • Intervention coverage: Under Privacy-Max, increasing k up to 20 reduces both OR and OS and yields the lowest PED, whereas broader intervention is not always better.

D.4 Field-wise Analysis of Policy Error Distance

Field-wise PED analysis shows that prompt-based baselines vary substantially across personal attributes, while REPAIR achieves lower and more consistent policy error across most fields. These results support more stable policy-conditioned disclosure control through intervention.

  • Field-wise policy error: Prompt-based baselines show large field-dependent variation in Policy Error Distance across personal attributes.This indicates reliance on field-specific default answer or refusal tendencies rather than consistent adaptation to user policies.
  • REPAIR performance: REPAIR achieves consistently lower PED across most fields on Gemma-3-4B.Lower PED indicates better policy adherence.
  • Mechanistic interpretation: REPAIR’s selected attention heads encode a policy-conditioned disclosure boundary that can be predicted from internal activations.The predicted state is translated into intended disclosure behavior through intervention.
  • Evaluation measure: Figure 12 compares average field-wise PED across policies, with lower values representing better policy adherence.The comparison highlights whether disclosure control remains stable across different personal fields.
Loading 2608.21209v1…