PaperlayerView exact revision on arXiv

Source-linked AI summary

Not All Entities are Created Equal: A Dynamic Anonymization Framework for Privacy-Preserving RAG

Xinyuan Zhu, Zekun Fei, Enye Wang, Ruiqi He, Jia Guo, Ruijie Wang, Zheli Liu, Qingkai Zeng

arXiv:2603.26074v3cs.CR

TL;DR

RAG over private or regulated knowledge bases can expose sensitive information, while uniform protection may overlook context and utility differences among entities. TRIP-RAG selectively anonymizes entities using contextual privacy and utility measures. Experiments report privacy protection comparable to full anonymization alongside improved retrieval and generation utility.

  • Problem

    Uniform entity protection overlooks differences in utility and context-dependent privacy leakage, although RAG knowledge bases can expose sensitive information.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    TRIP-RAG selectively anonymizes entities using a scoring mechanism that considers marginal privacy risk, knowledge divergence, and topical relevance.

    Source: §1 · p. 2

  • Results

    TRIP-RAG achieves privacy protection comparable to full anonymization while preserving more retrieval and generation utility; Recall@k improves by 9%-60%.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The evaluation framework can be extended to broader text anonymization and integrated with downstream anonymization methods for controllable-intensity privacy protection.

    Source: §6 · p. 9

    GLiNER extraction misses some entities in uncommon positions, and the evaluation is based exclusively on identified entity content.

    Source: §7 · p. 9

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) enhances the utility of Large Language Models (LLMs) by retrieving external documents. Since the knowledge databases in RAG are predominantly utilized via cloud services, private data in sensitive domains such as finance and healthcare faces the risk of personal information leakage. Thus, effectively anonymizing knowledge bases is crucial for privacy preservation. Existing studies equate the privacy risk of text to the linear superposition of the privacy risks of individual, isolated sensitive entities. The "one-size-fits-all" full processing of all sensitive entities severely degrades utility of LLM. To address this issue, we introduce a dynamic anonymization framework named TRIP-RAG. Based on context-aware entity quantification, this framework evaluates entities from the perspectives of marginal privacy risk, knowledge divergence, and topical relevance. It identifies highly sensitive entities while trading off utility, providing a feasible approach for variable-intensity privacy protection scenarios. Our theoretical analysis and experiments indicate that TRIP-RAG can effectively reduce context inference risks. Extensive experimental results demonstrate that, while maintaining privacy protection comparable to full anonymization, TRIP-RAG's Recall@k decreases by less than 35% compared to the original data, and the generation quality improves by up to 56% over existing baselines.

1 Introduction

TRIP-RAG addresses the mismatch between context-dependent privacy leakage and uniform entity anonymization, which can remove information needed for retrieval and generation. It selectively anonymizes entities using privacy risk and utility-related measures, and experiments report privacy protection comparable to full anonymization with stronger utility preservation.

1 Introduction

  • Private-domain RAG documents can expose sensitive information through retrieved passages, generated responses, or contextual associations.The introduction argues that sensitive information should be protected before indexing, retrieval, or generation.Source: §1 · p. 1
  • Uniform entity protection overlooks both entities' differing utility and context-dependent privacy leakage, and can degrade retrieval and generation utility.Anonymizing surrounding entities may change whether another entity contributes to privacy leakage.Source: §1 · p. 2
  • Entities with higher topical alignment or greater semantic deviation when anonymized are treated cautiously because they may contribute to retrieval and generation quality.Anonymization threshold adjustment allows the privacy-utility trade-off to vary across deployment requirements.Source: §1 · p. 2
  • TRIP-RAG selectively anonymizes entities in RAG knowledge bases instead of masking every detected entity.Its entity scoring jointly considers marginal privacy risk, knowledge divergence, and topical relevance.Source: §1 · p. 2
  • Recall@k improves by 9%-60%, BLEU by 3%-56%, and ROUGE-L by 5%-53% while TRIP-RAG provides privacy protection comparable to full anonymization.The experiments span datasets and backbone models.Source: §1 · pp. 2–3

2 Related Works

RAG systems can expose private information through knowledge-base storage and model outputs, motivating protection before documents enter the pipeline. Existing data-level approaches have trade-offs, while uniform entity protection motivates selective anonymization that balances privacy and utility.

2.1 Privacy Exposure in RAG

  • RAG can expose sensitive information during embedding, indexing, retrieval, or generation, so post-retrieval defenses cannot fully replace protection before indexing.The section describes cloud or proprietary embedding services and retrieval infrastructure as exposure points.Source: §2.1 · p. 3

2.2 Data-Level Privacy Protection in RAG

  • Synthesis-based methods avoid directly storing raw private documents but may require extensive generation and introduce information loss, factual drift, or hallucinations.These risks can undermine the faithfulness and retrievability required by RAG.Source: §2.2 · p. 3
  • LPRAG applies local differential privacy to detected entities, but DP perturbation may produce semantically imprecise or factually inconsistent replacements.The passage identifies these issues as problematic for factual accuracy, entity consistency, and retrieval discriminability.Source: §2.2 · p. 3
  • Existing data-level methods often protect detected entities uniformly, overlooking their differing roles in privacy leakage and RAG utility.Anonymizing a small subset of contextually critical entities may suffice, while further anonymization can remove useful topical cues and semantic structure.Source: §2.2 · p. 3

3 Problem Formulation

The formulation selects which entities in a knowledge-base text to anonymize, aiming to satisfy a privacy constraint while preserving task-relevant utility. It treats privacy risk as contextual and specifies an attacker model for residual reconstruction risk.

3.1 Problem Definition

  • Knowledge-base text anonymization selects entities for generalization, leaving entities outside the selected subset unchanged.The text-level mechanism applies the entity-level mapping to the selected entities.Source: §3.1 · p. 3
  • Contextual privacy leakage means anonymizing an appropriate subset of entities may prevent disclosure without anonymizing every entity.The passage frames leakage at the sentence level and says risk depends on relationships among textual elements.Source: §3.1 · p. 3
  • Utility-aware protection chooses an anonymization subset that meets a privacy constraint while preserving as much task-relevant utility as possible.The anonymized text is produced by applying the anonymization mechanism to the selected subset.Source: §3.1 · p. 3
  • Residual semantic privacy risk is the risk that protected target entities remain reconstructable from the remaining context.The formulation defines text utility as the total utility of preserved entities.Source: §3.1 · pp. 3–4
  • The optimization maximizes anonymized-text utility subject to residual privacy risk staying below an acceptable threshold.The formulation makes the privacy-utility trade-off explicit.Source: §3.1 · p. 4
  • Finding the optimal entity subset is combinatorial and NP-hard, so TRIP-RAG uses a context-aware greedy approximation.The passage points to Section 4 for the approximation strategy.Source: §3.1 · p. 4

3.2 Threat Model

  • The threat model considers a polynomial-time attacker seeking protected entities from anonymized text or RAG outputs, with knowledge of the anonymization mechanism.It includes untrusted-server and trusted-server scenarios; the privacy constraint requires reducing contextual reconstruction risk below the acceptable threshold.Source: §3.2 · p. 4

4 Methods

TRIP-RAG uses a two-stage pipeline to quantify entities’ contextual privacy and utility contributions, then selectively anonymize high-priority entities. Its security analysis links semantic indistinguishability to generalization classes with sufficiently large preimages.

4.1 Design and Workflow of TRIP-RAG

  • TRIP-RAG quantifies each entity’s privacy and utility contributions, then selectively generalizes high-priority entities to reduce privacy risks while preserving document structure and retrieval utility.The framework is designed as a two-stage pipeline: context-aware quantification followed by local generalization.Source: §4.1 · p. 4

4.1.1 Context-Aware Entity Quantification Stage.

  • Marginal privacy risk measures the reduction in document-level privacy risk after locally anonymizing an entity, using a privacy-aware language-model surrogate.The differential formulation accounts for surrounding context when estimating entity-level risk.Source: §4.1.1 · p. 4
  • Knowledge divergence measures the cosine distance between document representations before and after anonymizing an entity; a larger angular shift signals a greater contribution to knowledge structure.The representations are produced by a pre-trained text encoder.Source: §4.1.1 · p. 4
  • Topical relevance uses an entity as a simulated retrieval query and measures L2 divergence from the document topic; smaller divergence indicates greater retrieval relevance.The entity and document are represented in a vector space.Source: §4.1.1 · pp. 4–5

4.1.2 Local Generalization Stage.

  • The priority score combines marginal privacy risk with penalties for retrieval and knowledge divergence, and entities scoring above the dataset-specific threshold are selected for anonymization.The threshold is empirically determined using a small sample assessed by an LLM, with manual secondary evaluation of those assessments.Source: §4.1.2 · p. 5
  • For selected entities, TRIP-RAG maps original labels to coarse-grained safe descriptors, aiming to neutralize exact sensitive attributes while preserving syntactic coherence, factual consistency, and knowledge-base structure.The mapping is applied iteratively to the selected entities.Source: §4.1.2 · p. 5

4.2 Security Analysis

  • Under the stated negligible-distinguishability condition, the analysis argues that non-injective generalization with sufficiently large preimages achieves semantic indistinguishability against polynomial-time attackers.The condition compares an attacker’s ability to distinguish anonymized texts containing either of two entities in the same generalization class.Source: §4.2.1 · pp. 5–6

5 Experiment

Experiments across healthcare and workplace RAG evaluate retrieval, generation, and privacy, finding that TRIP-RAG balances privacy protection with stronger task utility than anonymization baselines. Ablations, sensitivity tests, and labeled-data analysis examine the scoring dimensions and their behavior.

5.1.1 Datasets and Evaluation Metrics.

  • The evaluation uses healthcare consultations and workplace question-answer pairs, measuring retrieval consistency, answer-text utility, and private-entity exposure.The retrieval corpora comprise 99% of each dataset; the remaining 1% is used for testing, with full generalization, paraphrasing, local differential privacy, and original data as comparisons.Source: §5.1.1 · p. 6, §5.1.2 · p. 6

5.2.1 Utility Evaluation.

  • 9%–60% higher Recall@k, 3%–56% higher BLEU, and 5%–53% higher ROUGE-L than baselines show that TRIP-RAG preserves retrieval and generation utility.The comparison covers both retrieval and generation metrics across the evaluated settings.Source: §5.2.1 · p. 6, Table 1 · p. 7

5.2.2 Privacy Protection Analysis.

  • 26% lower leakage on ChatDoctor and 48% lower leakage on EnronQA than original data accompany utility preservation, balancing privacy protection with downstream performance.Paraphrase has slightly lower leakage on EnronQA but significantly compromises utility.Source: §5.2.2 · pp. 6–7, Table 3 · p. 8

5.3 Case Study(RQ1)

  • In the consultation case, TRIP-RAG generalizes demographic attributes while preserving medications and treatment recommendations essential for medical reasoning.The example contrasts this selective treatment with semantic distortion, utility-critical entity removal, or weakened interaction context from other methods.Source: §5.3 · pp. 7–8

5.4 Ablation Study(RQ2)

  • 16% higher Recall@1 with topical relevance and 10% higher BLEU-1 with knowledge divergence than marginal-risk-only scoring support complementary roles for the dimensions.The full model improves Recall@1 by 23%, Recall@5 by 18%, and BLEU-1 by 12%.Source: Figure 3 · p. 8, §5.4 · pp. 8–9

5.5 Hyperparameter Sensitivity Analysis(RQ2)

  • γ/β=0.8 gives the best average BLEU in the tested hyperparameter search, balancing topical relevance’s retrieval gains with knowledge divergence’s semantic fidelity.The study fixes α=1 and varies β and γ over {0.2, 0.4, 0.5, 0.6, 0.8, 1.0}.Source: Figure 4 · p. 8, §5.5 · p. 9

5.6 Impact of Labeled Data on Marginal Privacy Risk(RQ3)

  • 0.9160 Spearman correlation with human judgments at 200 training samples indicates progressively stronger alignment as labeled sample size increases.Strong positive correlation is reached from 128 samples onward.Source: Figure 4 · p. 8, §5.6 · p. 9

6 Conclusions

The conclusion frames TRIP-RAG as context-aware anonymization for variable-intensity privacy protection. Its experiments report utility gains with almost no privacy impact, and its evaluation framework is presented as extensible to broader text anonymization.

6 Conclusions

  • The paper distinguishes text semantic-level privacy protection from scope-entity privacy protection and frames protection intensity around differences among entities.TRIP-RAG defines marginal privacy risk, knowledge divergence, and topical relevance to quantify those differences.Source: §6 · p. 9
  • Cross-dataset and multi-model experiments report utility improvement with almost no impact on privacy.The conclusion describes the scheme as quantifying and balancing privacy and utility.Source: §6 · p. 9
  • The evaluation framework is presented as extendable to broader text anonymization and integrable with downstream anonymization methods.The paper characterizes this as a controllable-intensity perspective for text semantic privacy protection.Source: §6 · p. 9

7 Limitation

The limitation section notes that GLiNER can miss private entities in uncommon positions, while the accompanying reduction establishes hardness for the anonymization optimization problem under a special case.

7 Limitation

  • Manual audits find that GLiNER misses some private entities in uncommon positions, and experiments rely exclusively on identified entity content.The authors state that this does not compromise their experimental findings.Source: §7 · p. 9

A Reduction of the Problem of Finding the Optimal Generalized Entity Subset

  • With scarce attacker background knowledge, generalizing any single entity suffices for the anti-reconstruction entropy constraint, making non-reconstructability non-binding.The optimization then selects entities subject to a total capacity limit.Source: A Reduction of the Problem of Finding… · p. 9
  • The reduction maps entity privacy to knapsack weight, utility contribution to value, and the privacy threshold to capacity, establishing the problem as strongly NP-Hard.Source: A Reduction of the Problem of Finding… · p. 9

B Information Overlap Among the Three Features

The three evaluation features have limited information overlap, while the privacy-score dataset uses trained human annotators and a four-level entity rubric.

B Information Overlap Among the Three Features

C.1 Data Sampling and Annotation Guidelines

  • Two NLP researchers independently annotated 200 randomly sampled, entity-rich snippets per dataset after training on GDPR privacy definitions.The cited regulation defines personal data in relation to an identified or identifiable person.Source: §C.1 · p. 10
  • Entities receive privacy-importance scores on a four-level scale, from directly identifying entities at Level 3 to general vocabulary at Level 0.Intermediate levels cover moderate or context-dependent risks and personal characteristics with high generalization.Source: §C.1 · p. 10

C.2 Comparison Between Language Model and Human Evaluation

The language model’s privacy scores align strongly with human annotations. A secondary human review found that its selected entity subsets included those identified by human evaluators, while retaining a utility advantage over naive baselines.

C.2 Comparison Between Language Model and Human Evaluation

  • Spearman correlations between model scores and human annotations reach 0.9139 on ChatDoctor and 0.9085 on Enronqa.The paper describes these evaluations as fundamentally consistent with human judgment.Source: §C.2 · p. 10

D Threshold Determination Details

  • The threshold prompt directs the LLM to assess combinatorial inference risks from intertwined contexts rather than treating entities in isolation.The prompt is available in the authors’ open-source codebase.Source: D Threshold Determination Details · p. 10
  • The LLM-selected entity subsets strictly include those identified by human evaluators, and downstream utility remains clearly better than naive baselines that anonymize all recognized entities.The LLM sometimes selects redundant entities, which leads to a generally lower overall score; the authors state that anonymizing supplementary entities introduces no additional privacy compromises.Source: D Threshold Determination Details · p. 10

E Proof of Semantic Security

The proof uses a game-based argument to establish semantic indistinguishability for generalized entities. It replaces entity-specific processing with deterministic labels and then considers a blind world independent of the challenge bit.

E Proof of Semantic Security

  • In the real-world game, the challenger selects two entities with the same class label and sufficient Ψ score, then outputs the transformed text for a randomly chosen entity.The adversary guesses which entity was selected.Source: E Proof of Semantic Security · p. 10
  • The deterministic-replacement game substitutes the class label for entity features, preserving an output distribution identical to the real-world game.The stated reason is that the replacement mechanism is deterministic for a given input and context.Source: E Proof of Semantic Security · p. 10
  • In the blind-world game, the output depends on the template and class labels rather than the challenge bit, leaving the adversary to guess randomly.The proof’s conclusion compares the real-world and blind-world guessing advantages.Source: E Proof of Semantic Security · p. 10
  • The proof concludes that the advantage is zero, establishing absolute semantic indistinguishability for generalized entities.The argument uses a game-based proof technique standard in modern cryptography.Source: E Proof of Semantic Security · p. 10

F Introduction to Experimental Baseline

The experiments compare TRIP-RAG with redaction, paraphrasing, and local differential privacy perturbation. The disclosure describes Gemini Deep Research as a literature-discovery aid and AI as a CSV-to-table formatting aid, while attributing the research and analysis to the authors.

F Introduction to Experimental Baseline

  • The baselines include generalized label replacement, LLM-based context filtering or rewriting, and local differential privacy perturbation.LPRAG uses word, phrase, and number perturbation modules, with a global privacy budget of 5.0 in the reproduction.Source: F Introduction to Experimental Baseline · p. 11

GenAI Usage Disclosure

  • Gemini Deep Research helped discover prior work on locally private entity perturbation, while literature synthesis and critical analysis were performed manually.The disclosure identifies the discovered study as He et al..Source: GenAI Usage Disclosure · p. 11
  • Generative AI translated CSV files into LaTeX tables, while the datasets, analyses, and scientific interpretations were human-authored.The authors state that the fundamental research contributions, algorithmic design, empirical analysis, and manuscript drafting were completed without AI involvement.Source: GenAI Usage Disclosure · p. 11

Related papers

Loading 2603.26074v3…