Source-linked AI summary
On the Impact of Anonymization on the Performance of Large Language Models
Tobias Deußer, Max Hahnbück, Lorenz Sparrenberg, Tobias Uelwer, Christian Bauckhage, Rafet Sifa
TL;DR
The paper asks how protecting PII through input anonymization affects LLM utility, a question with limited broad empirical evidence. It evaluates five models across eleven benchmarks and finds that anonymization effects vary sharply by model, task, and technique, motivating co-designed privacy-aware systems.
Problem
The impact of anonymizing LLM inputs to protect PII on downstream model utility is not well understood.
Method
The study compares five LLMs on original and pseudonymized inputs across eleven diverse benchmarks, with additional tests of anonymization techniques and prompting.
Results
Anonymization generally reduces performance, with larger drops for more capable models, catastrophic declines on RGB, improved TruthfulQA performance, and better results from reversible techniques than irreversible ones.
Takeaways & Limitations
Anonymization is not a one-size-fits-all solution and should be co-designed with the model and downstream task.
Takeaways & Limitations
The paper notes that carefully designed prompts may mitigate information loss, but such correction may amount to informed guessing rather than robust understanding.
Abstract
from arXiv · showhide
As large language models are increasingly deployed in sensitive domains, anonymizing input data to protect personally identifiable information has become a critical practice. However, the impact of this anonymization on model utility is not well understood. This paper presents a systematic empirical study of the trade-off between privacy and performance. We evaluate five prominent language models across eleven diverse benchmarks, comparing their performance on original versus pseudonymized inputs. Our results reveal that while anonymization generally degrades performance, the effect is highly nuanced. We find that more capable models, such as Qwen2.5-72B and GPT-4o mini, suffer the largest performance drops, suggesting a stronger reliance on specific entity information. The impact is also task-dependent: performance on TruthfulQA improves with anonymization, while retrieval-focused tasks like RGB experience a catastrophic decline. Further experiments show that reversible anonymization techniques that preserve entity uniqueness significantly outperform irreversible ones like redaction, and that explicitly prompting models about anonymization offers no discernible benefit. We conclude that anonymization is not a one-size-fits-all solution and must be co-designed with the model and task in mind to balance privacy and utility effectively. Our findings provide a crucial baseline for developing more robust, privacy-aware AI systems.
1 Introduction
The paper systematically examines how anonymization trades off privacy and LLM capability across models and NLP tasks. Its effects vary with model capability, task nature, and anonymization technique rather than following a uniform pattern.
- Anonymization generally degrades performance, but the magnitude depends on model capability, task type, and anonymization technique.
- More powerful models, including Qwen2.5-72B and GPT-4o mini, experience the largest performance drops after anonymization.The authors suggest these models rely more heavily on specific entity information.
- Retrieval-intensive RGB performance collapses after anonymization, whereas TruthfulQA performance consistently improves.The paper suggests removing named entities can reduce hallucinated incorrect associations on TruthfulQA.
- Reversible pseudonymization significantly outperforms irreversible redaction because it preserves entity uniqueness.
- Explicitly telling models that inputs were anonymized yields no discernible performance benefit.
- Anonymization should be co-designed with the model and downstream task rather than treated as a one-size-fits-all privacy solution.
2 Related Work
Prior anonymization research provides important but fragmented evidence across domains, tasks, and model generations. This paper broadens the evaluation scope to modern LLMs and diverse benchmarks.
- The study addresses the privacy-utility tension created by anonymizing text to protect PII in trustworthy AI applications.
- Modern LLMs are increasingly used for context-aware PII removal and replacement, including methods that distill proprietary-model knowledge into smaller open-source systems.
- Prior findings on anonymization vary, ranging from clinical performance losses to near-identical downstream performance after pseudonymization.
- Privacy evaluation frameworks now include broad multimodal benchmarks and domain-specific tools covering privacy and harmful behaviors.
- Existing studies often focus on clinical or financial domains, restricted NLP tasks, or earlier architectures such as BERT.
3 Methodology
The methodology compares five LLMs on original and anonymized inputs across eleven diverse benchmarks. A modular pipeline supports multiple datasets, models, anonymization strategies, and evaluation conditions.
- 3.1 Experimental Pipeline: The pipeline prepares benchmark data, anonymizes PII while retaining mappings, runs models on both conditions, and de-anonymizes outputs for evaluation.
- 3.2 Dataset and Benchmark Selection: Five LLMs are evaluated across eleven benchmarks spanning diverse linguistic and cognitive tasks.
- 3.2 Dataset and Benchmark Selection: The study evaluates 6,210 unique samples, generally capping benchmarks at 500 samples and multi-task benchmarks at 1,000.
- 3.2 Dataset and Benchmark Selection: Prompts are selected from official or established implementations, with few-shot prompts generally preferred and all prompt components anonymized consistently.
- 3.3 Experimental Setup: The core experiment compares each selected LLM on each benchmark using original versus anonymized inputs.
- 3.4.1 Anonymization Configuration: Primary pseudonymization replaces entities with category labels and consistent unique identifiers, such as “London” becoming “<LOC>-1”.The anonymization tool reports F1 above 88% and recall above 91%.
- 3.4.2 Alternative Anonymization Techniques: Additional comparisons test generalization, masking, randomization, and redaction on GPT-4o mini using MedQA, MUSR, and RGB.
- 3.5 Prompting Anonymized Inputs: Prompting experiments compare short and long anonymization prefixes against unprefixed anonymized and non-anonymized inputs.
4 Experiments
Across five models and eleven benchmarks, anonymization affects performance unevenly: stronger models and retrieval-intensive tasks often suffer larger losses, while TruthfulQA improves. Reversible techniques preserve utility better than irreversible methods, and explicit anonymization prompts provide no consistent benefit.
- 4.1 Impact of Pseudonymization on LLM Performance: RGB drops by 30–60 percentage points for most models, while ARC, HellaSwag, and IFEval typically drop by under 5 points.GPT-4o mini’s RGB score falls approximately 48 points, from 0.80 to 0.32.
- 4.1 Impact of Pseudonymization on LLM Performance: TruthfulQA is the only benchmark where all five models maintain or improve performance under anonymization, including an 8-point gain for Llama-3.1.The paper suggests that replacing named entities may reduce incorrect factual associations on this task.
- 4.1 Impact of Pseudonymization on LLM Performance: RGB’s retrieval-augmented tasks are especially vulnerable because abstract placeholders disrupt entity-dependent information integration and noise filtering.Its performance drops range from -0.22 for Teuken-7B to -0.47 for GPT-4o mini.
- 4.1 Impact of Pseudonymization on LLM Performance: Qwen2.5-72B’s average score drops 6.9 points after anonymization, compared with 2.3 points for Teuken-7B.The stronger models also achieve the highest non-anonymized averages and scores above 0.60 on 8 of 11 tasks.
- 4.1 Impact of Pseudonymization on LLM Performance: Entity counts and input length do not alone explain anonymization effects: MUSR has high entity statistics but minimal degradation, whereas MMLU-PRO and MedQA still decline noticeably.Correlation analyses found no statistically significant relationship between performance drops and total entities, unique entities, or input length.
- 4.2 Comparing Different Anonymization Techniques: Reversible methods generally preserve performance better than irreversible methods, while explicit prefixes about anonymization yield no significant or consistent improvement.On MuSR, pseudonymization scores 0.70 versus 0.58 for generalization and 0.52 for redaction; prefix length has little effect.
5 Conclusion
Anonymization produces a complex privacy–utility trade-off: its effects vary by model capability, task, and technique. The results favor preserving entity uniqueness and co-designing anonymization with downstream applications.
- Model and task dependence: More capable models, including Qwen2.5-72B and GPT-4o mini, suffer the largest performance drops after anonymization.The authors link this vulnerability to greater reliance on specific entity information.
- Model and task dependence: TruthfulQA performance improves for most models after anonymization, whereas retrieval-heavy RGB experiences a catastrophic decline.The authors associate the TruthfulQA improvement with reduced factual hallucination and RGB’s decline with sensitivity to precise entity context.
- Anonymization strategy: Reversible methods such as pseudonymization and randomization outperform irreversible methods such as redaction and generalization by preserving entity uniqueness.The comparison underscores entity uniqueness as important for model reasoning.
- Anonymization strategy: Explicitly prompting models that inputs are anonymized provides no discernible benefit, indicating that meta-information alone does not change their strategy.
- Practical implications: Anonymization should be co-designed with the model and downstream task rather than treated as a one-size-fits-all preprocessing step.The authors propose more granular controls and models robust to, or fine-tuned for, anonymized representations as future directions.