A Survey on Privacy Risks and Protection in Large Language Models

• LLMs process diverse data, including personal and sensitive information, while supporting instruction-following, writing, coding, and other applications.Source: Introduction · p. 2
• Privacy research focuses on extracting memorized training data, while model reasoning can also infer personal attributes from text.Source: Introduction · pp. 2–3
• Privacy protection methods include data cleaning, differential privacy, and confidential computing to reduce accidental data exposure.Source: Introduction · p. 3
• The survey analyzes eleven privacy risks and attacks, classifies them, and pairs each classification with mitigation techniques.Source: Introduction · p. 4
• It also evaluates existing technologies and proposes improvements and future guidance for protecting LLM security and user privacy.Source: Introduction · p. 4

• LLM workflows begin with collecting and preprocessing user language data, whose quality affects downstream model performance.Source: Architecture and vulnerabilities of LLMs · p. 5
• Privacy risks arise when collected or user-submitted sensitive information is processed, stored, and exposed during interaction or deployment.Source: Architecture and vulnerabilities of LLMs · p. 5
• Even anonymized prompts can be reverse-engineered to recover private information.Source: Architecture and vulnerabilities of LLMs · p. 5
• The survey classifies vulnerabilities as privacy attacks, privacy leakage, contextual leakage, and backdoor attacks.Source: LLMs Vulnerabilities · p. 5
• It examines privacy leakage and attacks targeting models, data, and users, including poisoning and backdoor methods that manipulate model behavior through malicious training samples.Source: LLMs Vulnerabilities · pp. 5–6

• Privacy leakage exploits LLM vulnerabilities to collect sensitive information, while privacy attacks breach defenses to obtain it.Source: Privacy Issues of LLMs · p. 6

• Users may enter names, contact details, identification numbers, and financial information that models can improperly use or leak.Source: Sensitive Information Leakage · pp. 6–7
• Users’ mishandling of sensitive prompts can expose information that may enter model knowledge bases or be stored by operators.Source: Sensitive Information Leakage · pp. 7–8
• Fine-tuning specialized LLMs with domain-specific knowledge introduces sensitive-information leakage concerns alongside domain adaptation.Source: Sensitive Information Leakage · p. 8
• Contextual leakage occurs when information flows violate contextual norms, such as disclosing healthcare records for unrelated promotional purposes.Source: Contextual Leakage · pp. 8–9
• GPT-4 achieved 84% top-1 and 95.1% top-3 accuracy on PersonalReddit personal-attribute identification, increasing privacy-leakage concerns.Source: Contextual Leakage · p. 9
• Personalization can infer user preferences and expose private information through customized content, targeted advertising, or recommendations.Source: Personal Preferences Leakage · p. 9

• Privacy attacks are classified into model-based, data-based, and user-based groups according to their targets and methods.Source: Privacy Attacks · p. 9

Model-based Attacks

• Backdoor attacks inject poisoned samples that can steal sensitive data or manipulate outputs when specific keywords trigger the hidden behavior.Poisoning can occur during pre-training or fine-tuning, creating privacy leakage after deployment.Source: Model-based Attacks · pp. 9–10
• Model inversion reconstructs private training samples by analyzing model outputs, parameters, and gradients.Attackers use reverse engineering to recover sensitive information from private datasets.Source: Model-based Attacks · p. 11
• Model stealing duplicates models fine-tuned on sensitive datasets by querying their responses and extracting internal information.The attacker can obtain confidential model details without direct access to the underlying dataset.Source: Model-based Attacks · p. 11

Data-based Attacks

• PCAT extracts client privacy data in Split Learning using only approximately 0.1%-5% of the private training set.The attack is described as transparent to the client and difficult to detect.Source: Data-based Attacks · p. 12
• Special Characters Attack uses character patterns to trigger memorized raw training data from language models.The reported mechanism links uninterrupted meaningless responses with the output of memorized data.Source: Data-based Attacks · pp. 12–13

User-based Attacks

• Membership and attribute inference attacks infer training-data membership or sensitive personal attributes from model outputs and API responses.Reported inferred attributes include personal identification details, medical records, and geographic location; MIA performance on LLMs remains unstable and largely unexplored.Source: User-based Attacks · pp. 13–14

Defense taxonomy

• Defense strategies are classified into protections against privacy leakage and protections against privacy attacks.This classification organizes the mechanisms reviewed in the survey.Source: Privacy Mitigation in LLMs · p. 14

Privacy leakage defenses

• Data cleaning reduces privacy risk by removing or anonymizing personally identifiable information before model use.The approach includes anonymization, data minimization, and related security practices.Source: Defense Against Privacy Leakage · p. 14

Inference protection

• Inference defenses include CONFAIDE, IOI, and InferDPt for assessing privacy reasoning, protecting decisions, and preserving document privacy during black-box generation.InferDPt also addresses information bias from differential privacy while enhancing remote and local model capabilities.Source: Defense Against Privacy Leakage · p. 15

Federated learning

• Federated learning enables collaborative training without directly sharing distributed data, but privacy leakage and stage-specific design remain concerns.Reported leakage includes class representative reconstruction, motivating precise optimization across federated-learning stages.Source: Defense Against Privacy Leakage · p. 16

Differential privacy

• Differential privacy adds random noise during processing and training to make personal details harder to infer from model training data.The survey notes that larger and more complex models may experience significant utility effects from this protection.Source: Defense Against Privacy Attacks · p. 17

Backdoor removal

• Fine-tuning, FT-SAM, pruning, and fine-tuning combinations can remove or weaken backdoors, while super fine-tuning further reduces member-leakage risk.FT-SAM promotes learning of backdoor neurons, and evaluations report that pruning and fine-tuning can weaken or eliminate backdoors.Source: Defense Against Privacy Attacks · p. 17

Cryptographic protection

• Cryptographic defenses provide provable security guarantees but can reduce performance, while secure multi-party computation may substantially slow Transformer inference.Obfuscation-based alternatives generally provide weaker and less predictable protection than encryption-based methods.Source: Defense Against Privacy Attacks · p. 18

Privacy-Preserving Model Compression

• Compression improves deployment efficiency but can expose sensitive information embedded in model weights or activations.Pruning, quantization, and knowledge distillation reduce storage and latency requirements, while distillation may transfer sensitive information through teacher outputs.Source: Privacy-Preserving Model Compression · pp. 19–20
• Federated learning could support collaborative, privacy-preserving compression without centralizing sensitive data.The approach aggregates model updates across clients rather than directly exposing underlying data.Source: Privacy-Preserving Model Compression · p. 20
• Privacy-aware pruning could reduce leakage by selecting parameters or neurons according to their contribution to privacy risk.The proposed pruning strategy incorporates privacy concerns into model-parameter selection.Source: Privacy-Preserving Model Compression · p. 20

Privacy Risk Evaluation

• Comprehensive risk-evaluation frameworks are needed to assess leakage across training, fine-tuning, and inference, including membership and attribute inference attacks.The survey also calls for model-auditing tools within these frameworks.Source: Privacy Risk Assessment · p. 20

Secure Knowledge Sharing Across LLMs

• Secure knowledge transfer across organizations requires methods that protect proprietary data while addressing leakage, model inversion, and unauthorized access risks.SMPC keeps combined data confidential during computation, while ZKPs verify knowledge or computation correctness without revealing the knowledge itself.Source: Secure Knowledge Sharing Across LLMs · p. 21

Interdisciplinary Approaches to Privacy Governance

• Privacy governance requires collaboration among AI researchers, legal experts, and policymakers, including compliance with data-erasure requirements.Benchmarks can give researchers and regulators tools to measure privacy and data-security protections in deployment.Source: Interdisciplinary Approaches to… · pp. 21–22

Conclusion

• The survey reviews how model inversion, training data extraction, and membership inference can expose sensitive information from LLMs.It focuses on privacy leakage and attacks as well as the mechanisms through which models inadvertently reveal data.Source: Conclusion · p. 22
• It categorizes privacy-preservation techniques including inference detection, federated learning, and confidential computing, evaluating their strengths and limitations.The review connects available defenses to the privacy risks they are intended to mitigate.Source: Conclusion · p. 22
• The survey identifies practical implementation challenges and emphasizes scalable, transparent, and efficient directions for future privacy protection.Its synthesis is intended to clarify the LLM privacy landscape and guide privacy-conscious AI systems.Source: Conclusion · p. 22