Source-linked AI summary
Privacy in Deep Learning: A Survey
Fatemehsadat Mireshghallah, Mohammadkazem Taram, Praneeth Vepakomma, Abhishek Singh, Ramesh Raskar, Hadi Esmaeilzadeh
TL;DR
Deep learning systems use sensitive, often crowdsourced data, creating risks of direct leakage and inference about training data or models. This survey organizes privacy threats and mitigation techniques across data aggregation, training, and inference, and finds that test-time inference privacy receives comparatively little attention. It therefore identifies test-time inference as a gap for future research.
Problem
Deep-learning datasets and models can expose sensitive information through direct breaches or inference attacks, while privacy protection is uneven across the data, training, and inference stages.
Method
The survey categorizes deep-learning privacy threats and reviews privacy-preserving mechanisms for data aggregation, training, and test-time inference.
Results
The survey finds an apparent disparity in research effort, with little attention devoted to privacy during the inference phase.
Takeaways & Limitations
Test-time inference privacy remains a stated gap in the literature and a direction for future research.
Takeaways & Limitations
Differential privacy can degrade performance, with less-represented subgroups losing more accuracy, and remote trusted enclaves may access raw user data.
Abstract
from arXiv · showhide
The ever-growing advances of deep learning in many areas including vision, recommendation systems, natural language processing, etc., have led to the adoption of Deep Neural Networks (DNNs) in production systems. The availability of large datasets and high computational power are the main contributors to these advances. The datasets are usually crowdsourced and may contain sensitive information. This poses serious privacy concerns as this data can be misused or leaked through various vulnerabilities. Even if the cloud provider and the communication link is trusted, there are still threats of inference attacks where an attacker could speculate properties of the data used for training, or find the underlying model architecture and parameters. In this survey, we review the privacy concerns brought by deep learning, and the mitigating techniques introduced to tackle these issues. We also show that there is a gap in the literature regarding test-time inference privacy, and propose possible future research directions.
1 Introduction
Deep learning’s deployment across sensitive domains creates privacy risks from direct exposure and inference attacks. This survey reviews existing threats and privacy-preserving methods while identifying limited attention to test-time inference privacy.
- 1 Introduction: DNN deployment in healthcare, finance, and other production systems can expose sensitive information such as genetic markers and confidential model parameters.The survey connects deep learning’s broad adoption with large, often sensitive datasets and confidential models.
- 1 Introduction: Attackers may directly access training data, queries, or model parameters, or indirectly infer membership, attributes, model properties, and training-data properties.Indirect examples include membership inference, model inversion, attribute inference, and property inference.
- 1 Introduction: Existing defenses primarily address data aggregation and private training, while only a handful protect test-time inference inputs.The survey groups defenses into data sanitization, private training, and test-time inference protection.
- 1 Introduction: The survey reviews privacy threats and mitigations, highlights a literature gap in test-time inference privacy, and excludes adversarial attacks and data poisoning.Its scope is privacy-specific vulnerabilities rather than all deep-learning security threats.
2 Existing Threats
Deep-learning privacy threats include direct breaches and indirect attacks that infer training participation, sensitive attributes, model details, or dataset properties. The survey organizes these attacks by exposure type, access requirements, and available model outputs.
- 2 Existing Threats: Indirect attacks include membership, model inversion, hyperparameter, parameter, and property inference, whereas direct threats expose information through curators, communication links, or clouds.The attack taxonomy distinguishes obtaining information directly from inferring or guessing it.
- 2 Existing Threats: Membership inference determines whether a given instance contributed to training, using black-box confidence scores, white-box loss thresholds, or even few-query label access.Different attacks relax requirements such as shadow models, target-model structure, confidence values, or distribution-matched data.
- 2 Existing Threats: Model inversion and attribute inference recover sensitive features or training examples from model outputs and nonsensitive attributes, including genomic information and reconstructed faces.The facial reconstruction example uses a person’s name and a facial-recognition system’s class confidence score.
- 2 Existing Threats: Model extraction and property inference can reveal confidential model parameters or architecture and specific patterns about the data used for training.The survey treats model ownership and memorized training information as privacy concerns.
3 Privacy-Preserving Mechanisms
Privacy-preserving mechanisms are organized around protecting data aggregation, model training, and test-time inference. The survey presents these as three major areas in the privacy-preservation landscape.
- 3 Privacy-Preserving Mechanisms: The survey divides privacy-preserving mechanisms into private data aggregation, private model training, and privacy protection for test-time inference.Aggregation protects contributors while collecting datasets; training mechanisms protect information about participants during model learning.
3.1 Data Aggregation
Data-aggregation privacy mechanisms range from identifier removal and k-anonymity to formal, cryptographic, and context-aware approaches. Differential privacy limits the effect of one participant’s data, while other frameworks tailor protection to deployment context.
- 3.1 Data Aggregation: Context-free privacy methods ignore intended data use, whereas context-aware methods model dataset statistics and can improve privacy or fairness through information degradation.Semantic security provides a cryptographic privacy requirement, and secure multiparty computation is based on it.
- 3.1 Data Aggregation: Naive anonymization removes identifiers such as names, addresses, and postcodes, but auxiliary information can enable re-identification and expose sensitive records.The Netflix Prize example illustrates de-anonymization using publicly available movie-rating information.
- 3.1 Data Aggregation: K-anonymity requires each participant’s quasi-identifier combination to match at least k−1 others, with the goal of impeding re-identification.Its guarantee is defined over available attributes and exact matching rows.
- 3.1 Data Aggregation: ϵ-Differential Privacy bounds output-probability changes between datasets differing in one element, approximating the effect of an individual opting out.The Laplace mechanism adds noise calibrated to a function’s global sensitivity, which can be difficult to determine for deep neural networks.
- 3.1 Data Aggregation: Differential privacy composes multiple mechanisms by adding their privacy budgets, while local differential privacy randomizes data before participants share it.Pufferfish and geo-indistinguishability adapt differential privacy to application-specific or location-based settings.
3.2 Training Phase
Private training methods are organized around differential privacy, homomorphic encryption, and secure multi-party computation. Differential privacy can intervene at multiple training-pipeline locations, while encryption-based approaches protect computation over encrypted data but face practical or functional constraints.
- 3.2 Training Phase: Private-training research is categorized into differential privacy, homomorphic encryption, and secure multi-party computation.The survey uses these three groups to organize the private-training literature.
- 3.2.1 Differential Privacy: Differential privacy can add randomization at the input, objective function, gradient updates, trained-model output, or labels.These are the five pipeline locations identified by the survey.
- 3.2.1 Differential Privacy: Gradient perturbation is widely used for private deep learning because gradient clipping addresses potentially unbounded gradient norms.The passage contrasts this approach with limitations of other perturbation strategies.
- 3.2.1 Differential Privacy: GDP provides a refined privacy analysis for Adam-based private DNN training and yields lower epsilon values than moments-accountant analysis.Bu et al. apply Gaussian Differential Privacy to analyze privacy-budget exhaustion without developing moments-accountant-style techniques.
- 3.2.1 Differential Privacy: Differential privacy reduces utility through noise and clipping, with smaller, less-represented population groups losing more accuracy.The survey reports disparate utility loss across subgroups of different sizes.
- 3.2.2 Homomorphic Encryption: Homomorphic-encryption methods approximate neural activation functions with low-degree polynomials to enable encrypted convolutional-network training.The reported approximations target ReLU, Sigmoid, and Tanh.
3.3 Inference Phase
Inference privacy protects data sent to already-trained models, but the survey finds substantially less work here than for training privacy. Existing approaches use differential privacy, encryption, secure computation, or information-theoretic feature and representation transformations, each with accuracy or efficiency trade-offs.
- Overview: Inference privacy has received less research attention than training privacy, despite targeting deployed Inference-as-a-Service systems that should not learn from user-provided data.The survey describes inference systems as performing designated tasks on user data without further learning.
- Information Theoretic Privacy: Information-theoretic methods reduce unnecessary information in transmitted inputs or intermediate representations while preserving the primary inference task.DPFE requires private labels and retraining, whereas Shredder learns additive noise distributions without changing or retraining a pretrained network.
- Differential Privacy: Differential privacy is uncommon for inference because its worst-case guarantees require high-intensity noise that can degrade pretrained-network performance.Arden combines data nullification and noise injection across an edge-cloud partition but requires noisy retraining of the entire network.
- Homomorphic Encryption: Homomorphic-encryption inference improves substantially over earlier systems, with GAZELLE reported as three orders of magnitude faster than CryptoNets.Other encrypted approaches approximate nonlinear activations or use preprocessing to improve efficiency while preserving accuracy.
- Secure Multi-Party Computation: Secure multi-party computation methods focus on reducing inference cost, with Chameleon achieving a 4.2× latency improvement over MiniONN and reported accuracy loss below 1% versus pretrained models.The cited methods primarily optimize heavy matrix-multiplication workloads.
- Information Theoretic Privacy: Both DPFE and Shredder reduce mutual information with original data; Shredder generally removes more, while DPFE performs slightly better for private-label inference.DPFE benefits from access to private labels during training, whereas Shredder is more general.
4 Privacy-Enhancing Execution Models and Environments
Privacy-enhancing execution models reduce exposure by keeping data local, splitting computation, aggregating updates securely, or executing code in trusted environments. They complement, rather than replace, dedicated privacy-preserving schemes and retain distinct practical limitations.
- Federated Learning: Federated learning keeps training data local while clients collaboratively train a model under central-server administration.The survey links this setting to focused collection and data minimization as ways to alleviate centralized-learning privacy risks.
- Federated Learning: Federated-learning workflows span problem identification, client instrumentation, optional simulation, federated training, model evaluation, and deployment.Deployment includes live A/B testing, manual quality assurance, and staged rollout.
- Federated Learning: Secure aggregation privately combines local model updates so no party, including the aggregator, receives another party’s update in cleartext.Federated learning is widely combined with secure multi-party computation and differential privacy.
- Split Learning: Split learning divides a neural network between client and server, sending cut-layer outputs, or smashed data, instead of processing the full network on one side.The cited figure contrasts vanilla splitting with a boomerang configuration that also withholds labels.
- Trusted Execution Environments (TEEs): Trusted execution environments provide attested, verifiable cloud execution with integrity and confidentiality, but remote enclaves still receive raw user data.Hardware breaches can therefore compromise privacy despite the trusted-execution model.
5 Conclusion
The survey synthesizes privacy threats and defenses across deep-learning data aggregation, training, and inference. Its central conclusion is that inference-phase privacy has received little attention relative to the other phases.
- Conclusion: The survey finds an apparent disparity in privacy research across data aggregation, training, and inference, with little attention to users’ privacy during inference.It frames the need for privacy protection against the backdrop of data-rich, compute-intensive machine learning using people’s sensitive information.