PaperlayerView exact revision on arXiv

Source-linked AI summary

More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence

Tianqing Zhu, Dayong Ye, Wei Wang, Wanlei Zhou, Philip S. Yu

arXiv:2008.01916v1cs.CRcs.LGstat.ML

TL;DR

The paper addresses the gap in documenting how differential privacy mechanisms and their properties can address AI problems beyond privacy preservation. It reviews applications across machine learning, deep learning, distributed learning, and multi-agent systems, finding uses for stability, fairness, security, communication control, and performance. The review also identifies limitations and areas requiring further research.

  • Problem

    Existing work had not systematically documented which differential privacy mechanisms and properties address AI problems beyond privacy preservation.

  • Method

    The paper reviews differential privacy applications in machine learning, deep learning, distributed deep learning, and multi-agent systems, including their strengths, limitations, and potential research areas.

  • Results

    The reviewed studies use differential privacy for privacy preservation as well as learning stability, fairness, security, communication control, and improved agent-learning performance.

  • Takeaways & Limitations

    Differential privacy has untapped applications across AI, while more efficient mechanisms and further study in new or dynamic environments remain worthwhile.

Abstract

from arXiv · show

Artificial Intelligence (AI) has attracted a great deal of attention in recent years. However, alongside all its advancements, problems have also emerged, such as privacy violations, security issues and model fairness. Differential privacy, as a promising mathematical model, has several attractive properties that can help solve these problems, making it quite a valuable tool. For this reason, differential privacy has been broadly applied in AI but to date, no study has documented which differential privacy mechanisms can or have been leveraged to overcome its issues or the properties that make this possible. In this paper, we show that differential privacy can do more than just privacy preservation. It can also be used to improve security, stabilize learning, build fair models, and impose composition in selected areas of AI. With a focus on regular machine learning, distributed machine learning, deep learning, and multi-agent systems, the purpose of this article is to deliver a new view on many possibilities for improving AI performance with differential privacy techniques.

I. INTRODUCTION

AI’s growing reliance on data has introduced privacy, security, stability, fairness, and communication problems. The paper presents differential privacy as a framework whose randomization and privacy properties can address several of these issues across AI areas.

  • AI’s increasing reliance on data has created problems including privacy violations, security issues, model instability, unfairness, and communication overhead.
  • Differential privacy limits the impact of any one individual’s record on aggregate outputs, preventing inference about that record even when other records are known.
  • Randomization in differential privacy extends its use beyond privacy preservation to stable learning and other emerging AI problems.
  • A. Examples: In machine learning, differential privacy mechanisms can resample data for fairness, add noise to models for privacy, and generate testing data to reduce overfitting.
  • A. Examples: In multi-agent systems, differential privacy can transfer knowledge, restrict communication through privacy budgets, and reduce the impact of malicious agents.
  • B. AI areas: The paper organizes AI into machine learning, deep learning, and multi-agent systems, while treating robotics, computer vision, and NLP as application areas.

C. Differential privacy in AI areas

Differential privacy offers AI-relevant properties beyond privacy preservation, including stability, security, fairness, and composition. The paper surveys how these properties have been explored across machine learning, deep learning, and multi-agent systems, while noting that applicability remains open in some areas.

  • Utility and scope: Differential privacy mechanisms can extend AI algorithms through calibrated randomization, with utility potentially increasing when privacy also contributes to stability, security, or fairness.The paper notes that privacy preservation normally has a utility cost, but added properties may improve ultimate performance in settings such as federated learning or fairness.
  • Properties of differential privacy: Differential privacy mechanisms provide privacy preservation, stability, security, fairness, and composition as properties relevant to AI.Privacy hides individuals in aggregate information; stability limits outcome changes from modifying one record; fairness can reduce dependence on sensitive attributes; composition controls steps and communication loads.
  • Applications across AI areas: In machine learning, differential privacy has been applied to private learning, stability, and fairness; deep-learning work has focused primarily on privacy, with distributed and federated learning also investigated.The paper’s table-based overview distinguishes the properties explored across its AI disciplines.
  • Applications across AI areas: In multi-agent systems, differential privacy has been used to guarantee privacy, provide security, and ensure composition.These applications are presented as part of the paper’s cross-disciplinary survey.
  • Utility and scope: Blank cells in the surveyed property table indicate unexplored applications rather than proof that differential privacy cannot be applied there.The paper leaves open the possibility that additional AI problems may benefit from differential privacy’s advantages.
  • Paper contributions: The paper summarizes differential-privacy properties, shows their application to diverse AI areas, and explores further possibilities for integrating the two fields.Its stated contributions cover machine learning, deep learning, and multi-agent systems.

B. Randomization: Laplace mechanism

The Laplace mechanism protects numeric outputs by adding independent noise, while Gaussian noise provides another mechanism associated with L2 sensitivity. The section places these mechanisms within the broader set of differential-privacy tools summarized in the paper.

  • Laplace mechanism: The Laplace mechanism adds independent noise to numeric outputs and provides epsilon-differential privacy.Its definition applies the mechanism to a function over a dataset.
  • Gaussian mechanism: The Gaussian mechanism adds zero-mean isotropic Gaussian noise and provides epsilon, delta-differential privacy.The noise is sampled from a Gaussian distribution and is tied to L2 sensitivity.
  • Overview: Table I summarizes the properties of differential privacy explored across artificial-intelligence areas.The table is presented as an overview alongside the mechanism discussion.

D. Exponential mechanism

The exponential mechanism randomizes non-numeric outputs by sampling according to a score function that measures output quality. Its privacy behavior depends on the application-specific score and its sensitivity, within broader differential-privacy composition principles.

  • Exponential mechanism: The exponential mechanism randomizes results for non-numeric queries using a score function that evaluates output quality.The score function is application-dependent, so different applications can require different score functions.
  • Exponential mechanism: The exponential mechanism’s score function and sensitivity determine how candidate outputs are randomized under differential privacy.The mechanism is defined using q(D, φ) and the sensitivity of the score function.
  • Composition: Parallel composition applies mechanisms to disjoint dataset subsets and yields a privacy guarantee determined by the largest privacy budget.For mechanisms with guarantees ϵ_i, the combined guarantee is max{ϵ_1, ..., ϵ_m}.
  • Composition: Sequential composition adds privacy budgets across a series of randomized computations on the same dataset.If each of m mechanisms provides an ϵ guarantee, the combined guarantee is (m · ϵ)-differential privacy.
  • Private learning: Private-learning research extends non-private algorithms with Laplace or exponential mechanisms, noise in objective functions, and related learning procedures.The surveyed procedures include empirical risk minimization, prediction, Bayesian inference, and multi-armed bandits.
  • Learning stability: Learning stability is linked to generalization because stable predictions change little when training data is slightly modified, reducing over-fitting.Differential privacy is described as stronger than previous stability notions and as possessing adaptive composition guarantees.

2) Differential privacy in learning stability:

Differential privacy connects randomized data analysis with learning stability, fairness, and privacy, but fairness applications remain constrained by similarity measurement and composition challenges. The surveyed evidence emphasizes both practical mechanisms and unresolved research problems.

  • Learning stability: Differentially private interfaces can provide statistically valid estimates on holdout data during adaptive analysis, supporting stable learning.The interface restricts access to testing data while allowing unrestricted access to training data.
  • Learning stability: For sufficiently large testing sets, the interface returns vf such that |vf − y| ≤ τ with probability at least 1 − β.τ is the analyst’s chosen error and β is the confidence parameter.
  • Learning stability: Randomization during analysis can improve learning stability by obfuscating observations, removing data points, or switching labels before results are selected.The raw dataset is used only to compute final reported values in the described approach.
  • Fairness: Fairness methods use randomization to treat similar individuals similarly, preserve useful attributes, or reduce information about protected-subgroup membership.The approaches include Lipschitz-based distributions and learned intermediate representations.
  • Fairness: Designing a fairness score function remains difficult because differential-privacy fairness assumes similarity between individuals, yet similarity is hard to estimate across the feature universe.The paper identifies similarity measurement and score-function design as key obstacles requiring further exploration.
  • Fairness and privacy: Fairness and privacy preservation are difficult to achieve simultaneously because fairness concerns attributes while privacy concerns individual record membership.One proposed solution adds different amounts of differentially private noise to polynomial coefficients of a constrained objective function.
  • Open problems: Fairness research remains concentrated on classification, while applications to online learning, bandits, and reinforcement learning need further investigation.The paper also identifies composition fairness as a challenge because fair classifiers and decisions may not compose reliably.

D. Summary of differential privacy in machine learning

Differential privacy can address privacy, stability, fairness, and utility-related challenges in machine learning and deep learning, but these benefits involve trade-offs and unresolved limitations.

  • Machine learning: Differential privacy can improve machine-learning stability and fairness through private access to testing data and differentially private resampling.The paper links private testing-set access to stability and fresh private resampling to fairness.
  • Limitations: Open issues include privacy–utility trade-offs, limited applicability of stable learning to non-regularized losses, and fair-learning mechanisms that require unavailable sample utilities.The paper calls for better generalization with regularized loss functions and new mechanisms for fair learning.
  • Machine learning: Sampling from the data universe can improve machine-learning performance to some extent.The paper also proposes generating testing data and using small-database mechanisms for accurate sampling.
  • Deep learning: Deep-learning inference defenses add noise at the dataset, loss, gradient, or model-weight stages, targeting attribute and membership inference attacks.Dataset perturbation primarily targets attribute inference, while loss and gradient perturbation target membership inference; weight perturbation can target both.
  • Deep learning: Gradient perturbation is common because gradient clipping supplies the sensitivity bound required before applying differential privacy.The paper identifies gradient noise as the most common deep-learning approach.
  • Deep learning: Dataset noise can defend against attribute inference but may significantly reduce model utility, whereas loss or gradient noise only slightly affects utility and mainly resists membership inference.Noise added to weights or classes can resist both attack types but may drastically harm utility after training.

B. Differential privacy in distributed deep learning

Distributed deep learning applies differential privacy by perturbing inputs, outputs, objectives, gradients, or weights, with mechanisms designed to preserve privacy while supporting collaborative learning.

  • Noise-placement mechanisms: Distributed deep-learning privacy methods add noise at multiple stages, including input datasets, output classes, loss functions, gradients, and model weights.The reviewed methods extend privacy protection across different parts of the distributed learning pipeline.
  • Input perturbation: Distributed Bayesian learning can combine secure multiparty communication with client-side Gaussian noise and secret sharing.Each client divides its data and noise into shares before sending them to a server.
  • Objective and gradient perturbation: Collaborative deep-learning methods can share model parameters or small parameter subsets instead of datasets, while perturbing objectives or gradients to preserve privacy.Functional mechanisms perturb neural-network objective functions, and gradient-sharing systems support joint model training without dataset exchange.
  • Gradient and weight perturbation: Differentially private distributed stochastic gradient descent clips gradients, adds Gaussian noise, and tracks cumulative privacy cost with privacy and moment accountants.Other approaches perturb leader gradients, aggregated models, or gradient updates in distributed settings.
  • Noise allocation: A heterogeneous Gaussian mechanism redistributes noise across network layers and gradients to seek a trade-off between model utility and privacy loss.A noise redistribution vector changes the variance of the Gaussian distribution.
  • Output perturbation: PATE combines independently trained teacher models, an aggregation mechanism, and a student model while adding Laplace noise to output classes.The method protects private data labels and has been applied to generative adversarial networks.

3) Summary of differential privacy in distributed deep learning:

Distributed and federated learning face synchronization, collusion, policy-compliance, and update-leakage problems. Differential privacy is presented as a way to address these concerns while preserving collaborative model training.

  • Distributed deep learning: Synchronization in distributed deep learning can reduce learning rates or training efficiency, and differential privacy is proposed as a tool for modeling coordination among privacy-constrained modules.The coordination problem can be represented as a multi-player cooperative game in which differential privacy helps achieve equilibria.
  • Distributed deep learning: Joint differential privacy can protect an individual user even when all other users collude against that user.This addresses the non-collusion assumption common in existing distributed-learning methods.
  • Distributed deep learning: Differential privacy may help users truthfully report their types and follow recommendations when privacy policies govern data access.The paper identifies policy noncompliance as a separate challenge for distributed learning.
  • Federated learning: Federated learning keeps training data on local devices, but model updates can still leak information about the data used during training.Local storage alone does not provide a strong privacy guarantee when conventional learning methods expose model parameters.
  • Federated learning: Federated-learning defenses use differential privacy to hide whether an individual client participated, including random client selection and Gaussian noise on local-model differences.Other approaches combine differential privacy with secure multiparty computation or quantization to address privacy, collusion, and communication costs.
  • Federated learning: Private distributed data-analysis protocols can produce meaningful aggregate statistics with small total error even when some users fail to respond.The cited approach uses computational differential privacy with an untrusted aggregator.

3) Summary of differential privacy in federated learning:

Federated learning avoids transferring training data but remains exposed to attacks, learning constraints, and communication or device limitations. Differential privacy is presented as a tool for privacy, security, and communication control in distributed learning.

  • Federated learning keeps training data on mobile devices, addressing basic privacy concerns without transferring data to a cloud centre.
  • Federated learning faces attacks including user-membership inference, message tampering, and model poisoning.
  • Its learning challenges include accuracy, scalability, efficiency, and fault tolerance.
  • System-level constraints include communication efficiency, mobile-device power and computation, and infrastructure reliability.
  • Most surveyed distributed and federated deep-learning studies use the Gaussian mechanism because differentiability is needed to calculate learning-model gradients.
  • Differential privacy alone may not fully protect models or training data because GAN-based adversaries can generate samples resembling a victim’s private dataset.

1) Differential privacy to improve the security of the reinforcement learning:

In multi-agent reinforcement learning, differential privacy addresses malicious advice and communication overhead by randomizing adviser selection and limiting communication through the privacy budget. The approach is reported to improve performance over broadcast-based learning when malicious agents are present.

  • Security in multi-agent reinforcement learning: Differential privacy reduces the impact of malicious agents by randomizing which neighbors are selected for advice.Exponential or Laplace mechanisms can be applied using neighbors’ reward histories.
  • Security in multi-agent reinforcement learning: An adviser-selection algorithm uses the exponential mechanism after an agent receives advice from both honest and malicious neighbors.
  • Security in multi-agent reinforcement learning: The privacy budget controls communication because the system stops communicating when the budget is exhausted.
  • Security in multi-agent reinforcement learning: Compared with broadcast-based learning, differential privacy achieves better agent-learning performance with less communication overhead when malicious agents are present.Performance is normally evaluated using total system reward and convergence rate.
  • Related multi-agent applications: Differentially private auction mechanisms use exponential-mechanism-based randomization for prices, winners, or payments while targeting privacy and allocation objectives.The surveyed mechanisms cover spectrum and cloud-resource auctions.

1) Summary of differential privacy in auctions:

Differential privacy in auctions protects bidding information while supporting revenue, strategy, and allocation objectives, but conventional mechanisms assume direct auctioneer access to buyers. Social-network auctions create additional privacy problems during bid propagation and through exposed relationships.

  • Auction mechanisms: Existing auction approaches typically assume that a seller or auctioneer can directly interact with all potential buyers.
  • Social-network auctions: In social-network auctions, bids must propagate through potentially competing intermediaries, creating privacy risks and the possibility of fake bids that mislead sellers.
  • Social-network auctions: Bid-propagation trajectories can reveal information about a buyer’s social relationships.
  • Game-theoretic applications: Differential privacy research in game theory targets both performance properties such as stability and equilibrium and the privacy of agents.
  • Game-theoretic applications: Differentially private game mechanisms can support approximate equilibria, truthful reporting, robustness to malicious users, and efficiency under changing populations.

3) Summary of differential privacy in game theory:

The paper surveys differential privacy as a tool for privacy, security, stability, and performance across game theory and interacting-agent systems. It also identifies scope boundaries, including limited dynamic-environment research and unresolved online meta-learning requirements.

  • Limitations and future directions: Current game-theory research mainly studies static environments, leaving changes in strategies and utilities within dynamic environments open.
  • Game theory: Game-theory research using differential privacy mainly addresses performance, including stability and equilibrium, or agent privacy.
  • Multi-agent systems: Most surveyed multi-agent studies use differential privacy to protect interaction information, while some instead target malicious agents or agent performance.

E. Multi-agent systems

The paper examines how differential privacy can address privacy and knowledge-sharing challenges in multi-agent systems, including action advising, knowledge transfer, and reasoning. It also identifies limitations in existing approaches and directions for future research.

  • Action advising: Action advising traditionally requires a teacher agent to have visited the student’s current state, but differential privacy may relax this assumption for similar states.Similar states are treated as neighbouring datasets, while generated advice is treated as query output.
  • Knowledge transfer: Differential privacy is proposed as a lower-overhead alternative to homomorphic cryptosystems for privacy-preserving knowledge transfer in resource-constrained systems.The passage specifically gives wireless sensor networks as an example of a resource-constrained setting.
  • Reasoning: Reasoning can query private known facts, creating a privacy-preservation challenge when reasoning is combined with AI systems such as neuro-symbolic learners.Existing work mainly uses reasoning to prove privacy guarantees for programs rather than to protect facts during reasoning.
  • Action advising: MADRL action advising lets students query a teacher’s knowledge base, but the large state space can make that knowledge base difficult to manage.This contrasts with regular multi-agent reinforcement learning, where the teacher offers an action directly.
  • Scope and outlook: The article reviews differential privacy across selected AI areas, discusses current studies’ strengths and limitations, and identifies untapped research opportunities.Its stated focus includes machine learning, deep learning, and multi-agent learning.
Loading 2008.01916v1…