PaperlayerView exact revision on arXiv

Source-linked AI summary

AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning

Jinyuan Jia, Neil Zhenqiang Gong

arXiv:1805.04810v2cs.CRstat.ML

TL;DR

Attribute inference attacks expose private attributes through public data, while existing defenses are either computationally intractable or costly in utility. AttriGuard uses a two-phase adversarial-noise mechanism with probabilistic selection, and experiments report stronger defense with small utility loss. Its scope includes unresolved questions about detecting and removing the added noise and limitations in approximating game-theoretic optimization.

  • Problem

    Attribute inference attacks threaten privacy because classifiers can infer private attributes from users’ public data, while existing defenses face computational or utility costs.

  • Method

    AttriGuard finds minimum attribute-targeted noise with an adapted evasion attack, then probabilistically selects among those noises under a convex utility-constrained formulation.

  • Results

    75% lower inference accuracy is achieved by modifying at most 4 rating scores on average, and AttriGuard uses significantly less noise than existing defenses at equal reduction.

  • Takeaways & Limitations

    The paper demonstrates that evasion attacks can serve as defensive privacy techniques and that AttriGuard substantially outperforms existing defenses empirically.

  • Takeaways & Limitations

    Noise detectability and removal are not the focus of the work, and the proposed approximation of the game-theoretic formulation is described as not meaningful.

Abstract

from arXiv · show

Users in various web and mobile applications are vulnerable to attribute inference attacks, in which an attacker leverages a machine learning classifier to infer a target user's private attributes (e.g., location, sexual orientation, political view) from its public data (e.g., rating scores, page likes). Existing defenses leverage game theory or heuristics based on correlations between the public data and attributes. These defenses are not practical. Specifically, game-theoretic defenses require solving intractable optimization problems, while correlation-based defenses incur large utility loss of users' public data. In this paper, we present AttriGuard, a practical defense against attribute inference attacks. AttriGuard is computationally tractable and has small utility loss. Our AttriGuard works in two phases. Suppose we aim to protect a user's private attribute. In Phase I, for each value of the attribute, we find a minimum noise such that if we add the noise to the user's public data, then the attacker's classifier is very likely to infer the attribute value for the user. We find the minimum noise via adapting existing evasion attacks in adversarial machine learning. In Phase II, we sample one attribute value according to a certain probability distribution and add the corresponding noise found in Phase I to the user's public data. We formulate finding the probability distribution as solving a constrained convex optimization problem. We extensively evaluate AttriGuard and compare it with existing methods using a real-world dataset. Our results show that AttriGuard substantially outperforms existing methods. Our work is the first one that shows evasion attacks can be used as defensive techniques for privacy protection.

1 Introduction

AttriGuard addresses impractical attribute-inference defenses with a computationally tractable two-phase method that uses adversarial noise and probabilistic selection. On real-world data, it reduces inference accuracy with few modifications and less noise than existing defenses.

  • Attribute inference attacks use classifiers to infer private attributes from public data such as page likes or rating scores.
  • Game-theoretic defenses are computationally intractable for high-dimensional public data, while correlation-based methods can incur large utility loss.
  • AttriGuard first finds minimum noise for each attribute value by adapting evasion attacks to privacy-protection requirements.
  • It then samples an attribute value using a probability distribution and adds the corresponding Phase I noise to the user’s public data.
  • By modifying at most 4 rating scores on average, AttriGuard reduces attacker inference accuracy by 75% across several defense-unaware and adaptive attacks.
  • AttriGuard adds significantly less noise than existing defenses when achieving the same reduction in attacker inference accuracy.

2 Related Work

Prior work spans attacks across social, recommender, mobile, and side-channel settings, alongside game-theoretic, heuristic, and differential-privacy defenses. These approaches trade privacy guarantees, computational tractability, or utility.

  • Attribute inference attacks use public measurements to infer private attributes such as location, political view, gender, or sexual orientation.
  • Recommender-system attackers can use users’ rating-score feature vectors and classifiers such as logistic regression to predict gender.
  • Mobile and side-channel studies infer user locations or other attributes from power consumption, processing time, and sensor measurements.
  • Game-theoretic defenses provide theoretical privacy guarantees but require computationally intractable optimization for attribute inference attacks.
  • Correlation-based defenses modify highly correlated public-data entries, but existing tractable methods can incur large utility loss and may require private attribute access.
  • Local differential privacy protects records under a different privacy goal and can add much larger noise than AttriGuard at the same inference accuracy.

3 Problem Formulation

The formulation models a defender that adds policy-constrained random noise to public data so an unknown classifier’s attribute inference approaches a target distribution within a utility budget. It defines users, attackers, mechanisms, and tractable loss measures.

  • 3.1 User: A user publishes public vector x while protecting private attribute s, whose m possible values are indexed from 1 to m.
  • 3.1 User: Noise-type policies specify whether the defender may modify existing nonzero entries, add changes to zero entries, or combine these operations.
  • 3.2 Attacker: The attacker applies a machine-learning classifier to noisy public data, while the defender does not know the attacker’s classifier.
  • 3.3 Defender: The defender is client-side software that adds noise through a randomized mechanism M to the user’s public data.
  • 3.3 Defender: The defender learns classifier C and uses its output distribution as a proxy for the attacker’s inference distribution q.
  • 3.3 Defender: KL divergence is selected because it yields a convex optimization problem, while expected distance measures utility loss under randomized noise.
  • 3.3 Defender: The defense problem chooses mechanism M* under a noise policy and utility budget β to bring the classifier-induced distribution q toward target distribution p.
  • 3.3 Defender: The L0 utility metric counts modified entries, corresponding to changed ratings or added or removed page likes; L2 can measure rating-change magnitude.

4 Design of AttriGuard

AttriGuard reduces the complexity of randomized defense design by grouping noises according to the classifier’s inferred attribute value. It finds minimum policy-compatible noises with adversarial-machine-learning techniques, then optimizes their sampling probabilities under utility constraints.

  • 4.1 Overview: The two-phase framework groups noises by the attribute value inferred after perturbation, reducing an exponentially parameterized mechanism to representative noises.For group Gi, any noise produces the same inferred attribute value; the mechanism’s probability for that value is the probability of sampling from Gi.
  • 4.2 Phase I: Finding ri: Phase I finds minimum noise ri that makes classifier C predict target attribute value i while respecting noise-type-policy P.The L0 norm is used to measure noise and thus utility loss.
  • 4.1 Overview: AttriGuard demonstrates that evasion attacks can serve as defensive techniques for protecting privacy against attribute inference attacks.This reframes classifier evasion from an offensive method into a privacy-defense mechanism.
  • 4.2 Phase I: Finding ri: PANDA selects policy-compatible entry modifications using a saliency map and step size τ, increasing or decreasing entries according to which change better moves classification toward i.For Modify Exist and Add New, no feasible ri may exist; the method then extends to Modify Add.
  • 4.3 Phase II: Finding M*: Phase II samples one representative noise ri according to mechanism M*, making the classifier’s output probabilities equal the noise-selection probabilities Mi.The simplified mechanism has at most m nonzero parameters.
  • 4.3 Phase II: Finding M*: The Phase II optimization is convex, so M* is a global minimum and can be obtained using KKT conditions followed by solving a single-variable nonlinear equation with Newton’s method.The mechanism balances the target probability distribution against the utility-loss constraint.

5 Evaluations

The evaluation uses a real-world app-review dataset to test AttriGuard across attack settings, parameter choices, and comparisons with existing defenses. Results show reduced inference accuracy, favorable noise and runtime trade-offs, and smaller utility loss than alternatives.

  • Experimental setup: The dataset contains 16,238 users, each rating 23.2 apps on average, represented by 10,000-dimensional review vectors.Users who rated at least 10 apps were retained; 90% formed the training dataset and the remainder formed the testing dataset.
  • Phase I parameterization: PANDA is recommended for Phase I because it balances noise, success rate, and runtime better than the compared evasion attacks.PANDA adds smaller noise than JSMA and is about 800 times and 4,000 times faster than CW for LR-D and NN-D, respectively.
  • Effectiveness of AttriGuard: With LR-D as defender, modifying 3–4 rating scores on average makes several attacks less effective than the baseline attack.LRA-A is more robust than the other evaluated attacks, while LR-A decreases fastest as the utility-loss budget increases.
  • Parameter sensitivity: The target distribution p_t outperforms p_u, yielding lower attacker inference accuracy, especially at relatively large utility-loss budgets.The advantage is attributed to p_t incorporating attribute information from the training dataset, whereas p_u assumes no attribute knowledge.
  • Parameter sensitivity: Using NN-D as the defender lowers attacker accuracy at the same utility-loss budget because same-category classifier transfer is more likely.The attack in this comparison is NN-A.
  • Parameter sensitivity: Training-set overlap has a small impact: attacker accuracy is around 0.10 with identical datasets and remains below 0.15 with no overlap.The comparison uses NN-A, LR-D, and a utility-loss budget of 4.
  • Noise-type policies: Modify Add outperforms Add New, which outperforms Modify Exist, because the policies differ in how flexibly they modify or add ratings.Users review only a small fraction of apps, making Add New more flexible than Modify Exist in this dataset.
  • Comparison with existing defenses: AttriGuard adds substantially smaller noise than all compared defenses at the same inference accuracy and also yields lower recommendation-precision loss.BlurMe and ChiSquare use correlation heuristics, QPM changes public data substantially, and LDP-SH targets a different privacy goal.

6 Discussions and Limitations

The discussion identifies limits of approximating game-theoretic defenses and considers extensions for multiple attributes and dynamic public data.

  • Approximating the game-theoretic optimization: Approximate solutions to the intractable game-theoretic optimization problem remain an open direction.The authors report that QPM incurs larger utility loss than AttriGuard and that a direct AttriGuard-based approximation would be uninformative.
  • Approximating the game-theoretic optimization: A direct AttriGuard approximation of the game-theoretic formulation becomes meaningless because fixing x makes the objective constant.Under that construction, any probabilistic mapping satisfying the utility-loss budget is an approximate solution.
  • Multiple attributes: Multiple private attributes may enable stronger attacks through correlations between attributes.The proposed response is to design the target probability distribution from the attributes’ joint probability distribution.
  • Dynamic public data: The work focuses on one-time public-data release, whereas historical releases could give attackers more information for dynamic-data inference.Extending the framework to dynamic public data is identified as future work.

7 Conclusion and Future Work

The conclusion presents AttriGuard as a practical two-phase defense using optimized evasion attacks and randomized noise selection, with favorable empirical results and several future directions.

  • Conclusion: AttriGuard finds minimum noise for each attribute value in Phase I and randomly selects one corresponding noise in Phase II.Phase I adapts evasion attacks to privacy protection, while Phase II uses random selection to mislead inference.
  • Conclusion: Real-world experiments show defense against attribute inference attacks with small utility loss and significant improvement over existing defenses.The conclusion also identifies adversarial machine learning as useful for privacy protection.
  • Future Work: Future work includes detecting added noise, improving approximate game-theoretic solutions, and extending AttriGuard to dynamic and non-relational public data.Social graphs are given as an example of non-relational public data.

A Game-Theoretic Formulation

The game-theoretic formulation models privacy protection as choosing a probabilistic mapping from true to noisy public data while limiting expected utility loss against an optimal inference attack.

  • A Game-Theoretic Formulation: The formulation considers location inference in which the public and private data are both users’ true locations, with obfuscation to a fake location.The attacker seeks to infer the true location after obfuscation.
  • A.1 Notations: The private attribute s and public data x have joint distribution Pr(s,x), while mapping f produces noisy x′ with probability f(x′|x).The mapping is represented as a matrix whose dimensions depend on the domain size of x.
  • A.2 Privacy Loss: Privacy loss is defined by dp(s, ŝ), which measures the loss associated with the attacker’s inferred attribute value.The metric can assign loss according to the relationship between the true and inferred values.
  • A.3 Utility Loss: Utility loss dq(x,x′) measures the change between true and noisy public data, including the L0 norm counting modified entries.Expected utility loss is defined using the marginal distribution Pr(x) and mapping f.
  • A.4 Defender’s Strategy: The attacker observes x′, uses f and Pr(s,x), and computes a posterior distribution over the private attribute s.The attacker then selects an inferred value ŝ based on that posterior.
  • A.4 Defender’s Strategy: For a chosen ŝ, conditional expected privacy loss is ∑s Pr(s|x′)dp(s, ŝ), and the attacker’s maximum conditional loss is then considered.The formulation proceeds by averaging this quantity over the distribution of x′.
  • A.4 Defender’s Strategy: The defender minimizes unconditional expected privacy loss subject to a utility-loss budget β by selecting the probabilistic mapping f.The auxiliary quantity yx′ represents the maximum expected loss for each noisy output.
  • A.4 Defender’s Strategy: The resulting optimization can be transformed into a linear program, but its computation is exponential in the public-data vector’s dimensionality.This makes the formulation computationally intractable for high-dimensional practical data.

A.5 Limitations

The game-theoretic formulation is computationally intractable in practice because its cost grows exponentially with the dimensionality of the public data vector.

  • A.5 Limitations: The optimization cost is exponential in the dimensionality of the public data vector.Recommender-system vectors can be high-dimensional because they include item ratings and zeros for unrated items.
Loading 1805.04810v2…