Source-linked AI summary
Exacerbating Algorithmic Bias through Fairness Attacks
Ninareh Mehrabi, Muhammad Naveed, Fred Morstatter, Aram Galstyan
TL;DR
Fairness measures’ robustness to deliberate attacks has received less attention than accuracy in adversarial machine learning. This paper proposes anchoring and influence poisoning attacks that target fairness, and reports that they affect fairness measures, with the influence attack on fairness the most effective among the attacks evaluated.
Problem
The robustness of fairness measures to malicious perturbations is understudied, although adversarial machine learning has mostly focused on accuracy.
Source: Introduction · p. 1Method
The paper proposes anchoring attacks that place opposite-label poisoned points near targets and influence attacks that maximize covariance between sensitive attributes and decision outcomes.
Source: Introduction · p. 1, Anchoring Attack · p. 3, Methods · p. 4Results
The influence attack on fairness is the most effective evaluated attack on fairness measures, and anchoring outperforms Solans et al. in most cases.
Source: Results · pp. 5–6Takeaways & Limitations
Experiments indicate that accuracy-oriented poisoning attacks are not suitable for fairness attacks, supporting the paper’s focus on attacks designed for fairness.
Source: Introduction · p. 1The effectiveness of defenses against fairness-targeting attacks and the design of stronger such attacks remain open questions identified for future research.
Source: Conclusion and Future Work · p. 7
Abstract
from arXiv · showhide
Algorithmic fairness has attracted significant attention in recent years, with many quantitative measures suggested for characterizing the fairness of different machine learning algorithms. Despite this interest, the robustness of those fairness measures with respect to an intentional adversarial attack has not been properly addressed. Indeed, most adversarial machine learning has focused on the impact of malicious attacks on the accuracy of the system, without any regard to the system's fairness. We propose new types of data poisoning attacks where an adversary intentionally targets the fairness of a system. Specifically, we propose two families of attacks that target fairness measures. In the anchoring attack, we skew the decision boundary by placing poisoned points near specific target points to bias the outcome. In the influence attack on fairness, we aim to maximize the covariance between the sensitive attributes and the decision outcome and affect the fairness of the model. We conduct extensive experiments that indicate the effectiveness of our proposed attacks.
Introduction
Fairness measures have received growing attention, but their robustness to malicious perturbations is understudied while adversarial machine learning has mostly targeted accuracy. The paper proposes two families of fairness-targeted poisoning attacks and reports experiments showing their effectiveness.
Introduction
- Fairness measures’ robustness to random or malicious perturbations remains understudied, unlike accuracy, which adversarial machine learning has mostly targeted.The authors note that fairness attacks could be used to influence sensitive decisions, including government and credit decisions.Source: Introduction · p. 1
- The paper proposes anchoring attacks that bias decision boundaries and influence attacks that target fairness by maximizing covariance between outcomes and sensitive attributes.Anchoring does not modify the attacker loss; influence attacks can target fairness and accuracy through a regularized adversarial loss.Source: Introduction · p. 1
- Experiments across three datasets and different fairness measures show the attacks affect fairness, while accuracy-oriented poisoning attacks are unsuitable for fairness attacks.The authors also report comparisons with baseline models and concurrent fairness-attack work.Source: Introduction · p. 1
Background on Poisoning Attacks
The background formulates poisoning as an adversary selecting poisoned training points to raise test loss, subject to a poison budget and a feasible set.
Background on Poisoning Attacks
- The adversary’s training set combines clean data Dc with poisoned data Dp, whose size is a fraction ϵ of the clean set.Poisoned points are restricted to feasible set Fβ, selected by a defense mechanism using anomaly detection.Source: Background on Poisoning Attacks · pp. 1–2
- A poisoning attack optimizes over poisoned data points to maximize the adversary’s test loss.The feasible set is intended to help poisoned points blend with natural data and evade anomaly detection.Source: Background on Poisoning Attacks · p. 2
- The described attack samples positive and negative instances and makes class-balance-weighted copies to form the poisoned points.The passage defines the positive and negative clean-data counts used for this allocation.Source: Background on Poisoning Attacks · p. 2
Poisoning Attacks against Fairness
The section sets up poisoning against fairness and describes two strategies: influence attacks that modify the adversarial loss and anchoring attacks that place opposite-label poisons near selected targets.
Poisoning Attacks against Fairness
- The fairness setup assumes two demographic groups and restricts poisoned points to the advantaged or disadvantaged group.The groups are designated advantaged and disadvantaged, with poisoning confined to points belonging to either group.Source: Poisoning Attacks against Fairness · p. 2
Influence Attack on Fairness
- The influence attack modifies an adversarial loss to maximize covariance between sensitive features and signed distance from the decision boundary.Its loss combines accuracy and fairness terms, with λ controlling their trade-off; the experiments set λ = 1.Source: Influence Attack on Fairness · p. 2
- The described influence-attack sampling targets increased bias, though an adversary could instead skew the disadvantaged group toward advantage.The paper follows the former direction for its goal of increasing bias and harming fairness.Source: Influence Attack on Fairness · p. 3
Anchoring Attack
- Anchoring generates a poison near a clean target with the same demographic group but the opposite label, aiming to skew the decision boundary.Targets may be sampled uniformly at random or chosen as popular points; the figure depicts poisons near their targets.Source: Anchoring Attack · p. 3, Figure 1 · p. 3
Evaluation
The evaluation tests fairness-targeted attacks across three datasets, fairness measures, and baselines. The results find influence attack on fairness most effective overall for attacking fairness measures, while its regularization parameter shapes fairness outcomes.
Evaluation
- The evaluation measures accuracy, statistical parity, and equality of opportunity, and compares the proposed attacks with accuracy-focused and fairness-optimized baselines.The attacks are evaluated on German Credit, COMPAS, and Drug Consumption datasets.Source: Evaluation · p. 4, Methods · p. 4, Datasets · p. 4
Datasets
- The experiments use gender as the sensitive attribute and an 80-20 train-test split across German Credit, COMPAS, and Drug Consumption.The prediction tasks are good-versus-bad credit, two-year recidivism, and lifetime cocaine use, respectively.Source: Datasets · p. 4
Measures
- Statistical parity captures predictive outcome differences between advantaged and disadvantaged groups, while equality of opportunity captures their true-positive-rate differences.Both measures are used alongside accuracy to assess the attacks.Source: Measures · p. 4
Methods
- The influence attack on fairness maximizes covariance between signed distance from the decision boundary and sensitive features, whereas anchoring places poisoned points near same-group, different-label targets.The non-random anchoring variant selects a target with the most same-group, same-label neighbors; the random variant selects a target at random.Source: Methods · p. 4
Results
- Influence attack on fairness is the most effective of the evaluated attacks at attacking fairness measures, while the accuracy-focused influence attack does not attack fairness well.The proposed influence attack on fairness and anchoring attack also outperform Solans et al. in affecting fairness measures in most cases.Source: Results · pp. 5–6, Figure 2 · p. 5
- Increasing λ makes the influence attack on fairness affect fairness measures more; at lower λ, it behaves similarly to the accuracy-focused influence attack.The changes with λ are more significant at ϵ = 1 than at lower values such as ϵ = 0.1.Source: Results · p. 6, Figure 3 · p. 6
Related Work
The related work spans fair machine learning and adversarial machine learning, whose intersection motivates attacks that target fairness. The paper contrasts its two attacks with prior fairness-poisoning work.
Adversarial Machine Learning
- Data poisoning manipulates training instances, while evasion attacks create test-time examples; prior poisoning work focused on accuracy, whereas this paper treats fairness as a performance measure too.Source: Adversarial Machine Learning · p. 7
Fair Machine Learning
- Fair machine learning research develops fairness measures and mitigation methods across preprocessing, in-processing, and post-processing; this work uses established measures and draws on fair-classification loss functions.Source: Fair Machine Learning · p. 7
Adversarial Fair Machine Learning
- Compared with Solans, Biggio, and Castillo (2020), anchoring uses no loss function, while IAF introduces a different fairness-oriented loss and exposes a fairness–accuracy trade-off through its hyperparameter.Source: Adversarial Fair Machine Learning · p. 7
Conclusion and Future Work
The paper introduces anchoring and influence-based poisoning attacks to target fairness, with different speed, subtlety, and trade-off-control properties. It identifies stronger attacks and defenses as future research directions.
Conclusion
- The two attacks complement each other: gradient-based IAF is slower and harsher, while gradient-free anchoring is considerably faster and more subtle.Source: Conclusion and Future Work · p. 7
- Unlike anchoring, IAF can control the fairness–accuracy trade-off; the paper also notes that anchoring cannot explicitly control it.Source: Conclusion and Future Work · p. 7
Future Work
- Future work includes adapting defenses to fairness-targeting attacks, assessing whether accuracy-attack defenses apply, and exploring stronger attacks and other poisoning strategies.Source: Conclusion and Future Work · p. 7
Ethics Statement
The ethics statement frames fairness-targeting attacks as a dual-use concern and calls for research toward more robust and fair machine-learning models.
Ethics Statement
- The proposed attacks are intended to support robustness of fairness, but could also enable attackers to harm fairness in existing machine-learning systems.Source: Ethics Statement · p. 8
- The paper combines fairness and adversarial machine learning, fields it describes as having safety, security, and ethics implications, and anticipates collaboration toward more robust and fair models.Source: Ethics Statement · p. 8