Source-linked AI summary
Poisoning Attacks on Fair Machine Learning
Minh-Hao Van, Wei Du, Xintao Wu, Aidong Lu
TL;DR
Attacking fair machine learning models has received less attention, and generating poisoning samples remains challenging because each sample’s impact on accuracy loss or fairness violation is difficult to quantify. PFML provides three online training-data attacks and evaluates them on real datasets. The reported results show reduced model accuracy and flexible targeting of accuracy or fairness, with performance comparisons against baselines.
Problem
Generating poisoning samples that attack algorithmic fairness remains challenging because each sample’s impact on accuracy loss or fairness violation is difficult to quantify.
Source: Introduction · p. 1Method
PFML uses online adversarial sampling, labeling, and feature modification to poison training data and quantify candidate-point impacts on accuracy loss and fairness violation.
Source: Introduction · p. 1Results
PFML attacks significantly reduce accuracy versus benign models and show stronger accuracy or fairness attack performance than the reported baselines at the corresponding attack-focus settings.
Source: Evaluation of PFML with Equalized Odds · pp. 5–6, Evaluation of PFML with Demographic… · p. 6Takeaways & Limitations
The attack focus can be adjusted between prediction accuracy and fairness violation using α.
Source: Evaluation of PFML with Equalized Odds · p. 5, Evaluation of PFML with Demographic… · p. 6The studied attacks are indiscriminate; targeted attacks, counterfactual fairness, and robust defenses are left for future work.
Source: Conclusions and Future Work · p. 7
Abstract
from arXiv · showhide
Both fair machine learning and adversarial learning have been extensively studied. However, attacking fair machine learning models has received less attention. In this paper, we present a framework that seeks to effectively generate poisoning samples to attack both model accuracy and algorithmic fairness. Our attacking framework can target fair machine learning models trained with a variety of group based fairness notions such as demographic parity and equalized odds. We develop three online attacks, adversarial sampling , adversarial labeling, and adversarial feature modification. All three attacks effectively and efficiently produce poisoning samples via sampling, labeling, or modifying a fraction of training data in order to reduce the test accuracy. Our framework enables attackers to flexibly adjust the attack's focus on prediction accuracy or fairness and accurately quantify the impact of each candidate point to both accuracy loss and fairness violation, thus producing effective poisoning samples. Experiments on two real datasets demonstrate the effectiveness and efficiency of our framework.
Introduction
The paper introduces PFML, a framework for poisoning fair machine learning models by targeting prediction accuracy or fairness. Its three online attacks select, relabel, or modify training samples, with candidate-point impacts assessed on both accuracy loss and fairness violation.
Introduction
- PFML addresses the challenge of quantifying each poisoning sample’s impact on accuracy loss and fairness violation in a trained fair machine learning model.The framework lets attackers adjust whether they focus on decreasing prediction accuracy or increasing fairness violation.Source: Introduction · p. 1
- The three online attacks—adversarial sampling, adversarial labeling, and adversarial feature modification—perturb training data while leaving test data unchanged.Sampling selects from a candidate attack dataset with the clean data’s underlying distribution; labeling flips labels, and feature modification changes selected features.Source: Introduction · p. 1
- PFML targets group-based fairness notions including demographic parity and equalized odds, and experiments on two real datasets demonstrate effectiveness and efficiency.Source: Introduction · p. 1
Background
Fair machine learning minimizes training loss while constraining empirical fairness under a chosen group-based notion. The poisoning approach selects training points iteratively from a feasible set, updates model parameters, and can sample, relabel, or modify points.
Fair Machine Learning
- Fair machine learning minimizes empirical accuracy loss subject to a fairness constraint, with fairness defined relative to a sensitive feature.The setup assumes a binary sensitive feature; demographic parity requires predicted labels to be independent of it.Source: Fair Machine Learning · pp. 1–2
- A δ-fair classifier has empirical fairness gap no greater than the user-specified threshold δ; demographic parity and equalized odds are two fairness notions.Equalized opportunity is a weaker form of equalized odds, requiring non-discrimination only within the advantaged outcome group.Source: Fair Machine Learning · pp. 1–2
Data Poisoning Attack
- The poisoning attacks assume access to training data during preparation and knowledge of the model structure and fairness constraint; attackers select candidate features from data sharing the clean set’s underlying distribution.The attacker can sample, relabel, or modify a fraction of training data to reduce test accuracy.Source: Data Poisoning Attack · p. 2
- The online algorithm iteratively selects a feasible poisoning point with the highest loss under the current parameters, adds it to the poisoning set, and updates the model parameters.Adversarial labeling flips selected labels, while adversarial feature modification changes selected feature vectors.Source: Data Poisoning Attack · p. 2
Data Poisoning Attack on FML
The section formulates poisoning as a weighted accuracy–fairness attack, relaxes fairness constraints for optimization, and describes an iterative attack procedure and its complexity.
Problem Formulation
- The attacker maximizes a weighted combination of prediction loss and fairness loss, with α controlling the focus and γ scaling the losses.The user’s inner optimization minimizes classification loss subject to a fairness constraint; the attacker’s outer optimization selects poisoning data.Source: Problem Formulation · p. 2
- Figure 1 tracks accuracy loss and equalized-odds fairness loss as PFML-AS adds poisoning samples on COMPAS; the text motivates γ to align their scales.The caption specifies α = 0.8, and the prose describes increasing the number of generated samples from 1 to 211.Source: Problem Formulation · pp. 2–3, Figure 1 · p. 3
Convex Relaxation of Fairness Constraint
- The framework relaxes demographic parity using decision-boundary fairness and equalized odds by balancing risk across sensitive groups.For demographic parity, the proxy uses covariance between the sensitive attribute and signed distance to the decision boundary; the equalized-odds relaxation uses group- and label-specific datasets.Source: Convex Relaxation of Fairness Constraint · p. 3
Attack Algorithm
- PFML iteratively selects a point with high weighted accuracy-loss and fairness-violation impact, adds it to the poisoning set, and updates the model.The framework includes adversarial sampling, adversarial label flipping, and adversarial feature modification.Source: Attack Algorithm · p. 3
- The attack loop has time complexity O(ϵnm), where m is the size of the feasible poisoning set.The text reports that execution time increases linearly with ϵ and that PFML takes two or three orders of magnitude less time to generate poisoning samples than baseline models.Source: Attack Algorithm · p. 4
- The authors characterize prior fairness-impact weighting choices as heuristic and present a framework extending beyond equalized odds and two attack types.Their stated additions include three group-based fairness notions and adversarial feature modification.Source: Attack Algorithm · p. 4
Experiments
On COMPAS and Adult, PFML attacks affect test accuracy and fairness across equalized-odds and demographic-parity settings, with outcomes varying by attack focus and fairness threshold. Results also show sensitivity to poisoning ratio and penalty parameter, and statistical tests report significant advantages over baselines.
Experiments
- Experiments use COMPAS and Adult, SVM classifiers, fair reduction and post-processing, and five-run averages on held-out test data.The COMPAS and Adult datasets use race and gender, respectively, as sensitive attributes.Source: Experiments · pp. 4–5
Evaluation of PFML with Equalized Odds
- PFML attacks reduce accuracy and tune accuracy–fairness effects under equalized odds, outperforming sampling and baseline attacks across fair-reduction and post-processing models.Feature modification and label flipping outperform adversarial sampling; increasing α shifts the attack toward accuracy.Source: Evaluation of PFML with Equalized Odds · pp. 5–6
Evaluation of PFML with Demographic Parity
- Under demographic parity, PFML attacks outperform RS, LF, and HE on accuracy reduction and outperform INFL, KKT, and min-max at large α for accuracy or small α for fairness.The reported pattern is similar to equalized odds; the table covers fair reduction and post-processing on COMPAS.Source: Table 2 · p. 5, Evaluation of PFML with Demographic… · p. 6
- With stricter fair-reduction thresholds, accuracy falls under both fairness notions, but fairness gaps diverge: they rise for equalized odds under PFML-AS/AF and fall for demographic parity.For equalized odds, PFML-AM’s fairness gap instead falls as δ decreases; the authors call for theoretical analysis of robustness.Source: Evaluation of PFML with Demographic… · p. 6, Evaluation of PFML with Equalized Odds · p. 6
Sensitivity Analysis of Hyperparameters
- As poisoning ratio ϵ increases on COMPAS, fair-reduction accuracy falls and fairness gaps rise; PFML generates poisoning samples two or three orders of magnitude faster than baselines.With the same ϵ, PFML degrades accuracy more than INFL, KKT, and min-max, while causing similar or higher unfairness in most scenarios.Source: Table 3 · p. 6, Sensitivity Analysis of Hyperparameters · p. 6
- For post-processing with δ = 0, larger λ shifts PFML-AS toward fairness, producing a larger fairness gap and a smaller accuracy drop.The comparison uses equalized odds on COMPAS.Source: Table 4 · p. 6, Sensitivity Analysis of Hyperparameters · p. 6
Significance Testing
- Across 42 pairwise tests, 39 p-values are below 0.01 and the remaining three are below 0.1, supporting statistical significance over baselines.The tests compare accuracy reduction and fairness; the reported cases cover demographic parity with α = 1 and equalized odds with α = 0.Source: Table 5 · p. 6, Significance Testing · pp. 6–7
Summarized Results of Adult Dataset
- On Adult, accuracy generally falls and fairness gaps generally rise with increasing ϵ across PFML-AS, PFML-AF, and PFML-AM.At δ = 0.1 and α = 0.8, reported execution times range from 187.4 s to 1399.2 s and are below baseline times.Source: Figure 2 · p. 7, Summarized Results of Adult Dataset · p. 7
Related Work
Prior poisoning work perturbs training data, while early attacks on fair machine learning models target demographic disparity and equalized odds.
Related Work
- Test-time attacks perturb test examples, whereas train-time poisoning changes training data to affect the learned model.Data poisoning can insert, modify, or delete training points.Source: Related Work · p. 7
- Influence, KKT, and min-max attacks manipulate poisoned labels and features using different optimization objectives [18].Influence attacks increase test loss, KKT attacks target decoy parameters, and min-max attacks maximize train loss as a proxy for test loss.Source: Related Work · p. 7
- Earlier fair-model attacks addressed demographic disparity through gradient-based, anchoring, and influence attacks (Solans, Biggio, and Castillo 2020; Mehrabi et al. 2020).Chang et al. (2020) targeted equalized odds using approximate algorithms for a bi-level optimization problem.Source: Related Work · p. 7
Conclusions and Future Work
The paper's attacks address model accuracy and fairness, but their scope remains limited to indiscriminate attacks and the fairness notions considered. The authors identify targeted attacks, additional fairness notions, and robust defenses as future directions.
Conclusions and Future Work
- The three proposed attacks are special cases of gradient-based, indiscriminate attacks.The authors propose extending the approach to targeted attacks that cause errors on specific test examples.Source: Conclusions and Future Work · p. 7
- Future work includes extending the approach to fairness notions such as counterfactual fairness.The paper also identifies robust defenses, including multi-gradient optimization and robust learning, as future work.Source: Conclusions and Future Work · p. 7