To be Robust or to be Fair: Towards Fairness in Adversarial Training

Han Xu; Xiaorui Liu; Yaxin Li; Anil K. Jain; Jiliang Tang

Source-linked AI summary

To be Robust or to be Fair: Towards Fairness in Adversarial Training

Han Xu, Xiaorui Liu, Yaxin Li, Anil K. Jain, Jiliang Tang

arXiv:2010.06121v2Oct 13, 2020cs.LGstat.ML

TL;DR

Adversarial training can introduce severe disparities in accuracy and robustness between data groups, even when datasets are balanced. The paper empirically and theoretically studies this phenomenon and proposes Fair Robust Learning (FRL), whose variants reduce worst-class errors in experiments.

Problem
Adversarial training improves robustness but can create severe accuracy and robustness disparities between classes, raising a robust fairness problem.
Method
The paper combines empirical studies, theoretical analysis of unequal class difficulty, and the Fair Robust Learning framework with reweighting and remargin strategies.
Results
FRL variants reduce worst-class standard and robust errors, with FRL (Reweight) achieving around 10% reduction in worst-class standard error compared with vanilla methods.
Takeaways & Limitations
Adversarial defenses should account for class-wise fairness because robust-error minimization can disproportionately hurt harder classes.
Takeaways & Limitations
Remargin-based methods usually incur a 1–2% larger average standard error than PGD Adversarial Training or TRADES (1/λ = 1).

Abstract

from arXiv · show

Adversarial training algorithms have been proved to be reliable to improve machine learning models' robustness against adversarial examples. However, we find that adversarial training algorithms tend to introduce severe disparity of accuracy and robustness between different groups of data. For instance, a PGD adversarially trained ResNet18 model on CIFAR-10 has 93% clean accuracy and 67% PGD l-infty-8 robust accuracy on the class "automobile" but only 65% and 17% on the class "cat". This phenomenon happens in balanced datasets and does not exist in naturally trained models when only using clean samples. In this work, we empirically and theoretically show that this phenomenon can happen under general adversarial training algorithms which minimize DNN models' robust errors. Motivated by these findings, we propose a Fair-Robust-Learning (FRL) framework to mitigate this unfairness problem when doing adversarial defenses. Experimental results validate the effectiveness of FRL.

1. Introduction

Adversarial training improves robustness but can create substantial class-wise disparities in both accuracy and robustness, even on balanced data. The paper studies why this occurs and proposes Fair Robust Learning (FRL) to mitigate it.

Motivation: Adversarial training usually creates large disparities in class-wise accuracy and robustness, whereas naturally trained models show similar class performance.The disparity is reported on balanced CIFAR10 data and in other datasets, architectures, and adversarial training algorithms.
Research question: The paper defines this disparity as the “robust fairness” problem and asks why adversarial training produces it while natural training does not.The question concerns both accuracy and robustness differences between classes.
Analysis: Mixture-Gaussian studies use classes with different difficulty levels to compare class-wise performance under natural and adversarial training.These conceptual examples support the paper’s empirical and theoretical analysis of the disparity.
Findings: Adversarial training favors easier classes and sacrifices performance on difficult classes, producing more pronounced class-wise discrepancies.This mechanism motivates a dynamic debiasing approach for adversarial settings.
Contributions: FRL is proposed to mitigate fairness issues in adversarial training, alongside conceptual examples intended to explain why the problem can arise generally.The contributions include discovering the robust fairness problem and studying its potential reasons.

2. Preliminary Studies

Preliminary CIFAR10 studies show that adversarial training produces severe class-wise disparities in standard and robust errors, unlike natural training. The disparity reflects weaker treatment of intrinsically difficult classes and appears across several settings.

Adversarial training: 32.8% standard error and 82.4% robust error occur for “cat” under PGD adversarial training, versus 6.1% and 34.3% for “car”.The PGD-trained model’s average errors are 15.5% standard and 56.4% robust.
Comparison: Adversarial training therefore creates strong standard and robustness disparities that are negligible in natural training.The observed difference concerns class-wise error performance.
Class difficulty: Adversarial training increases standard errors more for naturally difficult classes such as “dog” and “cat” than for easier classes such as “car” and “ship”.It also provides poorer robust-error reductions for the difficult classes, motivating the hypothesis that hard classes become harder.
Generality: Similar fairness disparities appear with WRN28, SVHN, l2-norm attacks, and Randomized Smoothing, suggesting the phenomenon generalizes across settings.The reported settings vary the architecture, dataset, attack norm, and defense method.

3. Theoretical Analysis

Theoretical analysis shows that adversarial training can worsen class-wise performance disparities by shifting the decision boundary toward harder classes. Experiments on simulated Gaussian data support the theoretical predictions.

Motivation: Adversarial training favors easier classes and sacrifices performance on harder classes, creating larger class-wise discrepancies than natural training.The paper studies whether this disparity is inherent to adversarial training and finds consistent empirical evidence.
3.1. A Binary Classification Task: The Gaussian-mixture model assigns different classification difficulties by giving the two classes variances in the ratio σ+1 : σ−1 = K : 1, with K > 1.The classes are centered at −θ and θ, while the larger variance makes class “+1” less compact.
3.1. A Binary Classification Task: Rnat(fnat, −1) < Rnat(fnat, +1), so the optimal natural linear classifier already has higher standard error on the higher-variance class.The class-wise difference disappears when K = 1.
3.2. Optimal Linear Model to Minimize Robust Error: Robust training shifts the decision boundary closer to the harder class “+1” and farther from the easier class “-1”, exacerbating the disparity.The robust classifier minimizes errors on perturbed class centers that are ε-distance closer to zero.
3.2. Optimal Linear Model to Minimize Robust Error: Rnat(frob, +1) > Rnat(fnat, +1), formally showing that adversarial training increases standard error for the harder class.The analysis limits the perturbation margin to 0 < ϵ < η.
3.2. Optimal Linear Model to Minimize Robust Error: 0 < brob < bnat, causing the robust classifier to predict more samples as class “-1”, reducing its error while increasing class “+1” error.The relation follows from g being monotone increasing on the relevant interval.

4. Fair Robust Learning (FRL)

FRL targets robust fairness by equalizing class-wise standard accuracy and adversarial robustness while minimizing overall robust error. It uses constraint-based debiasing and can adjust class weights or perturbation margins.

4.1. Objective of Robust Fairness: FRL seeks a robust model with parity of standard prediction accuracy and adversarial robustness across classes.The framework treats both accuracy and robustness disparities as fairness objectives.
4.1. Objective of Robust Fairness: Equalized Accuracy requires class-conditional standard error to approximate overall error, while Equalized Robustness imposes the analogous condition for robust error.Equalized Robustness is introduced as a fairness property for robustly trained models.
4.1. Objective of Robust Fairness: FRL constrains each class’s standard and robust errors to remain within small positive tolerances τ1 and τ2 of the corresponding averages.These constraints are intended to prevent an obvious worst-performing group.
4.1. Objective of Robust Fairness: The method separates standard error from boundary error, enabling independent treatment of unfairness in standard accuracy and adversarial vulnerability.Boundary error measures the probability that a class-i sample lies close to the decision boundary and can be attacked.
4.2. Fair Robust Learning: FRL alternates multiplier updates based on class-wise constraint violations with model optimization, increasing training weight for errors from underperforming classes.The procedure starts from a pretrained robust model and evaluates class-wise errors on a separate validation set.
4.3. ReMargin for Robust Fairness: Because boundary-error reweighting alone may not reduce boundary error, FRL (Remargin) enlarges the perturbation margin for the affected class during adversarial-example generation.Reweighting and Remargin can also be combined.

5. Experiment

Experiments on CIFAR10 and SVHN evaluate FRL strategies for reducing worst-class standard and robust errors. Reweight, Remargin, and their combination expose different fairness trade-offs, with the combination generally strongest for worst-class robustness.

Experimental setup: FRL experiments compare Reweight, Remargin, and Reweight+Remargin on CIFAR10 and SVHN using adversarial-learning benchmarks.The study uses PreAct-ResNet18 results, with WRN28 results deferred to the appendix.
CIFAR10 results: 10–15% lower worst-class robust error is achieved by FRL (Reweight + Remargin) than vanilla adversarial training on CIFAR10.The combined strategy is also reported as the most effective for reducing worst-class boundary and robust errors.
CIFAR10 results: Around 10% lower worst-class standard error is achieved by FRL (Reweight), although it does not adequately equalize boundary errors.FRL (Remargin) is more effective for decreasing worst-class boundary error.
Trade-offs: 1–2% larger average standard error occurs with Remargin methods than with PGD Adversarial Training or TRADES on CIFAR10.The authors attribute this trade-off to larger perturbation margins and cap each margin at 16/255.
Trade-offs: 2–3% improvement in average robust error is reported for Remargin methods compared with vanilla methods, while average robust errors remain comparable overall.The associated increase in perturbation margin also produces smaller average boundary errors.
SVHN results: 1–2% decrease in average standard error is observed for every FRL method on SVHN, while Reweight minimizes worst-class standard error and Reweight+Remargin minimizes worst-class robust error.The SVHN findings are described as similar to those on CIFAR10, with no average-standard-error increase from Remargin.
Ablation study: Upweighting the deer class lowers its boundary error but increases its standard error, whereas increasing the margin lowers boundary and robust errors without increasing standard error.This motivates combining Reweight and Remargin to improve worst-class robustness.

6. Related Work

The paper situates robust fairness within machine-learning fairness and imbalanced-data research while distinguishing its setting from prior work. It identifies class-wise accuracy and robustness disparity in balanced datasets as a distinct adversarial-training problem.

Fairness in machine learning: Prior fairness work distinguishes prediction outcome disparity from prediction quality disparity.The paper frames its focus as disparity in prediction quality under adversarial conditions.
Imbalanced datasets: Unlike imbalanced-data settings, the paper reports that robust unfairness can occur in balanced datasets with equal class representation.This motivates new scopes and methods for studying the phenomenon.
Robust learning: The paper describes itself as the first study of unfairness in the adversarial setting and notes independent work on unequal robustness across deep-learning tasks.Its emphasis is adversarial-training algorithms and mitigation methods for more balanced robustness.

7. Conclusion

The paper concludes that adversarial training can create class-wise disparities in standard accuracy and adversarial robustness. It proposes FRL as a mitigation framework and validates it on benchmark datasets.

Conclusion: Adversarial training can cause serious disparity in standard accuracy and adversarial robustness between data classes.The conclusion describes this property as established empirically and theoretically.
Conclusion: FRL is proposed as a framework to mitigate fairness issues from adversarial training.The paper validates FRL on benchmark datasets.

A.1. More Results of Preliminary Studies

Additional experiments examine robust fairness across architectures, datasets, attack norms, and defense methods. The reported observations are similar across these settings.

Additional settings: Additional settings include WRN28, l2-norm attacks, Randomized Smoothing, and the SVHN dataset.These experiments extend the preliminary studies beyond the original architecture, attack, defense, and dataset settings.
Findings: Similar fairness phenomena are observed across the additional settings, suggesting generalization to other algorithms, architectures, datasets, and attacks.The authors characterize this as a likely generalization based on the reported experiments.

A.1.1. ROBUST FAIRNESS IN WRN28 IN CIFAR10

Under WRN28 models on CIFAR10, adversarial training shows the same robust fairness issue observed with other architectures: class-wise performance differs substantially compared with natural training.

WRN28 models on CIFAR10 exhibit robust fairness issues under adversarial training.The passage reports similar class-wise behavior to the PreAct ResNet18 results.
Natural and PGD adversarial training are compared using class-wise standard error and robust error.Tables 4 and 5 provide the corresponding error changes for the two training procedures.
The comparison identifies unequal effects of adversarial training across classes.The passage presents this unequal class-wise effect as evidence of the robust fairness problem.

A.1.2. ROBUST FAIRNESS IN l2-NORM ADVERSARIAL TRAINING

The robust fairness phenomenon also appears when adversarial training targets l2-norm attacks on CIFAR10, with class-wise errors evaluated under a specified PGD procedure.

l2-norm adversarial training on CIFAR10 exhibits robust unfairness across classes.The evaluation targets an l2-norm 0.5 attack.
The study reports class-wise standard errors and robust errors for the l2-norm setting.These measurements are used to confirm disparity under this attack type.
Robustness evaluation uses PGD with step size 0.1 for 20 steps.

A.1.3. ROBUST FAIRNESS FOR CERTIFIED DEFENSES

Robust fairness issues extend to certified defenses and multiple experimental settings, while the theoretical analysis explains how robust optimization can amplify class difficulty differences. FRL is reported to improve worst-class performance and mitigate unfairness, with reweighting plus remargining addressing more error dimensions than reweighting alone.

Certified defenses: Randomized Smoothing, a certified defense, shows the same class-wise disparity phenomenon as traditional adversarial training.The method minimizes the probability of nearby adversarial examples and thereby implicitly minimizes overall robust error.
Certified defenses: Randomized Smoothing has high standard and robustness errors on “cat” and “dog” but relatively low errors on “car” and “ship”.
Additional settings: On SVHN with PreAct ResNet18, adversarial training produces strong class-wise disparities absent from natural training.Classes “3” and “8” have the largest standard-error increases and the least robust-error decreases after adversarial training.
Theoretical analysis: The theoretical analysis concludes that robust optimization unequally influences two classes and creates severe performance disparity.For the model distribution considered, robust classifiers exacerbate the class that already has higher natural-training error.
Theoretical analysis: The extended theoretical model predicts that robust optimization increases the standard error of the harder class at a larger rate.This extension includes robust and non-robust features so both classes’ errors can increase.
Fair Robust Learning: FRL improves worst-class standard and robustness performance, while reweighting plus remargining better equalizes boundary and robust errors than reweighting alone.Reweighting is usually most effective for equalizing standard performance but is insufficient for all error types.