PaperlayerView exact revision on arXiv

Source-linked AI summary

Fair Mixup: Fairness via Interpolation

Ching-Yao Chuang, Youssef Mroueh

arXiv:2103.06503v1cs.LGcs.CYstat.ML

TL;DR

Fairness constraints estimated on finite training samples may fail to generalize at evaluation time. Fair mixup addresses this by regularizing predictions along mixup-generated paths between sensitive groups, and it improves fairness generalization across tabular, vision, and language benchmarks. The paper also analyzes the method theoretically and reports improved accuracy and fairness tradeoffs in experiments.

  • Problem

    Finite-sample fairness constraints may be satisfied during training yet fail to generalize to evaluation data.

  • Method

    Fair mixup regularizes model predictions along interpolated paths between sensitive groups using mixup or latent-space Manifold Mixup samples.

  • Results

    Fair mixup improves generalizability for demographic parity and equalized odds across tabular, vision, and language benchmarks.

  • Takeaways & Limitations

    Regularizing smooth prediction changes along group-interpolation paths improves fairness generalization while experiments also report improved accuracy tradeoffs.

  • Takeaways & Limitations

    The illustrative attractive-attribute experiment concerns subjective classifiers and is not ethical.

Abstract

from arXiv · show

Training classifiers under fairness constraints such as group fairness, regularizes the disparities of predictions between the groups. Nevertheless, even though the constraints are satisfied during training, they might not generalize at evaluation time. To improve the generalizability of fair classifiers, we propose fair mixup, a new data augmentation strategy for imposing the fairness constraint. In particular, we show that fairness can be achieved by regularizing the models on paths of interpolated samples between the groups. We use mixup, a powerful data augmentation strategy to generate these interpolates. We analyze fair mixup and empirically show that it ensures a better generalization for both accuracy and fairness measurement in tabular, vision, and language benchmarks.

1 INTRODUCTION

Fair mixup addresses the poor evaluation-time generalization of fairness constraints by regularizing predictions along interpolated paths between sensitive groups. The method uses mixup-based augmentation and improves fairness-metric generalization across tabular, vision, and language benchmarks.

  • Finite-sample fairness constraints satisfied during training may not generalize to testing data.
  • Fair mixup regularizes models on interpolated distributions connecting sensitive groups instead of directly restricting group disparity.The interpolated path represents a gradual transition between group distributions.
  • Mixup constructs the interpolated samples, while its simple form yields a closed-form smoothness regularizer that is easy to optimize.The approach can also use latent-space Manifold Mixup, although input interpolates may leave the natural data manifold.
  • Fair mixup improves generalizability for both demographic parity and equalized odds on tabular, computer-vision, and natural-language benchmarks.

2 RELATED WORK

Prior work defines fairness through individual or group criteria and uses data augmentation as implicit regularization. Fair mixup applies mixup-based regularization specifically to group-fairness constraints and targets their generalization.

  • Fairness definitions are commonly divided into individual fairness, which treats similar users similarly, and group fairness, which measures subgroup statistical parity.
  • Existing fair-classifier constraints are data-dependent, so training-time satisfaction can differ from evaluation-time behavior.Prior game-theoretic approaches address generalization but may be difficult to scale to complex model classes.
  • Mixup augments training data with convex combinations of examples, while Manifold Mixup performs interpolation in latent space to improve generalization.
  • Fair mixup extends mixup-style regularization from general learning scenarios to group fairness and fairness-constraint generalization.

3 GROUP FAIRNESS

The paper focuses on demographic parity and equalized odds as group-fairness constraints, while motivating a dynamic augmentation strategy because static training gaps may not generalize.

  • The setup uses inputs X, binary labels Y, binary sensitive attributes A, and prediction scores from a model f.
  • Demographic parity requires predictions to be independent of the sensitive attribute, whereas equalized odds additionally conditions independence on the label.Demographic parity can ignore correlations between labels and sensitive attributes.
  • Fairness metrics can be incorporated into penalized optimization alongside the classification loss.
  • Small training values of demographic-parity disparity do not necessarily generalize at evaluation time, motivating dynamic fairness regularization.

4 DYNAMIC FORMULATION OF FAIRNESS: PATHS BETWEEN GROUPS

The dynamic formulation replaces a static endpoint disparity with a path-based view of prediction behavior between groups. Fairness is encouraged by penalizing prediction changes along the interpolation path, whose arc length upper-bounds demographic-parity disparity.

  • 4 DYNAMIC FORMULATION OF FAIRNESS: PATHS BETWEEN GROUPS: An interpolator T connects samples from group distributions P0 and P1 through a continuous path indexed by t ∈ [0, 1].The endpoints satisfy T(x0, x1, 0) = x0 and T(x0, x1, 1) = x1.
  • 4 DYNAMIC FORMULATION OF FAIRNESS: PATHS BETWEEN GROUPS: The expected prediction µf(t) records model behavior while traveling from group 0 to group 1 along that path.
  • 4 DYNAMIC FORMULATION OF FAIRNESS: PATHS BETWEEN GROUPS: The derivative dµf(t)/dt measures prediction sensitivity to the relaxed continuous sensitive attribute t.If f is invariant to the sensitive attribute, this derivative should remain small along the path.
  • 4 DYNAMIC FORMULATION OF FAIRNESS: PATHS BETWEEN GROUPS: A small endpoint disparity does not guarantee small derivatives throughout the path because prediction changes can fluctuate between groups.
  • 4.1 SMOOTHNESS REGULARIZATION: Fairness regularization therefore penalizes the derivative along the interpolation path to make predictions more invariant to t.
  • 4.1 SMOOTHNESS REGULARIZATION: The smoothness regularizer is the arc length of µf(t), whereas demographic-parity disparity is the endpoint distance between µf(0) and µf(1).
  • 4.1 SMOOTHNESS REGULARIZATION: By Jensen’s inequality, demographic-parity disparity is bounded above by the path regularizer, supporting smoothness regularization as a stronger constraint.

5 FAIR MIXUP: REGULARIZING MIXUP PATHS

Fair mixup regularizes prediction changes along interpolated paths between sensitive groups, using mixup or manifold mixup to impose fairness through smoothness. Its analysis gives closed-form optimization and characterizes fair mixup as changing feature-space geometry to ensure group fairness.

  • Interpolators: Mixup is a valid interpolator because it generates meaningful paths between groups and has an easily computed derivative.Input mixup uses linear interpolation, while manifold mixup constructs paths in latent space to better capture semantic changes.
  • Regularization: The fair mixup regularizer is optimized through the model Jacobian evaluated on mixup samples and the difference between paired inputs.This yields a closed-form expression for input mixup and reduces computational requirements by sampling one interpolation coefficient per batch.
  • Extensions: Fair mixup extends beyond demographic parity to equalized odds and other metrics based on expected group differences.The same formulation is applied conditionally for equalized odds and to metrics such as accuracy equality.
  • Theoretical analysis: The theoretical solution uses a positive-semidefinite matrix formed from derivatives of interpolated mean embeddings to define the fair mixup geometry.The matrix conditions the Jacobian along the interpolation path and supports a closed-form optimal solution.
  • Theoretical analysis: Fair mixup changes the decision-boundary geometry through a Mahalanobis metric that ensures group fairness, rather than projecting onto a subspace orthogonal to one group direction.This metric is defined through the covariance of time derivatives of mean embeddings along intermediate densities.

6 EXPERIMENTS

Across tabular, vision, and language benchmarks, fair mixup improves the test-set tradeoff between average precision and fairness metrics. Its interpolated paths are smoother or flatter, with shorter paths associated with smaller demographic-parity disparities.

  • Experimental setup: Fair mixup is evaluated on Adult, CelebA, and Toxicity classification using test-set AP and ∆DP/∆EO tradeoffs.The experiments vary the fairness regularization weight and compare against baselines that optimize fairness during training.
  • Adult: Fair mixup consistently achieves a better AP–∆DP tradeoff on the Adult dataset and performs slightly better than direct EO-gap regularization for ∆EO.It also achieves higher AP than ERM in the reported Adult comparison.
  • Path analysis: The Adult and Toxicity path visualizations link flatter or nearly linear calibrated paths with smaller ∆DP.On Adult, fair mixup achieves the shortest arc length; on Toxicity, it produces the smallest ∆DP.
  • CelebA: On CelebA, fair mixup consistently outperforms baselines across tasks, while manifold mixup further boosts performance and yields well-generalizing classifiers for both ∆DP and ∆EO.For the attractive task, fair mixup produces smoother input and latent paths; manifold mixup has the smallest latent-path slope and ∆DP.
  • Toxicity classification: On Toxicity classification, fair mixup achieves better AP–∆DP and AP–∆EO tradeoffs than the baselines.Its calibrated path is nearly linear with a small slope, whereas baselines fluctuate or form inverted-U curves; this corresponds to the smallest ∆DP.
  • Scope caveat: The attractive CelebA experiment is explicitly presented as illustrative, and the paper cautions that classifiers for subjective attributes are not ethical.This disclaimer limits how the attractive-attribute result should be interpreted.

7 CONCLUSION

The paper concludes that fair mixup improves the generalizability of fairness constraints by bridging sensitive groups with interpolated samples. It also identifies natural-manifold interpolation, additional fairness metrics, and generalization estimation as future directions.

  • Conclusion: Fair mixup bridges sensitive groups with interpolated samples to improve the generalizability of fairness constraints across modalities.The conclusion states this result across benchmarks with different modalities.
  • Future directions: The paper proposes generating interpolated samples on the natural data manifold using generative models or dynamic optimal transport paths.These are identified as future directions for fair mixup.
  • Future directions: Future work includes extending fair mixup to accuracy equality and estimating the generalization of fairness constraints.The conclusion lists both extensions explicitly.

A.2 PROOF OF PROPOSITION 2

The proof characterizes gap regularization and fair mixup through closed-form optimization solutions. Gap regularization projects the label-discriminating direction away from group-discriminating directions, while fair mixup uses a path-dependent matrix.

  • Gap Regularization: For a fixed embedding Φ, the optimal solution f∗ corresponds to a closed-form vector v∗.The derivation sets the first-order condition to zero and solves the resulting linear system.
  • Gap Regularization: Proposition 2 interprets the optimal solution as projecting the label-discriminating direction onto the subspace orthogonal to the group-discriminating direction.This projection prevents the classifier from using group-specific directions.
  • Fair Mixup: The fair mixup classifier is obtained in closed form after solving the regularized optimization problem.The proof proceeds from the path-based regularizer to the corresponding optimal classifier.
  • Fair Mixup: The proof defines a t-dependent mean embedding m_t along interpolations between groups and its derivative with respect to t.These quantities describe the path generated by mixup between distributions P_0 and P_1.
  • Fair Mixup: Fair mixup yields an optimal solution involving a positive-semidefinite matrix D formed from the outer products of mean-embedding derivatives along the path.The resulting first-order condition is (λ_1D + λ_2I_m)v∗ = δ_±.

B EXPERIMENT DETAILS

The experiments use tabular, vision, and language datasets with repeated training runs, validation-based model selection, and held-out evaluation. Dataset-specific preprocessing and model configurations vary across Adult, CelebA, and Toxicity Classification.

  • Adult: Adult experiments use one-hot encoding for discrete and quantized continuous attributes and an 80%/20% training-testing split.Each model is retrained 10 times with batch size 1000, and mean accuracy and fairness measurements are reported.
  • CelebA: CelebA experiments extract 512-dimensional features from ResNet-18 before training a two-layer ReLU predictor with hidden size 512.The original validation set is used for model selection, with accuracy and fairness evaluated on the testing set.
  • Toxicity Classification: Toxicity Classification uses a Jigsaw dataset containing 22,835 examples and reports mean metrics across 10 retraining runs.Models use batch size 200, validation-based selection, and Adam with learning rate 1 × 10^-3.

C.1 TRAINING PERFORMANCE

On the Adult training set, GapReg outperforms fair mixup because it directly optimizes the fairness metric. This training advantage supports the paper’s concern that fairness constraints satisfied during training may not generalize to evaluation.

  • Training Performance: GapReg outperforms fair mixup on Adult training performance because it directly optimizes the fairness metric.Figure 7 reports the tradeoff between average precision and ΔDP/ΔEO on the training set.
  • Training Performance: Training-set fairness constraints may fail to generalize at evaluation time.The observed training behavior motivates evaluating fairness on held-out data.

C.2 EVALUATION METRIC

The evaluation averages fairness disparities across thresholded predictions to avoid overestimating fairness from confidence outputs alone. Under this metric, fair mixup generally offers the strongest or competitive fairness-accuracy tradeoffs, including after reducing model size.

  • Evaluation Metric: A relaxed expectation-based metric can overestimate fairness when prediction confidence differs substantially between groups.A classifier may satisfy the expectation condition yet become highly unfair after binarization at threshold 0.5.
  • Evaluation Metric: Average ΔDP evaluates binarized predictors across multiple thresholds rather than relying on a single confidence-based condition.The paper uses thresholds T = [0.1, 0.2, · · ·, 0.9].
  • Evaluation Metric: Fair mixup achieves the best ΔDP tradeoff against the baselines on Adult, while its equalized-odds performance is similar to GapReg and better when ΔEO is small.These comparisons are reported in the average ΔDP and ΔEO evaluation.
  • Smaller Model Size: Reducing hidden size from 200 to 50 does not significantly change overall performance, while fair mixup continues to outperform baselines for ΔDP.For ΔEO, fair mixup and GapReg remain similar, with fair mixup better when ΔEO is small.
Loading 2103.06503v1…