Source-linked AI summary
A Convex Framework for Fair Regression
Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, Aaron Roth
TL;DR
Fairness in regression is less studied than in classification, despite the use of machine learning in consequential decisions and reports of unfair or discriminatory models. The paper introduces convex fairness regularizers spanning group to individual fairness, computes accuracy–fairness frontiers and the Price of Fairness across six datasets, and finds that trade-offs are domain-dependent. It concludes that practitioners must carefully choose fairness definitions and balance fairness against predictive accuracy for each application.
Problem
Fairness research has paid less attention to linear and logistic regression settings, where targets or predictions can be continuous.
Method
The paper applies a flexible family of convex fairness regularizers to linear and logistic regression, varying their weight to compute efficient accuracy–fairness frontiers and compare Price of Fairness across datasets.
Results
Across six datasets, detailed accuracy–fairness trade-offs and comparisons among fairness notions show domain dependence rather than prescriptive universals.
Takeaways & Limitations
Practitioners should define the relevant fairness notion for each application and determine the appropriate balance between predictive accuracy and fairness.
Takeaways & Limitations
The fairness regularizers can act as proxies for accuracy because they are small for any perfect regressor, and the framework assumes access to the sensitive feature during learning.
Abstract
from arXiv · showhide
We introduce a flexible family of fairness regularizers for (linear and logistic) regression problems. These regularizers all enjoy convexity, permitting fast optimization, and they span the rang from notions of group fairness to strong individual fairness. By varying the weight on the fairness regularizer, we can compute the efficient frontier of the accuracy-fairness trade-off on any given dataset, and we measure the severity of this trade-off via a numerical quantity we call the Price of Fairness (PoF). The centerpiece of our results is an extensive comparative study of the PoF across six different datasets in which fairness is a primary consideration.
1 Introduction
The paper addresses underexplored fairness in linear and logistic regression by introducing convex regularizers spanning group to individual fairness. It evaluates accuracy–fairness frontiers and the Price of Fairness across six fairness-sensitive datasets, finding that detailed trade-offs are domain-dependent rather than governed by universal rules.
- Fairness research has focused more on classification than on regression with continuous targets or predictions.
- The framework introduces convex fairness regularizers for linear and logistic regression, spanning group, intermediate, and individual fairness notions.The convex combined objectives permit efficient optimization.
- Varying the regularizer weight computes the efficient frontier between predictive accuracy and fairness for a dataset.These curves support domain-specific assessment of how mild or severe the trade-off is.
- The Price of Fairness numerically quantifies accuracy degradation as fairness increases, enabling comparisons across datasets, fairness notions, and protected-variable treatments.The study normalizes fairness loss across datasets so frontiers can be compared quantitatively.
- The empirical case study compares fairness–accuracy frontiers across six datasets where fairness is a primary concern.
- Across datasets, efficiency curves have broadly similar shapes, but detailed trade-offs and fairness-notion comparisons remain domain-dependent without prescriptive universals.PoF is generally higher for individual than group fairness, and is not generally much improved by using protected variables.
2 The Regression Setting
The paper formulates fairness for linear and logistic regression through convex regularizers that compare predictions across groups for similarly labeled instances. These notions range from uncompensated individual fairness to compensatory group and hybrid fairness, while increasing fairness regularization can lead to a trivial constant model.
- 2 The Regression Setting: The framework minimizes accuracy loss plus a weighted fairness penalty, with mean-squared error for linear regression, log loss for logistic regression, and ℓ2 regularization.The fairness weight is λ, while γ controls ℓ2 regularization.
- 2.1 A Convex Family of Fairness Regularizers: All fairness definitions compare predictions for cross-group pairs with similar labels, weighting prediction disparities by label similarity.The framework calls pairs from different groups cross pairs and pairs with similar labels similar cross pairs.
- 2.1 A Convex Family of Fairness Regularizers: Individual fairness penalizes every cross-pair disparity separately, whereas group fairness permits disparities to compensate across different similar cross-pairs.The proposed family spans group fairness and strong individual fairness, with hybrid constructions between these extremes.
- 2.1 A Convex Family of Fairness Regularizers: The sensitive feature supports either one model evaluated using group membership or separate models trained for the two groups.The regularizers are defined for the single-model setting and extend to separate models by substituting the group-specific model.
- Hybrid notions of fairness: Hybrid fairness averages treatment separately for positive and negative cross-pairs, allowing compensation only among instances with the same label.The positive and negative components can be weighted differently when one label type matters more.
- 2.2 Discussion of Our Notions of Fairness: Any constant regressor exactly minimizes these fairness regularizers, so increasing λ can move the solution toward a perfectly fair but trivial constant model.Perfect regressors also receive small fairness penalties, although perfect linear or nonlinear regressors are said to be rare in practice.
3 Related Work
Prior fairness research has largely emphasized classification, with several competing notions and choices about disparate treatment and where fairness enters the learning pipeline. This paper positions its approach as efficient in-processing fairness for regression, using regularization while studying accuracy–fairness trade-offs.
- 3 Related Work: Earlier fairness notions often conflict, and prior work has commonly divided along classification versus regression, individual versus group fairness, and disparate-treatment choices.These distinctions have led to different algorithms and optimization frameworks.
- 3 Related Work: Most prior work focused on classification, including statistical parity and equality of odds, while this paper emphasizes regression settings.Equality of odds is intuitive for classification but difficult to assess empirically for regression.
- 3 Related Work: Regression work has studied equal means, balanced residuals, and causal notions of impartial estimates; the paper’s group fairness notion weights cross-group comparisons by label similarity.Choosing d(y, y′) as a constant recovers equal means.
- 3 Related Work: Fairness can be enforced through preprocessing, in-processing, or postprocessing, with in-processing incorporating fairness as a constraint or objective term during training.This paper uses in-processing by encoding fairness as a regularizer.
- 3 Related Work: The literature debates whether models should allow disparate treatment, including separate models or sensitive attributes, because such treatment has also been argued necessary for fairness.The paper explicitly considers single-model and separate-model settings.
4 A Comparative Empirical Case Study
The study applies convex fairness regularizers to linear and logistic regression across six datasets, examining accuracy–fairness frontiers and Price of Fairness patterns. Results show substantial dataset- and regularizer-dependent variation, with separate models offering little consistent out-of-sample benefit.
- Study design: Six datasets include real-valued and binary-valued prediction tasks, using linear or logistic regression with protected race or gender features.The datasets vary in size, dimensionality, and minority-population proportion.
- Efficient frontiers: Varying the fairness weight λ traces each dataset’s efficient frontier, while logistic predictions are evaluated with mean squared error.As λ increases, models converge to the best constant predictor that minimizes fairness penalties.
- Efficient frontiers: Trade-offs differ sharply across datasets: individual fairness is gradual on four datasets but nearly costless initially on COMPAS and Default before accuracy costs rise dramatically.This diversity persists even after normalizing fairness losses with Price of Fairness.
- Fairness notions: Individual fairness is generally more costly than group fairness, except on COMPAS and Default, where group unfairness may cost more for small unfairness levels.The comparison concerns the regime between λ = 0 and λ →∞.
- Model structure: Separate models barely improve, and sometimes worsen, the accuracy–fairness trade-off across almost all datasets and regularizers.Their apparent in-sample effects can reflect overfitting, while out-of-sample benefits remain limited.
- Price of Fairness: Price of Fairness varies by dataset, fairness penalty, and model structure, with notable single-versus-separate differences for group fairness on Law School and individual fairness on Adult.On Law School, single models have higher group-fairness PoF; on Adult, single models substantially reduce individual-fairness PoF.
5 Conclusions
The conclusions distinguish fairness regularization from ordinary complexity regularization because fairness is treated as a first-order objective. The study finds that dataset and fairness-notion choices qualitatively shape the trade-off, requiring application-specific decisions.
- 5 Conclusions: Complexity regularization trades goodness of fit for model complexity while serving generalization, whereas fairness regularization treats fairness as a first-order consideration.The paper studies fairness regularizers in regression settings where fairness is not subordinate to generalization.
- 5 Conclusions: The choice of dataset and fairness regularizer can qualitatively affect the accuracy–fairness trade-off.The comparison includes group, individual, hybrid, and other fairness notions.
- 5 Conclusions: Practitioners should define the fairness notion for their application and determine an appropriate balance between predictive accuracy and fairness.This conclusion is presented alongside theoretical results highlighting incompatibilities among fairness measures.
- 5 Conclusions: Figure 2 compares Price of Fairness across datasets, fairness regularizers, and single versus separate models.The figure supports cross-dataset comparison of normalized fairness costs.
A.1 Cross Validation for Picking γ
The appendix selects the ℓ2 regularization parameter γ separately for each fairness-weight value λ using 10-fold cross-validation. Selected γ generally increases with λ and is usually higher for separate models.
- Objective: For each λ, the optimization objective combines regression loss, fairness penalty, and γ||w||2.The loss is MSE for linear regression or logistic regression loss for binary-valued problems.
- Cross-validation procedure: Ten-fold cross-validation evaluates candidate γ values by training on nine folds and accumulating regularized test loss on the held-out fold.The procedure loops over candidate γ values and all ten folds.
- Parameter selection: The selected γ is the candidate with the smallest accumulated loss, yielding γ(λ) for the chosen fairness weight.This selection is performed after the candidate-and-fold loops finish.
- Empirical selection pattern: γ increases as λ increases, and separate models usually select higher γ than single models at fixed λ.The observed pattern links stronger fairness weighting and model structure to different selected complexity regularization.
A.2 Additional Details
Experiments use cross-validation with dataset-size-dependent sampling, while individual-fairness losses use a limited number of random cross-pairs for computational stability. The study prioritizes comparing fairness–accuracy trade-offs rather than maximizing predictive accuracy.
- Evaluation: Ten-fold cross-validation evaluates fairness and accuracy losses on held-out test data.Smaller datasets use one run; larger datasets use repeated runs on fresh random samples.
- Evaluation: For larger datasets, experiments sample 30–50% of observations and repeat cross-validation three times with new random samples.This design addresses increased runtime and CVXPY solver instability.
- Cross-pair sampling: Fairness losses defined over all n1 × n2 cross-pairs are approximated using 2 × Minority n random pairs.Using more pairs did not substantially improve efficiency curves, and many pairs destabilized binary individual-fairness optimization.
- Experimental consistency: The same cross-validation folds and cross-pairs are used across fairness notions to support comparable experiments.This controls these sampling choices when comparing regularizers.
- Scope: Linear models perform poorly on some datasets, because the study focuses on fairness–accuracy trade-offs rather than dataset-specific maximum accuracy.This is an explicit scope boundary of the empirical analysis.
A.3 Datasets
The study evaluates fairness-aware regression across six datasets covering education, sentencing, income, crime, recidivism, and credit-risk prediction. Sensitive attributes include race, gender, and community racial makeup, while preprocessing removes uninformative features and encodes categorical and missing values.
- Preprocessing: Preprocessing removes uninformative features, converts categorical variables to indicator variables, and adds missingness indicators without applying feature selection.Examples of removed features include various kinds of IDs.
- Economic prediction: The Adult dataset predicts whether income exceeds $50,000 using less sensitive attributes such as education and occupation, with groups formed by gender.The dataset is restricted to the Adult.data file.
- Sensitive attributes: Gender and race are treated as sensitive attributes in the income, law-school, recidivism, and credit-default settings.The Communities and Crime analysis instead uses community racial makeup, including a grouping based on whether blackPerCap exceeds another racial proportion.
- Crime prediction: The Communities and Crime dataset predicts violent-crime rates from community and law-enforcement features, motivating fairness concerns across communities with different racial makeup.The outcome combines murder, rape, robbery, and assault, and the analysis creates two groups using racial-composition information.
- Recidivism prediction: The COMPAS dataset predicts violent-crime recidivism within two years for selected Black and white defendants with risk scores assigned within 30 days of arrest.The passage also describes an additional restriction involving at least two years, but the supplied text is truncated.
- Dataset scope: The datasets span law-school bar-exam passage, prison-sentence length, income, violent-crime, recidivism, and credit-default prediction tasks.The Law School and Sentencing datasets concern legal outcomes; the other datasets address economic, criminal-justice, public-safety, and lending settings.