Source-linked AI summary
Fairness in multi-class multi-group classification problems via contextial coherent risk measures
Darinka Dentcheva, Xiangyu Tian
TL;DR
The paper addresses fair multi-class classification with overlapping groups defined by vector-valued sensitive attributes, while seeking to avoid violations of individual fairness and improve robustness. It uses contextual and systemic coherent risk measures with multi-level aggregation to encode these objectives. The proposed framework reports improved fairness metrics without sacrificing classification quality in the cited experiments.
Problem
Fairness criteria for multi-class, multi-group settings may miss differing error distributions across groups, while group-level fairness can permit inconsistent treatment of individuals.
Method
The paper uses contextual coherent risk measures and systemic risk aggregation to evaluate losses by sensitive-attribute context, class, and group, with nonlinear penalties for disparities.
Results
The proposed framework improves fairness metrics without reducing classification quality; CNACR attains a higher F1 score than every baseline at both attributes and corruption levels, with p < 10^-4 for each difference.
Takeaways & Limitations
Coherent-risk aggregation provides a structured framework for handling multi-class, multi-group fairness while incorporating robustness and individual-fairness considerations.
Takeaways & Limitations
The analysis focuses on random vectors representing losses; corresponding results for random variables representing gains are noted but not pursued.
Abstract
from arXiv · showhide
We propose a new design of fair classifiers for multi-class classification problems in the presence of vector-valued sensitive attributes. In that scenario each sensitive attribute has multiple values and forms several groups relevant to the fairness consideration. Naturally those groups are overlapping and one should also analyze the interaction of factors. Additionally, the decision makers aided by the classification should not violate individual rights at the expense of satisfying fairness metrics at the group level. We propose an approach using the theory and methods of coherent measures of risk aiming at resolving the fairness challenges. Further, we propose a specialized numerical method for solving the resulting optimization problem. The method scales well with the increase of the number of observations. Additionally, we note that the obtained classifier is robust with respect to corrupted data or to situation when data is scarce. We demonstrate the advantages of the proposed framework in comparison to the support-vector machine framework and other methods handling fairness.
1 Introduction
The paper addresses fairness in multi-class classification with multiple, overlapping sensitive-attribute groups while also considering individual fairness and robustness to imperfect data. It proposes coherent-risk-based methods that directly incorporate these concerns into classification.
- The proposed method handles multi-class classification and fairness across multiple groups, including intersectional groups.
- It additionally aims to provide individual fairness within each group, rather than relying only on group-level fairness.
- Existing fairness interventions include modifying training data or transforming model predictions after training.
- Many existing approaches do not address robustness to corrupted data, despite noisy labels, inaccurate sensitive attributes, and underrepresented minority groups.
- The paper uses contextual and systemic coherent risk measures to combine fairness enforcement with robustness and structured multi-class, multi-group handling.
2 Coherent measures of risk for groups’ and systems’ losses
The paper develops systemic coherent risk measures for vector-valued losses by evaluating component risks and aggregating them across groups, contexts, or classes. These aggregations can penalize disparities between component risks and connect risk minimization to distributionally robust optimization.
- Systemic coherent risk measures extend scalar risk evaluation to random vectors representing heterogeneous component losses.
- A systemic risk measure satisfies properties including convexity, monotonicity, positive homogeneity, and translation equivariance.
- The dual representation links minimizing a coherent risk measure with distributionally robust optimization over an ambiguity set of probability measures.
- The parameter κ controls risk aversion by enlarging the ambiguity set and permitting greater differences in probability weights.
- Systemic aggregation can impose an additional penalty when individual component risks deviate from their average, supporting fairness across components.
- Alternative aggregations can penalize components whose risks exceed a threshold, while linear expectation aggregation is a special systemic-risk case.
3 Fairness notions and challenges
The paper reviews fairness definitions and challenges in multi-class, multi-group classification, emphasizing that standard group metrics may miss differing error distributions and individual-level arbitrariness. It presents a conditional-distribution criterion and a chi-square test for detecting group dependence.
- Demographic parity, equalized odds, and equal opportunity impose different constraints on prediction rates across sensitive groups.
- Multi-class fairness is less straightforward because binary true-positive and false-positive concepts do not directly characterize all class-specific errors.
- Individual fairness critiques group criteria because equal group statistics can coexist with inconsistent or arbitrary treatment of individuals.
- The paper’s multi-class fairness condition requires the prediction distribution given the true label to be independent of group membership.
- Standard per-class metrics can miss unequal distributions of errors across groups, even when groups have equal recall or the same EO-ratio.
- Pearson’s chi-square test of homogeneity tests this conditional-distribution equality, with a large statistic indicating significant differences across groups.
4 Contextual risk and fairness
The paper uses contextual and systemic coherent risk measures to combine fairness across sensitive-attribute groups and classes while retaining individual fairness within groups. Its nested aggregation penalizes disparities both among groups within classes and among class-level risks.
- Contextual risk and fairness: The formulation is risk-averse and uses aggregation choices to enforce fairness within each class.The paper specifically describes penalties for deviation from average risk or excessive risk above a quantile.
- Contextual risk and fairness: The formulation evaluates class-specific losses conditionally on sensitive-attribute contexts, then aggregates contextual risks within each class and across classes.The construction defines contextual risks for each class and group, followed by class-level and outer systemic aggregation.
- Contextual risk and fairness: The middle aggregation penalizes within-class group disparity when conditional group risks differ.A mean-semideviation term increases class risk when one group has lower risk than another, discouraging unequal treatment within a class.
- Contextual risk and fairness: The outer aggregation addresses imbalance between class-level risks that a weighted average alone would not penalize.The proposed nonlinear outer aggregation extends the fairness-oriented risk structure beyond group comparisons within individual classes.
- Contextual risk and fairness: The resulting three-level composition remains a coherent systemic risk measure.Scenario losses are aggregated by inner risks, contextual risks within classes by middle aggregation, and class risks by outer aggregation.
5 Three-stage regularized decomposition
The paper solves the nested nonlinear risk optimization problem with a regularized multi-cut decomposition method. The algorithm builds scenario, class-level, and outer cuts iteratively, and its convergence follows from an established regularized decomposition theorem.
- Three-stage regularized decomposition: The method decomposes the three-level nonlinear risk structure into scenario, class-level, and outer subproblems.At each iteration, scenario risks are evaluated first, then class-level risks, and finally systemic risk across classes.
- Three-stage regularized decomposition: Dual representations generate cuts that approximate the outer, class-level, and scenario risks in the master problem.Class-level cuts approximate aggregators from below as functions of scenario risks, adding a new level relative to the earlier method.
- Three-stage regularized decomposition: The regularized master problem uses proximal and stability-center parameters while retaining indexed cut sets for each risk level.The parameters σ and β control regularization and proximal behavior, while w^k is the current stability center.
- Three-stage regularized decomposition: The proposed procedure iteratively solves subproblems, adds cuts, updates the center, and stops when the master value matches the current risk estimate.Inactive constraints can be removed before the next iteration.
- Three-stage regularized decomposition: The first and second stages can be collapsed into one master problem because the second stage only approximates class risks without changing the classifier parameter.The resulting variant accumulates all three cut sets in a single master problem.
- Three-stage regularized decomposition: Both decomposition methods inherit convergence from the convergence theorem for the regularized decomposition method.This convergence claim is stated for both the three-stage and collapsed formulations.
6 Numerical Experiments with Fair Risk-averse Classification
The experiments evaluate CNACR on corrupted sensitive attributes and cross-state distribution shifts, comparing it with linear-SVM-based fairness baselines across binary and multi-group settings. CNACR generally improves fairness while maintaining classification quality, although performance varies across states and fairness gains increase with more protected groups.
- Adult: Corrupted Sensitive Attributes: CNACR improves fairness on Adult race data, raising the EO-ratio from 0.7433 to 0.8756 and reducing the Gini coefficient from 0.0139 to 0.0066 versus plain SVM.The chi-square rejection rate also falls from 54% to 10%, with p < 10^-4 and CNACR ahead in 92 of 100 runs.
- Adult: Corrupted Sensitive Attributes: CNACR attains higher F1 scores than every baseline across both sensitive attributes and corruption levels, with each difference significant at p < 10^-4.These fairness improvements therefore do not reduce classification quality in the reported Adult experiments.
- ACSPublicCoverage: Distribution Shift across States: Across ACSPublicCoverage states, CNACR achieves the best EO-ratio and lowest Gini coefficient in binary-sex and five-group-race experiments.For race, the EO-ratio advantage is significant at p < 10^-4 against every baseline, with CNACR winning on 96 to 98 of 100 runs; the Gini advantage is significant with CNACR ahead on 98 or 99 runs.
- ACSPublicCoverage: Distribution Shift across States: CNACR’s F1 performance is state-dependent: it is best on five of eight race states and loses on TX, FL, and GA, while pooled F1 remains close to plain SVM.The same three states also account for weaker performance in the binary-sex experiment, suggesting sensitivity to state distributions.
- ACSPublicCoverage: Distribution Shift across States: CNACR’s fairness advantage grows with the number of protected groups, while competing constraints can overfit when many group-conditional rates rely on few positive examples.The authors report that constraining ten group-conditional rates simultaneously may overfit the fairness constraint.
- Within-group Fairness: Within-group unfairness decreases under CNACR in all four reported settings, whereas every constraint-based baseline increases it relative to plain SVM.The reductions are 2.6% and 3.2% for Adult sex, 5.4% and 5.6% for Adult race, and about 11% and 10% for ACSPublicCoverage.