Learning Controllable Fair Representations

Jiaming Song; Pratyusha Kalluri; Aditya Grover; Shengjia Zhao; Stefano Ermon

Source-linked AI summary

Learning Controllable Fair Representations

Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, Stefano Ermon

arXiv:1812.04218v3Dec 11, 2018cs.LGcs.AIstat.ML

TL;DR

Fair representation learning must preserve utility for unknown downstream tasks while limiting information about protected attributes. The paper formulates this as a mutual-information constrained optimization problem and introduces dual optimization to learn the model and fairness–expressiveness trade-off. The resulting method achieves higher expressiveness while satisfying user-specified unfairness limits, with empirical evidence of lower computational cost and support for multiple fairness notions.

Problem
Unknown downstream tasks require representations that remain useful while protecting sensitive attributes and balancing multiple fairness notions.
Method
The paper uses mutual-information objectives and hard fairness constraints, then jointly optimizes model parameters and Lagrange multipliers.
Results
The proposed dual method achieves higher expressiveness while satisfying user-specified unfairness limits, and empirical results support balancing multiple fairness notions at lower computational cost.
Takeaways & Limitations
Users can control representation fairness through interpretable constraints rather than manually selecting an indirect trade-off parameter.
Takeaways & Limitations
Existing fixed-multiplier approaches require manual hyperparameter searches because their multipliers do not provide an interpretable relationship to unfairness limits.

Abstract

from arXiv · show

Learning data representations that are transferable and are fair with respect to certain protected attributes is crucial to reducing unfair decisions while preserving the utility of the data. We propose an information-theoretically motivated objective for learning maximally expressive representations subject to fairness constraints. We demonstrate that a range of existing approaches optimize approximations to the Lagrangian dual of our objective. In contrast to these existing approaches, our objective allows the user to control the fairness of the representations by specifying limits on unfairness. Exploiting duality, we introduce a method that optimizes the model parameters as well as the expressiveness-fairness trade-off. Empirical evidence suggests that our proposed method can balance the trade-off between multiple notions of fairness and achieves higher expressiveness at a lower computational cost.

1 INTRODUCTION

The paper frames fair representation learning as a need for transferable, expressive representations that protect sensitive attributes while trading off multiple fairness notions. It introduces an information-theoretic constrained framework with user-controllable fairness limits and a dual method that learns the expressiveness–fairness trade-off.

Transferable representations must support unknown downstream tasks while preserving fairness toward sensitive attributes.
Expressiveness and fairness can conflict, and multiple fairness notions may not be simultaneously satisfiable.
The framework maximizes representation expressiveness subject to fairness constraints expressed through mutual information.
Existing approaches approximate the framework’s Lagrangian dual with fixed multipliers, requiring different representations for different fairness notions.
The proposed dual optimization method jointly optimizes model parameters and Lagrange multipliers to learn the expressiveness–fairness trade-off.
Empirically, the method provides user control through interpretable fairness constraints and balances multiple fairness notions with one representation at lower computational cost.

2 AN INFORMATION-THEORETIC OBJECTIVE FOR CONTROLLABLE FAIR REPRESENTATIONS

The paper formulates fair representation learning as maximizing information about observations while constraining information about sensitive attributes. It derives tractable bounds and practical constraints, with alternative guarantees that trade fairness tightness against expressiveness.

The representation z should be transferable across unknown downstream tasks while protecting sensitive attributes u.
Fairness is encouraged by minimizing Iq(z; u), while expressiveness is encouraged by maximizing Iq(x; z|u).
Conditioning expressiveness on u avoids encouraging information in x correlated with u to leak into z.
A hard constraint limits Iq(z; u) to a user-specified threshold ϵ, even if satisfying it reduces expressiveness or predictive power.
Because the mutual-information terms are difficult to estimate and optimize, the method replaces them with tractable variational bounds and Monte Carlo gradient estimates.
The practical formulation uses C1 and C2 to limit Iq(z; u), with C1 favoring fairness guarantees and C2 favoring expressiveness under an optimal adversary.

3 A UNIFYING FRAMEWORK FOR RELATED WORK

The framework views several existing fair-representation methods as approximations to a common Lagrangian-dual objective with fixed multipliers. Because these methods ignore explicit fairness limits, achieving a desired unfairness level requires manual hyperparameter search.

The framework covers methods based on clustering, adversarial training, and variational autoencoders with fairness regularization.
The reviewed methods are limited to binary sensitive attributes when fairness is measured through statistical parity.
Several existing methods learn fair representations using objectives that approximate the Lagrangian dual of the paper’s objective.
Existing approaches generally require separate representations for different fairness notions because they optimize dual objectives with fixed multiplier settings.
These methods use fixed multipliers as hyperparameters controlling the relative strengths of fairness constraints.
The values of ϵ do not affect the final solution in these approaches, so satisfying specific constraints requires computationally inefficient hyperparameter search.

4 DUAL OPTIMIZATION FOR CONTROLLABLE FAIR REPRESENTATIONS

The paper solves its constrained representation-learning problem through a dual objective that jointly optimizes model parameters and Lagrange multipliers. Strong duality holds over distribution space, while practical gradient updates adjust multipliers to enforce fairness constraints, subject to feasible limits.

The dual method jointly optimizes model parameters and Lagrangian multipliers to guarantee satisfaction of fairness constraints.
The multipliers λ and limits ϵ encode the strengths and permitted levels of the fairness constraints.
Strong duality holds for the distribution-space optimization when ϵ1 and ϵ2 are positive.
The proof uses convexity of KL divergence and mutual information to establish Slater’s conditions.
Gradient ascent on λ increases a multiplier when its constraint is violated, trading representation expressiveness for fairness.
With finite model capacity, an excessively small ϵ may make the specified fairness constraints infeasible.

5 EXPERIMENTS

The experiments test whether mutual-information objectives align with conventional fairness and accuracy measures, whether L-MIFR controls fairness through explicit budgets, and whether it improves expressiveness and efficiency over MIFR. Across these tests, L-MIFR provides controllable fairness, competitive or better representations, and lower computational cost.

Experimental setup: The experiments evaluate objective alignment, constraint control, MIFR versus L-MIFR, adversarial-training sensitivity, and balancing multiple fairness notions.Evaluation uses German credit, Adult, and Health datasets, with downstream prediction tasks and sensitive attributes varying across datasets.
Experimental setup: The study uses German credit, Adult, and Health datasets with downstream prediction tasks and sensitive attributes including age and gender.Health contains 18 sensitive-attribute configurations, preventing direct application of VFAE and LAFTR in this setup.
Mutual Information, Prediction Accuracy, and Fairness: Mutual information between inputs and representations correlates positively with test AUC, while mutual information with sensitive attributes correlates positively with ∆DP and therefore lower fairness.Principal-component features have slightly better AUC but significantly worse ∆DP fairness than the learned representations.
Controlling Representation Fairness with L-MIFR: Increasing a constraint budget ϵi increases its corresponding constraint value Ci while maintaining Ci < ϵi, enabling quantitative control over representation fairness.In Adult, ∆DP clearly increases with ϵi; the trend is less obvious in German because ∆DP is already very low.
Improving Representation Expressiveness with L-MIFR: L-MIFR outperforms MIFR in representation expressiveness despite MIFR using almost 25x the computational resources.One L-MIFR run per ϵ setting takes roughly the time of one MIFR run, apart from updating two scalar multipliers.
Hyperparameter sensitivity: L-MIFR’s final representations are not very sensitive to the number of adversarial-training iterations D, although larger D tends to produce solutions closer to ϵ2.This sensitivity analysis evaluates Adult at ϵ2 = 0.10 and Health at ϵ2 = 0.30.

6 DISCUSSION

The paper frames its mutual-information objective as a controllable approach to fair representation learning and uses dual optimization to improve expressiveness under user-specified unfairness limits. It also identifies extensions to other fairness notions as future work.

The proposed dual optimization method targets higher expressiveness while satisfying a user-specified limit on unfairness.
The mutual-information objective unifies and explains a range of existing fair-representation approaches through their connection to fixed-multiplier Lagrangian duals.
Future work includes formally and empirically extending the framework and dual optimization method to other fairness notions.
MIFR can approach the feasible set from outside, so its solution generally has high expressiveness.

A.1 Proof of Lemma 1

The proof rewrites an information-theoretic quantity using a reconstruction-related expression and bounds it using the non-negativity of KL divergence.

The derivation expresses the quantity through an expectation involving log qφ(x|z, u), log p(x|z, u), and conditional entropy Hq(x|u).
The bound provides the proof step connecting the expectation-based expression to the information-theoretic quantity.
The final inequality follows because KL divergence is non-negative.

A.2 Proof of Lemma 2

The proof establishes convexity of the constrained problem, constructs a feasible solution using independent noise, and invokes Slater’s condition to obtain strong duality.

The proof begins by verifying that the optimization problem is convex.
The primal reconstruction term is affine in qφ(z|x, u) and convex in pθ(x|z, u), while the first constraint is convex through KL divergence.
Convexity of KL divergence implies that the fairness-related mutual-information terms are convex in qφ(z|x, u).
A feasible solution sets z independently of u, making both KL constraints zero; this corresponds to random noise that captures nothing about u.
Because the constructed solution satisfies the constraints strictly, Slater’s condition holds and strong duality follows.

B Experimental Setup Details

The experiments compare MIFR and L-MIFR under specified constraint or multiplier settings, using adversarial neural-network training across three datasets with different training budgets and latent dimensions.

MIFR uses α = 1 with λ1 and λ2 grids totaling 25 configurations for the fairness constraints.
L-MIFR modifies ϵ1 and ϵ2 according to estimated dataset values to evaluate results for specified hyperparameters as other settings change.
Training uses Adam with initial learning rate 1e−3, β1 = 0.5, and a 0.98 learning-rate multiplier every 1000 iterations.
L-MIFR initializes λi to 1.0 and allows values from 0.01 to 100.
The experiments update the adversary ten times per representation or decoder update unless otherwise specified.
Adult and Health are optimized for 2000 epochs, whereas German is optimized for 10000 epochs.
The models are two-layer neural networks with 50-neuron softplus hidden layers, using latent dimension 10 for German and Adult and 30 for Health.

C Comparison with LAFTR

The comparison positions MIFR as a representation-learning method that can handle non-binary sensitive attributes and achieve a comparable accuracy–fairness trade-off to LAFTR. Across fairness notions, L-MIFR uses one representation while LAFTR often sacrifices other notions when optimized for a particular one.

Method differences: MIFR supports arbitrary sensitive-attribute types, whereas LAFTR is restricted to binary sensitive attributes.The methods also differ because MIFR does not require access to the downstream prediction task during representation learning.
Empirical comparison: 0.829 accuracy and ∆DP of 0.037 for MIFR versus 0.821 accuracy and ∆DP of 0.029 for LAFTR show comparable accuracy–fairness trade-offs.For this comparison, classification error was added to MIFR during training.
Multiple fairness notions: L-MIFR uses one representation across ∆DP, ∆EO, and ∆EOpp, whereas LAFTR often trades fairness on other notions when trained for one notion.LAFTR achieves better fairness on an individual notion when specifically trained for it, but may perform worse on other fairness notions.
Utility constraint: The downstream utility requirement can be encoded either in the primal objective or as a minimum accuracy constraint for predicting y from z.This formulation assumes access to labels for a particular task.
Fairness criteria: For downstream tasks, equalized odds requires low mutual information between z and u conditioned on y, while equalized opportunity conditions on y = 1.These information-theoretic criteria are intended to apply to any classifier over the representation z.