Source-linked AI summary
Fairlearn: Assessing and Improving Fairness of AI Systems
Hilde Weerts, Miroslav Dudík, Richard Edgar, Adrin Jalali, Roman Lutz, Michael Madaio
TL;DR
AI systems can create fairness-related harms across population groups, motivating tools that assess and mitigate those harms. Fairlearn combines disaggregated evaluation, fairness-constrained mitigation methods, and learning resources grounded in a sociotechnical perspective. The project’s scope includes helping practitioners examine harms and make trade-offs appropriate to their contexts.
Problem
AI systems may disproportionately harm minorities, historically disadvantaged populations, and other groups considered sensitive from ethical or legal perspectives.
Method
Fairlearn combines disaggregated evaluation, fairness metrics, mitigation algorithms, and learning resources that address fairness as a sociotechnical challenge.
Results
Fairlearn provides tools for evaluating model performance across population groups and mitigating fairness-related harms through methods that support fairness constraints.
Takeaways & Limitations
Practitioners can use Fairlearn to assess fairness-related harms, review mitigation strategies, and make trade-offs appropriate to their sociotechnical context.
Takeaways & Limitations
Fairlearn cannot fully debias a system or guarantee fairness because unfairness has complex societal and technical sources.
Abstract
from arXiv · showhide
Fairlearn is an open source project to help practitioners assess and improve fairness of artificial intelligence (AI) systems. The associated Python library, also named fairlearn, supports evaluation of a model's output across affected populations and includes several algorithms for mitigating fairness issues. Grounded in the understanding that fairness is a sociotechnical challenge, the project integrates learning resources that aid practitioners in considering a system's broader societal context.
1. Introduction
Fairlearn is an open source project that helps practitioners assess and improve AI fairness through a Python library and learning resources. It treats fairness as a sociotechnical challenge involving potential harms to different groups.
- Project scope: Fairlearn combines a Python library with learning resources to help data science practitioners assess and improve AI system fairness.The project and its components are MIT-licensed and available online, with an API designed to integrate with common Python libraries.
- Project scope: The project is community-driven, independently governed, actively developed, and open to contributions to its code and learning resources.Its governance includes a code of conduct adapted from the Contributor Covenant.
- Fairness perspective: The project focuses on negative impacts for groups defined by characteristics such as race, gender, age, or disability status.Its fairness perspective centers on fairness-related harms rather than a guarantee that systems can be fully debiased.
- Fairness perspective: Fairness is treated as a sociotechnical challenge because unfairness can arise from both societal and technical sources.Fairlearn therefore aims to help practitioners assess harms, review mitigation strategies, and make context-dependent trade-offs.
- Fairness-related harms: Fairlearn largely addresses allocation harms and quality-of-service harms affecting different population groups.Examples include unequal recommendations for high-risk care programs and different computer-vision error rates across demographic groups.
2. Fairness Assessment
Fairlearn supports fairness assessment by evaluating model performance across population slices and comparing fairness-related metrics. Its tools build on Python data-science conventions and include standard and extensible fairness metrics.
- Disaggregated evaluation: Disaggregated evaluation measures performance metrics, such as error rates, across different population groups to identify disproportionate negative impacts.This assessment addresses allocation and quality-of-service harms.
- MetricFrame: MetricFrame takes metric functions, y_true, y_pred, and sensitive_features, then exposes metrics for data slices as pandas Series or DataFrames.The API combines scikit-learn and pandas conventions.
- MetricFrame: MetricFrame compares metric values across groups using differences or ratios and supports plotting through pandas and matplotlib integration.These comparisons can be used for fairness and overall-performance analysis.
- Fairness metrics: The fairlearn.metrics module provides scalar functions such as demographic_parity_difference and equalized_odds_difference.These metrics quantify how much classifier predictions depart from the corresponding fairness criteria.
- Model comparison: Fairlearn also supports comparing multiple models and creating model-comparison scatter plots using overall-performance and fairness metrics.The plotting function represents each model as a point with coordinates for the two metric types.
3. Algorithmic Mitigation of Fairness-related Harms
Fairlearn provides mitigation methods that wrap standard machine-learning algorithms and operate before, during, or after model training. These methods optimize performance objectives subject to specified fairness constraints.
- Mitigation framework: Fairlearn mitigation methods are versatile meta-algorithms that wrap standard fairness-unaware machine-learning algorithms while following scikit-learn API conventions.The library includes methods for mitigating fairness-related harms across several stages of the modeling process.
- Pre-processing: Pre-processing methods transform input data before training, as CorrelationRemover does by removing linear correlation with sensitive features.CorrelationRemover can be incorporated into a scikit-learn pipeline.
- In-training: In-training reduction methods optimize a performance metric subject to fairness constraints while wrapping standard classification or regression algorithms.ExponentiatedGradient supports constraints such as an upper bound on differences between false negative rates.
- In-training: Adversarial mitigation jointly trains a predictor and an adversary so the predictor minimizes prediction loss while limiting sensitive-feature inference from its outputs.The two neural-network models can be defined using PyTorch or TensorFlow.
- Post-processing: Post-processing transforms a trained model’s outputs by selecting separate group thresholds to optimize an objective under fairness constraints.ThresholdOptimizer can target balanced accuracy subject to constraints such as false negative rate parity.
4. Learning Resources
Fairlearn’s learning resources complement technical tools by teaching practitioners to assess fairness within broader sociotechnical contexts. They combine conceptual guidance, realistic examples, and educational datasets.
- Learning objectives: The learning resources define objectives for what practitioners should know or do when assessing and improving AI fairness.These objectives were developed through a community-based effort.
- Learning objectives: The user guide integrates coding guidance with concepts such as construct validity and the risks of abstracting away social context.This integration is intended to keep technical and social aspects of AI fairness connected.
- Examples: Tutorials and Jupyter notebooks use real-world or realistic deployment contexts and focus on harms to real people.The examples are designed to avoid abstraction traps when teaching sociotechnical fairness.
- Datasets: Datasets in fairlearn.datasets serve an educational role by highlighting sociotechnical fairness issues in popular benchmark datasets.The user guide discusses fairness-related issues associated with several such datasets.
5. Conclusions
Fairlearn is built and maintained by contributors with varied backgrounds and expertise, reflecting a commitment to advancing fairer AI through broad participation.
- Fairlearn brings together contributors with diverse backgrounds and expertise to support progress toward fairer AI systems.