PaperlayerView exact revision on arXiv

Source-linked AI summary

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning

Ángel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, Duen Horng Chau

arXiv:1904.05419v4cs.LGstat.ML

TL;DR

Machine learning fairness auditing must identify disparities across many intersectional subgroups while handling competing fairness metrics. FairVis combines user-guided subgroup exploration with automated discovery and similarity-based comparison in coordinated visual views. The system is presented as helping users discover biases in income and recidivism datasets, although its current implementation does not support very large datasets well.

  • Problem

    Fairness auditing is difficult because models may perform disparately across demographic subgroups, while intersectional subgroup spaces and fairness metrics are large and complex.

  • Method

    FairVis is a mixed-initiative visual analytics system combining user-specified groups, clustering-based subgroup suggestions, and statistically similar subgroup exploration.

  • Results

    FairVis helps users discover biases in real income and recidivism prediction datasets through coordinated exploration of subgroup performance.

  • Takeaways & Limitations

    Interactive visualization can help users understand subgroup fairness and investigate potential inequities in machine learning models.

  • Takeaways & Limitations

    The current implementation scales to tens and hundreds of thousands of data points but does not support much larger datasets well.

Abstract

from arXiv · show

The growing capability and accessibility of machine learning has led to its application to many real-world domains and data about people. Despite the benefits algorithmic systems may bring, models can reflect, inject, or exacerbate implicit and explicit societal biases into their outputs, disadvantaging certain demographic subgroups. Discovering which biases a machine learning model has introduced is a great challenge, due to the numerous definitions of fairness and the large number of potentially impacted subgroups. We present FairVis, a mixed-initiative visual analytics system that integrates a novel subgroup discovery technique for users to audit the fairness of machine learning models. Through FairVis, users can apply domain knowledge to generate and investigate known subgroups, and explore suggested and similar subgroups. FairVis' coordinated views enable users to explore a high-level overview of subgroup performance and subsequently drill down into detailed investigation of specific subgroups. We show how FairVis helps to discover biases in two real datasets used in predicting income and recidivism. As a visual analytics system devoted to discovering bias in machine learning, FairVis demonstrates how interactive visualization may help data scientists and the general public understand and create more equitable algorithmic systems.

1 INTRODUCTION

Auditing fairness is difficult because disparities can arise across intersectional subgroups, while fairness requires navigating many metrics and a combinatorially large search space. FAIRVIS addresses this challenge with interactive subgroup exploration, automated suggestions, and similarity-based comparison.

  • Models deployed in consequential domains can show different predictive accuracy across demographic groups.
  • Intersectional bias concerns populations defined by multiple features, such as “Black Females,” and can be difficult to detect.
  • Auditing becomes unmanageable as the number of potential subgroups grows combinatorially and multiple fairness metrics must be considered.
  • FAIRVIS is a mixed-initiative visual analytics system for exploring user-specified and suggested subgroups across fairness and performance metrics.
  • 40% accuracy illustrates how intersectional subgroup performance can fall below the 66.6%–72.2% range observed for single-feature groups.
  • FAIRVIS recommends intersectional subgroups by clustering similar instances, identifying important features, and ranking anomalously low fairness metrics.
  • Users can compare statistically similar subgroups to investigate performance differences and form more general groups.

2 BACKGROUND IN MACHINE LEARNING FAIRNESS

Fairness research establishes that algorithmic bias has societal and mathematical dimensions. In particular, differing group base rates can make simultaneous satisfaction of multiple fairness definitions impossible, requiring explicit trade-offs.

  • Machine learning fairness research addresses the mathematical foundations and societal impacts of algorithmic bias.
  • Different group base rates make it mathematically impossible to fulfill all definitions of fairness simultaneously.
  • COMPAS illustrates the practical implications of incompatible fairness definitions in recidivism prediction.
  • Bias-mitigation approaches span data obfuscation, training regularization, and post-processing predictions, but fairness-metric choices remain necessary.

3 RELATED WORK

Prior work includes subgroup auditing, automated slice discovery, and visual analytics for machine learning. FAIRVIS builds on these directions while targeting complex subgroups and multiple fairness metrics for bias auditing.

  • Kearns et al. audit many subgroups but assume that the subgroup collection is predefined for the task.
  • Slice Finder generates granular subgroups top-down, whereas FAIRVIS discovers groups bottom-up by clustering instances without imposing feature structure.
  • Visual analytics systems support understanding, debugging, and analysis of machine learning models across parts of the workflow.
  • Existing visualization methods are limited by subgroup complexity and the number of performance metrics they support.
  • Google’s What-If tool combines dataset exploration with counterfactual explanations and fairness modifications.

4 DESIGN CHALLENGES AND GOALS

FAIRVIS is designed to help users audit known and unknown subgroups, contextualize performance across metrics, search large subgroup spaces, compare similar groups, and reason about potential bias causes.

  • Design Challenges: Users need tools that narrow the combinatorial search space while conveying multiple fairness metrics for each subgroup.
  • Known Subgroups: Known subgroups should be easy to generate and evaluate using user-selected feature combinations.
  • Performance Context: Subgroup performance should be contextualized against overall and other-group performance while retaining detailed views.
  • Subgroup Discovery: Suggested under-performing subgroups can make analysis more efficient when many feature combinations produce hundreds or thousands of groups.
  • Similar Subgroups: Similar-subgroup search should support feature-importance analysis and formation of more general groups.
  • Metric Trade-offs and Bias Causes: Interfaces should expose trade-offs among fairness metrics and highlight information that may suggest causes of biased behavior.

5 FAIRVIS: DISCOVERING INTERSECTIONAL BIAS

FAIRVIS is a mixed-initiative visual analytics system for auditing intersectional bias through coordinated views of user-defined and automatically generated subgroups. Its interface combines dataset exploration, subgroup creation, performance comparison, and detailed investigation.

  • FAIRVIS integrates four coordinated views for dataset exploration, subgroup generation, performance comparison, and detailed subgroup analysis.
  • Generated subgroups can be added to the Subgroup Overview, then compared and investigated in the Detailed Comparison View.
  • Feature Distribution View: The Feature Distribution View shows feature distributions and lets users create specific or multi-feature subgroups from selected values.
  • Feature Distribution View: Users can define subgroups at varying levels of generality by choosing any combination of features and values.

5.2 Subgroup Overview [G2]

The Subgroup Overview provides a coordinated, metric-centered view for comparing subgroup performance. Users select fairness or performance metrics, inspect subgroup rankings relative to the dataset average, and pin groups for cross-metric investigation.

  • The Subgroup Overview uses interactive strip plots to compare generated subgroups across selected performance metrics.
  • Visualizing multiple fairness metrics: Selecting a metric adds a strip plot containing all current subgroups and the corresponding dataset average.
  • Visualizing multiple fairness metrics: Users can choose among Accuracy, Recall, Specificity, Precision, Negative Predictive Value, several error rates, and F1 score.
  • Hovering over a subgroup highlights it across displayed plots, while clicking pins it for detailed comparison.
  • Choice of visual encoding: Strip plots emphasize subgroup performance relative to other groups and the overall dataset, while filtering helps manage crowded displays.

5.3 Suggested Subgroups [G3]

Suggested Subgroups help users search a combinatorially large subgroup space for potentially underperforming groups. FAIRVIS clusters instances, describes clusters through feature dominance, and lets users sort, filter, inspect, and pin suggestions.

  • FAIRVIS algorithmically generates suggested subgroups to reduce the burden of manually inspecting every feature combination.
  • Generating suggested subgroups: Clustering groups statistically similar instances, with user-selected K controlling the number and size of generated subgroups.
  • Describing suggested subgroups: Feature entropy ranks dominant features, with lower entropy indicating a distribution concentrated in one value.
  • Displaying suggested subgroups: Suggested-subgroup cards display dominant features, value distributions, subgroup information, and performance metrics.
  • Exploring suggested subgroups: Users can sort suggestions by a chosen fairness metric, filter by subgroup size, inspect detailed metrics, and pin groups for comparison.

5.4 Similar Subgroups [G4]

FAIRVIS finds similar subgroups to help users examine feature differences associated with performance and broaden problematic groups. It uses statistical divergence for distributions and modified counterfactual comparisons for user-specified groups.

  • Similar-subgroup exploration addresses the combinatorially large search space surrounding an already identified problematic group.
  • Finding similar subgroups: For user-specified subgroups, modified counterfactual comparisons change one or two feature values and identify groups with surprising performance differences.
  • Finding similar subgroups: For suggested subgroups, FAIRVIS measures similarity by summing Jensen-Shannon divergence across feature distributions.
  • Displaying similar subgroups: The Similar Subgroups view presents cards with subgroup size, relevant feature differences, and access to detailed comparison.
  • Displaying similar subgroups: Users can sort similar groups by fairness metric and filter them by size to focus the search on important, sufficiently large subgroups.

5.5 Detailed Subgroup Analysis and Comparison [G5]

The Detailed Comparison View lets users compare pinned and hovered subgroups’ performance, label balance, defining features, size, and feature distributions. These coordinated comparisons help explain performance differences and support exporting discovered subgroups for further analysis.

  • Performance and label balance: Users compare pinned and hovered subgroups directly through grouped bar charts showing selected performance metrics.The view also displays both groups’ ground-truth label balance.
  • Subgroup composition: The view compares the defining features and size of pinned and hovered subgroups.For user-specified groups, the table shows defining features and values; for suggested groups, it shows the five dominant feature values.
  • Performance and label balance: Comparing label balance can explain extreme metrics, such as high accuracy alongside zero sensitivity for a subgroup dominated by negative labels.A subgroup with 95% negative values can achieve 95% accuracy by classifying every instance as negative.
  • Feature distributions: Feature Distribution View overlays subgroup histograms with the overall distribution and, for two groups, with each other to reveal distributional similarity.These comparisons can indicate why a subgroup underperforms and suggest potential resolutions.
  • Export: Users can export pinned and hovered subgroups, including their composition and metrics, to a JSON file.This supports sharing discovered subgroups for subsequent analysis.

6 USE CASES

FAIRVIS supports audits of known and newly discovered subgroups in recidivism and income prediction, combining overview comparisons with detailed subgroup investigation. Its use cases show how subgroup metrics, feature distributions, and similar-group comparisons expose disparities and suggest hypotheses for corrective action.

  • 6.1 Auditing for Known Biases in Recidivism Prediction: In recidivism auditing, FAIRVIS lets users generate race-and-sex intersectional subgroups and compare their performance across fairness metrics.The overview revealed broad variation across subgroup metrics, motivating focused investigation of false positive rates.
  • 6.1 Auditing for Known Biases in Recidivism Prediction: 43% false positive rate for African-American males exceeded the dataset average of around 29%, identifying the subgroup for further investigation.The data scientist pinned this subgroup in the Subgroup Overview to compare it with other groups.
  • 6.1 Auditing for Known Biases in Recidivism Prediction: African-American males had an almost 60% positive recidivism base rate versus just over 40% for Caucasian males, complicating accuracy comparisons.The data scientist noted that the subgroup’s high base rate, highest false positive rate, and similar accuracy to Caucasian males indicated a need for more equitable results.
  • 6 USE CASES: FAIRVIS enabled the data scientist to confirm suspected recidivism disparities and export the implicated subgroups for corrective-action planning.The income case similarly used subgroup and similar-group investigation to formulate hypotheses about data size, base rates, and model expressiveness.
  • 6.1 Auditing for Known Biases in Recidivism Prediction: A suggested recidivism subgroup contained 249 instances and had a 39% false positive rate, with Caucasian females and a felony charge as defining characteristics.The observed label balance and subgroup composition led to hypotheses involving training priority and model expressiveness.
  • 6.2 Discovering Biases in Income Prediction: In income prediction, male and female accuracy differed by almost 10%, while the male subgroup had higher precision and recall despite lower accuracy.The imbalance was discovered through feature histograms showing that males comprised more than two-thirds of instances.
  • 6.2 Discovering Biases in Income Prediction: The similar income-prediction subgroup combined common marital-status and relationship values yet remained relatively small, helping contextualize its poor performance.FAIRVIS showed that “married-civ-spouse” was the most common marital-status value and “own-child” the third most common relationship value.

7 TECHNICAL IMPLEMENTATION

FAIRVIS is implemented as a web-based system using React, visualization libraries, Material.ui, and Python-based preprocessing and clustering.

  • 7 TECHNICAL IMPLEMENTATION: FAIRVIS uses React for the web interface, D3.js and Vega Lite for visualizations, and Material.ui for visual components and styling.These libraries support the system’s visual and interface construction.
  • 7 TECHNICAL IMPLEMENTATION: Python scripts handle preprocessing and clustering using scikit-learn’s K-means implementation.The clustering pipeline uses K-means to support subgroup analysis.

8 LIMITATIONS AND FUTURE WORK

FAIRVIS’s subgroup-generation technique still needs effectiveness validation and broader clustering support. Future work also targets additional data types, larger datasets, and automated bias resolutions.

  • Improving and measuring the effectiveness of the subgroup generation technique: The effectiveness of generated subgroup suggestions remains to be measured against groups users consider important.The authors propose collecting labeled datasets with model outputs and important underperforming subgroups for evaluation.
  • Improving and measuring the effectiveness of the subgroup generation technique: Future versions may add clustering methods such as subspace clustering to reveal groups defined by only a few features, especially in high-dimensional data.These methods could help users compare alternative subgroup discoveries and expose poorly performing groups.
  • Supporting more types of problems and data: FAIRVIS currently supports only binary classification and tabular data, with additional views needed for regression and expanded support for other data types.The authors specifically mention multiclass classification, graphical or textual data, and displaying images during audits.
  • Scaling to millions of instances: The current implementation scales to tens and hundreds of thousands of data points but does not support datasets with millions of instances well.The authors plan to improve subgroup-generation and suggestion efficiency for browser-based operation at larger scales.
  • Suggesting and providing automatic resolutions: Future work will add automatic bias resolutions, including post-processing steps and mechanisms that highlight or suggest potential issues.The paper notes that existing techniques can address bias after classifier prediction.
Loading 1904.05419v4…