Source-linked AI summary
FairSight: Visual Analytics for Fairness in Decision Making
Yongsu Ahn, Yu-Ru Lin
TL;DR
Data-driven decisions increasingly rank individuals, yet discrimination and opaque reasoning remain concerns, while fairness research has not translated into viable practice tools. FairSight integrates a fairness framework, visual analytics, and diagnostic methods across the decision pipeline, and its case and user studies report useful, understandable support for fairer decision making. The authors also identify visualization clutter, population-ratio sensitivity, and binary sensitive-group modeling as limitations.
Problem
Data-driven decisions about individuals can discriminate and lack explanations, while existing fairness measures and algorithms have not become comprehensive practical tools.
Method
FairSight integrates the FairDM fairness pipeline with visual analytics, fairness measures, feature-bias diagnostics, and global- and instance-level explanations.
Results
The case study and user study found FairSight useful for understanding, measuring, and mitigating bias, with a reported advantage over an existing tool.
Takeaways & Limitations
Fair decision tools need workflow-wide diagnosis and human scrutiny of features, not only fairness-aware algorithms and outcome measures.
Takeaways & Limitations
Visualization can clutter as instances increase, mislead under skewed group populations, and inadequately fit cases requiring more than binary protected and non-protected groups.
Abstract
from arXiv · showhide
Data-driven decision making related to individuals has become increasingly pervasive, but the issue concerning the potential discrimination has been raised by recent studies. In response, researchers have made efforts to propose and implement fairness measures and algorithms, but those efforts have not been translated to the real-world practice of data-driven decision making. As such, there is still an urgent need to create a viable tool to facilitate fair decision making. We propose FairSight, a visual analytic system to address this need; it is designed to achieve different notions of fairness in ranking decisions through identifying the required actions -- understanding, measuring, diagnosing and mitigating biases -- that together lead to fairer decision making. Through a case study and user study, we demonstrate that the proposed visual analytic and diagnostic modules in the system are effective in understanding the fairness-aware decision pipeline and obtaining more fair outcomes.
1 INTRODUCTION
FairSight addresses discrimination and opacity in pervasive ranking-based decisions by bringing fairness research into a practical, end-to-end decision-making tool. Its framework and visual analytics support understanding, measuring, diagnosing, and mitigating bias, with case and user studies indicating usefulness and advantages over an existing tool.
- Motivation: The paper responds to discrimination and limited explainability in data-driven decisions, including risks from biased personal profiles and black-box models.The introduction cites reported bias in recruiting and recidivism assessment as examples of the broader concern.
- FairDM: FairDM organizes fair decision making around understanding, measuring, identifying, and mitigating bias across group and individual fairness notions.The framework is intended to support a contemplative decision-making process rather than only apply isolated fairness algorithms.
- Fairness measures and explainability: FairSight introduces measures and explainability methods that assess bias, evaluate feature impacts, mitigate sources, and explain ranking fairness globally and for individual instances.The approach targets reasoning behind fair ranking decisions at both global and instance levels.
- FairSight: FairSight provides a viable visual analytic system for fair decision making across the machine learning workflow.Its workflow represents machine-learning phases as spaces mapping individuals and groups from one stage to another.
- Evaluation: A case study and user study evaluate FairSight in real-world decision-making practice and report effectiveness, understandability, and an advantage over an existing tool.The evaluation combines objective and subjective user-study evidence with a case study.
2 RELATED WORK
Prior work developed ranking-fairness measures, fair ranking algorithms, and explainable machine-learning techniques, but did not provide a comprehensive fairness workflow. FairSight extends this work by integrating fairness-aware explanation, diagnosis, and mitigation throughout decision making.
- Fair ranking: Earlier fair-ranking research mainly quantified discrimination or proposed re-ranking methods, while FairSight considers trade-offs among fairness notions and utility in decision making.The related work describes top-k and indirect-discrimination measures alongside several re-ranking approaches.
- Explainable machine learning: Existing explainable-machine-learning approaches addressed input–prediction interactions, but the paper identifies fairness-specific explainability as unaddressed.FairSight applies feature auditing and introduces neighborhood-comparison metrics for global- and instance-level fairness evidence.
- Fairness-supporting frameworks: Prior fairness tools combined explainable techniques and fair algorithms but lacked comprehensive guidance for actions across the decision pipeline.The paper contrasts this gap with FairSight’s guideline-based design and comparison against the What-if tool.
3 FAIR DECISION MAKING FRAMEWORK
FairDM models ranking-based decisions as a three-phase Data–Model–Outcome pipeline and prescribes four fairness-aware actions for each phase. FairSight operationalizes this framework with diagnostics for measuring, locating, explaining, and mitigating bias, especially in feature selection and ranking outcomes.
- Framework and ranking problem: FairDM treats fair decision making as a sequence of required actions that keep process and outcome fairness in check for top-k ranking.The framework begins with a top-k problem in which a decision maker selects the best k individuals from n candidates.
- Machine learning pipeline: The pipeline has Data, Model, and Outcome phases, with features representing candidates, a model mapping features to outcomes, and a ranking ordered by predicted qualification.The primary task is selecting top-k candidates from the resulting ordered list.
- Fairness-aware workflow: Each pipeline phase is supported by four actions: understand potential unfairness, measure bias, identify its sources, and mitigate it through diagnostic actions.This workflow is designed to examine both the process and outcome of ranking decisions.
- Measure: The framework quantifies fairness throughout the pipeline to detect consequential outcome bias and procedural bias in Data and Model phases.This extends measurement beyond the final ranking outcome.
- Identify: Feature inspection examines sensitive attributes and likely proxy variables, while per-feature auditing investigates pre-processing, in-processing, and post-processing bias.Feature-level diagnosis is treated as a prerequisite for informed mitigation.
- Mitigate: Mitigation addresses fairness-aware feature selection, less-biased model choice, and adjustment of ranking outcomes across Data, Model, and Outcome phases.The framework therefore links diagnosis to concrete interventions at each pipeline stage.
4 METHOD
FairSight models fair decision making as a pipeline across Input and Output spaces, measuring individual and group fairness through data, mapping, and outcome biases.
- 4.1 Understanding Bias: FairSight represents data and outcome phases as Input and Output metric spaces connected by a machine-learning mapping, allowing bias to be examined within spaces and between them.The Input space contains feature representations, the Output space contains ranked individuals, and the model maps I to O.
- 4.2 Measuring Bias: FairSight measures fairness across individual and group notions at both instance and global levels, alongside fairness and utility measures for ranking decisions.Its measure requirements organize evidence consistently with the space-mapping representation and support complementary fairness notions.
- 4.2.2 Individual Fairness: Individual fairness is assessed by distortion between Input-space distances and Output-space ranking distances, using nearest neighbors to quantify instance-level bias and averaging for global bias.The method uses four nearest neighbors in this work and normalizes absolute ranking differences by dataset size and neighborhood size.
- 4.2.2 Individual Fairness: Signed ranking distortion distinguishes whether an individual is disadvantaged or advantaged relative to similar nearest neighbors.This complements the magnitude-based rNN measure with the direction of ranking disparity.
- 4.2.3 Group Fairness: Group fairness evaluates separation in the data, distortion in the mapping, and outcome distribution across protected and non-protected groups.Group Separation uses symmetric Hausdorff distance, Group Skew measures between- versus within-group distortion, and outcome measures include ranking comparisons and top-k evaluation.
- 4.2.3 Group Fairness: Statistical parity is adopted for comparing different rankings because discounted measures such as GFDCG become less effective when the top-k threshold is large.GFDCG remains useful for comparing thresholds within the same ranking.
- 4.3 Mitigating Bias: FairSight provides three strategies and three types of methods for reducing bias across the machine-learning workflow.These strategies support mitigation across the system’s identified bias sources and decision phases.
5 DESIGN GOALS
The design goals require FairSight to support multiple fairness notions through consistent visual components and tools, beginning with sensitive-attribute and protected-group selection.
- R1. Consistent Fairness Examination: FairSight should let users examine different fairness notions in data-driven decisions through consistent visual components and interactive tools.The requirement explicitly covers both individual and group fairness.
- R1. Consistent Fairness Examination: Users should select a sensitive attribute and protected group while receiving an integrated representation consistent with individual and group fairness.The interface is intended to support pursuing different fairness notions through these selections.
R2. Facilitate the understanding and measuring of bias.
FairSight should help users understand where bias appears across the machine-learning process and summarize fairness and utility to support fair decisions amid trade-offs.
- R2. Understanding and Measuring Bias: FairSight should summarize fairness and utility levels so users can obtain fair decisions while considering trade-offs.The goal is to provide a compact assessment of both dimensions.
- R2. Understanding and Measuring Bias: Instance-level exploration should help users understand why individuals and groups are processed fairly or unfairly.The requirement emphasizes explanations for both individual and group outcomes.
R3. Provide diagnostic modules to identify and mitigate bias.
FairSight should diagnose potential feature sources of bias, support mitigation across machine-learning phases, and let users compare multiple rankings and their fairness–utility trade-offs.
- R3. Identify Bias: Feature evaluation should identify evidence about feature selection and fairness across the three machine-learning phases.The task focuses on examining features involved in preprocessing and other phases.
- R3. Mitigate Bias: Users should be able to mitigate bias to obtain better rankings through methods available across all three phases.This supports procedural fairness throughout the workflow.
- R4. Compare Rankings: FairSight should support comparison of multiple rankings to evaluate ranking decisions.The comparison requirement concerns alternative generated rankings.
- R4. Compare Rankings: Users should repeat the process to generate multiple rankings and evaluate fairness–utility trade-offs across them.The ranking workflow includes a list and plot of generated ranking outcomes.
6 FAIRSIGHT - SYSTEM OVERVIEW
FairSight organizes fair decision making into six visual analytic components spanning setup, ranking outcomes, fairness inspection, feature diagnosis, and ranking comparison. Together, these views help users measure fairness and utility, inspect individual and group bias, diagnose feature effects, and compare trade-offs.
- System architecture: FairSight integrates six visual analytic components that support fair decision making across the machine learning workflow.The system includes Generator, Ranking View, Global Inspection View, Local Inspection View, Feature Inspection View, and Ranking List View.
- Generator: Generator lets users select sensitive attributes, inspect group-based feature distributions and correlations, and choose standard or fairness-aware learning and mitigation methods.Available methods include Logistic Regression, Support Vector Machine, Ranking SVM, Additive Counterfactual Fairness, and FA*IR.
- Ranking View: Ranking View reports between-ranking and within-ranking fairness and utility measures while allowing users to inspect ranking trends and select top-k thresholds.Between-ranking measures compare generated rankings, whereas within-ranking measures help identify a suitable top-k threshold.
- Inspection Views: Global and Local Inspection Views expose fairness across workflow phases and provide individual-, group-, and neighborhood-level comparisons of bias and feature values.Global Inspection View separates individual and group fairness, while Local Inspection View compares an individual with nearest neighbors and group averages.
- Feature Inspection View: Feature Inspection View diagnoses potential feature bias through distortion distributions, outliers, feature correlations, and ranking changes after perturbation.Perturbed rankings encode previous top-k membership, before-perturbation rank, group membership, and target labels.
- Ranking List View: Ranking List View compares generated rankings using group fairness, individual fairness, and utility measures to make fairness–utility trade-offs explicit.Linear gauges show ideal and current scores for representative measures, supplemented by additional numerical measures.
7 CASE STUDY
The loan-approval case study shows how FairSight supports iterative fairness diagnosis and mitigation, from detecting group and individual bias to comparing rankings and adjusting the top-k threshold. Removing biased features improved fairness, and the final selection balanced higher fairness and utility.
- Iteration 2: Local inspection identified a woman with rNN = 0.67 and rNNgain = -0.13, linking individual disadvantage to differences in account status, marriage, and job features.The comparison used nearest neighbors to explain how the individual differed from surrounding cases.
- Ranking comparison: After mitigation, overall fairness improved to Group Separation = 0.09, Group skew = 1.01, and GFDCG = 1.26, while group bias measures became closer.The ranking comparison showed a trade-off among outcomes, but the last ranking achieved both higher fairness and utility scores.
- Iteration 6: Increasing the threshold to k = 50 improved within-ranking fairness by 0.14 while utility remained 0.82.The analyst used the fairness and utility trends near the selected threshold to finalize the ranking decision.
8 USER STUDY
The user study compared FairSight with Google’s What-if tool on fairness-aware decision tasks using objective accuracy and subjective ratings. FairSight achieved higher accuracy and significantly better understandability and usefulness, while What-if remained stronger for some instance-level explanations but lacked multiple-outcome comparison.
- Study design: Participants completed 12 fairness-stage tasks across three machine learning phases, plus questions assessing decision comparison and explanation.Accuracy measured whether users identified directly relevant information, while ratings used five-point understandability and usefulness scales.
- Results: FairSight achieved 95% overall accuracy versus 80% for What-if and significantly outperformed it in understandability and usefulness.The evaluation used 20 participants and compared performance across fairness-pipeline tasks.
- Evaluation dimensions: FairSight was evaluated across four fairness stages and three decision-making criteria: Fairness, Decision, and Explain.Figure 7 reports subjective ratings and accuracy for these evaluation dimensions.
- Comparative findings: What-if scored 3.5 for instance-level counterfactual reasoning and feature-importance explanations but lacked support for comparing multiple outcomes.This identifies a comparative strength and limitation of the baseline tool within the study.
- Subjective feedback: Participants reported that FairSight increased awareness of fairness, clarified changes across workflow phases, and supported diagnosis of direct or indirect discriminatory features.Subjective feedback emphasized the framework, visual overview, and feature-focused diagnosis.
9 DISCUSSION
FairSight’s discussion identifies pre-processing as the most critical stage for improving fairness, while emphasizing that fair decision making still requires human judgment and careful interpretation of visualizations. The system’s mapping-based representation connects fairer feature processing to less biased mappings and fairer ranking outcomes, but its visualizations have scalability and group-representation limitations.
- Importance of pre-processing stage: 80% of user-study exploration time focused on detecting and removing feature bias, while the first four case-study iterations substantially improved fairness scores during pre-processing.The authors argue that offering only fairness-aware algorithms and outcome measures is insufficient; practitioners also need comprehensive pre-processing support.
- Interaction between spaces: Fairer data representations produced less biased mappings and ultimately greater fairness in ranking outcomes than features without pre-processing.FairSight represents the machine-learning process as mappings between Data and Outcome spaces, making bias reinforcement across the pipeline visible.
- Subjectiveness of feature selection: Feature selection remains subjective because judging importance and fairness depends on domain knowledge, context, and differing perceptions of fairness.The authors note that no generally accepted criterion specifies the trade-off between fairness and utility in decision outcomes.
- Limitation: With more instances, the visualization becomes cluttered, and skewed group populations can make a fair ranking appear unfair.The system also models the sensitive attribute as a binary protected versus non-protected distinction, which may not fit some cases.
10 CONCLUSION
The paper presents FairDM and FairSight as a framework and visual analytic system for comprehensive fair data-driven decision making. Evaluation found the system useful for fairness work, while future work targets multiple groups and scalable ranking visualizations.
- 10 CONCLUSION: FairSight’s evaluation demonstrated usefulness for fairness work, and the case study showed that it could measure and mitigate bias on a well-known dataset.The conclusion positions FairSight as a viable system for bringing comprehensive fair decision making into real-world practice.
- 10 CONCLUSION: Future work will extend FairSight from binary sensitive attributes to multiple groups, sub-groups, and user-defined groups.This addresses the current representation of sensitive attributes as protected versus non-protected groups.
- 10 CONCLUSION: Future work will develop scalable visual representations of rankings for industry-scale datasets.Planned examples include more efficient matrix representations and reordering for presenting fairness.