PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools

Emily Black, Rakshit Naidu, Rayid Ghani, Kit T. Rodolfa, Daniel E. Ho, Hoda Heidari

arXiv:2309.17337v1cs.LGcs.AIcs.CY

TL;DR

In practice, fairness mitigation often reduces to enforcing an arbitrarily chosen metric rather than operationalizing a pipeline-aware approach. The authors survey pipeline fairness research and classify work by pipeline stage and whether it identifies, measures, mitigates, or documents a fairness problem. Identifying root causes of unfairness in the ML pipeline is challenging, even for individuals with specialized ML knowledge.

Abstract

from arXiv · show

While algorithmic fairness is a thriving area of research, in practice, mitigating issues of bias often gets reduced to enforcing an arbitrarily chosen fairness metric, either by enforcing fairness constraints during the optimization step, post-processing model outputs, or by manipulating the training data. Recent work has called on the ML community to take a more holistic approach to tackle fairness issues by systematically investigating the many design choices made through the ML pipeline, and identifying interventions that target the issue's root cause, as opposed to its symptoms. While we share the conviction that this pipeline-based approach is the most appropriate for combating algorithmic unfairness on the ground, we believe there are currently very few methods of \emph{operationalizing} this approach in practice. Drawing on our experience as educators and practitioners, we first demonstrate that without clear guidelines and toolkits, even individuals with specialized ML knowledge find it challenging to hypothesize how various design choices influence model behavior. We then consult the fair-ML literature to understand the progress to date toward operationalizing the pipeline-aware approach: we systematically collect and organize the prior work that attempts to detect, measure, and mitigate various sources of unfairness through the ML pipeline. We utilize this extensive categorization of previous contributions to sketch a research agenda for the community. We hope this work serves as the stepping stone toward a more comprehensive set of resources for ML researchers, practitioners, and students interested in exploring, designing, and testing pipeline-oriented approaches to algorithmic fairness.

1 INTRODUCTION

The paper argues that fairness mitigation often targets selected metrics rather than the sources of unfairness across the ML pipeline. It calls for practical tools and guidance, noting that even people with substantial ML background struggle to connect design choices to model behavior.

1 INTRODUCTION

  • Fairness mitigation often focuses on an arbitrarily chosen metric or ad hoc intervention, which can have little practical impact or worsen disparities.The paper contrasts these approaches with examining choices throughout the pipeline to identify and address underlying sources of unfairness.Source: §1 · pp. 1–2

aimtoalleviate[178,298].Priorscholarshiphasattributedtheseproblematictrendstothefactthatthetraditionalapproach

  • For undesirable fairness behavior, teams should search choices across the ML creation pipeline for plausible causes, evaluate alternatives such as changing the prediction target and retraining, and iteratively re-evaluate the model.Source: aimtoalleviate[178,298].Priorscholarshi… · p. 2

satisfactory, whereupon bias testing and model updates would continue throughout deployment.

  • Students with significant ML backgrounds struggled to identify harmful model behavior and propose appropriate mitigations, motivating a need for practical guidance.The authors report this observation from qualitative data gathered in a graduate-level fair-ML class and say it accords with their practitioner experience.Source: satisfactory, whereupon bias testing… · p. 2
  • The authors call for usable tools to measure discrimination, locate its sources in the pipeline, and match those sources with appropriate interventions.They argue conceptual awareness alone is insufficient and survey methods addressing specific design choices and pipeline stages.Source: satisfactory, whereupon bias testing… · pp. 2–3
  • The survey is intended as a practical resource for locating potential causes of unfairness and reviewing tools to detect, measure, and mitigate them.The authors also turned it into an interactive, community-maintained wiki for researchers, practitioners, and students.Source: satisfactory, whereupon bias testing… · p. 3

and testing system-level approaches to algorithmic fairness.2

The authors use their survey to outline a research agenda for operationalizing pipeline-aware fairness and introduce pipeline cards as an initial practitioner tool.

and testing system-level approaches to algorithmic fairness.2

  • Research effort has concentrated on statistical modeling, especially mitigation, while viability assessment, problem formulation, deployment, monitoring, and stage interactions remain understudied.The authors hypothesize that some neglected areas attract less work because they offer fewer opportunities for novel quantitative contributions or theoretical analysis.Source: and testing system-level approaches to… · p. 3
  • The authors identify a disconnect between research that identifies pipeline problems and work that mitigates their sources.They frame weighing imperfect design choices as a key collaboration avenue for ML experts and ethicists.Source: and testing system-level approaches to… · p. 3
  • The paper proposes pipeline cards as an initial tool for practitioners seeking to apply a pipeline-aware approach to fairness.The authors present them as an actionable step toward practice.Source: and testing system-level approaches to… · p. 4

2 THE NEED TO OPERATIONALIZE THE PIPELINE-AWARE APPROACH

The section argues that operationalizing a pipeline-aware approach is necessary to address discrimination in practice, using the Upstart case to illustrate its legal and practical relevance and a classroom study to examine students’ ability to diagnose harms and remedies. Student teams often identified these broadly, though some proposed specific interventions such as random instance selection or greater transparency.

2 THE NEED TO OPERATIONALIZE THE PIPELINE-AWARE APPROACH

  • Even students with prior ML experience struggled to identify plausible causes of unfairness and remedies, showing that conceptual pipeline knowledge alone does not ensure good practice.The section uses this finding to motivate clearer operational guidance and tools.Source: §2 · p. 4

2.1 Evidence from Regulatory Enforcement

  • Pipeline-aware analysis may help find less discriminatory alternatives while avoiding some legal concerns raised by fairness methods that directly use protected attributes to alter models.Such methods may use protected-attribute labels to compare models without directly enforcing a constraint with those attributes.Source: §2.1 · p. 4
  • Upstart investigators found a less discriminatory model by testing feature combinations, but the search took so long that Upstart updated its model before the investigation concluded.The discovered model had a performance drop, and whether it met the legal standard of equivalent performance remained unclear.Source: lishedandevidenceestablishesanLDA,acomp… · p. 5, §2.2 · p. 5
  • Searching only the feature-selection stage is likely expensive and inefficient; pipeline-aware tools could examine other sources of bias and support broader, more rigorous audits.The paper argues that operationalized pipeline-aware tools could help regulators and advocates enforce the use of less discriminatory systems.Source: lishedandevidenceestablishesanLDA,acomp… · p. 5, model with acceptable predictive… · p. 5

2.2 Evidence from the Classroom

  • The classroom activity involved 37 students with prior ML coursework, who received a pipeline lecture and then analyzed a societal domain in groups.Students discussed predictive tasks, harms, possible pipeline causes, and potential remedies.Source: §2.2 · pp. 5–6
  • Students found it difficult to translate real-world problems into predictive tasks, including distinguishing the construct of interest from its operationalization.For example, transplant allocation framed as who should receive an organ differs from predicting who would receive one at a particular hospital.Source: remedies for addressing those sources · p. 6
  • Students often described harms without specifying affected groups or a comparison baseline, leaving the claimed harm underspecified.Examples included housing price discrepancies without identifying who was harmed or relative to whom, and fake-news spread without stating whom it negatively affected.Source: remedies for addressing those sources · p. 6
  • Students struggled to connect observed harms to plausible causes and to map their proposed causes to remedies.One group attributed disparities in child-welfare assessments to coarse racial categories, although the paper notes no plausible mechanism linking that feature limitation to the disparity; the finance group cited developer bias.Source: remedies for addressing those sources · p. 6

remedies for addressing those sources

  • The Child Welfare team proposed randomly selecting instances for inclusion in the training data.Source: remedies for addressing those sources · p. 6
  • The online exam proctoring group proposed greater transparency, such as telling students what behavior triggers a cheating flag, to reduce errors.Source: remedies for addressing those sources · p. 6
  • Teams often attributed harm broadly to biased data and proposed more comprehensive data collection, but these high-level remedies were unlikely to yield concrete action.Another common proposal was human oversight, without specifying the implications of leaving final decisions to human decision-makers.Source: remedies for addressing those sources · p. 6

3 A SURVEY OF FAIRNESS RESEARCH ALONG THE PIPELINE

The authors survey fairness research across ML pipeline stages, organizing studies by stage and by whether they identify, measure, mitigate, or document a problem. They also discuss viability assessment and gaps in published work on decisions about whether to build systems.

3.1 Survey Methods and Organization.

  • The survey reviews fairness-related work from 2018–2022 across selected ML, fairness, HCI, and social-computing venues, supplemented by Google Scholar searches.The search produced approximately 1,000 papers, of which about 300 met the criteria for studying an aspect of the pipeline.Source: surveyoftherecentliterature,whichweclas… · p. 7

surveyoftherecentliterature,whichweclassifieddependinguponwhichareaofthepipelineanalyzed.Wegatheredpapers

  • The authors classify relevant studies by pipeline stage and by whether they identify, measure, mitigate, or present a case study of a fairness problem.They provide a full categorization in an appendix table and use selected examples in the survey discussion.Source: surveyoftherecentliterature,whichweclas… · p. 7
  • The survey is not intended to be comprehensive; it highlights well-known work and contributions addressing understudied areas.The authors say the broader categorization is available in the appendix.Source: surveyoftherecentliterature,whichweclas… · p. 7

3.2.1 Definition and Decisions. Viability assessments refer to a series of early investigations into whether including

  • Viability assessment considers whether an ML component is preferable to the status quo and whether a net-beneficial system is feasible with available resources.Questions include the policy goal, the model's role and scope, and organizational constraints.Source: §3.2.1 · p. 7, literatureondocumenting,ordetailingtheb… · p. 7

literatureondocumenting,ordetailingthebiasthatcanariseoutof,ormitigatingbiasfromtheviabilityassessmentprocess.

  • Published case studies detailing how decisions to build or not build were made are scarce, despite discussion of when to refuse to build.The authors also note organizational factors such as limited company support for ethical concerns.Source: literatureondocumenting,ordetailingtheb… · p. 8
  • The authors describe risks of proceeding without careful viability assessment, including advancing projects without establishing that deployment offers a net benefit in context.The cited discussion also raises the possibility of refusing to build an ML system.Source: literatureondocumenting,ordetailingtheb… · p. 8

knowledge, existing proposals, while promising, have not yet been evaluated in practice.

Problem formulation choices determine how a real-world goal becomes a prediction task, while data collection begins with decisions about whom and how to sample. The section proposes assessing validity and testing candidate choices as ways to investigate these decisions’ fairness implications.

3.3.1 Definition and Decisions. Problem formulation [234] is the translation of a real-world problem to a prediction task:

  • Problem formulation translates a real-world problem into a prediction task, with choices about the target, inputs, and prediction universe.For example, the universe specifies which people receive predictions.Source: §3.3.1 · p. 8

do”[75]:e.g.astheauthorsnote,establishingvaliditymaybedifficultinapredictivepolicingmodelthatpurportstopredict

  • Social-science validity and reliability concepts have been used to interrogate problem-formulation choices, with validity testing attempting to establish whether a system does what it purports to do.Source: §3.3.2 · p. 8
  • Using health care costs as a proxy for health care need adds to racial disparities in care allocation.The cited work found a disparity in the correlation between health care costs and sickness for white versus Black patients.Source: do”[75]:e.g.astheauthorsnote,establishi… · pp. 8–9
  • The authors propose protocols assessing internal, external, content, and predictive validity for targets, attributes, and prediction universes.They suggest quantitative tests alongside existing checklists and protocols, including bias tests across candidate prediction tasks and tests of whether targets predict related outcomes.Source: do”[75]:e.g.astheauthorsnote,establishi… · p. 9

3.4.1 Definition and Decisions. Data collection involves collecting or compiling data to train the model. This involves

  • Data collection starts with choices about the population to sample and how to collect its data.The passage also raises what measurements will be used.Source: §3.4.1 · p. 9

device will be used? How will we link pre-made data?

The section surveys how collection and preprocessing choices can shape model behavior, while identifying gaps in methods that address these choices directly. It highlights limited practical evaluation of collection interventions and understudied preprocessing effects.

haviorfromtheperspectiveofaccuracyandfairness[6,89,175].Interestingly,theyfindthatincreasingdatasetsize,oreven

platformssuchasMechanicalTurk[134,205,260].Therearealsoafewinstructivepapersthatdocumenthowsamplingand

  • The literature contains fewer studies that trace biased datasets to data-collection processes and show how to mitigate those processes.The section points to HCI work on collection failure modes, including crowdsourced labeling.Source: haviorfromtheperspectiveofaccuracyandfa… · p. 9
  • Interview-centered studies can illuminate collection problems but may omit low-level technical sources and provide limited operational guidance.The cited examples include record-linkage issues that lead to non-IID data dropping.Source: platformssuchasMechanicalTurk[134,205,2… · p. 10

3.4.3 Measurement, Mitigation, and Tools. There are myriad papers introducing methods of mitigating bias in model

  • Datasheets for datasets offer a worksheet for documenting potential failure and bias modes, but their real-world effectiveness at preventing collection mishaps is not established.The authors report preliminary evaluation papers, but none evaluating effectiveness in a real-world model-building scenario.Source: §3.4.3 · p. 10

3.5.1 Definition and Decisions. Data preprocessing refers to steps taken to make data usable by the ML model–e.g.,

  • Preprocessing makes data usable for an ML model through steps such as missing-value handling, transformations, and feature engineering.It also includes deciding which constructed features to use for prediction.Source: §3.5.1 · p. 10

3.5.2 Case Studies and Problem Identification. There are myriad ways for biases to enter through data preprocess-

  • Imputation strategies can yield different group accuracy disparities even when overall model performance is similar.The cited discussion also notes limited work on the fairness effects of imputing missing data or dropping rows with missing data.Source: §3.5.2 · p. 10
  • Preprocessing choices can change fairness behavior through shifts in the training distribution, while the fairness effects of data encoding remain underexplored.One translation study found that longer words led to lower performance and that more granular representations could mitigate disparities.Source: §3.5.2 · p. 11
  • Tools for isolating and removing bias from preprocessing steps beyond feature inclusion are understudied, and feature-selection methods can incur accuracy drops.An automatic feature-selection framework selects features that improve accuracy without reducing a user-specified fairness notion, but it saw considerable accuracy drops in this setting.Source: §3.5.3 · p. 11

manner.Wenotethatevenamongfeature-relatedmeasurementandmitigationtools,thereislittleworksurroundinghowto

The survey finds few tools for mapping preprocessing choices to fairness effects and suggesting mitigations. It also frames statistical modeling decisions as potential sources of downstream bias that warrant identification.

manner.Wenotethatevenamongfeature-relatedmeasurementandmitigationtools,thereislittleworksurroundinghowto

  • The survey found no tools that help practitioners map preprocessing choices to bias prevention and mitigation, though general data-exploration tools may be adaptable.The authors were unaware of existing systems that account for bias-related desiderata.Source: manner.Wenotethatevenamongfeature-relat… · p. 11

3.6 Statistical Modeling

  • Statistical modeling decisions include model type, learning rule, loss function, regularization, and training hyperparameters.The text also names architecture, optimization procedure, and whether to use a single model or an ensemble.Source: §3.6 · pp. 11–12

3.6.2 Case Studies and Problem Identification.

  • The authors note that fairness papers commonly intervene on loss functions or prediction processes, but fewer identify bias sources in modeling decisions.They observe that each such decision can lead to downstream bias.Source: §3.6.2 · p. 12

skewedpredictionsforcertainpopulations[171],ormayover-orunder-emphasizeoutliersorminoritypopulations.Model

The section describes how modeling and validation choices relate to fairness, while noting gaps in tools for evaluating their effects. It also reports concerns about benchmark reuse and the lack of on-the-ground studies of testing and validation sources of unfairness.

skewedpredictionsforcertainpopulations[171],ormayover-orunder-emphasizeoutliersorminoritypopulations.Model

  • Model type and loss-function choices can affect fairness, and models may not display acceptable desired behavior when it is not specified in the loss.The cited work also reports fairness degradations in robust and differentially private models.Source: skewedpredictionsforcertainpopulations[… · p. 12

3.6.3 Measurement, Mitigation, and Tools. Most of the focus on fairness interventions in statistical modeling is centered

  • Fair Bayesian Optimization selects hyperparameters to optimize accuracy subject to a fairness constraint, and experiments found regularization parameters especially influential for fairness.The section also reports that higher regularization led to higher fairness performance in experiments.Source: §3.6.3 · p. 12
  • The authors found no tools measuring how statistical modeling choices affect fairness, and few mitigations target modeling stages beyond the loss function.They note evidence that enforcing fairness constraints can degrade fairness according to those same metrics.Source: §3.6.3 · p. 12

3.7.1 Definition and Decisions. Model testing and validation refers to the processes by which a model is determined

  • Testing and validation choices include evaluation metrics, datasets, held-out data size, and the number of trials or folds.The section also cites a concern that many bias metrics are statistically biased estimators of the quantities they represent.Source: §3.7.1 · pp. 12–13

actuallyleadtodecreased fairnessbehavioralongthoseverysamemetricsinthemodel[298].Webelievetheremaybeother

  • Repeated use of benchmark datasets can overfit models and produce results unrepresentative of real-world fairness applications [237, 258]; the authors found no on-the-ground studies of testing-stage unfairness sources.The section also cites concerns that some fairness-literature experimental results may be incorrect.Source: §3.7.1 · p. 12, actuallyleadtodecreased… · p. 13

dictions[22,120,256,304],howeverwenotethattheseframeworksdonottargetwhichpartofthepipelinemaybeaddingto

The surveyed work spans fairness testing, deployment, human–ML decision systems, and runtime monitoring, but leaves important questions about how pipeline choices affect fairness and what practitioners should do in response.

basicbiasmitigationtotestingandvalidationproblems:nottestingforfairnessduringvalidationatall.Severalrecentworks

  • The survey found no papers measuring how testing and validation choices influence downstream fairness, and recommends exploring less-used datasets beyond established benchmarks.It also notes that existing fairness datasets may be vulnerable to overfitting.Source: basicbiasmitigationtotestingandvalidati… · p. 13

3.8 Deployment and Monitoring

3.8.2 Case Studies and Problem Identification. There has been a stream of theoretical work from the algorithmic fairness

  • Work on human–ML systems includes theoretical deferral frameworks, but the authors call for testing whether these methods work in deployed systems and whether they introduce bias.Prior findings conflict: allowing call-screen workers reduced Black–white disparities in child-welfare screen-in rates, while other work found human assistance reduced accuracy and workers exhibited bias.Source: §3.8.2 · p. 13
  • Studies of human assistance report mixed outcomes: call-screen workers reduced Black–white disparities, while Mechanical Turk assistance decreased accuracy and workers also exhibited bias.These findings motivate further testing and comparison of human-in-the-loop systems on deployed systems.Source: §3.8.2 · p. 13

3.8.3 Measurement, Mitigation, and Tools. While a few papers provide mitigation techniques and tools for fairness

  • The authors identify addressing bias from human involvement in ML decision systems as an important area for future work.They also note that surveyed papers offered no techniques for monitoring and preventing distribution shifts.Source: §3.8.3 · p. 13
  • Runtime fairness tools can measure drift and monitor traditional metrics, but the surveyed tools do not resolve when drift should trigger a fairness constraint versus a warning that deployment is unsuitable.One cited approach cannot implement individual fairness over individual inputs.Source: §3.8.3 · p. 14

4 DISCUSSION AND A PROPOSAL FOR PIPELINE CARDS

The discussion proposes a research agenda and introduces Pipeline Cards as a practical documentation framework for examining fairness-relevant choices across the ML pipeline.

developing a research agenda towards operationalizing pipeline-aware fairness techniques, and introducing Pipeline

  • The agenda calls for documenting real-world pipeline choices, bridging on-the-ground accounts of bias with FairML mitigation work, and studying interactions among pipeline decisions.It also identifies neglected pipeline areas, including viability assessment.Source: developing a research agenda towards… · p. 14
  • The agenda also calls for measurement techniques that catch bias entry points and guidance for choosing among biased design options.These needs are presented as part of five insights toward operationalizing pipeline-aware fairness.Source: developing a research agenda towards… · p. 14
  • Pipeline Cards document design choices along the ML pipeline to encourage practitioner introspection and transparent reporting about how AI systems are made.The authors present them as an actionable tool for practitioners using a pipeline-aware approach to fairness.Source: developing a research agenda towards… · p. 14
  • Pipeline Cards are under active development, including testing with practitioners and students to determine how best to prompt identification of potential pipeline problems.Source: developing a research agenda towards… · p. 14

5 CLASSROOM STUDY

The classroom study asked students with machine-learning backgrounds to analyze societal-domain prediction tasks through a pipeline-centered lens. Its table summarizes the tasks students selected and how teams formulated them.

5 CLASSROOM STUDY

  • The study included 37 students who had completed at least one machine-learning course; because enrollment was elective, participants were likely more aware of ML harms and more motivated to address them than a random sample.About 80% described their ML familiarity as intermediate, about 14% as advanced, and about 5% as elementary.Source: §5 · pp. 27–28
  • Table 2 overviews the teams’ predictive tasks and how they defined them, including a finance team’s task of assessing a prediction target.Source: Table 2 · p. 28, Studydesign.Studentswereintroducedtothe… · p. 28

Studydesign.StudentswereintroducedtotheMLpipelinethroughanapproximately45-minuteslonglecture.Thelecture

  • Students received an approximately 45-minute lecture on pipeline stages, then discussed a societal-domain prediction task, its harms, possible pipeline sources, and potential remedies.Teams selected among domains including employment, education, medicine, housing, child welfare, criminal justice, public safety, e-commerce, social media, and transportation.Source: Studydesign.Studentswereintroducedtothe… · p. 28

6 RESEARCH AGENDA

The research agenda identifies practical gaps in pipeline-aware fairness, including limited knowledge of real-world pipeline choices, weak links between problem identification and technical mitigation, and sparse coverage of interactions and measurement. It calls for documenting pipelines, connecting literatures, testing methods in real systems, developing measurement tools, and guiding context-sensitive choices among imperfect alternatives.

6 RESEARCH AGENDA

  • The agenda calls for documenting real-world pipeline decisions, connecting literatures, studying interactions, measuring harms, exploring neglected areas, and guiding choices among biased alternatives.It also recommends centralizing pipeline information so researchers can study how choices affect fairness behavior.Source: §6 · p. 29, techniques,suchasFairML;(3)theneedtostu… · p. 29

techniques,suchasFairML;(3)theneedtostudyinteractioneffectsacrossdecisionsmadealongthepipeline;(4)theneedto

  • Real-world ML pipeline choices are not centrally cataloged, yet mapping them is necessary to study, measure, and mitigate bias entering through those choices.The agenda encourages exploration and documentation of pipelines across applications.Source: techniques,suchasFairML;(3)theneedtostu… · p. 29

pipeline decisions they may have been unaware of impact fairness behavior.

videlittledirectionforcreatingoperationalizablesolutionstotheseproblems,oroftenevensufficientlydetailedinformation

  • The agenda urges testing pipeline-based mitigation methods in real-world systems to learn where they succeed, fail, and fit into deployed systems.Such testing is intended to reveal performance under different circumstances and failure modes.Source: videlittledirectionforcreatingoperation… · p. 29

muchoftheliteraturefailstoprovideinsightintohowdifferentbiasesandmitigationtechniquesalongthepipelineinteract.

how more contextualized notions of fairness are impacted by pipeline decisions?

effectstoworkinpractice?Thereareafewpapersthatshowtheimpactsoftheintersectionbetweenmultiplesourcesofbias

of pipeline decisions beyond the most common metrics.

7 PIPELINE CARDS

Pipeline cards provide questions for examining fairness-relevant choices across the ML development pipeline. They prompt teams to assess the problem and organizational readiness, scrutinize data and modeling choices, and consider testing, deployment, and monitoring.

7 PIPELINE CARDS

  • Pipeline cards frame fairness review as interrogation of choices across the machine-learning development pipeline.The questions range from whether an ML system fits the goal to how it is tested, deployed, and monitored.Source: §7 · p. 31, §7.1.1 · p. 31, §7.7.1 · p. 33

7.1 Fit and Capacity

  • Assess whether ML fits the policy goal, whether suitable proxy data and alternatives exist, and whether the organization has capacity and buy-in.The questions include trade-offs among goals and the resources needed to build and maintain the system.Source: §7.1.1 · p. 31, §7.1.2 · p. 31

7.2 Task Definition

  • Scrutinize the prediction proxy, its validity across populations, input-feature quality, and whether the modeled population fits the policy goal.The prompts ask about ground truth, proxy correlations, and possible under-representation in the problem universe.Source: §7.2.1 · p. 31, §7.2.2 · p. 31, §7.2.3 · p. 32

7.3 Data Collection

  • Check whether collected or reused data covers the relevant population and whether sampling, data quality, or record linkage differs across groups.The questions include condition-based data availability and unequal accuracy when matching people across datasets.Source: §7.3.1 · p. 32, §7.3.3 · p. 32

7.4 Data Preprocessing

  • Review missing-data handling, transformations, and feature construction for effects or errors that may differ across demographic groups.Examples include language-dependent transformation errors and features whose meaning may vary across groups.Source: §7.4 · p. 32

7.5 Statistical Modeling

  • Examine model type, learning-rule biases, and whether the loss function and constraints reflect project goals.The prompts note that some learning rules may over- or under-emphasize outliers or minority populations [13].Source: §7.5 · pp. 32–33, minority populations [13]) · p. 33

7.6 Testing and Validation

  • Evaluate model-selection metrics, fairness considerations, metric robustness, deployment-representative test data, and the full decision environment.The questions include trials or cross-validation and testing the model alongside human components.Source: §7.6.1 · p. 33, §7.6.3 · p. 33

7.7 Deployment and Monitoring

  • Consider how predictions enter decisions, human discretion and downstream actions, then plan monitoring, updates, shutdown procedures, and avenues to challenge decisions.The cards also prompt teams to identify other pipeline decision points and ask how they may affect fairness behavior.Source: §7.7.1 · p. 33, §7.7.2 · p. 33

Related papers

Loading 2309.17337v1…