Source-linked AI summary
Predictive Inequity in Object Detection
Benjamin Wilson, Judy Hoffman, Jamie Morgenstern
TL;DR
The paper asks whether state-of-the-art pedestrian detectors perform equitably across skin-tone groups, an important question as machine learning systems increasingly influence autonomous driving and other decisions. It annotates BDD100K with Fitzpatrick categories, compares model performance, and examines scene factors and loss reweighting. The reported evidence shows consistently higher precision for lower Fitzpatrick types, while the authors note limits from dataset size and the scope of their inequity measure.
Problem
The paper asks whether object-detection systems have equitable predictive performance for pedestrians with different skin tones, amid broader evidence of demographic disparities in machine-learning systems.
Method
The authors annotate BDD100K pedestrians with Fitzpatrick skin types, compare detection performance across groups and models, analyze time of day and occlusion, and test supervised-loss reweighting.
Results
Standard models consistently show 4–10% gaps in precision metrics, with LS outperforming DS across a wide range of learning rates.
Takeaways & Limitations
Predictive inequity appears across standard pedestrian-detection systems, including on relatively easy examples, and simple loss-term reweighting can reduce some observed inequity.
Takeaways & Limitations
High-confidence LS–DS comparisons require much larger datasets or many more sufficiently large annotated pedestrians, and the inequity measure does not capture every aspect of model behavior.
Abstract
from arXiv · showhide
In this work, we investigate whether state-of-the-art object detection systems have equitable predictive performance on pedestrians with different skin tones. This work is motivated by many recent examples of ML and vision systems displaying higher error rates for certain demographic groups than others. We annotate an existing large scale dataset which contains pedestrians, BDD100K, with Fitzpatrick skin tones in ranges [1-3] or [4-6]. We then provide an in-depth comparative analysis of performance between these two skin tone groupings, finding that neither time of day nor occlusion explain this behavior, suggesting this disparity is not merely the result of pedestrians in the 4-6 range appearing in more difficult scenes for detection. We investigate to what extent time of day, occlusion, and reweighting the supervised loss during training affect this predictive bias.
1. Introduction
Machine learning systems increasingly influence everyday decisions, including autonomous driving, where pedestrian recognition helps determine evasive actions. This paper asks whether detection performance differs across Fitzpatrick skin-tone groups and investigates data imbalance and loss prioritization as possible sources.
- Machine learning methods increasingly shape online content, products, emails, resource allocation, surveillance, and autonomous vehicles.Autonomous vehicles are presented as a major private-sector application that may reduce transit costs and personal vehicle reliance.
- Pedestrian and road-sign recognition can help determine when an autonomous vehicle must brake or swerve to avoid fatalities.
- The paper reports uniformly poorer detection performance for pedestrians with Fitzpatrick skin types 4–6 across several state-of-the-art object-detection models.The authors frame this as a potential unequal distribution of future autonomous-vehicle errors across demographic groups.
- The BDD100K training set contains roughly 3.5 times as many lower-Fitzpatrick-scored pedestrian examples as higher-scored examples.The authors identify both larger-subset generalization and standard loss functions prioritizing the larger subset as possible disparity sources.
- The paper distinguishes sampling, measurement, and weighting issues that are often conflated under the shorthand “biased data.”Sampling concerns population representation, while measurement concerns whether collected features adequately measure the intended outcome.
- The analysis investigates whether time of day, occlusion, or loss prioritization for the larger population affects the observed predictive disparity.
2. Related Work
Prior work documents predictive disparities across advertising, policing, facial recognition, and gender prediction systems. This paper extends that measurement focus to driving-centric pedestrian detection rather than broader ethical decision-making questions.
- Advertising systems and predictive-policing systems have been reported to show differing outcomes or predictive performance across demographic groups.
- Facial-recognition systems have received substantial attention for differing predictive qualities across women and people of color.
- Commercial gender-prediction software has been reported to have much worse accuracy on women with Fitzpatrick skin types 4–6 than on other groups.
- This work measures predictive imbalance in driving-centric object detection systems as several locations allow autonomous vehicles on public roads.The paper focuses on whether research-grade pedestrian-detection components detect pedestrians with different skin tones similarly, not on ethical dilemmas involving unavoidable crashes.
- Faster R-CNN and Mask R-CNN are described as state-of-the-art region-proposal methods that often underpin high-performing pedestrian-detection models.
3. Preliminaries
The paper frames its analysis around pedestrian detection, Fitzpatrick skin typing, and predictive disparity between groups. It defines predictive inequity through differences in per-instance loss while acknowledging that this measure does not capture every fairness dimension.
- The preliminaries introduce pedestrian detection, Fitzpatrick skin typing, and predictive disparity as the main concepts used in the work.
- Pedestrian detection remains important in systems ranging from security applications to autonomous cars, with occlusion, clothing changes, and lighting variation as common challenges.
- Fitzpatrick categories 1–3 generally correspond to lighter skin tones than categories 4–6 and were introduced to measure predisposition to ultraviolet-light burning.
- Predictive inequity is defined for a fixed partition of the person class as the difference in loss incurred by a model on LS versus DS members.
- The measure captures average additional loss for a random LS member versus a DS member, but does not perfectly capture all aspects of model behavior for all populations.
4. Evaluating Predictive Inequity
Because BDD100K lacks explicit skin-color labels, the authors augment it with Fitzpatrick annotations to quantify predictive disparities across skin-tone groups. They then evaluate standard recognition models and analyze disparity sources before testing a simple remedy.
- BDD100K’s pedestrian bounding boxes and class labels are augmented with Fitzpatrick skin types for predictive-inequity evaluation.The original dataset lacks explicit labeling of people by skin color.
- The evaluation quantifies disparity in predictive performance across groups of people with varying skin tones.
- The paper analyzes predictive inequity on the annotated benchmark and investigates its sources before proposing a simple remedy.The stated remedy is introduced later as loss-function reweighting.
4.1. Benchmarking Predictive Disparity for Pedestrian Detection
The paper augments BDD100K with Fitzpatrick skin-tone annotations to partition pedestrians for measuring predictive disparity. It uses filtered, consensus-based human labels because small or difficult crops produced inconsistent annotations.
- Annotation setup: BDD100K lacks demographic labels for pedestrians, so the study augments person bounding boxes with Fitzpatrick skin-tone categories.The annotation supports partitioning the dataset into LS and DS groups for predictive-disparity measurement.
- Annotation setup: Each cropped pedestrian was assigned LS, DS, unknown, or not a person, accommodating ambiguity from lighting, size, occlusion, and mislabeled boxes.The crops came from bounding boxes labeled as the person class.
- Filtering and reliability: Small pedestrian crops produced substantial within-annotator and between-annotator disagreement, motivating analysis on a filtered set with bounding-box area at least 10,000 pixels.The validation set above the cutoff was hand-labeled, while Turker tasks were created for 4,979 training pedestrians.
- Filtering and reliability: Three Turkers labeled each instance, and labels were retained only when at least two agreed; otherwise, the image was discarded for predictive-inequity evaluation.The response categories and consensus procedure are summarized in Figure 3.
- Dataset composition: The dataset contains more than three times as many LS individuals as DS individuals, motivating supervised-loss reweighting to assess representation-driven disparity.BDD100K was selected because its large driving-scene collection supports experimentation with real-time vision models.
4.2. Evaluating Predictive Inequity of Standard Object Detection Systems
The evaluation finds higher detection performance for LS than DS pedestrians across training sources, models, architectures, and learning settings. The disparity is especially pronounced under strict localization, although the available annotated evaluation sample limits high-confidence comparisons.
- Evaluation setup: Average Precision evaluates pedestrian detection on BDD100K validation images, using IoU thresholds to determine true positives.AP50 uses a 0.5 threshold, AP75 uses 0.75, and overall AP averages thresholds from 0.5 to 0.95.
- Statistical validity: The observed disparity persists across a wide range of learning rates, training settings, models, and training-process stages, but the annotated holdout is small.The authors report n_LS = 387 and n_DS = 100; detecting a true 0.05 AP gap with 0.95 probability would require n_LS = 12,000 and n_DS = 4,000.
- Training data importance: MS COCO and BDD100K training data show different overall trade-offs: MS COCO performs better on AP and AP50, while BDD100K performs better on AP75.Both training sources are therefore relevant for assessing person detection on the BDD100K validation set.
- Skin-tone comparison: Across both training sources, LS pedestrians achieve higher average precision than DS pedestrians.The comparison uses larger pedestrians in the BDD100K validation set labeled by Fitzpatrick skin-tone grouping and averages results over repeated trainings.
- Architecture and model selection: Across all tested models and backbones, performance on LS exceeds performance on DS, indicating the disparity is not specific to one architecture.The evaluation covers Faster R-CNN and Mask R-CNN models with multiple architecture backbones trained on MS COCO.
- Architecture and model selection: 11.2 percentage points separate average AP75 for LS and DS, falling from 67.1% to 55.9%; the AP50 gap is much smaller.The strongest disparity appears under the stricter localization metric, AP75.
4.3. Analyzing Sources of Predictive inequity
The paper examines whether occlusion and time of day explain the predictive-performance gap between LS and DS pedestrians. Removing occluded examples leaves the gap, while daytime results reproduce it and nighttime results are inconclusive because of the small DS sample.
- Potential sources: The analysis considers occlusion, time of day, and loss prioritization as potential sources of predictive inequality.Example detections in Figure 5 motivate these three factors for further analysis.
- Occlusion as a possible source of predictive inequity: AP, AP50, and AP75 improve for both LS and DS after removing occluded pedestrians, but AP and AP75 gaps remain.The authors conclude that co-occurrence with occluded people does not explain the LS–DS discrepancy.
- Time of day as a possible source of predictive inequity: Daytime performance is higher for LS than DS across AP, AP50, and AP75, with gaps similar to the full validation set.This preserves the observed disparity under daytime-only evaluation.
- Time of day as a possible source of predictive inequity: Nighttime DS performance exceeds LS performance, but only 15 of 144 sufficiently large nighttime pedestrians are DS.The authors caution that the nighttime comparison may reflect the small sample size.
5. Reducing Predictive Inequity
The paper tests whether unequal representation makes standard training prioritize LS loss over DS loss. Increasing DS loss weights reduces the total AP gap, while the AP75 gap remains about 4–6%, indicating partial mitigation without sacrificing LS performance.
- Loss weighting: A training subset 3.5 times larger can have up to 3.5 times as much impact on a uniformly weighted loss.The paper distinguishes this design effect from an information-theoretic shortage of data.
- Loss weighting: The reweighting study varies class weights in Faster R-CNN’s detection and classification loss to test whether LS receives excessive training emphasis.The generic loss takes image crops and true labels as inputs, with separate LS and DS instance sets.
- Reweighting results: For larger αDS values, the total AP gap decreases, while the LS-to-DS AP75 gap remains 4 to 6%.With αDS = 3, DS AP75 approaches the AP75 of unweighted LS, and LS performance is better under this weighting.
- Reweighting results: Careful reweighting may improve DS performance without sacrificing LS performance, and may even improve LS performance as a byproduct.Further analysis is needed to determine how reweighting affects stricter criteria within total AP.
6. Conclusion and Discussion
The paper introduces predictive inequity for pedestrian detection across Fitzpatrick skin-tone groups and finds higher precision for lower Fitzpatrick skin types. The disparity persists among large pedestrians and grows after removing occluded pedestrians, while loss reweighting partially mitigates it.
- Conclusion and Discussion: Standard object-detection models show higher precision on lower Fitzpatrick skin types than on higher skin types.The paper frames this disparity as predictive inequity in pedestrian detection.
- Conclusion and Discussion: The disparity appears on large pedestrians and grows when occluded pedestrians are removed.Thus, it is observed even on subsets described as relatively easy for object detectors.
- Conclusion and Discussion: Reweighting loss terms during learning can partially mitigate the disparity.This is presented as a simple learning-stage change rather than a complete resolution.
- Conclusion and Discussion: The study presents predictive inequity as a problem relevant to deploying driving-centric pedestrian-detection systems.Its scope is the comparative ability of research-grade detection components to detect pedestrians with different skin tones.
7. Appendix
The appendix describes Faster R-CNN’s proposal, classification, regression, and reweighting components, along with experimental settings and an additional MS COCO evaluation. It defines the losses and box parameterization used in the study.
- Faster R-CNN: Faster R-CNN first proposes likely object regions with an RPN, then classifies those regions and regresses bounding boxes.The appendix focuses on the separate detection network for its loss analysis.
- Box parameterization: Predicted and ground-truth box parameters are defined relative to positive anchors using center coordinates, widths, and heights.The appendix identifies the corresponding predicted, ground-truth, and anchor box quantities.
- Loss function: The detection loss combines classification and regularized ℓ1 regression terms for each example.The regression term compares parameterized predicted and ground-truth boxes.
- Reweighting implementation: The augmented loss assigns weights by attribute membership across LS, DS, non-person, and indeterminate-skin-color instances.A four-dimensional weight vector reparameterizes the objective, with each instance selecting its corresponding attribute index.
- Experimental settings: BDD100K experiments use PyTorch Faster R-CNN on one NVIDIA V100 for 3,350 iterations, with modified learning rate, batch size, and step schedule.The learning rate is 0.01 and the mini-batch size is 8 images.