Source-linked AI summary
Bias Behind the Wheel: Fairness Testing of Autonomous Driving Systems
Xinyue Li, Zhenpeng Chen, Jie M. Zhang, Federica Sarro, Ying Zhang, Xuanzhe Liu
TL;DR
Fairness in autonomous-driving pedestrian detection remains under-explored, despite its safety relevance. This paper tests eight deep-learning detectors across demographic groups and driving scenarios, finding especially large age disparities and nighttime gender bias, while some conditions improve fairness alongside performance.
Problem
Fairness testing of pedestrian detection has limited coverage across detectors, datasets, sensitive attributes, and environmental conditions, despite the potential for unequal treatment and harm.
Source: §1 · p. 2, §2.2 · p. 4Method
The study tests eight state-of-the-art deep-learning pedestrian detectors across four real-world datasets labeled for gender, age, and skin tone, and examines driving scenarios.
Source: §1 · p. 2Results
Children’s undetected proportions exceed adults’ by an average of 20.14%; nighttime tests show gender bias, while some conditions improve fairness and detection performance.
Source: §1 · p. 2Takeaways & Limitations
The findings show that pedestrian detectors can achieve enhanced fairness and detection performance under specific driving scenarios, challenging a commonly accepted fairness-performance trade-off.
Source: §1 · p. 2Annotating demographic attributes in foggy conditions remains challenging because weather can obscure skin tone and blur pedestrian features.
Source: §5.4 · p. 20
Abstract
from arXiv · showhide
This paper conducts fairness testing of automated pedestrian detection, a crucial but under-explored issue in autonomous driving systems. We evaluate eight state-of-the-art deep learning-based pedestrian detectors across demographic groups on large-scale real-world datasets. To enable thorough fairness testing, we provide extensive annotations for the datasets, resulting in 8,311 images with 16,070 gender labels, 20,115 age labels, and 3,513 skin tone labels. Our findings reveal significant fairness issues, particularly related to age. The proportion of undetected children is 20.14% higher compared to adults. Furthermore, we explore how various driving scenarios affect the fairness of pedestrian detectors. We find that pedestrian detectors demonstrate significant gender biases during night time, potentially exacerbating the prevalent societal issue of female safety concerns during nighttime out. Moreover, we observe that pedestrian detectors can demonstrate both enhanced fairness and superior performance under specific driving conditions, which challenges the fairness-performance trade-off theory widely acknowledged in the fairness literature. We publicly release the code, data, and results to support future research on fairness in autonomous driving.
1 INTRODUCTION
The paper addresses limited fairness testing of autonomous driving systems by evaluating state-of-the-art pedestrian detectors across demographic groups and driving scenarios. It finds substantial age disparities and nighttime gender bias, while some conditions can improve fairness alongside detection performance.
1 INTRODUCTION
- Fairness testing of autonomous driving systems remains under investigation, despite fairness being treated as a non-functional software property.The introduction connects detector disparities to discriminatory outcomes and possible harm to marginalized groups.Source: §1 · p. 2
- The study evaluates eight state-of-the-art deep-learning pedestrian detectors, manually labeling four real-world datasets with gender, age, and skin tone and examining brightness, contrast, and weather conditions.The annotated collection comprises 8,311 images, 16,070 gender labels, 20,115 age labels, and 3,513 skin tone labels.Source: §1 · p. 2
- Children’s undetected proportions exceed adults’ by an average of 20.14%; nighttime results also show more females than males going undetected, while brighter conditions can improve fairness and detection performance.Overall undetected-proportion gaps are 1.19% for males versus females and 0.44% for dark-skin versus light-skin groups.Source: §1 · p. 2
- The paper publicly releases its data, demographic labels, and code to facilitate future fairness research on autonomous driving systems.Source: §1 · p. 2
2 BACKGROUND AND RELATED WORK
The background reviews software fairness and autonomous-driving testing, then identifies pedestrian-detection fairness—especially across varied detectors, datasets, attributes, and environments—as an under-explored area. The study responds with a broad empirical evaluation across those dimensions.
2.1 Software Fairness
- Group fairness concerns equal treatment across groups; sensitive attributes partition populations, and age can distinguish privileged adults from unprivileged children in pedestrian detection.Source: §2.1 · p. 3
- Existing fairness-testing research largely centers on tabular data, while this study examines gender, age, and skin tone in pedestrian detection.Source: §2.1 · p. 3
2.2 Autonomous Driving Testing
- Autonomous-driving testing has addressed robustness and correctness, but fairness in pedestrian detection remains under-explored; prior studies varied in detector type, dataset scale, or attributes examined.Source: §2.2 · p. 4
- Prior work had not examined how brightness, contrast, and weather affect fairness, alongside limited variety in detectors, datasets, and sensitive attributes.Source: §2.2 · p. 4
- The study addresses these gaps using eight deep-learning detectors, four diverse datasets, three sensitive attributes, and scenarios varying in brightness, contrast, and weather.Source: §2.2 · p. 4
3 EXPERIMENTAL DESIGN
This section introduces the study’s research questions and experimental design.
3.1 Research Questions
The research questions ask whether pedestrian detectors show demographic disparities and how those disparities vary across driving scenarios. Figure 1 illustrates the experimental settings used to address them.
3.1 Research Questions
- RQ1 asks how widely studied pedestrian detectors differ in performance across gender, age, and skin-tone groups.Source: §3.1 · p. 4
- RQ2 examines fairness under different brightness, contrast, and weather conditions.Source: §3.1 · p. 5
- Figure 1 illustrates the experimental settings for answering the research questions.The section introduces the detectors, datasets, evaluation metric, statistical analysis, and experimental details.Source: §3.1 · p. 5
3.2 Pedestrian Detectors
The study evaluates eight widely studied deep-learning pedestrian detectors, spanning general object detectors and pedestrian-specific methods. The selected models include both one- and two-stage general detectors and four pedestrian-specific detectors.
3.2 Pedestrian Detectors
- The eight evaluated models comprise four general object detectors and four pedestrian-specific detectors.The general detectors are YOLOX, RetinaNet, Faster RCNN, and Cascade RCNN; the pedestrian-specific detectors are ALFNet, CSP, MGAN, and PRNet.Source: §3.2 · p. 5
- General detectors include both one-stage and two-stage designs, which trade detection speed against accuracy.YOLOX and RetinaNet are one-stage; Faster RCNN and Cascade RCNN are two-stage.Source: §3.2 · p. 5
- The pedestrian-specific models use varied mechanisms, including progressive refinement, anchor-free localization, and visible-area guidance for occlusion.ALFNet refines anchors, CSP locates center points, and MGAN and PRNet target occluded-pedestrian detection.Source: §3.2 · p. 6
3.3 Benchmark Datasets
The evaluation uses four real-world driving datasets and adds demographic and scenario labels to support analysis across people and conditions. The labeling procedure uses two annotators, with arbitration for disagreements.
3.3.1 Dataset Selection.
- Four real-world datasets provide street-level images with varied pedestrian poses, sizes, and occlusion scenarios.The selected benchmarks are CityPersons, EuroCity-Day, EuroCity-Night, and BDD100k; BDD100k contains greater skin-tone diversity than the European datasets described.Source: §3.3.1 · p. 6
3.3.2 Sensitive Attribute Labeling.
- Gender and age are manually labeled across datasets, while skin-tone analysis is limited to BDD100k, where pedestrian skin tones are more diverse.Two annotators independently assign binary male/female and child/adult labels; disagreements go to an arbitrator, and agreement is substantial for EuroCity-Day gender labeling and almost perfect for all other labeling tasks.Source: §3.3.2 · pp. 6–7
3.3.3 Scenario Labeling.
- Scenario labeling covers 5,933 of 8,311 images, and its two annotators achieve Cohen’s κ = 0.813.The paper reports this as almost-perfect agreement; fog and snow are not analyzed because those conditions are rare in the real-world datasets and have limited samples.Source: §3.3.3 · pp. 7–8
- Scenario labels cover day versus night, three RMS-contrast levels, and rainy versus non-rainy conditions.Contrast is measured on grayscale images and divided into three equal-range classes; images with indeterminate weather are left unlabeled.Source: §3.3.3 · p. 8
3.4 Evaluation Metric
The study measures group fairness using SPD and EOD, which coincide for pedestrian detection, and reports EOD. It uses miss rate to quantify undetected pedestrians and excludes AOD because the selected pedestrian subset complicates false-positive measurement.
3.4 Evaluation Metric
- Miss Rate (MR) is the proportion of pedestrians left undetected, with detection determined by box overlap exceeding 50% IoU.True positives are successfully detected ground-truth boxes and false negatives are undetected ground-truth boxes.Source: §3.4 · p. 9
- AOD is not used because focusing on a reasonable subset makes precise group-level false-positive counts difficult.The negative class may include instances that are actually pedestrians.Source: §3.4 · pp. 9–10
- SPD and EOD measure group differences in favorable detection outcomes and, in this setting, yield identical values, so the paper reports EOD.SPD compares favorable-outcome probabilities across groups; EOD compares true-positive rates for pedestrians.Source: §3.4 · pp. 9–10
3.5 Statistical Analysis
The study uses a two-proportion z-test to assess whether differences in group miss rates are statistically significant.
3.5 Statistical Analysis
- A two-proportion z-test treats group miss rates as different only when the p-value is below 0.05.For male and female miss rates, the null hypothesis is equality; a p-value below 0.05 leads to rejecting it.Source: §3.5 · p. 10
3.6 Experimental Details
The experiments use pre-trained pedestrian detectors, repeat runs ten times, and report averages. Implementation frameworks and hardware vary by detector.
Detector models
- General object detectors use the highest-accuracy pre-trained models from MMDetection’s model zoo, while pedestrian-specific detectors use models from their public repositories.The repositories are cited as, and the model zoo as.Source: §3.6 · p. 10
Repeated runs
- Each experiment is repeated ten times, and final results are the average across those iterations.Source: §3.6 · p. 10
Hardware and software
- The experiments use one NVIDIA GeForce RTX 2080 Ti GPU, with software frameworks and versions varying across detector groups.YOLOX, RetinaNet, Faster RCNN, and Cascade RCNN use PyTorch 1.8.1; CSP and MGAN use PyTorch 1.10.0; ALFNet and PRNet use Keras and TensorFlow.Source: §3.6 · p. 10
4 RESULTS
The results show substantial age disparities overall and fairness differences across driving conditions. Nighttime gender bias, contrast-level fairness, and rainy-weather effects vary across the tested detectors.
4.1 RQ1: Overall Fairness
- Children have a 20.14% higher miss rate than adults, while average miss rates are comparable across gender and skin tone.Age differences are significant in 30 of 32 dataset-detector combinations. Children and undetected pedestrians also tend to have smaller bounding boxes.Source: Table 7 · p. 11, §4.1 · p. 12, Figure 3 · p. 12
4.2 RQ2: Fairness in Different Scenarios
- RQ2 evaluates pedestrian-detector fairness across different real-world autonomous-driving scenarios.Source: §4.2 · p. 12
4.2.1 RQ2.1: Different brightness conditions.
- At night, six of eight detectors show significant bias against females, while age bias also increases from day to night.The average child-adult miss-rate difference rises from -22.45% during the day to -25.48% at night; skin-tone bias remains statistically insignificant in both conditions.Source: §4.2.1 · pp. 13–14, Table 8 · p. 14
4.2.2 RQ2.2: Different contrast levels.
- Level 3 yields the fairest results across gender, age, and skin tone, although overall detection performance has no clear pattern with contrast.Level 2 produces the most biased results on average.Source: §4.2.2 · pp. 15–16, Table 9 · p. 15
4.2.3 RQ2.3: Different weather conditions.
- Rain increases miss rates across demographic groups, while three of four pedestrian-specific detectors no longer show significant age bias; the fairness gain is marginal.Source: §4.2.3 · p. 16
5 DISCUSSION
The discussion finds that driving conditions can coincide with both improved pedestrian-detection fairness and performance, while age-bias findings remain significant under tested changes to the adult-data distribution. It outlines research and policy implications alongside limitations in data, evaluation choices, and annotations.
5.1 Fairness-performance Trade-off
- Daytime detection improves while absolute EOD falls across age, gender, and skin tone; highest contrast also improves fairness and sometimes performance, while rain has little fairness impact.These findings offer a counterpoint to the fairness-performance trade-off; brightness and contrast adjustments may improve both outcomes.Source: §5.1 · pp. 16–17, Table 10 · p. 17
5.2 Impact of Data Imbalance
- Age bias remains significant across all tested adult-data proportions (80%, 60%, 40%, and 20%), despite children being underrepresented.The analysis varies adult sample sizes while keeping the number of child samples constant.Source: §5.2 · pp. 17–18, Table 11 · p. 18
5.3 Implications
- Researchers can explore brightness and contrast editing, adaptive and multi-objective fairness methods, while accounting for detectors’ difficulty with small pedestrians and broader demographic annotation.The discussion also proposes image augmentation and brightness- or contrast-aware training approaches.Source: §5.3.1 · pp. 18–19
5.3.2 Implication for developers.
- Policy makers should consider regulations and protections for children and females, whom current models are more likely to overlook.Source: §5.3.3 · p. 19
5.4 Threats to Validity
- Validity is bounded by possible manual-label subjectivity, the selection of eight representative detectors, and four real-world datasets; demographic annotation under foggy conditions remains difficult.Two annotators and an arbitrator resolve labeling conflicts, and the authors report high inter-rater agreement.Source: §5.4 · pp. 19–20
- The analysis uses EOD, which yields observations equivalent to SPD, rather than AOD; evaluated attributes are limited to gender, adult-or-child age, and skin tone.These attributes were selected because they are more readily observable in vehicle-camera images.Source: §5.4 · p. 20
6 CONCLUSION
The paper evaluates fairness across eight state-of-the-art pedestrian detectors and four datasets, finding age and nighttime gender biases but balanced overall detection by skin tone. It releases annotated real-world datasets to support future fairness research.
- The study tests eight state-of-the-art pedestrian detectors across four widely studied datasets for fairness by gender, age, and skin tone.Source: §6 · p. 20
- Children show particularly significant detection bias, and detectors exhibit gender bias at night, while overall detection is balanced between light- and dark-skin individuals.Source: §6 · p. 20
- The authors release real-world pedestrian-detection datasets labeled for gender, age, and skin tone to support future fairness research.Source: §6 · p. 20