Source-linked AI summary
LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark
Irem Yoldas, Martim Brandão, Jie Zhang, Odinaldo Rodrigues
TL;DR
The paper asks whether general-purpose “common sense” models used in AVs inherit human biases in pedestrian yielding, a question relevant to fairness and public trust. It proposes AEBE and SC audits with an open benchmark, finding statistically significant yielding differences across seven demographic dimensions in both LLMs and VLMs. The results support including bias in AV model evaluation and safeguarding, while the SC test remains less robust than AEBE.
Problem
Evidence is limited on whether LLMs and VLMs used for AV decision making inherit human bias in pedestrian-yielding scenarios, despite concerns about biased model behavior.
Method
The paper proposes AEBE tests for LLMs and SC tests for VLMs, applying them in an open benchmark of pedestrian-yielding scenarios.
Results
Both LLMs and VLMs show statistically significant yielding biases associated with pedestrian gender, ethnicity, religion, disability, age, skin tone, and socio-economic status.
Takeaways & Limitations
AV developers should treat bias as a model-evaluation and safeguarding factor to help preserve public trust and safety on public roads.
Takeaways & Limitations
The SC test is less robust than AEBE and may be influenced by scene context and model reasoning capabilities.
Abstract
from arXiv · showhide
Public trust in Autonomous Vehicles (AVs) may depend not only on technical success but also on the fairness of their decision making. While a recent trend in AV research involves using general purpose "common sense" models to guide AV decision making, the degree to which these inherit human biases in driving is still understudied. Given that psychology studies have shown human driver biases exist, such as lower pedestrian-yielding rates to Black pedestrians in the US, we argue that analyses of model bias should also be part of AV evaluation. Concretely, in this paper we propose two new bias testing methodologies for Large Language Models (LLMs) and Visual-Language Models (VLMs)-"All Else Being Equal" tests and "Self-Consistency" tests-in order to assess bias in pedestrian-yielding decisions. Our findings show that both LLMs and VLMs make yielding decisions which are influenced by pedestrian gender, ethnicity, religion, disability, age, skin tone and socio-economic status. While the type and degree of bias is different from model to model, we highlight common patterns-and raise questions about the "common sense" model paradigm, particularly the need to either revise the paradigm or address issues of downstream bias.
I. INTRODUCTION
The paper examines whether “common sense” LLMs and VLMs used in AVs inherit human biases in pedestrian-yielding decisions. It proposes bias audits and an open benchmark after identifying significant demographic differences in model yielding.
- “Common sense” models may provide human-like decisions across varied driving conditions but can also contain biased and discriminatory behavior.This motivates testing such models before deployment in AVs.
- The paper addresses limited evidence on whether LLMs and VLMs inherit human driver bias in pedestrian-yielding scenarios.The authors identify a lack of datasets isolating pedestrian characteristics from other driving factors.
- The authors propose two audit methodologies and an open benchmark for evaluating yielding bias in LLMs and VLMs.One method varies pedestrian characteristics while holding other scenario details constant; the other measures associations between model-estimated characteristics and driving decisions.
- Current evaluated models show statistically significant yielding bias across gender, ethnicity, religion, disability, age, skin tone, and socio-economic status.Disability is reported as the most consistent bias across the evaluated models.
A. Human Driver Bias in Yielding
Prior psychology and AV research documents demographic and behavioral influences on human drivers’ pedestrian-yielding decisions. These findings motivate testing whether LLMs and VLMs reproduce implicit human driver bias.
- Pedestrian condition, gender, age, eye contact, and smiling behavior affect human drivers’ yielding behavior.Eye contact positively affects yielding among male drivers in the cited studies.
- Black pedestrians experience approximately 30% higher waiting time than White pedestrians, according to cited studies of human driver bias.The paper presents this as potentially subconscious bias in driving behavior.
- Driver age, education level, and gender produce stronger variations in driving behavior than geographical area type in the cited research.
- Prior evaluations have also examined gender, age, racial, and adversarial-prompt biases in VLMs and LLMs.
- Existing AV bias studies primarily examine unequal pedestrian-detection performance caused by dataset imbalance and underrepresented classes.The paper distinguishes these detection-focused studies from its evaluation of yielding decisions.
III. DRIVING-BIAS AUDITING METHODOLOGY
The paper introduces two complementary audits for AV language models: AEBE for LLMs and SC for VLMs. Both assess whether pedestrian-yielding decisions differ across demographic groups under comparable conditions.
- The All Else Being Equal test compares driving decisions across scenarios identical except for pedestrians’ personal characteristics.It is suited to text-based datasets that can be synthetically modified to vary demographic descriptions.
- The Self-Consistency test measures associations between VLM driving decisions and estimated pedestrian characteristics across scenarios the model judges to have similar conditions.The audited VLM estimates the driving decision, pedestrian characteristics, and scenario condition.
- AEBE is used for LLMs, while SC is used for VLMs to assess yielding bias characterized by lower yielding rates for specific demographic groups.
- The benchmark draws real driving scenarios from nuImages, JAAD, and PIE, retaining images with a single ground-truth pedestrian annotation.The single-pedestrian selection is intended to avoid interference between characteristics of different pedestrians.
B. Generating AEBE Tests for LLMs
The LLM benchmark constructs controlled text scenarios from real pedestrian-crossing images, then varies demographic descriptions while preserving the underlying scenario. The resulting AEBE conditions cover multiple individual and intersectional characteristics.
- 3,346 images were manually selected from 23,812 common scenarios because pedestrians were close to the car and visibly intended to cross.Crossing intention was verified from whether pedestrians began crossing in the following frame.
- Qwen-3-VL generated text descriptions for the selected driving scenarios before the descriptions were manually reviewed.
- Figure 1 presents AEBE for LLMs and SC for VLMs as audits of yielding-decision differences across demographic groups.
- Manual review removed demographic mentions and scenarios that implicitly described the need for the driver to stop before generating AEBE variants.The review replaced gender-specific words with “the pedestrian” when appropriate.
- The AEBE benchmark creates an original condition and variants that add gender, ethnicity, religion, disability, age, skin tone, socio-economic status, or demographic combinations.The combinations include gender-disability, gender-ethnicity, and gender-age conditions.
- The demographic categories and labels were selected from cited census, reference, and prior-work sources, including intentionally potentially offensive wording.
C. Generating SC Tests for VLMs
The Self-Consistency test identifies comparable VLM scenarios and compares stopping decisions across inferred pedestrian demographic characteristics.
- SC tests use 23,812 common scenarios to evaluate VLM pedestrian-yielding decisions.
- A scenario is treated as comparable when the VLM predicts that the pedestrian intends to cross and that stopping is required and sufficient for safe crossing.
- The test independently prompts VLMs for stopping decisions and pedestrian demographic characteristics, then compares decisions by demographic characteristic.
A. Experimental Setup
The experiments evaluate multiple LLMs and VLMs, using mostly locally run 7–13B models alongside GPT-4o, with deterministic zero-shot decisions based on log-odds.
- The evaluation includes four LLMs—Qwen-3, Llama-3.1, Mistral and GPT-4o—and four VLMs—Qwen-2.5-VL, Qwen-3-VL, LLaVA-NeXT and SPHINX.
- Locally run models use 7–13B versions because of resource constraints and challenges running larger models in AVs; GPT-4o was accessed through APIs.
- All models are evaluated zero-shot, with the highest-probability answer from log-odds outputs used as the deterministic decision.
- Implementations use HuggingFace Transformers for the listed local models and the official OpenAI API for GPT-4o.
B. LLM evaluation results
LLM yielding decisions vary across demographic conditions and models, with especially low yielding for paralyzed pedestrians and statistically significant differences across several characteristics and combinations.
- Qwen-3 has a median stop rate of approximately 70%, while other models’ medians correspond to stopping almost 100% of the time.
- Qwen-3 shows low yield rates for gender-disability, gender-ethnicity and gender-age combinations, indicating compounding effects across dimensions.
- Qwen-3 yields least to female pedestrians at 61% and most when no gender is indicated at 92%, with almost all pairwise comparisons statistically significant.
- Qwen-3 yields most to unclear, able-bodied, normal and nondisabled pedestrians, and least to paralyzed, nonspeaking and ADHD pedestrians.
- Mistral yields at 98% for female and 97% for male pedestrians, while GPT-4o yields significantly less often to Muslim, Christian and Sikh than Jewish pedestrians.
C. VLM evaluation results
VLM self-consistency tests produced low overall yielding rates and substantial variation across models and pedestrian demographics. Detailed results show statistically significant differences associated with skin tone, gender, and ethnicity.
- Self-Consistency results: VLM median yield rates ranged from 1% to 30%, with Qwen-3-VL highest and LLaVA-NeXT lowest.GPT-4o was excluded after refusing demographic questions.
- Model-specific biases: Qwen-3-VL yielded more often to female than male pedestrians, at 35% versus 28%.The difference was statistically significant.
- Model-specific biases: LLaVA-NeXT yielded to 50% of “Indian” pedestrians but below 4% for most other ethnicity predictions.The difference was statistically significant.
- Model-specific biases: SPHINX yielded most often to “Indian” pedestrians at 21%, while “Black” and “Southeast asian” pedestrians each received 20%.The latter two groups were significantly higher than East asian, Hispanic, and White pedestrians at 10%.
V. IMPLICATIONS
The results indicate that yielding decisions are associated with pedestrian characteristics across models, creating implications for fairness, safety, accountability, and public trust. The authors therefore call for bias testing as part of AV evaluation and safeguarding.
- Implications: Yielding decisions were statistically associated with pedestrian gender, ethnicity, religion, disability, age, skin tone, and socio-economic status.Models differed in the types and degrees of bias they exhibited.
- Safety and trust: Lower yield rates can lengthen pedestrian waiting times and may increase impatience and crossing risk.The paper connects these safety concerns to previously reported traditional-driving findings.
- Safety and trust: Bias inequalities could undermine public confidence in AVs by reinforcing perceptions of systemic discrimination and inadequate safeguards.The paper frames this concern in the context of trust that has not yet been fully gained.
- Evaluation and safeguards: The authors call for AEBE, SC, and similar bias tests to increase fairness, safety, and accountability in AV development.They also propose extending the benchmarks to other AV decision-making tasks.
VI. CONCLUSIONS
The paper concludes that its two audit methodologies reveal statistically significant demographic associations in LLM and VLM pedestrian-yielding decisions, with patterns varying across models. It recommends treating bias as an AV evaluation and safeguarding factor while noting important testing limitations.
- Conclusions: The proposed AEBE and SC methodologies benchmark yielding bias in LLMs and VLMs for AV driving.They were applied specifically to pedestrian-yielding scenarios.
- Conclusions: Statistically significant yielding differences were associated with gender, ethnicity, religion, disability, age, skin tone, and socio-economic status.Disability showed consistently lower yielding for some groups across most models.
- Conclusions: Models often predicted yielding more frequently when demographic information was absent or identity predictions were unclear.The paper identifies filtering and activation steering as possible mitigation directions.
- Implications: The authors argue that AV developers should consider bias in model evaluation and safeguarding to preserve public trust and road safety.This conclusion follows the reported implications for safety and confidence in AV technology.
- Limitations: The SC test is less robust than AEBE and may be influenced by scene context and model reasoning capabilities.The study focused on assessment rather than mitigation, leaving root causes and mitigation effects for future work.