Source-linked AI summary
Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practice
Amir Rasouli, John K. Tsotsos
TL;DR
Autonomous vehicles must communicate with pedestrians and understand their intentions, but pedestrian behavior depends on interacting demographic, environmental, social, and traffic factors. The paper surveys classical and autonomous-vehicle studies, reviews behavioral-study methods and practical interaction systems, and identifies contradictions, evaluation gaps, and future research directions.
Problem
Autonomous vehicles face a social interaction void, while pedestrian behavior is shaped by multiple factors whose extent and interrelationships lack a holistic account.
Method
The paper comprehensively surveys classical driver–pedestrian and autonomous-vehicle studies, examines behavioral-study methods, and reviews communication and pedestrian-intention systems.
Results
The review identifies pedestrian-behavior factors, connects findings across disciplines, and highlights contradictions and limited cross-modality evaluation in communication research.
Takeaways & Limitations
Effective interaction research must account for pedestrian behavior as a social phenomenon and evaluate communication mechanisms with larger, more diverse participant samples.
Abstract
from arXiv · showhide
One of the major challenges that autonomous cars are facing today is driving in urban environments. To make it a reality, autonomous vehicles require the ability to communicate with other road users and understand their intentions. Such interactions are essential between the vehicles and pedestrians as the most vulnerable road users. Understanding pedestrian behavior, however, is not intuitive and depends on various factors such as demographics of the pedestrians, traffic dynamics, environmental conditions, etc. In this paper, we identify these factors by surveying pedestrian behavior studies, both the classical works on pedestrian-driver interaction and the modern ones that involve autonomous vehicles. To this end, we will discuss various methods of studying pedestrian behavior, and analyze how the factors identified in the literature are interrelated. We will also review the practical applications aimed at solving the interaction problem including design approaches for autonomous vehicles that communicate with pedestrians and visual perception and reasoning algorithms tailored to understanding pedestrian intention. Based on our findings, we will discuss the open problems and propose future research directions.
I. INTRODUCTION
Autonomous vehicles create a social interaction void in traffic, making pedestrian understanding and communication central to safe urban operation. This survey connects behavioral research with practical systems for studying, communicating with, and predicting pedestrian behavior.
- Replacing human drivers with autonomous control systems creates a social interaction void because driving requires interaction among road users for traffic flow and safety.
- Poor social understanding in autonomous vehicles has been associated with traffic accidents and erratic behaviors toward pedestrians.
- Pedestrian crossing decisions are influenced by demographics, road conditions, social factors, and traffic characteristics, but their interrelationships lack a holistic account.
- The review also covers practical systems for communicating with pedestrians and understanding or predicting their behavior.
- The paper surveys classical driver–pedestrian studies and autonomous-vehicle studies, connecting traffic-interaction research across disciplines.
- Behavioral methods include questionnaires, traffic reports, on-site observation, laboratory recordings, naturalistic recording, interviews, and Wizard of Oz experiments.Questionnaires can suffer from response and recall biases, while naturalistic recordings permit repeated review and multiple observers; Wizard of Oz studies disguise or hide the human controller.
III. PEDESTRIAN BEHAVIOR STUDIES
The paper organizes pedestrian behavior research into classical studies involving human drivers and studies involving autonomous vehicles. Figures 3 and 4 summarize the data-collection methods used in these two categories.
- Pedestrian behavior studies are divided into classical studies and studies involving autonomous vehicles.Classical studies examine pedestrians interacting with human drivers, whereas autonomous-vehicle studies examine interactions involving autonomous vehicles.
- Figure 3 summarizes data-collection methods used in classical pedestrian behavior studies.
- Figure 4 summarizes data-collection methods used in pedestrian behavior studies involving autonomous vehicles.
A. Classical Studies
Classical studies identify pedestrian demographics, group behavior, social norms, and environmental context as interrelated influences on crossing decisions. The literature also shows that research coverage is uneven across factors.
- Framework: The survey presents these factors as a connected system rather than independent variables, with Fig. 5 mapping categories, sub-factors, interconnections, and influence direction.The reviewed literature is a subset of the field because an exhaustive survey would be prohibitive.
- Social Factors: Group size shapes pedestrian and driver behavior: groups often cross together, receive more yielding, accept shorter gaps, and move more slowly when denser.Children crossed as a group in more than 80% of observed cases; denser groups generally reduce pedestrian speed.
- Social Factors: Social norms influence behavior and intention prediction beyond formal traffic rules, which alone do not guarantee safe interaction.Norms can diverge from legal speed limits and have been linked to accidents even when drivers were not legally at fault.
- Social Factors: Imitation can spread both lawful and unlawful pedestrian behavior, with stronger effects for law violations and variation by the imitated person’s social status.A law-adhering or law-violating pedestrian increases the likelihood that others behave similarly, while fancy clothing was associated with more imitation.
- Demographics: Demographics affect caution, attention, speed, and risk assessment: women and older pedestrians are generally more cautious, while younger pedestrians are more variable.Men look more often at vehicles, whereas women attend more to traffic lights and other pedestrians; older pedestrians walk slower and assess vehicle speed less accurately.
- State and Physical Context: Pedestrian state and road conditions alter speed and perception, including differences between walking and standing and faster movement under several crossing conditions.Walking pedestrians may estimate speed and distance better than standing pedestrians; speed also varies with road structure, group size, age, and time of day.
2) Environmental Factors:
Environmental and traffic factors affect pedestrian crossing through road design, signals, weather, traffic gaps, waiting, and communication. Their effects can be context-dependent, and the classical literature leaves several factors underaddressed in recent work.
- Physical Context: Signals and crossings alter attention, trajectory, speed, yielding, and law compliance, while road geometry changes risk, attention, and accepted gaps.Pedestrians looked at vehicles 69.5% of the time at signalized intersections and 86% at unsignalized intersections; they also crossed diagonally without signals.
- Environmental Conditions: Environmental conditions affect perception and caution: bad weather worsens speed estimation, while low illumination reduces visual functions such as resolution, contrast, and depth perception.The passage also reports greater caution among elderly pedestrians and women in warm than cold weather.
- Dynamic Factors: Gap acceptance depends on vehicle speed, vehicle distance, pedestrian characteristics, group size, culture, law compliance, and street width.Time To Collision combines vehicle speed and distance; reported average pedestrian gap acceptance is between 3-7s.
- Dynamic Factors: Waiting time has disputed effects on gap acceptance and should be considered with personal characteristics rather than treated as an isolated predictor.One account links longer waits to frustration and lower gap acceptance, while another finds waiting time alone insufficient to explain changes.
- Communication: Communication is ambiguous and context-dependent, yet pedestrians rely on it during crossing; ineffective communication contributes to more than a quarter of traffic conflicts.Signals include gaze, gestures, vehicle speed changes, and yielding behavior, with eye contact or stepping onto the road often preceding yielding at unsignalized crossings.
- Research Coverage: Recent research increasingly studies communication, attention, trajectory, and culture, while lighting, road conditions, vehicle type, past experience, social status, and pedestrian flow remain underaddressed.The survey identifies an uneven distribution of attention across environmental and social factors.
B. Studies in the Context of Autonomous Driving
Studies of pedestrian interaction with autonomous vehicles examine communication, attention, perceived risk, and related behavioral factors, but coverage remains narrower than in classical research.
- Autonomous-vehicle studies divide behavioral factors into pedestrian and environmental categories and map their connections in a decision-making framework.
- Communication and attention are central themes, with empirical work linking inability to communicate and establish eye contact to higher perceived risk.
- 38% improvement in resolving deadlocks was observed when a remotely controlled vehicle used an intent display, with prior familiarization increasing trust.
- Intent displays were inconsistent: one study found only 12% of decisions influenced and longer decision times, while informative speed displays outperformed advisory signals.
- Pedestrians’ crossing decisions remain strongly shaped by vehicle speed and distance, while vehicle appearance and friendliness are less influential.
- Compared with classical studies, autonomous-driving research rarely addresses signal, location, road structure, gap acceptance, social norms, group size, pedestrian speed, and street width.
A. Communicating with Pedestrians
Autonomous vehicles communicate with pedestrians through explicit displays, vehicle motion, wireless links, human-like cues, and instrumented roads, each with distinct implementation concerns.
- V2X exchanges entities’ pose, speed, and location in real time, while V2P extends this communication to pedestrians’ smartphones and wearable devices.
- V2P systems can issue imminent-collision warnings, but raise privacy concerns and may be rejected by pedestrians who see them as shifting responsibility for accidents.
- Explicit communication includes displays, road projections, LED patterns, advisory messages, moving eyes, and human-like gestures that convey vehicle intent or recognition.
- Communication concepts range from Mercedes-Benz zebra-crossing projections and Mitsubishi directional indicators to AutonoMI tracking lights and AEVITA moving eyes.
- Smart roads combine sensors and lighting to detect crossings, weather, and hazards, then use visual effects to inform road users about potential threats.
B. Understanding Pedestrians’ Intentions
Pedestrian-intention estimation combines movement, pose, street context, social context, and classification models, but available data and perception assumptions constrain current approaches.
- Intention-estimation methods use pedestrian dynamics, context, pose, motion history, head orientation, goals, group size, signals, and distances to infer crossing decisions.
- Trajectory-only methods can miss imminent crossings when pedestrians start suddenly, change direction, stop, or wait while checking traffic.
- Social-context models incorporate awareness, curb distance, trajectory, and interpersonal relationships, including head orientation and social forces.
- The contextual model was based on scripted experiments in a narrow, non-signalized street, limiting the breadth of its evaluation setting.
- A contextual crossing model used road characteristics, traffic signals, zebra crossings, and pedestrian state, while another used street-zone descriptors with an SVM.
- There is no clear trend in whether algorithms use pedestrian dynamics or contextual information, partly because training data availability and type vary.
V. WHAT’S NEXT
The paper introduces open problems in pedestrian behavior and traffic social interaction as a basis for future research directions.
- The section frames open problems identified throughout the paper as the basis for future research directions in traffic social interaction.
1) Classical studies of pedestrian behavior:
Classical pedestrian-behavior studies identify many influential factors, but their findings can conflict because behavior depends on interacting cultural, temporal, and contextual conditions.
- 38 factors can potentially influence pedestrian behavior, with age, gender, group size, and gap acceptance among the most studied.
- Studies generally agree on many relationships, including how group size influences gap acceptance and how demographics shape behavior.
- Findings are often contradictory for communication, imitation, attention, and waiting-time effects on gap acceptance.
- Cultural differences and changing socioeconomic, technological, and traffic conditions limit direct transfer of findings across regions and time periods.
- Because behavioral factors are strongly interrelated, studies should use multimodal designs that account for chain effects rather than isolated subsets.
2) Pedestrian behavior and autonomous vehicles:
Research on pedestrian behavior toward autonomous vehicles is growing but remains limited in scale and participant diversity, producing some contradictory findings and leaving classical-study implications incomplete.
- Autonomous-vehicle pedestrian studies remain relatively few compared with classical pedestrian-behavior studies.
- Pedestrians may behave differently around autonomous vehicles, so classical findings cannot alone characterize autonomous-vehicle interactions.
- Larger-scale studies with broader demographics and classical behavioral-study considerations are needed to support autonomous-vehicle design.
- Samples are often smaller than 100 participants and disproportionately composed of university students, limiting demographic breadth.
3) Communicating with road users:
Communicating with pedestrians remains an open interface-design problem, especially because cross-modality effectiveness is rarely evaluated with sufficiently large and diverse samples.
- The main unresolved interface question is which communication modality is most effective for autonomous vehicles communicating with pedestrians.
- Existing studies commonly test whether communication matters or compare strategies within a single modality rather than across modalities.
- Cross-modality evaluations are rare, and available studies have used no more than 10 participants.
- The literature therefore calls for larger studies involving participants from diverse backgrounds.
4) Understanding pedestrians’ intention:
Pedestrian-intention systems remain limited by insufficient context, perception, and data diversity, while effective universal systems must operate across varied traffic conditions.
- Current intention-estimation algorithms use limited contextual information and often lack visual-perception methods for scene analysis.
- Their data are often scripted or insufficiently diverse to represent varied traffic scenarios.
- Effective systems should identify relevant scene elements, reason about their interconnections, and infer road users’ upcoming actions.
- Universal systems should handle different street structures, traffic signals, and crosswalk configurations, requiring behavioral data from varied conditions and geographic locations.