PaperlayerView exact revision on arXiv

Source-linked AI summary

Agreeing to Cross: How Drivers and Pedestrians Communicate

Amir Rasouli, Iuliia Kotseruba, John K. Tsotsos

arXiv:1702.03555v1cs.RO

TL;DR

The paper addresses limited evidence about how pedestrians and drivers communicate during crossings and presents JAAD, a visual dataset paired with an analysis of joint attention and crossing behavior. More than 90% of pedestrians used attention before crossing, while crossing also depended on TTC, driver reaction, and crosswalk structure.

  • Problem

    Urban autonomous vehicles must communicate with road users, but existing approaches do not fully account for contextual factors such as driver reaction, vehicle speed, or crossway structure.

    Source: §I · p. 1, A. Studies of driver and pedestrian… · p. 2

  • Method

    The authors introduce JAAD and analyze annotated pedestrian actions and visual attention, including looking and glancing, across varied crossing scenarios.

    Source: §I · p. 1, §IV · p. 3

  • Results

    More than 90% of cases involved pedestrian attention before crossing, while crossing depended on street structure, driver reaction, and TTC.

    Source: §VI · p. 6

  • Takeaways & Limitations

    Pedestrian crossing communication includes attention, explicit gestures, and responses to driver actions, rather than a single communication event determining crossing.

    Source: §VI · p. 6

Abstract

from arXiv · show

The contribution of this paper is twofold. The first is a novel dataset for studying behaviors of traffic participants while crossing. Our dataset contains more than 650 samples of pedestrian behaviors in various street configurations and weather conditions. These examples were selected from approx. 240 hours of driving in the city, suburban and urban roads. The second contribution is an analysis of our data from the point of view of joint attention. We identify what types of non-verbal communication cues road users use at the point of crossing, their responses, and under what circumstances the crossing event takes place. It was found that in more than 90% of the cases pedestrians gaze at the approaching cars prior to crossing in non-signalized crosswalks. The crossing action, however, depends on additional factors such as time to collision (TTC), explicit driver's reaction or structure of the crosswalk.

I. INTRODUCTION

Fully autonomous urban driving remains unsolved partly because vehicles must communicate with road users amid formal rules, informal cues, and challenging conditions. This paper introduces JAAD and analyzes pedestrian crossing behavior and non-verbal communication.

  • Urban autonomous driving remains unsolved, with communication among road users a major dilemma in chaotic traffic scenes.Source: §I · p. 1
  • Pedestrians often use informal non-verbal communication and anticipation alongside official traffic rules when deciding whether to cross.Source: §I · p. 1
  • JAAD is a novel visual dataset for detecting and analyzing pedestrian behavior while crossing or attempting to cross under varied conditions.Source: §I · p. 1
  • The paper analyzes pedestrians’ actions and non-verbal cues across crossing scenarios, including effects associated with crosswalk structure, driver behavior, and vehicle distance.Source: §I · p. 1

A. Studies of driver and pedestrian interaction

Prior studies examine driver yielding, awareness, pedestrian decisions, and factors such as TTC, road geometry, weather, and eye contact. Existing predictive approaches often emphasize motion or crude body-language cues without incorporating crossing context.

B. Existing Datasets

Existing pedestrian datasets provide detection-oriented annotations, but the authors identify no dataset facilitating study of pedestrian crossing behavior. They therefore conduct a small-scale naturalistic driving study to extract non-verbal communication data across situations.

  • KITTI, the Caltech benchmark, and the Daimler dataset provide pedestrian-detection ground truth such as bounding boxes, stereo information, sensor readings, and occlusion tags.Source: B. Existing Datasets · p. 2
  • The authors report no existing datasets designed to facilitate study of pedestrians’ crossing behavior.Source: B. Existing Datasets · p. 2
  • Relevant psychological data is often collected at selected locations through researchers’ direct observation.Source: B. Existing Datasets · p. 2
  • Because raw Naturalistic Driving Study data is restricted, the authors conducted a small-scale naturalistic driving study and extracted non-verbal communication data.Source: B. Existing Datasets · p. 2

III. THE JAAD DATASET

JAAD comprises video clips from extensive driving footage covering varied urban scenarios, times, weather, lighting, and pedestrian interactions. The dataset combines targeted bounding-box annotations with textual behavioral annotations and metadata.

  • 346 high-resolution JAAD video clips of 5–15 seconds were extracted from approximately 240 hours of driving videos collected across several locations.Source: §III · p. 2
  • The clips primarily represent urban and suburban scenarios, with fewer rural examples and fewer interactions involving other drivers.Source: §III · p. 2
  • Scenarios include individual and group crossings, pedestrian occlusion, and pedestrians walking along the road.Source: §III · p. 2
  • Videos span different times of day and varied weather and lighting conditions, including challenging sun glare, snow, and rain.Source: §III · p. 2
  • Annotations include bounding boxes for interacting or driver-relevant cars and pedestrians plus textual observations created with BORIS.Source: §III · p. 2
  • Each clip records weather, time of day, pedestrian age and gender, location, and whether the crossing is designated.Source: §III · p. 3

IV. THE DATA

The dataset reveals highly varied pedestrian crossing behavior and organizes it into nine groups based on initial state and whether attention or crossing occurs. It also distinguishes two forms of visual attention by duration and purpose.

  • More than 100 distinct action patterns occur in pedestrian crossing and no-crossing scenarios.Common sequences such as “standing, looking, crossing” and “crossing, looking” cover only half of completed crossings.Source: §IV · p. 3
  • The patterns are split into 9 groups according to the pedestrian’s initial state and whether attention or crossing is occurring.Source: §IV · p. 3
  • Attention is defined as the pedestrian’s first environmental assessment while expressing an intention to approaching vehicles.The paper treats this initial assessment as non-verbal communication.Source: §IV · p. 3
  • Looking typically lasts at least 1 second and may involve assessing the car or establishing eye contact, whereas glancing is a brief traffic assessment.Glancing usually lasts less than a second and occurs when the pedestrian is confident the vehicle poses little immediate danger.Source: §IV · p. 3

V. OBSERVATIONS AND ANALYSIS

The analysis excludes incomplete or ambiguous crossing cases, while Table II presents the behavioral patterns observed in the data.

  • Incomplete crossing scenarios are omitted because they do not show the full event and make behavior at the crossing point difficult to assess.Source: §V · p. 3
  • Action cases are omitted because pedestrian intentions are ambiguous when people are far from or not approaching the curb.Source: §V · p. 3
  • Table II lists the behavioral patterns observed in the data.Source: Table II · p. 4

A. Forms of non-verbal communication

Pedestrians primarily communicate crossing intentions through looking or glancing, while other signals are rarer and often respond to driver actions. Their responses may appear as behavioral changes rather than explicit signals.

B. Attention occurrence prior to crossing

Pedestrian attention before crossing varies with TTC, crosswalk designation, street width, and pedestrian age. Attention-free crossings are uncommon and concentrated in safer-looking contexts, while gaze duration changes nonlinearly with TTC.

C. Crossing action post attention occurrence

Whether pedestrians cross after signaling attention depends on crosswalk structure and the driver’s response. Pedestrians are more likely to proceed when designated pathways or signals are present and when unsignalized drivers slow or stop.

VI. CONCLUSIONS

Pedestrians communicate crossing intentions primarily through attention toward approaching vehicles, while nodding and hand gestures are less common responses to driver actions. Crossing decisions also depend on street structure, driver reaction, and time to collision, motivating richer analysis of pedestrian gait and driver behavior.

  • In more than 90% of cases, pedestrians use attention toward approaching vehicles to communicate an intention to cross.Looking is the most prominent form of attention-based communication.Source: §VI · p. 6
  • Nodding and hand gestures occur less often and commonly respond to the driver’s action by expressing gratitude, acknowledgement, or yielding.These explicit communication forms were observed in 15% of cases.Source: §VI · p. 6
  • Crossing depends on street structure, the driver’s reaction to communication, and time to collision rather than necessarily following the first communicated intention.Source: §VI · p. 6
  • Future analysis should examine pedestrian gait with and without attention and record driver gestures, eye movements, and vehicle-state changes.Source: §VI · p. 6

Related papers

Loading 1702.03555v1…