PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios

Baiming Chen, Xiang Chen, Wu Qiong, Liang Li

arXiv:2004.06531v2cs.ROcs.LGstat.ML

TL;DR

Static naturalistic testing is inefficient because risky events are rare, and existing datasets may miss scenarios. The paper proposes adaptive adversarial evaluation using ensemble reinforcement learning and nonparametric Bayesian clustering; in lane-change tests, generated environments significantly degraded both rule-based and learning-based models.

  • Problem

    Naturalistic testing is inefficient because risky events are rare, while static datasets may omit scenarios not represented in the data [6].

    Source: §I · p. 1

  • Method

    The framework uses ensemble deep reinforcement learning to generate adversarial environments and a nonparametric Bayesian method to cluster diverse risky scenarios.

    Source: §I · p. 2

  • Results

    99.2% success rate in naturalistic environments became 90.6% crash rate in adversarial environments for Mrl, and adversarial environments degraded both Mgap and Mrl.

    Source: C. Results of Adversarial Training · p. 7

  • Takeaways & Limitations

    The authors describe adversarial evaluation as a promising way to supplement current autonomous-vehicle evaluation frameworks.

    Source: §V · p. 9

Abstract

from arXiv · show

Autonomous vehicles must be comprehensively evaluated before deployed in cities and highways. However, most existing evaluation approaches for autonomous vehicles are static and lack adaptability, so they are usually inefficient in generating challenging scenarios for tested vehicles. In this paper, we propose an adaptive evaluation framework to efficiently evaluate autonomous vehicles in adversarial environments generated by deep reinforcement learning. Considering the multimodal nature of dangerous scenarios, we use ensemble models to represent different local optimums for diversity. We then utilize a nonparametric Bayesian method to cluster the adversarial policies. The proposed method is validated in a typical lane-change scenario that involves frequent interactions between the ego vehicle and the surrounding vehicles. Results show that the adversarial scenarios generated by our method significantly degrade the performance of the tested vehicles. We also illustrate different patterns of generated adversarial environments, which can be used to infer the weaknesses of the tested vehicles.

I. INTRODUCTION

The paper addresses inefficient, static evaluation by proposing adaptive adversarial environments that respond to tested-vehicle behavior. It combines ensemble reinforcement learning and nonparametric Bayesian clustering to generate and organize risky scenarios, with lane-change tests showing increased collisions for both rule-based and learning-based models.

I. INTRODUCTION

  • Naturalistic field tests are inefficient because risky events are rare, and static datasets may omit patterns that never occur in the collected data.The evaluation results therefore depend on dataset completeness, which the paper says is difficult to verify.Source: §I · p. 1
  • Earlier adaptive methods still depend heavily on initial datasets and generate initial conditions rather than time-sequential interactions,.Existing adversarial learning also faces traffic interactions that are not fully competitive and a need to find varied risky patterns, not only one optimum –[11].Source: §I · p. 1
  • The framework treats surrounding vehicles as adversaries that make time-sequential decisions based on the tested ego vehicle’s behavior.The comparison describes this as closed-loop generation, adapting risky environments to the tested vehicle.Source: §I · p. 2, Figure 1 · p. 2
  • Non-zero-sum rewards model mixed cooperation and competition, while ensemble training and nonparametric Bayesian clustering represent and group diverse adversarial policies.The clustering approach addresses an unknown number of potential local optima.Source: §I · p. 2
  • Adversarial environments significantly increase collision rates for both rule-based and learning-based lane-change models.The paper reports that the generated environments significantly degrade both types of tested models.Source: §I · p. 2

II. LANE-CHANGE SCENARIO

The paper uses a four-vehicle lane-change scenario to evaluate its adversarial method, motivating the choice by the safety relevance and frequency of lane changes. Initial conditions draw on naturalistic data and specified distributions, while the ego vehicle must complete a left merge within distance and time limits.

II. LANE-CHANGE SCENARIO

  • Sudden lane changes caused about 17.0% of severe crashes from 2010 to 2017, motivating the scenario’s safety relevance.Lane changes are also described as frequent and important for autonomous driving and microscopic traffic simulation –.Source: §II · p. 2
  • The scenario has one ego vehicle making a left lane change among a follow, leader, and target vehicle; only the ego vehicle moves laterally.The lane is 3.2 meters wide, and vehicles are 1.85 meters wide and 4.83 meters long for collision detection.Source: §II · p. 2, Figure 2 · p. 2
  • Initial leader–ego and target–follow distances are sampled from the SPMD empirical range distribution, while other distances and vehicle speeds use Gaussian distributions.The ego–follow spacing has mean 0 m and standard deviation 5 m; vehicle speeds have mean 10 m/s and standard deviation 4 m/s.Source: §II · pp. 2–3, Figure 3 · p. 3
  • The ego vehicle must complete the lane change within 300 m and 30 s, and success requires its whole body to enter the target lane.The surrounding vehicles’ initial longitudinal behavior is controlled using the Intelligent Driver Model (IDM).Source: §II · p. 3

III. ADVERSARIAL EVALUATION APPROACH

The approach generates adversarial environments in which surrounding vehicles try to prevent the ego vehicle from completing its task, then clusters the environments to provide risky patterns. The authors present this process as a way to supplement and accelerate traditional evaluation.

III. ADVERSARIAL EVALUATION APPROACH

  • Surrounding vehicles act as adversaries that try to prevent the ego vehicle from finishing its task.The generated environments are then clustered to provide risky patterns for evaluating the ego vehicle.Source: §III · p. 3
  • The authors state that adversarial evaluation can significantly accelerate and supplement traditional evaluation procedures.Source: §III · p. 3

A. Adversarial Environment Generation

The framework trains adversarial environments to supplement missing parts of naturalistic-data distributions. It uses diverse local optima and a traffic-rule penalty to generate challenging scenarios while keeping surrounding vehicles rule-abiding.

A. Adversarial Environment Generation

  • The method seeks adversarial environments to supplement naturalistic-data distributions that are limited in coverage and risky events.The paper notes that systems developed from such databases may be overfitted to scenarios that appear in the data.Source: A. Adversarial Environment Generation · p. 3
  • The scenario is formulated as a Markov game, with the ego and surrounding vehicles as agents; the adversaries share a reward and cooperate against the ego.The paper treats the ego as part of the adversaries’ environment, reducing their problem to an MDP, and states that V2X infrastructure makes the cooperation assumption achievable.Source: A. Adversarial Environment Generation · p. 3
  • DDPG trains the adversaries, using an actor for deterministic actions and a critic for expected return.The off-policy update allows an experience replay buffer to reduce time correlation and improve sample efficiency; slowly updated target networks are also used for stability.Source: A. Adversarial Environment Generation · p. 4
  • The adversarial reward combines the negative ego reward with a traffic-rule penalty, encouraging interference while keeping environment vehicles rule-abiding.The penalty is weighted by β, which determines the environment vehicles’ rationality.Source: A. Adversarial Environment Generation · p. 4
  • Training multiple randomly initialized DDPG agents without exploration collects diverse local optima as candidate adversarial scenarios.Training stops when a local optimum is reached or an episode’s cumulative reward reaches a boundary indicating a challenging environment.Source: A. Adversarial Environment Generation · p. 4

B. Environment Clustering

The framework clusters adversarial environments by the state distributions they generate, rather than by their high-dimensional policies. DP-Means accommodates an unknown number of clusters, with Jensen-Shannon divergence measuring distribution distance.

B. Environment Clustering

  • Policies are clustered through Monte Carlo estimates of their generated state distributions because the deep-network policies are too high-dimensional to cluster directly.Each distribution describes scenarios when the ego vehicle is in a particular adversarial environment.Source: B. Environment Clustering · p. 4
  • DP-Means addresses the unknown cluster count by allowing an arbitrary number of clusters rather than requiring a preset count.It is a hard-clustering approach related to the K-Means objective, derived from a Dirichlet process mixture model.Source: B. Environment Clustering · pp. 4–5
  • Jensen-Shannon divergence measures distances between state distributions during clustering.The algorithm assigns a distribution to its nearest cluster unless its minimum distance exceeds λ, in which case it starts a new cluster.Source: B. Environment Clustering · p. 5
  • A heuristic sets λ by iteratively adding distributions farthest from the current set, then using the maximum distance in the last round.After clustering, the method extracts adversarial patterns for the tested vehicle.Source: B. Environment Clustering · p. 5

IV. SIMULATION

The simulation uses CARLA to demonstrate adversarial evaluation in a lane-change scenario with one tested ego vehicle and three surrounding adversarial vehicles.

IV. SIMULATION

  • CARLA provides vehicle-dynamics models, configurable sensors, and scenario generation for the simulation.The paper uses this environment to demonstrate its adversarial evaluation approach.Source: §IV · p. 5
  • The CARLA scenario contains an ego vehicle and three adversarial vehicles: follow, leader, and target.The ego vehicle is attempting a left lane change.Source: §IV · p. 5, Figure 4 · p. 5

A. Ego Vehicle Lane-Change Controllers

The evaluation uses two lane-change controllers: a gap-acceptance rule-based model and a reinforcement-learning-trained model.

A. Ego Vehicle Lane-Change Controllers

1) Gap Acceptance Model.:

The section presents a gap-acceptance lane-change model and a hierarchical reinforcement-learning model. Both achieve success rates above 99% in naturalistic environments, where direct evaluation is described as low-efficient.

1) Gap Acceptance Model.:

  • The gap-acceptance model assesses target-lane vehicle positions and speeds to judge whether the gap is sufficient, while front spacing to the leader matters for crash avoidance.Source: 1) Gap Acceptance Model · p. 5

2) Reinforcement Learning Model.:

  • The reinforcement-learning model uses deep Q-learning to decide whether to start a lane change, with lattice planning for trajectories and model predictive control for vehicle control.Source: 2) Reinforcement Learning Model · p. 5
  • Both lane-change models achieve success rates higher than 99% in naturalistic environments, while Mrl converges after 500 training episodes.Direct evaluation in those environments is described as low-efficient.Source: 2) Reinforcement Learning Model · pp. 5–6, Figure 5 · p. 6

B. Adversarial MDP Setting

The adversarial MDP trains environment vehicles to challenge the ego vehicle, and the learned environments substantially degrade both tested lane-change models. The results also distinguish Mgap's crash avoidance from Mrl's difficulty generalizing to risky scenarios.

B. Adversarial MDP Setting

  • The MDP uses a nine-dimensional state with perfect observation and three longitudinal control actions for the adversarial vehicles.Its adversarial reward negates the ego reward and adds a traffic-rule term; breaking traffic law receives −50.Source: B. Adversarial MDP Setting · p. 6

C. Results of Adversarial Training

D. Clustering and Visualization

The visualizations show that generated adversarial scenarios concentrate on states rare in naturalistic traffic and represent riskier conditions. Their patterns differ by tested model, revealing ways adversaries exploit each model's weaknesses.

D. Clustering and Visualization

  • Adversarial state distributions are narrower than naturalistic ones and usually concentrate on states rare in naturalistic traffic, indicating riskier scenarios.The state distributions are reduced to two dimensions for visualization using principal component analysis.Source: Figure 9 · p. 7, D. Clustering and Visualization · pp. 7–8
  • For Mgap, adversaries commonly block ahead of the ego vehicle while the following vehicle minimizes the lane-change gap; Mgap avoids crashes in most experiments but has 7.1% lane-change success.Source: Figure 10 · p. 8, D. Clustering and Visualization · p. 8
  • Mrl can act too aggressively or too conservatively in adversarial scenarios; the distribution mismatch with its training environments is linked to a 90.6% crash rate.Source: D. Clustering and Visualization · pp. 8–9

E. Influence of the Reward Configuration

The adversarial environments significantly degrade the performance of rule-based and learning-based lane-change models. The reward weight β tunes adversarial-vehicle rationality and the resulting crash behavior.

E. Influence of the Reward Configuration

V. CONCLUSION

  • The authors set β = 1 for diverse behaviors that are neither too aggressive nor too conservative, and identify more realistic assumptions as a future direction.They also propose using extracted weaknesses to improve autonomous-driving strategies.Source: E. Influence of the Reward Configuration · p. 9, §V · p. 9

E. Influence of the Reward Configuration

V. CONCLUSION

  • The framework's adversarial environments significantly degrade both rule-based and learning-based lane-change models' performance.Source: §V · p. 9
Loading 2004.06531v2…