PaperlayerView exact revision on arXiv

Source-linked AI summary

EPSILON: An Efficient Planning System for Automated Vehicles in Highly Interactive Environments

Wenchao Ding, Lu Zhang, Jing Chen, Shaojie Shen

arXiv:2108.07993v1cs.RO

TL;DR

Independent prediction and planning can miss interactions among traffic participants, while POMDP planning can be computationally demanding. EPSILON combines guided behavior planning with safety-aware multi-agent simulation and corridor-based motion optimization. The authors validate it in real dense city traffic and report safe, smooth behavior without excessive conservatism.

  • Problem

    Independent prediction does not model mutual influence between the ego vehicle and other traffic participants, while real-world driving also involves uncertain behavior and noisy perception.

    Source: §I · p. 1

  • Method

    EPSILON uses guided semantic-action branching and safety-aware multi-agent forward simulation for behavior planning, followed by motion planning with a spatio-temporal semantic corridor.

    Source: §I · p. 2, B. Guided Action Branching · pp. 6–7, §IX · p. 18

  • Results

    EPSILON was validated on a real vehicle in dense city traffic, and the authors report safe, smooth driving without excessive conservatism.

    Source: §I · p. 2, §IX · p. 18

  • Takeaways & Limitations

    The system converts behavior decisions into safe, comfortable executable trajectories while retaining consistency with behavior planning.

    Source: §IX · p. 18

Abstract

from arXiv · show

In this paper, we present an Efficient Planning System for automated vehicles In highLy interactive envirONments (EPSILON). EPSILON is an efficient interaction-aware planning system for automated driving, and is extensively validated in both simulation and real-world dense city traffic. It follows a hierarchical structure with an interactive behavior planning layer and an optimization-based motion planning layer. The behavior planning is formulated from a partially observable Markov decision process (POMDP), but is much more efficient than naively applying a POMDP to the decision-making problem. The key to efficiency is guided branching in both the action space and observation space, which decomposes the original problem into a limited number of closed-loop policy evaluations. Moreover, we introduce a new driver model with a safety mechanism to overcome the risk induced by the potential imperfectness of prior knowledge. For motion planning, we employ a spatio-temporal semantic corridor (SSC) to model the constraints posed by complex driving environments in a unified way. Based on the SSC, a safe and smooth trajectory is optimized, complying with the decision provided by the behavior planner. We validate our planning system in both simulations and real-world dense traffic, and the experimental results show that our EPSILON achieves human-like driving behaviors in highly interactive traffic flow smoothly and safely without being over-conservative compared to the existing planning methods.

I. INTRODUCTION

Real-world planning must handle uncertainty and interaction that are difficult to reproduce in simulation, while many prior methods rely on independent prediction or remain computationally expensive. EPSILON combines interaction-aware behavior planning with optimization-based motion planning and extends validation to dense real-world traffic.

I. INTRODUCTION

  • Real-vehicle planning must handle perception noise, occlusion, and stochastic traffic behavior that are difficult to reproduce in simulation.The paper frames closed-loop execution in an imperfect world as a challenge for methods validated only in simulation or on datasets.Source: §I · p. 1
  • Independent prediction cannot model mutual influence between ego and surrounding vehicles, which matters in interactive scenarios such as gap merging.The paper notes that prediction cannot be perfect because traffic behavior is stochastic and onboard perception is noisy.Source: §I · pp. 1–2
  • EPSILON’s motion planner uses a spatio-temporal semantic corridor to encode complex constraints and optimizes a safe, smooth trajectory that follows the behavior plan.The trajectory uses piecewise Bézier curves, whose convex-hull and hodograph properties support safety and dynamic feasibility.Source: §I · p. 2
  • EPSILON extends prior behavior and motion planners and into an integrated system, with guided branching to make interaction-aware planning more efficient.The paper also introduces a more flexible forward simulation model with a safety mechanism for over-aggressive or uncooperative traffic participants.Source: §I · p. 2
  • The paper reports extensive validation on a real automated vehicle in dense city traffic without an HD-map, relying on onboard sensors.Earlier behavior and motion planning work cited by the authors was limited to simulation.Source: §I · p. 2

II. RELATED WORK

  • Prior geometric behavior-planning methods commonly rely on deterministic prediction, while POMDP approaches face high computational cost in complex driving environments –.The paper contrasts its guided branching with MPDM’s single-stage selection of long-duration semantic actions, which restricts maneuverability.Source: §II · pp. 2–3

III. SYSTEM OVERVIEW

EPSILON separates coarse behavior decisions from local trajectory optimization, while coupling multi-agent anticipation into behavior planning. The resulting decision guides a safe, dynamically feasible motion plan for closed-loop execution.

III. SYSTEM OVERVIEW

  • The system has guided branching, scenario realization, and evaluation; closed-loop multi-agent forward simulation realizes combinations of ego actions and other agents’ intentions.The flexible driving model is intended to support safe, human-like behavior around noisily rational participants.Source: §III · p. 3
  • The motion layer builds a spatiotemporal semantic corridor around the behavior plan’s initial guess, then optimizes a piecewise Bézier trajectory for safety and dynamic feasibility.A trajectory server schedules replanning and sends control commands to close the execution loop.Source: §III · p. 3

IV. PROBLEM FORMULATION

  • The behavior layer outputs a horizon-length sequence of discrete states for all agents, and the motion layer turns it into a smooth, safe trajectory for the ego vehicle.The behavior layer receives the observation and local environment; the motion layer receives the decision and environment.Source: §IV · p. 4
  • The driver model represents surrounding agents’ high-level decisions and can be user-defined or data-learned, with latent intentions or aggressiveness supporting varied driving styles.The formulation separates the ego vehicle’s controlled action from probabilistic actions of surrounding agents.Source: §IV · p. 5
  • The POMDP belief update couples planning and prediction because hidden states for other agents are estimated step-by-step from observations.Driver models use all agents’ states and observations, making belief updates interactive even when per-agent transitions are independent.Source: §IV · p. 5
  • The behavior layer’s belief tree grows exponentially with depth, so domain knowledge is used to simplify decision-making for real-time driving.The motion layer then solves a local trajectory-optimization problem to realize the multi-agent decision at finer resolution.Source: §IV · p. 5

A. Motivating Example

EPSILON uses guided branching to focus POMDP planning on a small set of plausible, high-level driving decisions. Semantic actions and a domain-specific policy tree make the search more flexible while limiting its growth.

A. Motivating Example

  • Conditional focused branching selects potentially risky agent intentions conditioned on the ego policy, avoiding evaluation of every combination.The approach extends the policy across multiple future decision points rather than using one semantic action for the entire horizon.Source: A. Motivating Example · pp. 5–6
  • Semantic actions guide exploration away from the vast, mostly low-likelihood primitive-action space while preserving potential high-level decisions.The motivation is to use domain knowledge in action branching rather than explore a fine-grained control lattice.Source: A. Motivating Example · p. 5, Figure 3 · p. 6

B. Guided Action Branching

  • Semantic actions are human-interpretable maneuvers that predefined controllers execute as closed-loop primitive controls over potentially longer durations.Context-dependent controllers map semantic actions to primitive actions at each step.Source: B. Guided Action Branching · p. 6
  • The DCP-Tree starts from the ongoing action and limits each policy sequence to one action change per planning cycle.Replanning enables back-and-forth behavior across cycles, while the restricted tree's leaf count grows linearly with its height.Source: B. Guided Action Branching · p. 7
  • The planner evaluates root-to-leaf action sequences as candidate policies and selects the one with maximum utility.Policy evaluations are independent and can be parallelized.Source: B. Guided Action Branching · p. 7

C. Guided Observation Branching

Guided observation branching reduces policy-evaluation cost by focusing on risky agents and fixing their sampled intentions during forward simulation. The resulting simulation remains closed-loop, but relies on simplifying assumptions about observability and other agents' hidden states.

C. Guided Observation Branching

  • The planner assumes a fully observable ego state, noise-free observations, deterministic agent policies, and deterministic kinematic transitions for efficient policy evaluation.Control noise may be injected if computational budget permits.Source: C. Guided Observation Branching · p. 7
  • Other agents' hidden states and semantic actions remain fixed during forward simulation to avoid belief updates with exponential complexity.The authors report satisfactory future anticipation under this assumption.Source: C. Guided Observation Branching · p. 7
  • Although sampled intentions remain fixed, agents' actions depend on the simulated previous state, preserving closed-loop interaction.Scenario utilities are weighted by the corresponding initial-belief values.Source: C. Guided Observation Branching · p. 7
  • Other agents' semantic actions represent center-line choices, with a lightweight classification network estimating probabilities across a varying number of urban center-lines.The paper contrasts this urban estimator with a rule-based predictor used in highway environments.Source: C. Guided Observation Branching · p. 7
  • Conditional focused branching conditions on the ego policy, branching over risky agents' intentions while assigning low-risk agents their most probable intention.This reduces evaluation from all possible intention combinations to selected scenarios.Source: C. Guided Observation Branching · p. 8, Figure 5 · p. 8

D. Multi-agent Forward Simulation

EPSILON couples forward simulation with behavior planning so that other agents’ responses are modeled in relation to the ego vehicle’s candidate maneuvers. Its context-aware ego controller supports flexible lane changes, while the lightweight simulation model is reported to outperform two state-of-the-art predictors on multi-modal prediction.

D. Multi-agent Forward Simulation

  • The forward simulation is coupled to behavior planning, so each candidate ego maneuver has a corresponding interactive anticipation of other agents.This contrasts with independent prediction, which does not account for the effect of the ego vehicle’s future motion on other agents.Source: D. Multi-agent Forward Simulation · p. 8
  • The context-aware controller handles action branching by adapting lane-changing motion to traffic configurations, whereas simple pure pursuit can fail or produce risky maneuvers.The figure identifies the context-aware outcomes as having a higher success rate.Source: Figure 6 · p. 8
  • The context-aware ego controller supports realistic, traffic-adaptive lane changes, reducing the action sequences needed and improving forward-simulation success.Other traffic participants retain a simple model-based controller; the context-aware controller is used for the controlled vehicle.Source: D. Multi-agent Forward Simulation · p. 8
  • The lightweight forward simulation is reported to outperform two state-of-the-art trajectory-prediction methods on multi-modal prediction.The paper describes this as evidence that the simulation is accurate enough for anticipating the future.Source: D. Multi-agent Forward Simulation · p. 8
  • The resulting multi-agent prediction is conditional on the controlled vehicle’s decision, a form of prediction that the paper distinguishes from traditional prediction methods.The cited prior work is described as showing improved predictor performance when an agent’s potential future plan is considered.Source: D. Multi-agent Forward Simulation · p. 9

E. Safety Mechanism

The safety mechanism addresses risk underestimation when real drivers deviate from predefined models. It combines RSS-based control overrides during simulation with a safe backup-policy check before executing a behavior policy.

E. Safety Mechanism

  • Risk may be underestimated when real-world drivers deviate from predefined models, while large safety margins can restrict flexibility in highly interactive environments.The paper characterizes real-world drivers as stochastic and noisily rational.Source: E. Safety Mechanism · p. 9
  • An RSS-based safety module checks simulated ego states and controller outputs, overriding a control signal when it fails the proper-response criterion.The fallback signal follows RSS proper response when the state is unsafe and the controller output does not meet that criterion.Source: E. Safety Mechanism · p. 9
  • A policy is executed only if at least one feasible safe backup policy exists; if none does, the planner signals failure and triggers low-level protection such as AEB.Backup choices can be prioritized to match the maneuver, such as using lane-change cancellation as the backup for a lane-change policy.Source: E. Safety Mechanism · p. 9

F. Policy Selection

Policy selection evaluates a limited set of closed-loop scenarios and selects the policy with the best weighted reward. The motion planner then optimizes a trajectory within an SSC, while semantic actions and a DCP-Tree organize behavior-planning choices.

F. Policy Selection

  • The behavior planner evaluates policies by the weighted rewards of their simulated scenarios and selects the best policy, with policy and critical-scenario evaluations carried out in parallel.The reward combines efficiency, safety, and navigation.Source: F. Policy Selection · p. 9

VI. TRAJECTORY GENERATION WITH SPATIO-TEMPORAL SEMANTIC CORRIDOR

  • The motion planner optimizes a smooth, safe, dynamically feasible trajectory within an SSC around the behavior planner’s state-sequence decision.Its objective adds trajectory-to-reference-state similarity to jerk minimization to reduce potential disagreement between the behavior and motion layers.Source: §VI · p. 9
  • The SSC-based trajectory objective combines smoothness and similarity costs in a quadratic program, adapts across semantic-element combinations, and feeds infeasibility back to behavior planning.The quadratic program can be solved with off-the-shelf solvers.Source: §VI · p. 10

A. Semantic-level Actions and DCP-Tree

  • Semantic actions combine available adjacent-lane targets with aggressive, moderate, or conservative velocity-controller settings.The lateral action set depends on driving context, and the longitudinal controller parameters define different maneuver styles.Source: A. Semantic-level Actions and DCP-Tree · p. 10
  • The DCP-Tree uses five semantic-action steps of one second each, with closed-loop simulation at 0.2-second resolution for a five-second planning horizon.The tree depth is five.Source: A. Semantic-level Actions and DCP-Tree · p. 10

B. Context-aware Forward Simulation Model

The context-aware forward simulation uses different controllers for lane keeping and lane changes. For lane changes, it chooses a collision-free reference path and a target gap, then applies feedback control with an RSS safety check.

B. Context-aware Forward Simulation Model

  • Lane keeping uses pure pursuit steering and IDM speed control, while lane changes couple steering and speed control to account for vehicles in current and adjacent lanes.For lane changes, the method uses reactive pure pursuit and a gap-informed velocity controller.Source: B. Context-aware Forward Simulation… · p. 10
  • When the target lane is occupied, the controller tracks an alternate collision-free path that moves toward the lane without crossing its marking while maintaining obstacle clearance.When the adjacent lane is free, the target lane centerline is used instead.Source: B. Context-aware Forward Simulation… · p. 10
  • The lane-change speed controller selects a target gap and desired longitudinal state using the target-lane leader and follower, preferred cruise velocity, and safe spacing and headway parameters.The desired state remains defined when either the target-lane leader or follower is absent.Source: B. Context-aware Forward Simulation… · pp. 10–11
  • Feedback control drives the ego vehicle toward the desired longitudinal state, while the commanded acceleration is limited by the IDM acceleration for ego-lane distance keeping.The controller output is the minimum of the tracking acceleration and the IDM acceleration aid.Source: B. Context-aware Forward Simulation… · p. 11
  • The lateral and longitudinal commands are checked against RSS proper response, and unsafe outputs are overridden with a safe control signal.The check is applied when the ego vehicle is RSS-dangerous at the current simulation step.Source: B. Context-aware Forward Simulation… · p. 11

C. Intention Estimator

The intention estimator assigns scores to candidate center-lines from lane geometry and the target vehicle’s motion history. Policy evaluation combines efficiency, safety, and navigation costs to rank candidate policies.

C. Intention Estimator

  • Candidate center-lines are scored from sampled lane positions and the target vehicle’s trajectory history, with the scores normalized into an intention distribution.The lane encoder uses 70 points per center-line; an LSTM encodes trajectory positions before an MLP scores each candidate.Source: C. Intention Estimator · pp. 11–12

D. Policy Evaluation

  • Each policy’s reward is a weighted sum of scenario rewards built from efficiency, safety, and navigation costs.The total cost is the negative reward, with weights balancing the cost terms.Source: D. Policy Evaluation · p. 12
  • Safety evaluation penalizes collisions and RSS-dangerous states, while navigation cost rewards matching the user’s command and the previous cycle’s decision.An RSS-dangerous state is penalized according to its difference from the safe velocity interval.Source: D. Policy Evaluation · p. 12

VIII. EXPERIMENTAL RESULTS

The experiments assess semantic-level actions and forward simulation on Argoverse, then examine how intention estimation affects those predictions. The method performs especially well when evaluated with multiple trajectory hypotheses.

1) Semantic-level action and forward simulation:

  • Semantic-level actions couple forward simulation to planning, enabling the planner to realize diverse future scenarios conditioned on the ego vehicle’s decision.The comparison tests whether this simplification loses fidelity against learning-based trajectory predictors.Source: 1) Semantic-level action and forward… · p. 13
  • The evaluation uses Argoverse motion forecasting data, predicting three seconds of target-vehicle motion from two seconds of observations and assessing six-hypothesis minADE and minFDE.The dataset’s training, validation, and test splits have no geographical overlap.Source: 1) Semantic-level action and forward… · p. 13
  • With one trajectory, the method achieves 2.58 ADE and 5.7 FDE; with multiple hypotheses, it achieves 1.41 minADE and 2.48 minFDE, outperforming all baselines.The single-hypothesis results outperform NN and LSTM ED but trail two state-of-the-art baselines; the multi-hypothesis comparison includes those baselines.Source: Table II · p. 13, 1) Semantic-level action and forward… · p. 14
  • Replacing the intention estimator with a uniform predictor significantly worsens ADE, FDE, minADE, and minFDE, while an oracle estimator yields only a relatively small improvement.The authors conjecture that predefined controllers become the accuracy bottleneck when intention estimates are highly accurate.Source: Table III · p. 13, 1) Semantic-level action and forward… · p. 14

2) Impact of coupling prediction and planning:

The benchmark tests whether a planner can merge around a broken-down vehicle while balancing travel efficiency and safety. Coupled prediction and planning is most advantageous in dense, aggressive traffic.

2) Impact of coupling prediction and planning:

  • The benchmark presents a broken-down vehicle ahead, requiring a lane change within limited preparation distance to avoid waiting in a hazard.Changing lanes also involves balancing the risk of sudden braking by a vehicle ahead against the risk of a rear collision if the maneuver takes too long.Source: 2) Impact of coupling prediction and… · p. 14, Figure 10 · p. 14
  • Simulated traffic varies in aggressiveness and willingness to yield, and those settings are unknown to the ego planner.The evaluation measures average velocity and RSS safety cost over the first 300 planning cycles, roughly 15 seconds.Source: 2) Impact of coupling prediction and… · pp. 14–15
  • The decoupled baseline predicts other vehicles without accounting for the ego vehicle’s future action, then evaluates action sequences using the same pipeline as EPSILON.This preserves interactions among other traffic participants while excluding the ego decision’s effect on them.Source: 2) Impact of coupling prediction and… · p. 15
  • In aggressive traffic with a 1.75 m cooperative range, EPSILON merges safely at 10.8 m/s with a safety cost of 16.8, while decoupled planning cannot find a gap and waits in the hazard.In less aggressive, more cooperative traffic, both systems merge successfully with high efficiency while staying safe.Source: Table IV · p. 14, 2) Impact of coupling prediction and… · p. 15

3) Impact of the safety mechanism:

The experiments assess the safety mechanism, computational cost, and real-world driving performance. The system runs at 20 Hz in simulation and achieves frequent lane-change success in dense traffic, though congestion and noisy observations remain challenging.

3) Impact of the safety mechanism:

  • Without the safety mechanism, the average safety cost reaches 147.2 in aggressive, less cooperative traffic—8.7 times EPSILON’s cost—even as the vehicle averages 10.7 m/s.The gap is small in loose, cooperative traffic but large under the more difficult condition.Source: 3) Impact of the safety mechanism · p. 15

4) Computation Time Cost:

  • Both planning layers stay within 50 milliseconds, allowing EPSILON to run stably at 20 Hz in simulation.Behavior-planner time rises with the number of surrounding vehicles, while motion-planner time remains basically constant.Source: 4) Computation Time Cost · p. 16, Figure 12 · p. 16

B. Qualitative Analysis

  • Across four real-world drives, lane-change success exceeds 80% overall, with comfortable acceleration ranges and human-like maneuvers observed.Trial 3, which encountered traffic jams, had a 62.5% lane-change success rate and three human interventions.Source: B. Qualitative Analysis · p. 16, Table V · p. 17
  • Congestion and sensor noise from occlusion can degrade evaluations and decisions; the authors also report that proactive tuning contributed to aggressive maneuvers.The reported interventions include mistimed active lane changes under noisy observations and plans blocked by aggressive drivers.Source: B. Qualitative Analysis · p. 17

1) Intelligent gap finding and merging:

EPSILON demonstrates flexible lane changes and cut-in handling in dense, interactive traffic while maintaining safety and avoiding overly conservative behavior. Its conclusion ties these behaviors to guided branching, a safety mechanism, and spatiotemporal semantic-corridor motion planning validated on a real vehicle.

1) Intelligent gap finding and merging:

  • Driving knowledge in semantic actions lets EPSILON adapt lane changes to traffic and find suitable merging gaps more flexibly than MPDM or pure rule-based methods.The reported comparison describes greater maneuverability than MPDM and greater generality than pure rule-based decision making.Source: 1) Intelligent gap finding and merging · p. 17

2) Interaction-aware overtaking:

  • During overtaking, a longitudinal evasive maneuver increases the gap from a competing follower while preserving room to abandon the maneuver safely.The example reports that the vehicle completes the merge after the follower stops accelerating, avoiding overly conservative behavior while keeping interaction within a safety envelope.Source: 2) Interaction-aware overtaking · p. 17

3) Low-speed automatic lane change in dense traffic:

  • In low-speed dense traffic, EPSILON completes two consecutive active lane changes despite large errors in the observed speed of a nearby following vehicle.The account reports that interaction consideration and the safety mechanism preserve safety without overly conservative behavior under this observation noise.Source: 3) Low-speed automatic lane change in… · pp. 17–18

4) Cut-in handling and automatic lane change:

  • EPSILON handles two aggressive cut-ins smoothly, then overtakes the vehicles and switches to a more efficient lane without leaving others no space.The passage attributes the handling to intention estimation and forward simulation, and notes comfortable stop-and-go behavior.Source: 4) Cut-in handling and automatic lane… · p. 18

IX. CONCLUSION

  • Real-vehicle validation in dense city traffic shows EPSILON is neither overly conservative nor over-aggressive around noisily rational traffic participants.The system combines guided branching, a safety mechanism for multi-agent forward simulation, and spatiotemporal semantic corridors for safe, comfortable executable trajectories.Source: §IX · p. 18

Related papers

Loading 2108.07993v1…