PaperlayerView exact revision on arXiv

Source-linked AI summary

A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning

Liting Sun, Cheng Peng, Wei Zhan, Masayoshi Tomizuka

arXiv:1707.02515v1cs.AIcs.LGcs.SYeess.SY

TL;DR

Long-term MPC can provide safe, feasible driving policies, but its computational complexity can be unacceptable for real-time use. The paper combines a learned policy layer with short-horizon optimization and representative features. Example scenarios demonstrate fast planning with long-term smoothness and safety.

  • Problem

    Long-term MPC can provide safe, feasible driving policies, but its computational complexity can be unacceptable for real-time implementation.

    Source: §I · p. 1

  • Method

    A neural-network policy layer imitates long-term MPC trajectories from representative features, while a short-horizon optimization layer tracks them and ensures short-term safety and feasibility.

    Source: §I · pp. 1–2

  • Results

    Examples of overtaking, car-following, and straight-going demonstrate fast planning of safe motions with long-term smoothness.

    Source: A. Overtaking Maneuver · p. 7, C. Straight Going · p. 7, §VI · p. 8

  • Takeaways & Limitations

    The authors conclude that the framework can imitate long-term MPC and plan safe motions extremely fast while retaining long-term smoothness.

    Source: §VI · p. 8

Abstract

from arXiv · show

For safe and efficient planning and control in autonomous driving, we need a driving policy which can achieve desirable driving quality in long-term horizon with guaranteed safety and feasibility. Optimization-based approaches, such as Model Predictive Control (MPC), can provide such optimal policies, but their computational complexity is generally unacceptable for real-time implementation. To address this problem, we propose a fast integrated planning and control framework that combines learning- and optimization-based approaches in a two-layer hierarchical structure. The first layer, defined as the "policy layer", is established by a neural network which learns the long-term optimal driving policy generated by MPC. The second layer, called the "execution layer", is a short-term optimization-based controller that tracks the reference trajecotries given by the "policy layer" with guaranteed short-term safety and feasibility. Moreover, with efficient and highly-representative features, a small-size neural network is sufficient in the "policy layer" to handle many complicated driving scenarios. This renders online imitation learning with Dataset Aggregation (DAgger) so that the performance of the "policy layer" can be improved rapidly and continuously online. Several exampled driving scenarios are demonstrated to verify the effectiveness and efficiency of the proposed framework.

I. INTRODUCTION

The section frames a central tension: long-term MPC can produce safe, efficient policies, but its computational cost can prevent real-time use. It proposes learning MPC’s policy and pairing it with short-horizon optimization, using representative features and online DAgger to support rapid improvement.

I. INTRODUCTION

  • The planning-and-control problem must jointly address safety and feasibility, driving quality, and limited response time.The section names smoothness, passenger comfort, and time-efficiency as desirable driving qualities.Source: §I · p. 1
  • Long-horizon MPC supports safety and feasibility, but its computational burden can make real-time implementation difficult.Terminating optimization at a runtime cap can sacrifice optimality and require a warm start.Source: §I · p. 1
  • The framework learns long-term MPC driving policies in a neural-network policy layer, then tracks their trajectories with short-horizon optimization for execution-layer safety and feasibility.This division avoids solving the most time-consuming long-horizon optimization online.Source: §I · p. 1
  • Sampled-DAgger runs expert MPC alongside the learned policy, labels sampled failure cases, aggregates them into the training data, and retrains the network.The stated aim is to improve the policy’s fidelity to long-term MPC.Source: §I · p. 2
  • The authors distinguish their approach from end-to-end systems that learn from human drivers and map sensory inputs directly to driving actions.They identify concerns about feasibility, safety, smoothness, policy optimality, redundant image information, and long training times that impede online DAgger.Source: §I · p. 2
  • The framework uses representative middle-level features and a small network to make rapid, continuous online DAgger improvement practical.The authors report that a single-hidden-layer static network can be trained in minutes on a CPU.Source: §I · p. 2

II. PRELIMINARIES

The planning-and-control module uses a policy layer to convert representative features into reference trajectories and an execution layer to optimize short-term control. Its long-term MPC formulation considers a two-lane scenario with a front car and an adjacent-lane car.

A. Overview of the Hierarchical Structure

  • The policy layer extracts representative features from perception and uses a neural network to generate reference trajectories that imitate long-term MPC.The execution layer solves a short-horizon optimization problem to generate control actions and further guarantee short-term feasibility and safety.Source: A. Overview of the Hierarchical… · pp. 2–3
  • The framework is described as responding extremely fast while achieving long-term smoothness and safety similar to the expert MPC policy.The overview assigns short-term feasibility and safety to the execution layer.Source: A. Overview of the Hierarchical… · p. 3

B. Long-Term Model Predictive Control

1) Objective Function:

The long-term MPC seeks fast progress toward a goal while balancing goal distance and control changes. It constrains vehicle motion to remain kinematically and dynamically feasible and collision-free.

1) Objective Function:

  • The objective penalizes distance to the goal and changes in control variables while seeking to drive as fast as the speed limit allows.The MPC aims to reach high-level goal positions as quickly as safety and feasibility permit.Source: 1) Objective Function · p. 3, 2) Constraints · p. 4

2) Constraints:

  • The MPC combines kinematic, dynamic, and collision-free safety constraints in a nonlinear optimization problem.It produces a long-horizon optimal full-state trajectory and corresponding control actions.Source: 2) Constraints · pp. 3–4
  • A bicycle model constrains the predicted states and controls; the initial predicted state matches the ego vehicle’s current state, with a 0.1-second time step.The total car length is L=Lr+Lf.Source: 2) Constraints · p. 3
  • The G-G diagram bounds longitudinal and lateral accelerations within a feasible region to enforce dynamic feasibility.The figure depicts acceleration boundaries for this feasibility constraint.Source: 2) Constraints · p. 3, Figure 3 · p. 3
  • Collision safety includes constraints on vehicle configuration near static obstacles such as curbs and safe distances from surrounding cars.The safe distance is defined as R=Lr+Lf+ϵ.Source: 2) Constraints · pp. 3–4
  • The MPC also imposes physical constraints on the steering angle; when safe and feasible, it may overtake a slow lead car if the adjacent lane is clear, otherwise following it.The stated examples describe behavior under the long-term MPC’s goal-seeking objective.Source: 2) Constraints · p. 4

C. Short-Term Optimization in the Execution Layer

The execution layer uses a short-horizon optimization to track the policy layer’s reference trajectory. Its horizon is shortened to reduce computation load.

C. Short-Term Optimization in the Execution Layer

  • The execution layer uses a horizon of Nshort=5 to reduce computation load.Its optimization problem is formulated similarly to the long-term MPC, with a much shorter horizon.Source: C. Short-Term Optimization in the… · p. 4
  • Instead of driving toward a goal point, the execution layer tracks the reference trajectory supplied by the policy layer.The trajectory is represented by reference positions across the short horizon.Source: C. Short-Term Optimization in the… · p. 4
  • The tracking cost represents error between the ego vehicle’s predicted positions and the reference trajectory.The supplied passage identifies the cost term as tracking error.Source: C. Short-Term Optimization in the… · p. 4

III. IMITATION LEARNING OF THE LONG-TERM OPTIMAL DRIVING POLICY

The long-term MPC optimization is generally too time-consuming for real-time applications because of its nonlinearities and equality and inequality constraints. The paper introduces a learning-based policy layer to imitate its optimal policy, aiming to improve time efficiency while maintaining optimality to some extent.

III. IMITATION LEARNING OF THE LONG-TERM OPTIMAL DRIVING POLICY

  • The long-term MPC is generally too time-consuming for real-time applications because it has nonlinearities and equality and inequality constraints.The paper frames this as a time-efficiency problem.Source: §III · p. 4
  • A learning-based policy layer imitates the optimal policy produced by the long-term MPC.The policy layer is described in Section II-A.Source: §III · p. 4
  • The learning-based policy layer is introduced to address time efficiency while maintaining optimality to some extent.The passage introduces this section as covering details of the learning process.Source: §III · p. 4

A. Feature Extraction

The feature set compresses perception into on-map motion, goal, and safety variables for the policy network. It represents lane geometry with sampled reference-center deviations and encodes the goal using the predicted horizon-end position.

A. Feature Extraction

  • The feature set covers on-map motion, goal, and safety variables, including the ego-vehicle’s current location and speed, goal distance, and relative positions and velocities to obstacles or cars.Source: A. Feature Extraction · p. 4
  • Lane geometry is represented by curb distances and ego-vehicle deviations from spatially sampled points along the proximate reference-lane center.The sampling intervals may increase with distance to cover a longer range and improve sensitivity to small yaw-angle changes.Source: A. Feature Extraction · p. 4
  • With m=10, the method generates ten additional on-map-motion-related features; its speed-related feature represents current speed and margin to the predefined speed limit.The feature definitions also describe x-direction vehicle velocity using the previous steering angle input.Source: A. Feature Extraction · pp. 4–5
  • The goal-related feature is the x-direction distance from the predicted horizon-end position to the goal point.Source: A. Feature Extraction · p. 5, Figure 4 · p. 5
  • The surrounding-vehicle safety features assume that the two surrounding vehicles move at constant velocities throughout the long-term horizon.Source: A. Feature Extraction · p. 5

B. Structure of the Policy Layer

The policy layer uses a single-hidden-layer neural network with extracted features as inputs. It learns proximate trajectories from the long-term MPC policy, reducing network size while supplying short-term references to the execution layer.

B. Structure of the Policy Layer

C. Learning via Sampled-DAgger Process

Sampled-DAgger iteratively improves the policy layer by selectively querying the long-term expert MPC on-policy and aggregating labeled examples. The procedure uses a pre-defined safety criterion to select cases and is reported to support rapid, continuous policy updates.

C. Learning via Sampled-DAgger Process

IV. GENERALIZATION TO MORE COMPLICATED SCENARIOS

The learned policy, trained in an abstract scenario, can be applied to more complex driving when combined with a decision-making module. Virtual curbs and adjacent cars adapt the abstract scenarios to multi-lane conditions.

IV. GENERALIZATION TO MORE COMPLICATED SCENARIOS

  • Continuous multi-lane driving can be reduced, according to environment settings and driving decisions, to a sequence of abstract scenarios where the learned policy applies.Source: §IV · p. 6
  • When lane markers can be crossed, virtual curbs define abstract two-lane scenarios for the selected reference lane.The described cases use different virtual curb configurations for the selected lanes.Source: §IV · p. 6
  • When lane markers cannot be crossed, virtual adjacent cars are added to discourage aggressive overtaking and keep the policy on the reference lane.Source: §IV · p. 6
  • Combining the decision-making module with virtual curbs and adjacent-lane cars supports generalization to more practical and complicated driving scenarios.Source: §IV · pp. 6–7

V. ILLUSTRATIVE EXAMPLES

The illustrative simulations test the proposed framework in overtaking and report the runtimes of its policy and execution layers. In the overtaking scenario, the learned policy plans ahead, unlike a short-horizon MPC run in the same setting.

V. ILLUSTRATIVE EXAMPLES

  • The illustrative examples are presented to demonstrate the framework’s effectiveness and efficiency.The overtaking comparison uses the same scenario for the proposed framework and short-horizon MPC.Source: §V · p. 7, A. Overtaking Maneuver · p. 7
  • Simulations used a 1/10-scale RC car at dt=0.1s; worst-case runtimes were 6.42×10−6s for the policy layer and 0.0766s for the execution layer.The authors state that real-time responses are guaranteed in this setup.Source: §V · p. 7

A. Overtaking Maneuver

  • Compared with same-scenario short-horizon MPC, the framework plans an early, smooth overtaking maneuver, while MPC brakes and steers sharply, repeatedly hits curbs, and stops.The learned policy starts accelerating and turning left early, then accelerates to the speed limit and returns to its reference lane when safe.Source: Figure 8 · p. 7, Figure 9 · p. 7, A. Overtaking Maneuver · p. 7

B. Car Following

In car-following and straight-going examples, the framework follows safely and maintains its lane, while customized DAgger improves a policy that initially fails in an unfamiliar straight-going scenario. The conclusions report fast, safe motions with long-term smoothness across the demonstrated scenarios and describe virtual features for more complex settings.

B. Car Following

  • In no-passing car-following, a virtual adjacent-lane car lets the same policy handle the scenario; the ego-vehicle decelerates to the front car’s end speed and remains collision-free.The virtual car is used when the left lane is unavailable for safety or because of road settings.Source: B. Car Following · p. 7, Figure 10 · p. 8
  • Data insufficiency leaves the car-following velocity profile insufficiently smooth, and the authors say more DAgger iterations may improve it.The caveat concerns the ego-vehicle’s velocity profile in Fig. 10.Source: B. Car Following · p. 7

C. Straight Going

  • With no surrounding vehicles, the learned policy accelerates from rest to the speed limit without departing from the reference lane.This is the straight-going example shown in Fig. 11.Source: C. Straight Going · p. 7, Figure 11 · p. 8
  • With customized DAgger, aggregated on-policy data improves a straight-going policy that initially fails and collides with curbs when unfamiliar features confuse it.The improved policy learns to go straight as fast as possible when safe.Source: C. Straight Going · pp. 7–8, Figure 12 · p. 8

VI. CONCLUSIONS

  • The conclusions characterize the framework as imitating long-term MPC and planning safe motions extremely fast with long-term smoothness across overtaking, car-following, and straight-going examples.These scenarios are presented as verification examples.Source: §VI · p. 8
  • Virtual features generalize the learned policy to more complicated driving scenarios; intersections and curvy roads are identified as future extensions.The paper also proposes studying interactions among multiple autonomous cars.Source: §VI · p. 8
Loading 1707.02515v1…