PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving

Zhiyu Huang, Haochen Liu, Jingda Wu, Chen Lv

arXiv:2207.10422v2cs.RO

TL;DR

Autonomous-driving systems often separate prediction from planning and rely on manually specified cost functions, despite the need for safe, smooth, and socially compliant trajectories. DIPP integrates a neural predictor with a differentiable nonlinear optimizer and learnable costs, and experiments show stronger open- and closed-loop performance than baseline approaches, with joint training outperforming separate prediction-planning training.

  • Problem

    Prediction and planning are often separated, while planning cost functions are difficult to specify and tune for safe, smooth, socially compliant autonomous driving.

  • Method

    DIPP jointly trains a Transformer-based predictor and differentiable optimization-based motion planner, allowing planning loss to update predicted agent trajectories and cost weights.

  • Results

    DIPP outperforms baseline methods in open- and closed-loop tests, while joint prediction-planning training outperforms a separate trained prediction module.

  • Takeaways & Limitations

    The framework provides planning-aware prediction and maintains performance under closed-loop distributional shift in complex urban driving scenarios.

Abstract

from arXiv · show

Predicting the future states of surrounding traffic participants and planning a safe, smooth, and socially compliant trajectory accordingly is crucial for autonomous vehicles. There are two major issues with the current autonomous driving system: the prediction module is often separated from the planning module and the cost function for planning is hard to specify and tune. To tackle these issues, we propose a differentiable integrated prediction-planning framework (DIPP) that can also learn the cost function from data. Specifically, our framework uses a differentiable nonlinear optimizer as the motion planner, which takes as input the predicted trajectories of surrounding agents given by the neural network and optimizes the trajectory for the autonomous vehicle, enabling all operations to be differentiable, including the cost function weights. The proposed framework is trained on a large-scale real-world driving dataset to imitate human driving trajectories in the entire driving scene and validated in both open-loop and closed-loop manners. The open-loop testing results reveal that the proposed method outperforms the baseline methods across a variety of metrics and delivers planning-centric prediction results, allowing the planning module to output trajectories close to those of human drivers. In closed-loop testing, the proposed method outperforms various baseline methods, showing the ability to handle complex urban driving scenarios and robustness against the distributional shift. Importantly, we find that joint training of planning and prediction modules achieves better performance than planning with a separate trained prediction module in both open-loop and closed-loop tests. Moreover, the ablation study indicates that the learnable components in the framework are essential to ensure planning stability and performance.

I. INTRODUCTION

Autonomous driving requires safe, socially compatible, and human-like decisions, but existing approaches separate prediction from planning or sacrifice explicit safety and reliability. DIPP integrates these components while learning planning costs from real-world driving data.

  • Safe, socially compatible, and human-like decisions are fundamental capabilities for autonomous vehicles.
  • Traditional autonomous-driving stacks treat prediction and planning as separate sequential modules despite their interdependence in interactive urban traffic.
  • End-to-end holistic models implicitly handle prediction-planning interactions but lack safety guarantees and reliability for safety-critical decision-making.
  • DIPP makes prediction aware of downstream planning while preserving the established prediction-planning structure and learning from planning error.
  • The framework integrates prediction and planning, learns the cost function from real-world data, and is validated in both open-loop and closed-loop driving tests.
  • Existing trajectory-prediction models often forecast agents independently, which can be computationally inefficient and produce inconsistent results.

B. Motion planning

The paper situates DIPP between interpretable optimization-based planning and learning-based interaction modeling. Its planner explicitly optimizes the AV trajectory using predicted agent states, scene context, and a weighted cost function.

  • B. Motion planning: Learning-based neural planners are efficient but suffer from limited interpretability, generalization, stability, and safety guarantees.
  • B. Motion planning: Interactive driving requires joint reasoning about other agents’ future behaviors and the AV’s plan to produce active, human-like decisions.
  • B. Motion planning: Cross-entropy planning methods can require repeated prediction queries for sampled action sequences, potentially slowing planning.
  • B. Motion planning: Holistic end-to-end models capture interactions implicitly but cannot explicitly compute the planning-prediction feedback loop from offline training or guarantee safety.
  • B. Motion planning: The scene representation includes the AV, diverse interacting participants, historical agent states, and contextual information such as maps and traffic lights.
  • B. Motion planning: DIPP formulates motion planning as nonlinear least squares over AV actions, using predicted agent states and weighted individual costs.

B. Multi-agent prediction and initial plan

DIPP predicts multimodal joint futures and an initial AV plan from agent histories and scene context, then passes these outputs to a differentiable optimizer. Planning loss gradients update both predicted behaviors and cost weights.

  • B. Multi-agent prediction and initial plan: The predictor uses historical agent states and scene context as its two input data types.
  • B. Multi-agent prediction and initial plan: LSTM encoding is used for short-term agent histories because it improves final prediction performance and computational efficiency relative to Transformers.
  • B. Multi-agent prediction and initial plan: The differentiable framework combines predicted surrounding-agent states, an initial AV plan, and a learnable cost function to explicitly optimize the AV trajectory.
  • B. Multi-agent prediction and initial plan: Two interaction graphs model agent-agent and agent-map relationships using attention-based encoders.
  • B. Multi-agent prediction and initial plan: The decoder predicts possible future trajectories for surrounding agents and future AV control actions, translating AV actions into trajectories with a kinematic model.
  • B. Multi-agent prediction and initial plan: The planner minimizes squared weighted residual costs, and final planning error backpropagates through optimization to predicted agent states and cost weights.
  • B. Multi-agent prediction and initial plan: The AV state is derived from acceleration and steering controls through a differentiable kinematic bicycle model.

3) Cost function:

The cost function combines travel efficiency, ride comfort, traffic-rule adherence, and safety. Safety focuses computation on interactive agents identified in the Frenet frame, while distances are calculated in Cartesian coordinates.

  • Weights: The cost function contains weighted feature terms evaluated across the planning horizon, with traffic-rule costs sampled every other timestep to reduce computation.
  • Travel efficiency: Travel-efficiency cost encourages reaching the destination quickly without exceeding the lane speed limit.It is defined from the AV speed and the speed limit.
  • Ride comfort: Ride comfort is represented through longitudinal acceleration, longitudinal jerk, steering angle, and steering change rate.
  • Traffic rules: Traffic-rule costs promote staying near the lane centerline, following lane direction, and obeying traffic lights.Red-light violations are represented with a hinge-loss penalty and can receive a large cost weight.
  • Safety: Safety identifies interactive agents whose predicted positions enter the route’s conflict area, then evaluates safe distances for those agents.Agents are projected into the Frenet frame for interaction selection, while distance calculations use the Cartesian frame.
  • Safety: A hinge loss ensures the safety distance is sufficiently large, with the minimum requirement defined by agent lengths plus a safety gap.

4) Differentiable optimization:

The planner solves a nonlinear optimization problem with differentiable Gauss–Newton iterations. A fractional update step mitigates divergence, allowing prediction outputs and cost weights to participate in end-to-end differentiation.

  • Optimization: Gauss–Newton solves the nonlinear planning problem through iterative least-squares optimization.Each iteration linearizes the nonlinear objective around the current control variables.
  • Optimization: The linearized system is solved with Cholesky decomposition of the normal equations to obtain the control update.
  • Optimization: Using a step size 0 < α < 1 applies only part of the update and mitigates divergence when the objective does not decrease monotonically.
  • Differentiability: Because the optimization is fully differentiable, prediction components and cost-function weights can be integrated into the planner.The resulting architecture supports differentiation through the planning procedure.

D. Learning process

DIPP trains a prediction network and differentiable planner together using real-world driving data. The planner uses likely joint predictions and an initial AV plan, while losses update both network parameters and cost weights.

  • Forward process: The planner receives other agents’ predictions, the highest-probability initial plan, and the cost function, then iteratively outputs an optimized AV trajectory.
  • Losses: Training combines prediction, score, imitation, and planning-cost losses in an end-to-end objective.The four terms respectively supervise agent trajectories, future probabilities, AV imitation, and planning behavior.
  • Prediction training: Prediction loss uses the joint future whose summed displacement error across agents is smallest, with smooth L1 losses applied to that future.
  • Prediction training: The scoring loss trains probabilities for alternative futures, where p_k denotes the probability of future k.
  • Planning training: The planner uses most-likely agent trajectories for safety evaluation and the AV’s predicted control actions for initialization, then compares its output with the ground truth for imitation loss.
  • Optimization: Algorithm 1 backpropagates the total loss to update the prediction parameters and cost weights with Adam.

A. Dataset

The framework is evaluated on a large-scale urban-driving dataset using open-loop and closed-loop tests. The evaluation covers safety, traffic-rule compliance, vehicle dynamics, human-driving similarity, and prediction accuracy.

  • Dataset: Waymo Open Motion Dataset contains 103 354 unique 20-second urban-driving scenes with map data and agent tracks.The selected subset preserves the original data distribution through random sampling.
  • Dataset: Frames use a 7 s window with 2 s of history and a 5 s planning/prediction horizon, sliding with a 1-second stride.
  • Evaluation: Open-loop testing compares each planned AV trajectory with the ground-truth future, while closed-loop testing uses a log-replay simulator that executes the first planned action at each step.
  • Metrics: Safety is measured by collision rate, using planned AV trajectories against ground-truth agent futures in open loop and simulator collisions in closed loop.
  • Metrics: Traffic-rule violations count frames in which the AV deviates from its route or passes a red-light stop line.
  • Metrics: Vehicle-dynamics metrics average absolute longitudinal acceleration, longitudinal jerk, and lateral acceleration over each scene.
  • Metrics: Human-driving similarity uses position errors from open-loop plans or closed-loop rollouts at specified future timesteps, while prediction uses joint ADE and FDE.

C. Comparison baselines

The comparison evaluates DIPP against imitation-learning, reinforcement-learning, and model-based planning baselines, while specifying the network outputs, learnable costs, and training and testing settings.

  • Baselines: The study compares DIPP with vanilla imitation learning, imitation learning with a prediction sub-task, separated prediction-planning, CQL, and IDM.The baselines cover direct policy learning, multi-task prediction, separately trained prediction and planning, offline reinforcement learning, and model-based control.
  • Baselines: Vanilla imitation learning directly generates one AV trajectory from the scene context, without predicting other agents.It uses the same dataset and hyperparameters as the proposed method for a fair comparison.
  • Model outputs: The network outputs 3 possible joint futures for all agents and their probabilities.For other agents, the model predicts displacements relative to their current locations.
  • Model outputs: A small MLP learns cost-function weights from a fixed dummy input, while traffic-light and collision weights remain fixed at large values.The fixed safety-related weights are larger than two orders of magnitude and are unlearnable.
  • Training and testing: Training uses 20 epochs with 5 epochs of planner-free pretraining, while testing permits up to 50 motion-planner iterations.Training uses batch size 32 and Adam with an initial learning rate of 2e-4; testing uses step size 0.2 and tolerance 1e-2.

V. RESULTS AND DISCUSSIONS

The results show that DIPP produces multimodal, planning-aware predictions and safe, rule-compliant trajectories in varied urban interactions. Open-loop comparisons indicate that integrating prediction with planning improves alignment with human-like behavior over separate prediction results.

  • Multi-modal joint prediction: DIPP predicts multiple futures for interacting agents, assigning higher probability to the joint trajectory closest to ground truth in one representative scenario.In another unsignalized-intersection scenario, the predicted AV behavior varies with the other agent’s predicted behavior.
  • Qualitative results: The planner handles smooth turns, pedestrian yielding, red lights, and unprotected left turns while producing safe, traffic-rule-compliant trajectories.The qualitative examples include vehicles, cyclists, and pedestrians.
  • Qualitative comparison: In open-loop comparisons, planner outputs adhere to road structure whereas unconstrained neural-network trajectories gradually deviate from the lane centerline.The comparison covers interactions with a vehicle and a cyclist.
  • Qualitative comparison: DIPP produces planning-aware predictions that help the planner generate trajectories closer to human driving than separate prediction results.In a merging scenario, the predicted vehicle yields at a stop sign; with a cyclist, the planner slows down to avoid potential risk even when the prediction is not exact.
  • Quantitative results: Open-loop evaluation reports lower collision and traffic-rule violation rates than imitation-learning methods, while planning-based methods provide lower acceleration and jerk.Imitation-learning methods also show higher off-route rates and worse lateral acceleration, whereas multi-agent prediction imitation learning has the smallest planning error.

B. Closed-loop test

The closed-loop evaluation tests replanning in replayed urban scenes, showing that the integrated method handles interactive and adversarial situations while its ablations clarify the components and settings needed for reliable planning.

  • Evaluation setup: Closed-loop testing rolls out planned AV trajectories for 15 seconds in 100 replayed scenes while other agents follow their original tracks.The AV replans every 0.1 seconds and executes the first action from each planned trajectory.
  • Closed-loop behavior: The framework handles cruising, traffic lights, smooth turns, and adversarial vehicle actions while avoiding collisions in the reported scenarios.The adversarial actions arise when other vehicles respond to the AV deviating from its original track.
  • Baseline comparison: Imitation-learning methods and CQL cannot complete a single test scene, whereas planning-based methods perform significantly better on the offline evaluation.The reported explanation for imitation-learning failures is distributional shift and compounding errors; CQL is described as difficult to train and label.
  • Baseline comparison: The separate prediction-planning baseline performs similarly but worse than the proposed method, highlighting the advantage of integrating prediction with planning.The comparison is reported in the closed-loop evaluation discussion.
  • Ablation study: Learnable initialization is most important for open-loop planning closeness to human trajectories, while prediction becomes more critical for closed-loop safety and human likeness.The paper attributes the closed-loop emphasis to interactive scenarios requiring accurate prediction of other agents’ future states.
  • Ablation study: All three learnable components—initialization, cost function, and prediction—are needed to maintain planning performance, stability, safety, and reliability.Removing initialization or cost learning can destabilize optimization and make viable solutions harder to find.

D. Discussions

DIPP integrates prediction, learnable costs, and differentiable planning into an end-to-end framework trained on real-world data. It shows strong open- and closed-loop performance, outperforming separated approaches while remaining limited by runtime and single-mode behavior prediction.

  • Discussion: DIPP integrates prediction, cost-function learning, and optimization-based planning into a fully differentiable, decision-focused pipeline.The framework trains the whole pipeline to optimize final planning performance directly from real-world driving data.
  • Discussion: DIPP outperforms separated planning-and-prediction methods in both planning and prediction performance, supporting joint end-to-end training.This advantage is reported in both open-loop and closed-loop evaluations.
  • Discussion: The method demonstrates strong open-loop and closed-loop performance and robustness against distributional shifts and adversarial agents despite offline training.Closed-loop testing also indicates better performance than the pipeline method, while planning-based methods outperform imitation-learning methods in that setting.
  • Limitations and future work: Testing runtime averages 1.78 s per sample, with Jacobian computation for least-squares optimization identified as the framework’s bottleneck.The prototype runs in 0.012 s during training and 1.78 s during testing under the reported hardware and iteration settings.
  • Limitations and future work: The framework uses only the most probable behavior prediction, leaving uncertainty and multimodality for future contingency-planning extensions.The authors propose incorporating all modalities and timestep uncertainties into future planner inputs.
  • Discussion: The framework uses a Transformer-based predictor and differentiable motion planner to produce planning-aware predictions and autonomous-vehicle trajectories.Predicted agent trajectories, an initial plan, and learnable costs are passed into the differentiable optimizer.
Loading 2207.10422v2…