PaperlayerView exact revision on arXiv

Source-linked AI summary

PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving

Jie Cheng, Yingbing Chen, Qifeng Chen

arXiv:2404.14327v1cs.RO

TL;DR

Learning-based planning remains less effective than rule-based planning, and imitation learning faces challenges including limited lateral behavior and shortcut learning. PLUTO combines a query-based architecture, differentiable auxiliary losses, and contrastive imitation learning with new augmentations; on nuPlan, it achieves state-of-the-art closed-loop performance and surpasses rule-based PDM for the first time.

  • Problem

    Learning-based planners have remained behind rule-based planning, while imitation learning also faces challenges including limited lateral behavior and shortcut learning.

    Source: §I · pp. 1–2

  • Method

    PLUTO combines a query-based architecture for lateral and longitudinal planning with differentiable auxiliary losses and contrastive imitation learning using new data augmentations.

    Source: §I · pp. 1–2

  • Results

    PLUTO achieves state-of-the-art closed-loop performance on nuPlan and surpasses rule-based planner PDM for the first time.

    Source: §I · p. 2

  • Takeaways & Limitations

    The authors present PLUTO’s closed-loop performance as a significant milestone for data-driven planning with imitation learning.

    Source: §I · p. 2

    PLUTO predicts a single trajectory for each dynamic agent; meaningful joint multimodal predictions and their efficient use in planning remain future work.

    Source: §VI · p. 13

Abstract

from arXiv · show

We present PLUTO, a powerful framework that pushes the limit of imitation learning-based planning for autonomous driving. Our improvements stem from three pivotal aspects: a longitudinal-lateral aware model architecture that enables flexible and diverse driving behaviors; An innovative auxiliary loss computation method that is broadly applicable and efficient for batch-wise calculation; A novel training framework that leverages contrastive learning, augmented by a suite of new data augmentations to regulate driving behaviors and facilitate the understanding of underlying interactions. We assessed our framework using the large-scale real-world nuPlan dataset and its associated standardized planning benchmark. Impressively, PLUTO achieves state-of-the-art closed-loop performance, beating other competing learning-based methods and surpassing the current top-performed rule-based planner for the first time. Results and code are available at https://jchengai.github.io/pluto.

I. INTRODUCTION

PLUTO addresses shortcomings in imitation-based planning with a query-based architecture, auxiliary losses, and contrastive learning with new augmentations. Evaluated on nuPlan, it achieves state-of-the-art closed-loop performance and surpasses the rule-based planner PDM for the first time.

I. INTRODUCTION

  • A query-based architecture fuses lateral and longitudinal planning to generate many proposals, targeting learning-based planners’ difficulty with lane changes and obstacle avoidance.The authors identify limited explicit lateral behavior modeling as a source of this difficulty.Source: §I · p. 1
  • Differentiable interpolation enables broad auxiliary losses with efficient batch-wise computation for vector-based models.Prior approaches used heatmap outputs or differentiable rasterizers, limiting their efficient application to vector-based models.Source: §I · p. 1
  • New augmentations target open-loop/closed-loop discrepancies and shortcut learning while regulating driving behavior and enhancing interaction learning.The paper adds techniques beyond perturbations, which teach recovery from small deviations, and dropout, which discourages shortcuts.Source: §I · p. 1
  • Contrastive Imitation Learning distinguishes original from modified inputs, such as inputs excluding leading vehicles, to address imitation learning’s difficulty recognizing causal relationships.The framework is designed to do so without substantially complicating training as reinforcement learning or a data-driven simulator would.Source: §I · p. 2
  • PLUTO achieves state-of-the-art closed-loop performance on nuPlan and surpasses the rule-based planner PDM for the first time.The authors describe this as the first time a learning-based planner has surpassed the existing state-of-the-art rule-based planner.Source: §I · p. 2

A. Imitation-based Planning

  • Mid-to-mid planning uses post-perception features and real-world training data, while many end-to-end approaches depend on high-fidelity simulation and face sim-to-real concerns.The paper’s framework is designed for planning and jointly models longitudinal and lateral behavior.Source: A. Imitation-based Planning · p. 2

B. Contrastive Learning

PLUTO frames planning as generating multiple AV trajectories and agent predictions from scene context, while encoding agent histories, static obstacles, and map polylines. Its contrastive-learning discussion extends representation learning to planning and implicit vehicle interactions.

B. Contrastive Learning

  • Contrastive learning compares similar and dissimilar representations, and PLUTO applies it to planning to improve driving-behavior prediction and model implicit vehicle interactions.The paper also introduces data augmentations to define the contrastive task by generating negative samples.Source: B. Contrastive Learning · pp. 2–3

A. Problem Formulation

  • PLUTO generates multiple multimodal AV planning trajectories and predictions for dynamic agents, with a scoring module selecting the final trajectory using scene context.The scene includes the AV, dynamic agents, static obstacles, a high-definition map, and traffic-related context.Source: A. Problem Formulation · p. 3

B. Input Representation and Scene Encoding

  • Agent histories are represented through changes between consecutive states, then condensed into embeddings with a neighbor-attention Feature Pyramid Network.The state features include position, heading, velocity, bounding-box dimensions, and an observation indicator.Source: B. Input Representation and Scene… · p. 3
  • Scene encoding combines AV, agent, obstacle, and polyline embeddings with global-position and semantic-attribute embeddings before Transformer encoding.The global positional embedding uses Fourier features; semantic attributes include agent types, lane speed limits, and traffic-light statuses.Source: B. Input Representation and Scene… · p. 4

C. Multi-modal Planning Trajectory Decoding

PLUTO decodes multimodal trajectories by combining lateral reference-line queries with longitudinal queries, then applying factorized attention. Teacher-forced supervision trains trajectory and score outputs, while an auxiliary prediction head predicts dynamic-agent trajectories.

C. Multi-modal Planning Trajectory Decoding

  • The decoder addresses multimodal driving behaviors by combining lateral and longitudinal actions in a semi-anchor-based structure, avoiding direct anchor-free queries associated with mode collapse and training instability.The paper motivates this decomposition with choices such as following a slower vehicle or changing lanes to overtake.Source: C. Multi-modal Planning Trajectory… · p. 4
  • Reference lines derived from nearby connected lanes form lateral queries, while learnable anchor-free queries represent longitudinal behaviors.The reference-line process searches lane topology, truncates connected centerlines to a uniform length, and resamples them consistently.Source: C. Multi-modal Planning Trajectory… · p. 4
  • Factorized self-attention integrates lateral and longitudinal information across queries while reducing the high computational complexity of direct self-attention over their combined set.The supplied passages state that direct attention becomes prohibitively costly as the numbers of reference lines and longitudinal queries increase.Source: C. Multi-modal Planning Trajectory… · p. 4
  • Decoder layers apply lateral and longitudinal self-attention plus query-to-scene cross-attention, then MLPs produce AV trajectory points and scores.Each decoded trajectory point has six channels: position, heading represented by cosine and sine, and velocity.Source: C. Multi-modal Planning Trajectory… · p. 4
  • Teacher forcing selects a target reference line and longitudinal query from the ground-truth endpoint, then trains trajectory regression and score classification with smooth L1 and cross-entropy losses.A two-layer MLP also produces a single-modal prediction for each dynamic agent; these predictions provide dense supervision and help eliminate unsuitable planning proposals during post-processing.Source: C. Multi-modal Planning Trajectory… · p. 5

D. Efficient Differentiable Auxiliary Loss

PLUTO adds differentiable auxiliary penalties for constraints such as staying within the drivable area and avoiding collisions. Its method queries cost maps through interpolation, enabling batch-wise computation without rendering trajectories into images.

D. Efficient Differentiable Auxiliary Loss

  • Differentiable interpolation supports auxiliary training constraints because imitation learning alone may not prevent collisions or departures from the drivable path.The paper contrasts its approach with differentiable rasterization, whose computational and memory demands limit output resolution, time intervals, and planning horizons.Source: D. Efficient Differentiable Auxiliary… · p. 5
  • The method represents non-drivable areas with an ESDF cost map and obtains signed distances for vehicle covering circles by projection and bilinear interpolation.A penalty is applied when a circle's signed distance falls below its radius; epsilon sets a safety threshold.Source: D. Efficient Differentiable Auxiliary… · pp. 5–6
  • The loss calculation is differentiable and efficient batch-wise, and can use continuously queryable cost representations beyond ESDFs, such as potential fields.The approach also applies to collision penalties with a change to the cost-map construction.Source: D. Efficient Differentiable Auxiliary… · p. 6

E. Contrastive Imitation Learning Framework

CIL uses positive and negative augmentations to shape latent representations, with six scenario changes designed to regulate driving behavior and interaction learning.

Contrastive Imitation Learning Framework

  • CIL pairs positive and negative samples, projects their embeddings, increases agreement with the positive, decreases agreement with the negative, then decodes original and positive samples for imitation and auxiliary losses.Positive augmentations preserve the original ground truth; negative ones alter causal structure and can make it invalid.Source: E. Contrastive Imitation Learning… · p. 6
  • For each minibatch, the framework creates three times as many samples and applies a softmax-based triplet contrastive loss across their embeddings.The original and positive samples use the unmodified ground-truth trajectory; negative samples contribute only to contrastive learning because their original ground truth may be invalid.Source: E. Contrastive Imitation Learning… · p. 6

Data augmentations

  • The six carefully crafted augmentation functions define the contrastive task and are designed with minimal inductive bias for broad applicability.The framework is intended to provide implicit reward signals reinforcing fundamental driving principles.Source: E. Contrastive Imitation Learning… · pp. 6, 8
  • State perturbation and non-interactive-agent dropout are positive augmentations intended to teach recovery from small deviations and discourage imitation of irrelevant agents.Interactive agents are identified by whether their future bounding boxes intersect the AV’s trajectory.Source: E. Contrastive Imitation Learning… · pp. 6–7
  • Negative augmentations remove leading or interactive agents, insert a leading vehicle into the AV’s path, or invert traffic lights to train responses to altered interactions and rules.The stated aims include learning leading-following behavior, less intuitive interactions, and adherence to traffic-light rules.Source: E. Contrastive Imitation Learning… · pp. 7–8

F. Planning and Post-processing

PLUTO selects a deterministic trajectory from its multimodal outputs through closed-loop simulation and rule-based scoring, without altering the selected trajectory.

Planning and Post-processing

  • PLUTO first retains the top K multimodal trajectories by confidence score, then simulates each using an LQR tracker and kinematic bicycle model.The authors assess simulated rollouts rather than direct planned trajectories to narrow discrepancies between plans and execution.Source: F. Planning and Post-processing · p. 8
  • A rule-based evaluator scores rollouts for progress, comfort, and traffic-rule adherence, uses agent predictions to exclude at-fault collisions, and combines its score with learned confidence.The final trajectory maximizes the combined score, with α as a fixed weighting factor.Source: F. Planning and Post-processing · p. 8
  • The post-processing module is a trajectory selector, not an optimizer: it leaves the original planning trajectory unchanged.The authors describe it as a proxy for injecting human preference or control and as a lower-bound safety assurance.Source: F. Planning and Post-processing · p. 8

A. Experiment Setup

The experiments use nuPlan’s real-world dataset and closed-loop benchmark to compare PLUTO with rule-based, pure-learning, and hybrid planners.

Experiment Setup

  • nuPlan contains 1,300 hours of real-world driving across up to 75 labeled scenario types; evaluation uses Val14’s 1,090 scenarios from 14 types.The standardized training split contains 1M frames, and a few scenarios that initialized as failed in reactive simulations are filtered out.Source: A. Experiment Setup · p. 8
  • The study focuses on closed-loop performance, using the non-reactive closed-loop score as its primary overall metric.The benchmark also defines open-loop and reactive closed-loop scores; closed-loop scores combine multiple metrics, including collision, drivable-area, comfort, progress, speed-limit, and driving-direction measures.Source: A. Experiment Setup · pp. 8–9
  • Comparisons cover three baseline families: rule-based planners, pure-learning planners, and hybrid methods with post-processing.PDM-Closed is described as the 2023 nuPlan challenge winner and current state of the art; PlanTF is identified as the pure-learning state of the art.Source: A. Experiment Setup · p. 9

B. Implementation Details

PLUTO’s implementation uses a shared parameter setting for two variants, with training reported both with and without CIL.

Implementation Details

  • 45 hours with CIL and 22 hours without CIL were required to finish training.The reported setup used four RTX3090 GPUs, batch size 128, and 25 epochs.Source: B. Implementation Details · p. 9
  • Training used AdamW with weight decay 1e−4, a learning rate warmed to 1e−3 over three epochs and then cosine-decayed, and loss weights of 1.0.Planning and historical-data horizons were set to 8 seconds and 2 seconds, respectively.Source: B. Implementation Details · p. 9
  • Table I lists parameters for PLUTO† and PLUTO, while Table II presents Val14 closed-loop results with all metrics higher-is-better.The supplied table captions do not state individual parameter values or benchmark outcomes.Source: Table I · p. 9, Table II · p. 10

V. RESULTS AND DISSCUSION

PLUTO surpasses the leading rule-based planner PDM-Closed for the first time, while PLUTO† outperforms prior pure-learning baselines. The methods’ performance closely aligns with the log-replay expert, and PLUTO is designed to support lateral maneuvers.

A. Comparison with State of the Art

  • 94.13 to 96.18 (Collisions), 90.73 to 93.28 (TTC), and 96.79 to 98.53 (Drivable) are PLUTO†’s improvements over PlanTF.The passage reports improvements across nearly all evaluated metrics, particularly safety-related metrics.Source: A. Comparison with State of the Art · p. 10
  • PLUTO surpasses PDM-Closed, the current state-of-the-art rule-based model, for the first time; its performance closely aligns with the log-replay expert (93.68 vs. 93.21).The passage describes PLUTO as attaining the highest scores across all baselines.Source: A. Comparison with State of the Art · p. 10
  • PLUTO’s longitudinal-lateral query architecture supports maneuvers such as lane changes, which are restricted in PDM-Closed’s speed-planning focus.The authors say this behavioral advantage will be illustrated in the qualitative results.Source: A. Comparison with State of the Art · p. 10

B. Qualitative Results

PLUTO’s qualitative examples show varied interactive driving, while ablations report gains from its training components and design choices. The final M5 variant has the best overall performance, with a comfort trade-off tied to more safety-checker emergency stops.

B. Qualitative Results

  • Across five test scenarios, PLUTO changes lanes, stops at a red light, yields around an obstacle, and bypasses a stationary vehicle before returning to its route.Fig. 5 shows four snapshots at five-second intervals in each 15-second scenario.Source: B. Qualitative Results · p. 10, Figure 5 · p. 11
  • Other examples show PLUTO waiting for a safe left turn in heavy traffic and overtaking a slower vehicle, behaviors the authors describe as human-like.The paper summarizes the scenarios as diverse, interactive driving and contrasts them with simplistic speed-planning methods.Source: B. Qualitative Results · p. 10

C. Ablation Studies

  • M1’s state dropout improves nearly all metrics over M0, while M2 raises Collision from 97.37 to 97.98 and Drivable from 95.92 to 98.38.M3 adds a reference-line-free decoding head for situations such as parking lots without reference lines.Source: C. Ablation Studies · p. 11
  • M4’s contrastive imitation learning raises performance from 90.69 to 91.66, approaching the expert’s performance.The passage identifies M4 as trained with the proposed contrastive imitation learning framework.Source: C. Ablation Studies · p. 11
  • M5 attains the best overall performance, with a minor Comfort trade-off from more safety-checker emergency stops that enhance safety-related metrics.The supplied passages do not give M5’s overall score.Source: C. Ablation Studies · pp. 11–12
  • Among four tested variants, 12 longitudinal queries performs best; the authors say more queries can become redundant for the 8-second planning horizon.Ablations use a nuPlan subset with 20 scenarios for each of 14 scenario types.Source: C. Ablation Studies · pp. 11–12
  • The authors select K = 20 for coarse trajectory selection and α = 0.3 for combining rule-based and learning-based scores.The combined scoring approach outperforms either the rule-based-only score or the purely learning-based score; the learning-based score improves on α = 0.Source: C. Ablation Studies · p. 12, Table VI · p. 12
  • Learned agent prediction outperforms PDM-Closed’s constant-velocity prediction, although the learned model predicts only one modal trajectory per agent.The authors say the single-modal prediction works well in practice.Source: C. Ablation Studies · pp. 12–13, Table VII · p. 12

VI. CONCLUSION

PLUTO combines a query-based architecture, differentiable-interpolation auxiliary loss, and contrastive imitation learning with data augmentation. On real-world driving data, it sets a new closed-loop benchmark and surpasses the previously best-performing rule-based planner; the paper identifies single-trajectory agent prediction as a limitation.

VI. CONCLUSION

  • PLUTO combines query-based architecture, differentiable-interpolation auxiliary loss, contrastive imitation learning, and data augmentation to address imitation-learning challenges.The architecture supports longitudinal and lateral driving behaviors, while the training framework is intended to improve desired behaviors and interaction understanding.Source: §VI · p. 13
  • On real-world driving datasets, PLUTO sets a new closed-loop performance benchmark and surpasses the previously best-performing rule-based planner.Source: §VI · p. 13
  • The approach predicts one trajectory per dynamic agent; meaningful joint multimodal prediction and efficient integration into planning remain future research areas.Post-processing cannot handle cases where all generated trajectories are unusable.Source: §VI · p. 13
Loading 2404.14327v1…