Source-linked AI summary
DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planning
Wenhao Yao, Zhenxin Li, Shiyi Lan, Zi Wang, Xinglong Sun, Jose M. Alvarez, Zuxuan Wu
TL;DR
Selection-based planners can evaluate multiple trajectories but still struggle to identify the best among many subtle alternatives and underrepresented maneuvers. DriveSuprim addresses these issues with coarse-to-fine filtering, rotation-based augmentation, and soft-label self-distillation, achieving strong results across NAVSIM and Bench2Drive. Its multi-stage extension, however, does not yield additional improvements and challenging corner cases remain open.
Problem
Selection-based planners must distinguish the safest trajectory among thousands of candidates, including subtle alternatives and rare turning scenarios.
Method
DriveSuprim combines coarse-to-fine candidate refinement, rotation-based augmentation, and soft-label self-distillation for end-to-end trajectory selection.
Results
DriveSuprim outperforms prior methods on NAVSIM and Bench2Drive benchmarks, with reported gains of 3.6% and 1.5% on NAVSIMv1 and NAVSIMv2 and scores of 83.02 Driving Score and 60.00 Success Rate on Bench2Drive.
Takeaways & Limitations
The framework improves discrimination of hard negatives and precise selection, including scenarios with sharp turns.
Takeaways & Limitations
Extending the two-stage filtering strategy to multiple stages yields no additional improvement, and challenging corner cases remain areas for improvement.
Abstract
from arXiv · showhide
Autonomous vehicles must navigate safely in complex driving environments. Imitating a single expert trajectory, as in regression-based approaches, usually does not explicitly assess the safety of the predicted trajectory. Selection-based methods address this by generating and scoring multiple trajectory candidates and predicting the safety score for each. However, they face optimization challenges in precisely selecting the best option from thousands of candidates and distinguishing subtle but safety-critical differences, especially in rare and challenging scenarios. We propose DriveSuprim to overcome these challenges and advance the selection-based paradigm through a coarse-to-fine paradigm for progressive candidate filtering, a rotation-based augmentation method to improve robustness in out-of-distribution scenarios, and a self-distillation framework to stabilize training. DriveSuprim achieves state-of-the-art performance, reaching 93.5% PDMS in NAVSIM v1 and 87.1% EPDMS in NAVSIM v2 without extra data, with 83.02 Driving Score and 60.00 Success Rate on the Bench2Drive benchmark, demonstrating superior planning capabilities in various driving scenarios.
Introduction
DriveSuprim addresses three weaknesses in selection-based planning: difficulty distinguishing subtle trajectory differences, directional bias against turns, and unstable binary safety decisions. It uses coarse-to-fine refinement, rotation-based augmentation, and self-distillation to improve trajectory selection.
- Regression-based methods predict a single expert-mimicking trajectory without explicitly evaluating multiple safety-critical alternatives.
- Selection-based methods generate and score diverse candidates using safety metrics, enabling selection of safer trajectories than single-trajectory regression.
- Selection models struggle to distinguish optimal trajectories from thousands of similar-looking alternatives because easy negatives dominate training supervision.
- Only 18% of NAVSIM ground-truth trajectories involve turns exceeding 30 degrees, contributing to weaker performance on large-angle turns.
- Binary safe/unsafe labels create hard thresholds that can make models overly sensitive to minor trajectory-feature changes.
- DriveSuprim progressively refines candidate selection and combines rotation-based augmentation with self-distillation to address these weaknesses.
Related Works
Related work progresses from modular and regression-based driving systems toward selection-based planning. DriveSuprim extends this paradigm with progressive candidate-space refinement and driving-specific augmentation for complex maneuvers.
- End-to-end methods unify perception and planning in one optimizable network that maps raw sensor inputs directly to driving trajectories.
- Selection-based methods score diverse candidate trajectories using safety-focused metrics such as PDM scores.
- Unlike prior single-shot selection from fixed candidate sets, DriveSuprim progressively narrows the candidate space to improve selection precision.
- Iterative refinement has been used in optical flow, motion estimation, and object detection, motivating progressive refinement for trajectory selection.
- DriveSuprim applies targeted corruption-based augmentation to address domain shifts associated with overrepresented straightforward driving trajectories and complex turns.
Methods
DriveSuprim selects trajectories through coarse filtering followed by fine-grained scoring, while rotation augmentation and self-distillation address directional imbalance and training stability.
- Preliminaries: The planner encodes sensor inputs and a fixed vocabulary of candidate trajectories, scores their quality, and selects the highest-scoring trajectory.Candidate quality can reflect imitation distance, safety, and traffic-rule metrics.
- Coarse-to-Fine Trajectory selection: Coarse filtering scores the full trajectory vocabulary and retains top-k candidates for subsequent refinement.The retained candidates and their refined features form the input to fine-grained fitting.
- Coarse-to-Fine Trajectory selection: Fine-grained scoring applies another Transformer decoder to filtered candidates containing many hard negatives, then selects the highest final-layer score.The refinement decoder produces scores at decoder layers and optimizes them against ground-truth trajectory scores.
- Rotation-based data augmentation: Rotation augmentation simulates ego-vehicle rotation by horizontally transforming sensor views, diversifying scenarios across vehicle orientations.For each scenario, the method samples θ from U[−Θ, Θ], shifts a pseudo-panoramic view, crops the input, and rotates the human trajectory by −θ.
- Self-distillation with Soft-labeling: Self-distillation stabilizes training by using teacher-generated soft labels while the student trains on original and augmented inputs.The teacher receives original data, whereas the student uses both original and augmented data; inference uses the teacher model to output planning trajectories.
Experiments
Experiments evaluate DriveSuprim on NAVSIM and Bench2Drive, with ablations and visual analyses examining its performance, real-time capability, and behavior in challenging turns.
- NAVSIM: 93.5% PDMS is achieved on NAVSIM v1 with the ViT-Large backbone, while ResNet34 reaches 89.9% PDMS and surpasses DiffusionDrive by 1.8%.On NAVSIM v2, DriveSuprim surpasses previous state-of-the-art methods by 1.7%, 0.9%, and 1.5% on EPDMS.
- Efficiency: 27.2 FPS with ResNet34 and 12.5 FPS with ViT-Large provide real-time planning capability.
- Bench2Drive: 83.02 Driving Score and 60.00 Success Rate are achieved on Bench2Drive, alongside 238.78 Efficiency and 20.89 Comfortness.Bench2Drive evaluates closed-loop planning on 220 short CARLA v2 routes.
- Ablation Studies: Multi-stage refinement improves PDMS by 1.0%, while augmented data and self-distillation add 0.3% and 0.4%, respectively.The ablation study evaluates the proposed modules against a baseline.
- Ablation Studies: Layer-wise scoring and trajectory filtering improve EPDMS by 0.7%, compared with 0.3% from increasing decoder layers from 3 to 6.This comparison attributes the larger gain to the coarse-to-fine mechanism rather than model-size increase.
- Turning Scenarios: DriveSuprim shows more pronounced improvements in turning scenarios than near-straight scenarios, while augmentation makes trajectory frequencies more similar across directions.Qualitative results include safer overtaking near a crossroad and smoother sharp-turn trajectories than Hydra-MDP++.
Conclusion
DriveSuprim is an end-to-end planning framework that combines coarse-to-fine selection with rotation-based augmentation and soft-label self-distillation. Experiments on NAVSIM and Bench2Drive show substantial improvements over prior methods, including better handling of hard negatives and sharp turns.
- DriveSuprim combines coarse-to-fine selection, rotation-based data augmentation, and soft-label self-distillation for end-to-end planning.
- The framework improves discrimination of hard negatives and precise trajectory selection, particularly in scenarios involving sharp turns.
- Experiments on NAVSIM and Bench2Drive show that DriveSuprim outperforms prior methods by a substantial margin.
A. More information about Datasets and Metrics
NAVSIM provides separate training and testing splits and evaluates trajectories with rule-based safety and driving metrics. Bench2Drive complements it with short closed-loop routes and multiple driving-quality metrics.
- NAVSIM contains 103k navtrain samples and 12k navtest samples for model training and testing.
- NAVSIM v1 uses PDMS, which aggregates simulator-derived rule-based subscores through multiplication and weighted averaging.
- NAVSIM v1 includes no collisions, drivable area compliance, ego progress, time-to-collision, and comfort subscores.
- NAVSIM v2 adds driving direction compliance, traffic light compliance, lane keeping, and extended comfort, while revising comfort to history comfort.
- Bench2Drive evaluates 220 short routes across 44 interactive scenarios using Success Rate, Driving Score, Efficiency, and Comfortness.
B. Supplementary Implementation Details
The supplementary implementation uses several image backbones, staged training schedules, metric-specific inference coefficients, and score-based trajectory selection. Final trajectory scores combine predicted metric scores before filtering and selection.
- DriveSuprim uses ResNet34, VoVNet, and ViT-Large image encoders with backbone-specific input resolutions.ResNet34 and VoVNet use 2048×512 inputs, while ViT-Large uses 1024 × 256 inputs and Depth Anything pretraining.
- Pretrained-backbone models train for 6 epochs, whereas the scratch-trained ResNet34 model trains for 10 epochs.EMA momentum schedules differ across the training pipelines.
- During inference, predicted scores across metrics are linearly combined into a final score used to filter and select trajectories.Imitation and multiplier metrics use logarithmic score transformations, while weighted-average metrics apply a coefficient before an additional logarithmic aggregation.
- λavg is 8.0 for NAVSIM v1 and 6.0 for NAVSIM v2.
C. Supplementary Experiment Result
Supplementary experiments identify effective refinement, soft-label, camera, self-distillation, and efficiency settings. Results indicate that the selected configuration improves refinement outcomes while preserving real-time planning capability.
- A 3-layer refinement Transformer decoder filtering 256 trajectories achieves the best refinement result.
- A 0.15 teacher soft-label clipping threshold and 1-meter soft imitation shift produce the best EPDM score.
- The 5-camera setting yields the best performance among the evaluated FOV settings.The five-camera configuration uses left, front-left, front, front-right, and right cameras.
- Self-distillation is compared with temperature-scaled cross-entropy and vanilla label smoothing to address hard binary labels.The comparison uses temperature 2.0 and a label-smoothing coefficient of 0.1.
- DriveSuprim adds approximately 7M parameters relative to Hydra-MDP++ while maintaining real-time planning capability.The reported FPS decrease from the parameter increase is described as acceptable.
D. Visualization
Visualizations compare DriveSuprim with a classic selection-based method across challenging interactions and sharp turns. The figures also show camera layouts and trajectory-distribution changes associated with rotation-based augmentation.
- In an overtaking-before-crossroad example, DriveSuprim completes the maneuver while the classic selection-based method turns left and may cause a collision.
- The qualitative results report precise trajectories in complex scenarios and accurate handling of sharp turns.
- The trajectory-distribution comparison counts high-score trajectories using ground-truth PDM scores and normalizes frequency by the most frequent trajectory.
- Figure 5 uses five cameras: front, front-left, left, front-right, and right.
- Figure 6 compares human ground truth, a classic selection-based trajectory, and DriveSuprim across challenging and sharp-turning scenarios.The ground-truth trajectory is green, the classic selection-based trajectory is red, and the DriveSuprim trajectory is blue.
E. Limitations and Future Work
DriveSuprim’s two-stage coarse-to-fine filtering improves performance on NAVSIM and Bench2Drive, while further multi-stage filtering adds no improvement and challenging cases remain open.
- Two-stage coarse-to-fine trajectory filtering enhances model performance on the NAVSIM and Bench2Drive benchmarks.
- The augmentation mitigates directional bias by increasing previously underrepresented turning trajectories in the dataset.
- Multi-stage extension of the filtering strategy yields no additional improvement.
- Performance remains improvable in challenging or corner-case scenarios.