Source-linked AI summary
Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Driving
Ziying Song, Caiyan Jia, Lin Liu, Hongyu Pan, Yongchang Zhang, Junming Wang, Xingyu Zhang, Shaoqing Xu, Lei Yang, Yadan Luo
TL;DR
End-to-end autonomous driving can suffer from unstable one-shot trajectories and occlusion-sensitive single-frame perception. MomAD introduces trajectory and perception momentum through TTM, MPI, and robust denoising, and reports stronger long-term consistency and lower collision rates across evaluations.
Problem
One-shot trajectory prediction may produce unstable control and remain vulnerable to occlusions when perception relies on single-frame observations.
Method
MomAD uses Hausdorff-distance TTM for historical trajectory matching, MPI for historical query interaction, and robust instance denoising for noisy perception features.
Results
MomAD maintains long-term consistency for at least 3 seconds, reduces collision rate by 26% and TPC by 0.97m versus SparseDrive over 6 seconds, and improves Bench2Drive success rate by up to 16.3%.
Takeaways & Limitations
Trajectory and perception momentum provide a planning framework that improves temporal consistency and robustness across standard, turning, and closed-loop driving evaluations.
Abstract
from arXiv · showhide
End-to-end autonomous driving frameworks enable seamless integration of perception and planning but often rely on one-shot trajectory prediction, which may lead to unstable control and vulnerability to occlusions in single-frame perception. To address this, we propose the Momentum-Aware Driving (MomAD) framework, which introduces trajectory momentum and perception momentum to stabilize and refine trajectory predictions. MomAD comprises two core components: (1) Topological Trajectory Matching (TTM) employs Hausdorff Distance to select the optimal planning query that aligns with prior paths to ensure coherence;(2) Momentum Planning Interactor (MPI) cross-attends the selected planning query with historical queries to expand static and dynamic perception files. This enriched query, in turn, helps regenerate long-horizon trajectory and reduce collision risks. To mitigate noise arising from dynamic environments and detection errors, we introduce robust instance denoising during training, enabling the planning model to focus on critical signals and improve its robustness. We also propose a novel Trajectory Prediction Consistency (TPC) metric to quantitatively assess planning stability. Experiments on the nuScenes dataset demonstrate that MomAD achieves superior long-term consistency (>=3s) compared to SOTA methods. Moreover, evaluations on the curated Turning-nuScenes shows that MomAD reduces the collision rate by 26% and improves TPC by 0.97m (33.45%) over a 6s prediction horizon, while closedloop on Bench2Drive demonstrates an up to 16.3% improvement in success rate.
1. Introduction
End-to-end driving integrates perception and planning, but one-shot trajectory prediction can be unstable and vulnerable to occlusions. MomAD addresses these issues by introducing trajectory and perception momentum for coherent, long-horizon planning.
- End-to-end frameworks integrate perception and planning but require long-horizon prediction of static environments and surrounding-agent interactions.
- Trajectory prediction is inherently stochastic because other road users’ intentions and road conditions are unpredictable.
- MomAD uses trajectory momentum to align candidate trajectories with prior predictions, reducing abrupt ego-path shifts and improving control consistency.
- MomAD uses perception momentum to aggregate historical context and attend to map elements and surroundings over time, capturing intentions missed by single-frame observations.
- TTM uses Hausdorff Distance to select trajectories aligned with prior planning, while MPI cross-attends historical queries to improve long-horizon generation.
- 26% lower collision rate and 0.97m lower TPC were reported against SparseDrive over a 6-second horizon on Turning-nuScenes.
2. Related Work
Prior end-to-end methods integrate upstream tasks but commonly use deterministic trajectory prediction. MomAD frames temporal planning consistency as a central problem and combines robust perception with momentum-aware planning.
- End-to-end autonomous driving methods learn from raw sensor data to generate planning trajectories or driving commands, eliminating manual feature extraction.
- MomAD’s architecture combines sparse scene representation, robust instance denoising, TTM, and MPI for momentum-aware planning.
- UniAD integrates perception, prediction, and planning information, while VAD uses vectorized representations and instance-level planning constraints.
- Deterministic trajectory prediction in prior methods fails to account for trajectory diversity and may introduce risks through intermediate regression results.
- The work focuses on temporal planning consistency and addresses it using momentum planning.
3. Method
MomAD builds sparse perception features and refines multimodal trajectories through TTM and MPI. TTM selects historically aligned candidates, while MPI injects temporal query context; denoising limits the effect of noisy perception features.
- 3. Method: Sparse perception encodes multi-view image features into instance features for road agents and map elements through detection, tracking, and online mapping blocks.
- 3.1. Topological Trajectory Matching (TTM): TTM explicitly selects the candidate trajectory that best matches the prior path among multimodal trajectories to ensure temporal coherence.
- 3.1. Topological Trajectory Matching (TTM): MomAD typically uses 6×3 candidate trajectories, representing six proposals for each left, right, and straight command.
- 3.1. Topological Trajectory Matching (TTM): TTM uses Hausdorff distance because it captures local and global trajectory structure through maximum deviation between point sets.
- 3.1. Topological Trajectory Matching (TTM): Selecting the minimum-distance trajectory enforces continuity and makes predictions less prone to sudden shifts.
- 3.2. Momentum Planning Interactor (MPI): MPI cross-attends the selected planning query with historical multimodal queries, enriching it with long-term spatiotemporal context before trajectory regeneration.
- 3.2. Momentum Planning Interactor (MPI): The refined query is combined with instance features and ego-anchor information, then processed by the planning head to generate updated multimodal trajectories.
- 3.3. Robust Instance Denoising: Controlled training perturbations and an encoder-decoder transformer learn to denoise unstable instance features from detectors and changing maps.
4. Experiments
Experiments on nuScenes, Turning-nuScenes, Bench2Drive, and nuScenes-C evaluate MomAD’s planning stability, long-horizon performance, closed-loop behavior, perception, and robustness. Across these settings, MomAD improves temporal consistency and several planning and perception metrics, with ablations examining denoising and historical-frame usage.
- Experimental Setup: TPC measures disparity between current and historical predicted trajectories, providing a dataset-wide statistical assessment of continuity across frames.The metric masks trajectories beyond their overlapping time period.
- Planning Results: 0.30m, 0.53m, and 0.78m are MomAD’s TPC improvements at 1s, 2s, and 3s on nuScenes, respectively.MomAD also reports 0.60m L2 error and 0.09% collision rate on nuScenes.
- Long Trajectory Prediction: 0.97m (32.45%) is MomAD’s 6s TPC decrease versus SparseDrive on Turning-nuScenes, alongside a 0.85m (25.30%) L2-error decrease and 0.26% collision-rate decrease.At 4s and 5s, MomAD also decreases TPC by 0.17m (11.04%) and 0.73m (31.60%), respectively.
- Closed-Loop Evaluation: 16.3% and 8.4% are MomAD’s success-rate improvements over the VAD multi-modal variant and SparseDrive on Bench2Drive.Comfortness, measuring trajectory smoothness, also improves by 7.2% and 5.3%.
- Ablation Studies: Historical-query fusion improves the t = 2 ablation to 0.76m L2 (Avg), 0.32% Col. (Avg), and 0.63m TPC, while adding more frames at t = 3 reduces the improvement.The reported explanation is uncertainty introduced by departing historical features.
5. Conclusion and Future Work
MomAD improves planning stability and robustness through momentum-aware trajectory prediction, while teacher forcing still limits trajectory diversity through mode collapse.
- 5. Conclusion and Future Work: MomAD uses trajectory and perception momentum with TTM and MPI to improve temporal coherence and enrich long-horizon context.The framework is evaluated on nuScenes and Turning-nuScenes for collision reduction and trajectory consistency against state-of-the-art methods.
- 5. Conclusion and Future Work: Teacher-forcing-based trajectory regression induces mode collapse, leaving a gap in trajectory diversity.The paper identifies diffusion models and speculative decoding as future directions for improving diversity while balancing efficiency.
A. Appendix
The appendix contains supplementary descriptions of MomAD, the Turning-nuScenes dataset, implementation, planning results, robustness analysis, and visualizations.
- A. Appendix: The supplementary material covers contributions, Turning-nuScenes construction, implementation details, additional planning results, robustness analysis, and visualizations.These topics are organized across Appendix A.1 through A.6.
A.1. Contributions
The contributions introduce MomAD as a momentum-planning framework and propose TTM and MPI to use historical guidance for more consistent planning.
- A.1. Contributions: MomAD is an end-to-end autonomous driving framework that uses trajectory and perception momentum to enhance current planning with historical guidance.The stated goal is to address temporal inconsistency, planning stability, and robustness.
- A.1. Contributions: TTM and MPI are proposed as modules for aligning candidate trajectories with past paths and supporting momentum-aware planning.The passage identifies Hausdorff Distance as part of TTM’s trajectory-alignment mechanism.
A.2. The Detail of Turning-nuScenes dataset
Turning-nuScenes is curated from nuScenes validation data to evaluate stable planning during turns, using trajectory and coordinate-based selection criteria.
- A.2. The Detail of Turning-nuScenes dataset: Turning scenarios are challenging because vehicles must rapidly and accurately change direction while maintaining stable planning.The dataset was created because no dataset specifically targeted turning-scenario evaluation.
- A.2. The Detail of Turning-nuScenes dataset: The dataset construction maps sample tokens to scene tokens and extracts each sample’s next three seconds of ego trajectory.The extracted future trajectory is denoted Tfut.
- A.2. The Detail of Turning-nuScenes dataset: Potential turns are selected by detecting significant x-coordinate changes, using ε = 25 as the default judgment threshold.The selection assumes the ego vehicle’s driving direction aligns with the world-coordinate y-axis.
- A.2. The Detail of Turning-nuScenes dataset: Turning-nuScenes contains 17 scenes and 680 samples spanning intersections, T-junctions, roundabouts, traffic islands, and alleyway turns.The resulting collection covers diverse urban turning scenarios.
A.3. Implementation Details
MomAD is implemented with a two-stage training process and evaluated using planning results on Turning-nuScenes and long-horizon nuScenes settings. The implementation combines multi-task losses with specified perception, motion, and planning configurations.
- Training Procedure: MomAD trains sparse perception first, then jointly trains perception, motion, and planning without freezing perception weights.Stage 1 learns sparse scene representations; Stage 2 uses losses from detection, tracking, mapping, motion prediction, and planning.
- Evaluation Settings: The planning experiments include Turning-nuScenes validation results and 6-second-horizon long-trajectory evaluation on nuScenes and Turning-nuScenes.The reported tables follow the ST-P3 evaluation metric.
- Task Losses: Detection uses Focal Loss for classification and L1 Loss for box regression.The cited implementation passage specifies the detection-loss composition but not the full objective equation.
- Task Losses: Online mapping uses the loss formulation adopted from VAD and SparseDrive, with λc and λr set to 2 and 0.25, respectively.The passage identifies the source formulation and its coefficient settings.
- Task Losses: Motion and planning use ADE to select the positive trajectory, classify remaining trajectories as negative, and predict ego state with Focal and L1 losses.The trajectory with the lowest ADE is designated positive; the remaining trajectories are negative.
A.4. More Planning Results
Additional experiments evaluate MomAD in corrupted, turning, congested, and long-horizon scenarios. The reported results emphasize robustness, smoothness, and temporal consistency across challenging planning conditions.
- Planning Results: MomAD’s TPC evaluation identifies temporal-consistency challenges in existing end-to-end methods that rely on one-shot trajectory prediction.The cited analysis reports these challenges in turning scenarios and frames MomAD as addressing planning stability and robustness.
- Robustness Analysis: MomAD outperforms SparseDrive on nuScenes-C across detection, tracking, mapping, motion, and planning under diverse corruptions.Average improvements are 22.9% for detection, 27.1% for tracking, 25.1% for mapping, 24.2% for motion, and 40.0% for planning.
- 3-Second Prediction: MomAD’s qualitative 3-second results show successful large-angle turns without collisions across varied turning scenarios.The visualization includes selected optimal, multi-modal proposal, and optimal motion trajectories.
- 6-Second Prediction: MomAD maintains smooth and consistent predicted trajectories over a 6-second horizon under challenging conditions.The passage attributes this performance to incorporating historical trajectory data.
- Multi-Frame Results: The qualitative study compares planning across multiple frames using ground truth and predictions at t−1, t, and t+1.The supplied visualization labels include GT, Pred (t−1), Pred (t), and Pred (t+1).