PaperlayerView exact revision on arXiv

Source-linked AI summary

MomADv2: Reliable Temporal Memory for End-to-End Autonomous Driving

Ziying Song, Shengkai Zhang, Lin Liu, Peiliang Wu, Lei Yang, Dongyang Xu, Bin Sun, Li Wang, Shaoqing Xu, Caiyan Jia, Yadan Luo

arXiv:2608.23405v1cs.CVcs.RO

TL;DR

Long-horizon temporal memory can become inconsistent with current driving intentions and mislead planning, motivating selective memory use. MomADv2 combines reliable state-space memory with flow-matching trajectory refinement, improving long-horizon planning consistency across open- and closed-loop benchmarks and reducing 6-second collision rate by 15.6% over MomAD.

  • Problem

    Temporal memory improves planning continuity but can become inconsistent with current intentions under scene changes, temporal discontinuities, or command shifts, so reliable history selection remains necessary.

  • Method

    MomADv2 uses selective state-space memory to filter historical planning queries by temporal continuity and command consistency, while flow-matching refinement corrects local trajectory deviations and accumulated errors.

  • Results

    MomADv2 improves long-horizon planning consistency and trajectory accuracy across NAVSIM, Bench2Drive, and nuScenes, reducing the 6-second collision rate by 15.6% over MomAD.

  • Takeaways & Limitations

    Reliable temporal memory selection and complementary trajectory refinement support safer, more consistent long-horizon planning across open- and closed-loop autonomous-driving evaluations.

Abstract

from arXiv · show

Long-horizon planning is critical for safe autonomous driving in complex scenarios. Existing methods improve planning continuity with temporal memory, but such memory may become invalid and mislead decisions when the driving command changes. Thus, selectively leveraging useful history while suppressing command-inconsistent memory remains a key challenge. To address this issue, we propose MomADv2, a reliable state-space memory framework for long-horizon end-to-end autonomous driving. At its core, MomADv2 introduces a Selective State-Space Planning Memory Query Module, which filters historical planning queries based on temporal continuity and command consistency, selects planning modes relevant to the current command, and models the evolution of planning intentions through a selective state-space mechanism. To further alleviate local trajectory deviations and error accumulation in long-horizon planning, we design a Flow-Matching Trajectory Residual Refiner. It learns a continuous residual correction field from the refined planning output to the expert trajectory, enabling fine-grained trajectory refinement while preserving the stability of anchor-based planning. Extensive experiments on closed-loop NAVSIM and Bench2Drive, as well as open-loop nuScenes, demonstrate that MomADv2 improves long-horizon planning consistency and reduces the average collision rate by 15.6% over MomAD under 6-second planning.

Introduction

Long-horizon autonomous driving benefits from temporally consistent planning, but blindly reused history can become inconsistent with current commands. MomADv2 addresses this with selective memory and flow-matching refinement, improving planning safety and consistency.

  • Long-horizon planning anticipates future behavior, helping autonomous vehicles perceive risks and generate smoother, more stable, and safer trajectories.
  • Temporal memory preserves planning consistency across consecutive frames and can reduce mode instability and trajectory jitter.
  • Command shifts, temporal discontinuities, and dynamic scene changes can make historical information inconsistent with current intentions and mislead planning.
  • MomADv2 filters historical planning queries using temporal continuity and command consistency, selects relevant modes, and models evolving planning intentions.
  • MomADv2 combines selective reliable-memory inheritance with flow-matching trajectory refinement to suppress invalid interference and correct local deviations.
  • 15.6% lower average 6-second collision rate is reported for MomADv2 versus MomAD across experiments on NAVSIM, Bench2Drive, and nuScenes.

Related Work

Related work advances end-to-end autonomous driving through unified representations, efficient decoders, generative planning, large models, and world models. State-space models provide an additional route for efficient long-range sequence modeling.

  • Existing E2E-AD research includes unified frameworks, efficient planning decoders, generative trajectory modeling, reliable planning optimization, large-model driving cognition, and world models.
  • Historical information may conflict with current driving intentions and cause negative interference, motivating selective retention of reliable memory.
  • State-space models capture long-range dependencies through latent dynamical systems, selective state transitions, and hardware-aware linear-complexity computation.

Methodology

MomADv2 combines selective state-space memory with flow-matching residual refinement to improve reliable, stable long-horizon planning. Its memory filters and aligns historical queries before trajectory corrections refine anchor-based planner outputs.

  • MomADv2 processes multi-view images into sparse scene representations, then uses an anchor-based decoder to produce multi-modal predictions and initial ego planning candidates.
  • The Selective State-Space Memory Query Module filters historical planning queries and enhances current queries using reliable, command-relevant temporal information.It stores raw planning queries and baseline command-specific trajectories before temporal enhancement.
  • Historical retrieval is restricted to the current high-level command and aligned by trajectory consistency rather than fixed candidate indices.Temporal trajectory distance compensates for frame-to-frame offsets when selecting the most relevant historical query.
  • Reliability-aware temporal modeling aggregates aligned historical queries using validity, trajectory similarity, and temporal decay before selective state-space encoding.The encoder models transitions from historical planning intentions to the current intention while preserving candidate-wise semantics.
  • Historical memory activates only when sufficient valid, trajectory-consistent information exists, while reliability-gated norm-constrained updates limit query perturbations.The enhanced query is added to the current query and passed to the planning refinement head.
  • The Flow-Matching Trajectory Residual Refiner learns a query-conditioned correction field from planner-refined trajectories toward expert trajectories, preserving anchor-based planning stability.Displacement predictions are converted to absolute trajectories before flow refinement, with stop-gradient used to prevent the auxiliary objective from destabilizing the planner.

Experiments

MomADv2 is evaluated across open-loop and closed-loop benchmarks, where it achieves strong long-horizon planning accuracy, safety, and driving performance. Ablations show that selective temporal memory and trajectory refinement provide complementary benefits, while excessively long memory can degrade performance.

  • Open-Loop nuScenes: MomADv2 achieves the best nuScenes 6-second results, with average L2 error of 1.21 m and collision rate of 0.76%.It improves average L2 error by 11.7% and collision rate by 11.6% over the strongest prior method.
  • Closed-Loop Bench2Drive: MomADv2 improves Bench2Drive performance over MomAD from 47.91 to 52.32 DS and from 18.11% to 24.24% SR.It reaches 78.82 DS and the best Comfortness of 31.23.
  • Closed-Loop NAVSIM: MomADv2 achieves strong NAVSIMv1 performance with 99.0 NC, 97.1 DAC, 95.5 TTC, 83.2 EP, and 89.9 PDMS.These metrics jointly reflect closed-loop planning performance across safety, progress, and stability dimensions.
  • Closed-Loop NAVSIM: MomADv2 obtains the best overall EPDMS of 39.5 on NAVSIMv2 navhard and 87.9 on navtest, ranking first on several component metrics.On navtest, it ranks first on DDC, EP, and EC; on navhard, it maintains strong DAC and TTC performance.
  • Ablation Studies: SSM-Q raises PDMS from 84.0 to 86.9, while FM-Ref further increases it to 89.9.The ablation indicates complementary gains from reliable temporal memory and residual trajectory refinement.
  • Ablation Studies: The full selective memory strategy achieves a 0.76% collision rate, while K = 4 achieves the best PDMS of 89.9.Longer histories gradually degrade performance, suggesting that moderate memory is preferable to excessive historical context.
  • Qualitative Analysis: Consecutive visualizations show MomADv2 producing more temporally consistent trajectories than MomAD, with predictions closer to ground truth.The comparison spans t−1, t, and t+1 on nuScenes.

Conclusion

MomADv2 selectively preserves useful history and suppresses stale or command-inconsistent memory for long-horizon end-to-end autonomous driving. Its experiments report improved planning accuracy, temporal consistency, and driving safety, while future work targets command-free intention inference and uncertainty-aware memory selection.

  • Conclusion: MomADv2 filters historical planning queries by temporal continuity and command consistency, selects relevant planning modes, and models planning-intention evolution.The framework uses the Selective State-Space Planning Memory Query Module for reliable temporal memory.
  • Conclusion: The Flow-Matching Trajectory Residual Refiner learns a continuous residual correction field to mitigate local deviations and accumulated planning errors.It preserves the stability of anchor-based planning while refining trajectories.
  • Conclusion: Experiments on NAVSIM, Bench2Drive, and nuScenes demonstrate improved long-horizon planning accuracy, temporal consistency, and driving safety.
  • Limitation and Future Work: MomADv2 relies on predefined driving commands for memory selection and does not explicitly model uncertainty in future scene evolution.Future work proposes command-free intention inference, uncertainty-aware memory selection, and world-model-based future reasoning.

A.1 Broader Impacts

MomADv2 is presented as a framework for improving autonomous-driving safety and reliability through more consistent long-horizon planning and suppression of unreliable temporal memory.

  • Broader Impacts: Selective state-space memory preserves useful historical context while suppressing unreliable temporal memory.
  • Broader Impacts: The flow-matching trajectory refiner improves trajectory accuracy and smoothness.
  • Broader Impacts: These capabilities may support safer interactions, more stable decision-making, and greater robustness in complex traffic environments.

A.2 Contributions

The paper contributes a reliable temporal-memory framework for end-to-end autonomous driving that selectively preserves useful information and suppresses stale or command-inconsistent memory.

  • MomADv2 Framework: MomADv2 selectively preserves useful temporal information while suppressing stale or command-inconsistent memory.The design targets long-horizon planning consistency, trajectory stability, and robustness under dynamic driving conditions.
  • SSM-Q and FM-Ref: The paper introduces SSM-Q for reliable historical query filtering and FM-Ref for fine-grained trajectory correction.

A.3 Datasets

The paper evaluates MomADv2 on NAVSIM, Bench2Drive, and nuScenes, combining closed-loop interaction tests with open-loop trajectory prediction.

  • NAVSIM reports results on navtest and navhard settings, with navhard targeting challenging, safety-critical long-tail scenarios.NAVSIM-v1 navtest measures general planning ability, while NAVSIM-v2 navhard evaluates more difficult cases.
  • Bench2Drive evaluates closed-loop driving across 220 routes covering 44 interactive scenarios in diverse towns and weather conditions.Executed actions affect subsequent vehicle states and sensor observations, unlike open-loop log replay.
  • nuScenes provides 1,000 real-world driving scenes for open-loop planning evaluation using logged observations.The paper evaluates both short- and long-horizon ego trajectories without affecting subsequent scene evolution.

A.4 Evaluation Metrics

The evaluation combines benchmark-specific safety, feasibility, progress, comfort, interaction, accuracy, collision, and diversity metrics across closed- and open-loop settings.

  • NAVSIM: PDMS scores NAVSIM trajectories by aggregating safety, feasibility, progress, and comfort terms.Its multiplicative hard-penalty structure makes at-fault collisions and drivable-area violations dominate the final score.
  • NAVSIM: EPDMS extends NAVSIM scoring with driving-direction, traffic-light, lane-keeping, history-comfort, and extended-comfort terms.These additions provide finer-grained rule-compliance and comfort assessment.
  • Bench2Drive: Bench2Drive reports Success Rate and Driving Score, with Driving Score rewarding route progress while penalizing collisions and traffic violations.Efficiency, comfortness, and ability-wise scores provide additional closed-loop assessments.
  • nuScenes: nuScenes evaluates open-loop planning using L2 displacement error and collision rate, where lower values indicate more accurate and safer planning.Collision rate uses oriented ego footprints and heading estimated from consecutive trajectory points.
  • nuScenes: The time-conditioned Diversity Metric measures separation among multi-mode trajectory predictions, with higher values indicating more diverse modes.The metric is bounded within [0, 1] and applies only to multi-mode planners.

A.5 More Details of MomADv2

MomADv2 maintains a command- and continuity-aware historical memory, aligns trajectories across time, gates unreliable entries, and performs causal online inference.

  • Temporal-Link Construction and Memory Reset: The memory bank uses frame-level continuity identifiers to validate historical entries and resets at sequence or scene boundaries.It falls back to single-frame planning when no valid historical entry exists.
  • Memory Contents: MomADv2 stores raw planning queries and baseline command-specific trajectories before temporal enhancement.Candidates associated with different driving commands are excluded from retrieval.
  • Memory Contents: A FIFO memory stores at most K = 4 historical entries and uses read-before-write retrieval.Only pre-enhancement queries and baseline trajectories are stored, preventing self-retrieval and recursive correction amplification.
  • Temporally Shifted Trajectory Distance: Historical trajectories are time-aligned by removing elapsed waypoints, transforming coordinates, and interpolating when frame offsets are non-integer.The shifted distance compares predictions over the same absolute future times.
  • Reliability Gating: SSM-Q selects the minimum-distance historical candidate and assigns zero weight to invalid, discontinuous, or command-inconsistent entries.If all retrieved entries are rejected, the current query remains unchanged.
  • Causal Online Inference: Causal inference updates only the explicit memory state while model parameters remain fixed, and the flow refiner uses S = 2 Euler evaluations.The current query and baseline trajectory are appended only after producing the final planning result.

A.6 Implementation Details

MomADv2 is initialized from MomAD, adapts newly introduced memory and refinement modules, and is evaluated through benchmark results, ablations, and qualitative visualizations.

  • Training Protocol: NuScenes training uses two stages: MomAD pretraining followed by adaptation of the new SSM-Q and FM-Ref components.The image backbone, perception, motion prediction, and most planning-head parameters remain frozen during adaptation.
  • Bench2Drive Results: 49.02% mean Bench2Drive performance with expert feature distillation outperforms DriveAdapter and DIVER by 16.5%.Without distillation, MomADv2 improves the mean score from 22.67% to 25.84% compared with DIVER.
  • nuScenes Consistency: MomADv2 reduces average TPC error by 6.6% versus MomAD and achieves 1.04 m and 1.32 m errors at 4- and 5-second horizons.At 6 seconds, its 1.61 m TPC error matches the best-performing baseline.
  • nuScenes Short-Horizon Planning: 0.07% average collision rate matches the best short-horizon performance while MomADv2 records zero collisions at 1 second.Its average L2 error is 0.54 m, improving on MomAD and GraphWorld by 10.0% and 5.3%.
  • Module Ablations: Adding SSM-Q reduces trajectory error and collision rate across horizons, while FM-Ref further reaches 1.32, 1.82, and 2.40 m L2 errors at 4, 5, and 6 seconds.The corresponding collision rates are 0.72%, 1.33%, and 2.03%.
  • Architecture and Solver Ablations: Selective SSM/Mamba achieves the best planning accuracy and collision performance, while Euler improves PDMS from 89.0 to 89.9 and FPS from 38 to 43.These ablations compare temporal-memory architectures and flow-refinement solvers.
  • Qualitative Results: Qualitative visualizations show smoother, temporally consistent trajectories that better follow lane geometry and drivable-area constraints.The comparisons cover NAVSIM scenarios and 6-second nuScenes planning under turns and dense traffic.
Loading 2608.23405v1…