PaperlayerView exact revision on arXiv

Source-linked AI summary

ManeuverFlow: Maneuver Hypothesis Generation with Future Evidence for Trajectory Evaluation and Refinement in End-to-End Autonomous Driving

Yuhao Wei, Yong Chen, Chen Yang, Yingchen Wang, Delin Ouyang, Lingfeng Qi, Qi Lan, Zhanhua Zhang, Jie Li, Guofa Li

arXiv:2610.13409v1cs.RO

TL;DR

Complex scenes require planners to model diverse driving intentions, yet candidate generation and scoring face trade-offs between maneuver coverage, candidate quality, and computational cost. ManeuverFlow generates maneuver-specific hypotheses and evaluates and refines them with current and future evidence; it reports results across NAVSIM and WorldEngine benchmarks.

  • Problem

    Candidate-based planners face a fixed-budget challenge: compact pools limit maneuver coverage or discovery of high-quality trajectories, while larger pools increase generation and scoring costs.

    Source: §1 · p. 1

  • Method

    ManeuverFlow generates explicit maneuver-indexed hypotheses in one step, then uses trajectory-specific current/future evidence to score and refine selected candidates before joint re-scoring.

    Source: §1 · p. 2

  • Results

    ManeuverFlow-TF achieves 90.58 PDMS and 90.14 EPDMS on NAVSIM v1/v2 navtest, while LTF records the highest listed WorldEngine PDMS* scores under replayed and reactive traffic.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Complementary non-reactive NAVSIM and closed-loop WorldEngine evaluations demonstrate the effectiveness of the integrated planner.

    Source: §5 · p. 7

    The frozen-pool headroom diagnostic is not an end-to-end budget sweep, an ECFR recovery measurement, or deployed performance.

    Source: §5.4 · p. 15

Abstract

from arXiv · show

End-to-end planning in complex scenes must model diverse driving intentions and select the most appropriate trajectory using scene context and future risks. However, multi-query regression may collapse to one mode, while anchor-based and generative planners share a scaling bottleneck: small candidate sets limit maneuver coverage or discovery of high-quality trajectories, whereas large sets increase generation and scoring costs. We propose ManeuverFlow to generate trajectories for distinct intentions in one step and evaluate and refine them using current and future evidence. The model selects relevant maneuver modes, draws corresponding noise from the Gaussian mixture prior, and maps it to intention-specific trajectories through a maneuver-routed mixture-of-experts MeanFlow field. Current features and future features predicted by the world model are aggregated along each candidate path into trajectory-specific evidence for PDM-factor prediction. Using the same evidence, the refiner predicts local Frenet residuals for selected candidates; refined and retained candidates are then jointly re-scored for selection. With a lightweight TransFuser-style ResNet-34 backbone, ManeuverFlow achieves 90.6 PDMS and 90.1 EPDMS on NAVSIM v1/v2 navtest, respectively. On WorldEngine's recent 3DGS closed-loop benchmark, it attains the highest listed PDMS$^\ast$ scores of 75.19 and 74.58 under logged and IDM-controlled traffic, respectively. Code: https://github.com/wyh-autonomy/maneuverflow.

1 Introduction

Complex scenes require planners to preserve distinct driving intentions and select among candidate trajectories, but compact proposal budgets make scene-relevant generation and future-aware evaluation important. ManeuverFlow combines maneuver-structured generation with shared-future evidence, refinement, and joint selection.

1 Introduction

  • Learned queries may cluster in similar trajectory regions, while fixed anchors can spend proposals on modes weakly relevant to the current scene.A frozen-generator diagnostic also shows that larger candidate pools expose stronger candidates.Source: §1 · p. 1, Figure 2 · p. 2
  • Current-only scoring does not explicitly query predicted future states along candidate paths, while per-candidate future rollouts repeat BEV prediction across trajectories and time steps.The latter is the approach attributed to WoTE.Source: Figure 1 · p. 1, §1 · p. 2
  • The section frames the challenge as generating scene-relevant, mode-distinct hypotheses under a fixed budget, then using current and future evidence to evaluate, refine, and select them.The proposed motivation is improving candidate quality through scene-adaptive generation and evidence-conditioned local refinement.Source: §1 · p. 2
  • ManeuverFlow preserves maneuver-indexed hypotheses through scene-conditioned generation, trajectory-specific current/future evidence, local Frenet refinement, and joint re-scoring.M-GMP allocates over motion-defined components, M3F generates bases in one step, and Metric Prism and ECFR use evidence from a shared predicted future BEV field.Source: §1 · p. 2
  • ManeuverFlow-TF achieves 90.58 PDMS and 90.14 EPDMS on NAVSIM v1/v2 navtest, while LTF attains the highest listed WorldEngine PDMS* scores under replayed and reactive traffic.The WorldEngine scores are 75.19 and 74.58, respectively.Source: §1 · p. 2

2 Related Work

Related work spans end-to-end and generative planners, driving world models, and candidate evaluation and refinement. The section identifies retaining diverse hypotheses and coordinating their evaluation and refinement as open design challenges.

2.1 End-to-End and Generative Planning

  • Generative planners use structured latent spaces, diffusion, or flow matching, but organizing driving-specific motion modes while retaining explicit hypotheses for evaluation and refinement remains challenging.MeanFuser aggregates sampled proposals into one output, whereas MM-FM designs a Gaussian-mixture source and mode-dependent coupling for general multimodal generation.Source: §2.1 · p. 2

2.2 World Models for Planning

  • Candidate-conditioned world-model rollouts provide path-specific futures but require separate computation per candidate; shared forecasts amortize prediction without directly supplying path-specific evidence.The section identifies extracting trajectory-specific evidence from a shared forecast as an unresolved challenge.Source: §2.2 · pp. 2–3

2.3 Trajectory Refinement and Selection

  • Candidate evaluation and trajectory refinement have been strengthened in separate lines of work, while coordination within a common hypothesis pool remains less explored.The stated open design problem is to retain unrefined alternatives and compare them directly with refined trajectories under the same future-conditioned evaluator.Source: §2.3 · p. 3

3 Method

ManeuverFlow allocates and generates explicit maneuver-indexed trajectory hypotheses, then verifies and refines candidates using path-specific evidence from current and predicted future BEV features. A fixed-size candidate pool is jointly rescored for final selection.

3.2 Planning Context and M-GMP

  • M-GMP allocates candidates across scene-relevant motion components, and M3F transports their samples into trajectory bases in one step.M-GMP groups expert trajectories by displacement, mean heading, and speed trend; scene-conditioned component weights guide allocation, while maneuver-routed experts generate the bases.Source: §3 · p. 3, §3.2 · p. 4, §3.3 · p. 4

3 Method

  • The framework links scene context, maneuver generation, evidence-based scoring, and refinement in a hypothesize–verify–improve pipeline.The encoder supplies BEV and agent/maneuver context, while the M3F blocks attend to BEV–ego features and then agent memories.Source: §3 · p. 3, §3.2 · p. 3, §3.3 · p. 4

3.1 Problem Formulation

  • Each hypothesis is an Hτ-waypoint trajectory represented in the current ego frame; Metric Prism pre-scores N bases before the Refiner replaces R and retains N − R.The final fixed-size pool is rescored by Metric Prism.Source: §3.1 · p. 3

3.4 Future BEV Field and Metric Prism

  • A candidate-independent future BEV field is predicted once per planning step, while each trajectory retrieves path-specific current and future evidence for PDM-factor scoring.Metric Prism contextualizes candidate score tokens and uses factor-specific heads; the fixed log-PDM composition ranks candidates, while their evidence also conditions refinement.Source: §3.4 · p. 4

3.5 ECFR

  • ECFR predicts local residuals in a path-aligned Frenet frame, using the candidate’s evidence memory to refine selected base trajectories.Reference selection uses the highest evaluator-scoring candidate with the same maneuver index; invalid base–reference pairs receive zero residual targets and weights.Source: §3.5 · p. 5

3.6 Training Objectives and Inference

  • Training proceeds in two stages, with Stage 2 jointly optimizing inference-time modules using 64 generated candidates per scene for evaluator labels and refinement-reference selection.Candidate construction and supervision targets are detached.Source: §3.6 · p. 5
  • At inference, the planner keeps the top three maneuver components, allocates candidates across them, and transports component-specific samples into trajectories in one full-interval evaluation.Metric Prism pre-scores the bases, selected bases are replaced by refined outputs, and retained and refined candidates are jointly rescored from re-queried evidence.Source: §3.6 · p. 5

4 Experiments

Experiments evaluate ManeuverFlow on NAVSIM and WorldEngine, then examine its component contributions, proposal-budget behavior, and maneuver-structured hypotheses.

4.1 Experimental Setup

  • NAVSIM uses logged future scenes in a non-reactive protocol, while WorldEngine evaluates 288 reconstructed high-risk scenarios under logged replay and reactive IDM traffic.NAVSIM v1 reports PDMS and v2 reports EPDMS; WorldEngine assesses closed-loop success, progress, and PDMS* as the ego replans from updated observations.Source: §4.1 · p. 5

4.2 Main Results

  • 90.58 PDMS is ManeuverFlow-TF’s highest listed NAVSIM v1 score; camera-only LTF obtains 89.98 PDMS.Table 1 reports NAVSIM v1 navtest mean results.Source: §4.2 · p. 5, Table 1 · p. 6
  • 90.14 EPDMS and 88.75 EP are ManeuverFlow-TF’s highest listed NAVSIM v2 results without v2-specific adaptation.LTF reaches 89.70 EPDMS and leads NC, DAC, and TTC at 98.61, 98.13, and 97.96.Source: §4.2 · pp. 5–6, Table 2 · p. 6
  • 75.19 and 74.58 PDMS* are ManeuverFlow-LTF’s highest listed WorldEngine scores under logged and IDM-controlled traffic, respectively.The benchmark covers 288 high-risk scenarios; IDM updates surrounding agents reactively, and the ego replans from updated rendered observations.Source: §4.1 · p. 5, Table 3 · p. 6, §4.2 · p. 6

4.3 Ablation Studies

  • Candidate-based MeanFlow improves PDMS by 2.13 points, while M3F adds 0.44 over parameter-matched Dense-XL with M-GMP-8 fixed.M-GMP-8 exceeds KMeans-8 by 0.54 PDMS and 0.83 EP; the M0-to-M1 gain of 1.77 points changes both prior support and generator capacity.Source: §4.3 · p. 6, Table 4 · p. 7
  • Predicted-future evidence adds 1.58 PDMS and 2.29 EP before refinement; after ECFR, the complete model adds 0.55 PDMS and 0.86 EP over MCur4.ECFR adds 0.65 PDMS and 0.95 EP to the future-aware pipeline while NC decreases by 0.10 points; future verification and ECFR yield complementary but sub-additive gains.Source: §4.3 · p. 6

4.4 Qualitative Analysis

  • Across six urban scenes, routed components show coherent motion tendencies while spanning distinct, road-aligned lane and intersection alternatives.The paired BEV and camera views make the separation around lane choices and intersections visually inspectable.Source: Figure 5 · p. 7, §4.4 · p. 7

4.3 Ablation Studies

  • M-GMP recall reaches 98.30% at k = 3 and changes marginally thereafter; with k = 3, PDMS saturates beyond N = 64 as activation bounds rise and Proposal FPS falls.The authors identify N = 64 as a balanced operating point and summarize the lifecycle as maneuver routing, shared-future evidence, and maneuver-matched refinement.Source: §4.3 · p. 7, Table 5 · p. 7

5 Conclusion

ManeuverFlow maintains maneuver-indexed hypotheses through generation, evaluation, and refinement. Its shared predicted-future field supports path-specific scoring and refinement, and NAVSIM and WorldEngine evaluations demonstrate the integrated planner’s effectiveness.

5 Conclusion

  • ManeuverFlow maintains explicit maneuver-indexed hypotheses throughout generation, evaluation, and refinement.The conclusion frames this lifecycle as Hypothesize–Verify–Improve.Source: §5 · p. 7
  • M-GMP allocates a fixed proposal budget across scene-relevant motion components, and M3F transports samples into scene-adaptive hypotheses in one step.Source: §5 · p. 7
  • A shared predicted future BEV field supplies path-specific evidence for Metric Prism scoring and ECFR refinement, after which refined and retained hypotheses are jointly re-scored.The conclusion cites complementary NAVSIM non-reactive and WorldEngine closed-loop evaluations as evidence for the integrated planner.Source: §5 · p. 7

1 Hypothesize: Operational Maneuver Prior and Routed Transport

ManeuverFlow defines maneuver components from measurable trajectory motion and routes generation through those components. Soft component matching supplies the training route and Recall@k target, while deployment uses scene-predicted component weights.

1.1 Diff-Space and Prior Construction

  • The M-GMP uses eight rule-defined motion families fitted to NAVSIM navtrain expert trajectories using displacement, mean heading, and a speed-trend proxy.Every non-stop family has first-to-last distance Dn ≥5 m; the speed-trend proxy compares mean step lengths over equal early and late windows.Source: §1.1 · p. 10
  • The paper’s maneuver vocabulary represents measurable motion tendencies, not lane-topology labels; acceleration and deceleration use speed trend, while lane-change and turn names use mean heading.The speed-trend measure is a proxy rather than physical acceleration.Source: §1.1 · p. 10
  • M-GMP component statistics are estimated from assigned experts and then held fixed, with sampled waypoint states transformed into cyclically embedded M3F source codes.The normalized pre-embedding state has three dimensions, and the cyclic heading embedding yields a four-dimensional M3F code.Source: §1.1 · p. 10

1.2 Soft Component Association and Training Routes

  • Distance-based soft component similarities supervise matching and determine the generation route, while Recall@k measures coverage of the matched fitted component rather than semantic-label accuracy.The rule-assigned family and distance-matched component have distinct roles and are not assumed identical for each trajectory.Source: §1.2 · p. 11
  • Training independently samples six source proposals from the matched component and draws their shared interval pair from three specified regimes.The regimes are full-interval sampling, equal endpoints, and ordered endpoints from independent draws.Source: §1.2 · p. 11

2 Verify: Shared-Future Evidence and Factorized Ranking

ManeuverFlow uses a shared current observation and a candidate-independent predicted future field to retrieve path-specific evidence for ranking. Factor-specific predictions form the ranking score, and the same evidence also conditions refinement.

2.1 Future-Feature Targets

  • Future-feature targets are constructed from ego-frame observations aligned to the current frame, but future observations are confined to stop-gradient training and are unavailable at deployment.Targets cover 1–4 s at 2 Hz, with offsets +2, +4, +6, and +8.Source: §2.1 · p. 11

2.2 Trajectory-Guided Evidence Queries

  • A trajectory-guided probe samples current and time-aligned predicted BEV features along candidate paths, so candidates share one forecast while retrieving path-specific evidence.Future queries occur at waypoints h ∈ {2, 4, 6, 8}; unsupported samples are masked to zero.Source: §2.2 · p. 11

2.3 Factor-Wise Metric Prediction and Ranking

  • Metric Prism creates a score token for each candidate, jointly updates the candidate set, and cross-attends each token to its associated trajectory evidence.For N = 64 candidates, each trajectory receives one 512-D score token processed by a shared four-layer set-scoring stack.Source: §2.3 · pp. 12–13
  • Five factor-specific heads predict NC, DAC, EP, TTC, and Comf, whose predictions are combined by a fixed log-PDM score for inference ranking.The score’s omitted 1/12 normalization is a candidate-independent log offset and therefore preserves ranking.Source: §2.3 · pp. 12–13

3 Improve: Candidate Supervision and Frenet Refinement

Stage 2 trains verification and refinement on fresh candidate sets labeled by the NAVSIM v1 evaluator. Refinement targets are defined relative to the best-scoring candidate from the same maneuver family.

3.1 Stage-2 Candidates and Supervision

  • Stage 2 generates fresh sets of 64 candidates from the model’s top three predicted maneuver components and uses evaluator factor labels to supervise scoring and refinement references.Each candidate retains its generating maneuver index.Source: §3.1 · p. 13

3.2 Evaluator Labels and Refinement References

  • For each base candidate, the highest-scoring finite candidate with the same maneuver supplies its reference trajectory, and the base itself remains eligible.Invalid base–reference pairs receive zero residual targets and weights.Source: §3.2 · p. 13

3.3 Base-Local Frenet Targets

  • Refinement targets express reference-minus-base displacement in local tangent and normal directions, plus wrapped heading difference, and use a masked Smooth-L1 loss.Reference selection and residual targets are detached; the residual loss averages over valid candidates, waypoints, and three coordinates.Source: §3.3 · p. 13

4 Reproducibility: Interfaces, Optimization, and Deployment

The reproducibility section specifies the model interfaces, two-stage optimization, and deployment lifecycle. At inference, ManeuverFlow generates, scores, selectively refines, and re-scores a fixed set of hypotheses.

4 Reproducibility: Interfaces, Optimization, and Deployment

  • The reproducibility overview summarizes module interfaces and two-stage optimization, with Stage 2 initializing from Stage 1 and jointly optimizing inference-time modules.Stage 2 uses current-model candidates for evaluator labeling.Source: §4 · p. 13, Table S2 · p. 14, §4.3 · pp. 14–15

4.1 Input Construction

  • Both TF and LTF use three camera views and command, velocity, and acceleration inputs; TF adds rasterized LiDAR, while LTF uses a shared learnable latent BEV template.The camera views are cropped and resized to 256 × 1024; the latent template has four channels on a 256 × 256 grid.Source: §4.1 · p. 13

4.2 Multi-Resolution Perception Interface

  • The shared multi-resolution BEV interface supports auxiliary semantics and current evidence, conditions scene decoding and M3F, and supplies the world model’s future-prediction input.The reader projects pooled current and predicted-future features into its query space.Source: §4.2 · p. 14

4.3 Two-Stage Optimization

  • Stage 2 continues perception and generation objectives while shared BEV features support future prediction, scoring, and refinement; candidate labels, references, residuals, and future targets are detached.All inference-time modules are jointly optimized.Source: §4.3 · pp. 14–15

4.4 Deployment Lifecycle

  • At deployment, the model generates 64 bases, refines 16 selected candidates, retains 48, then re-queries and jointly re-scores all 64 before returning the highest predicted log-PDM candidate.The future BEV field is predicted once per planning step and shared across candidates.Source: §4.4 · p. 15

5 Statistical Reliability and Scoring Diagnostics

The section reports scorer retrieval on a fixed candidate pool, checkpoint variation, selector transfer to NAVSIM v2, and diagnostic limits on frozen-pool headroom. It also traces how two displayed final plans were produced.

5.1 Main-Result Variation

  • Across three independently trained checkpoints, the largest observed standard deviation is 0.14 on NAVSIM and 0.92 in WorldEngine.The paper cautions that these are not formal confidence intervals and that component ablations lack matched-seed uncertainty.Source: §5.1 · p. 15

5.2 Within-Pool Top-κ Retrieval of Metric Prism

  • 75.28% of scenes fall within one PDMS point of the Base64 oracle at rank one, rising to 83.18% by rank ten on the unrefined pool.These are pool-relative retrieval statistics, not a budget-matched cross-method scorer comparison.Source: §5.2 · p. 15

5.3 Fixed Selector Transfer to NAVSIM v2

  • NAVSIM v2 EPDMS is used as an empirical feasibility check for transferring the fixed v1-trained selector to the extended evaluator, not as a direct cross-version comparison.The v2-only factors contribute to EPDMS evaluation but are not added to the selector.Source: §5.3 · p. 15

5.4 Frozen-Pool Headroom Diagnostic

  • Frozen-pool nested prefixes measure headroom within one fixed draw, not independent-resampling scaling, an end-to-end budget sweep, ECFR recovery, or deployed performance.The comparison with the HumanAgent reference is an unpaired aggregate order statistic and establishes neither per-scene nor human-level superiority.Source: §5.4 · p. 15
  • The displayed final plans come from the full N = 64 base/refined pool after ECFR, evidence re-query, and Metric Prism re-scoring, rather than from the controlled expert sweep.Figure S3 varies the routed expert while holding the source state, scene context, and transport interval fixed.Source: §5.4 · p. 16, Figure S3 · p. 16

6 Qualitative Evidence and Claim Boundaries

The qualitative material illustrates route-conditioned trajectory variation and candidate organization, while the authors bound what these examples and evaluations establish. In particular, the paper distinguishes motion-level intentions from map-semantic labels and notes limits in interaction and transfer scope.

6.1 Controlled Expert Responses and Final-Plan Provenance

  • With source state and scene context fixed, routed experts vary lateral position and terminal position for lane changes, and turn onset, radius, and exit alignment for right turns.The selected-scene panels illustrate route-conditioned response separation and final-plan provenance, not the deployed candidate pool or semantic expert labels.Source: §6.1 · p. 16

6.2 Descriptive, Non-Budget-Matched Candidate Gallery

  • Figure S4 is a descriptive gallery across six shared scenes, not a budget-matched quantitative comparison; methods use different candidate counts and display conventions.ManeuverFlow shows 32 draws while reported results use 64; the other displayed sets are method-specific.Source: §6.2 · p. 16, Figure S4 · p. 17

6.3 Interpretation and Evidence Scope

  • “Intention” denotes a trajectory-derived motion hypothesis based on displacement, turning, or speed trend, not a separate map-semantic annotation task.Top-three routing may miss rare or compositional behavior, and the candidate-independent future field does not model responses to alternative ego plans.Source: §6.3 · p. 16
  • The qualitative examples and learned PDM factors establish neither cross-method superiority nor safety.The cumulative ablations evaluate the framework stages, while NAVSIM v2 transfer and WorldEngine replanning do not test independent-domain or real-vehicle generalization.Source: §6.3 · p. 16
Loading 2610.13409v1…