PaperlayerView exact revision on arXiv

Source-linked AI summary

Expand-and-Prune: Test-Time Candidate-Space Adaptation for End-to-End Autonomous Driving

Fabian Schmidt, Karol Fedurko, Markus Enzweiler, Abhinav Valada

arXiv:2610.23262v1cs.ROcs.CV

TL;DR

Sparse candidate sets may undersupport scene-relevant maneuvers, while fixed downstream budgets ignore prediction uncertainty. Expand-and-Prune adapts candidate support and capacity at test time without updating learned parameters. It improves planning performance across heterogeneous planners and benchmarks, including a higher LangAuto Driving Score for LAD-Drive.

  • Problem

    Sparse planners may lack candidates for scene-relevant maneuvers, and fixed downstream budgets do not account for prediction uncertainty.

    Source: §1 · p. 1

  • Method

    Expand-and-Prune expands native candidates using semantic intent and temporal memory, then adapts downstream candidate capacity using uncertainty in intermediate trajectory scores.

    Source: §1 · p. 2

  • Results

    Across heterogeneous planners and benchmarks, planning performance improves; LAD-Drive’s LangAuto Driving Score increases from 68.2 to 75.9.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Candidate-space adaptation transfers across sparse generative and dense selection-based planners while adaptive pruning reduces candidate-processing cost.

    Source: §5 · p. 8

    Intent-guided expansion is evaluated only on LAD-Drive, which has an explicit probabilistic action belief, and temporal memory uses only the immediately preceding frame.

    Source: §5 · p. 8

Abstract

from arXiv · show

Candidate-based end-to-end driving planners reason over a finite set of trajectory hypotheses. However, for sparse planners, a limited candidate set may provide insufficient support for scene-relevant maneuvers, while fixed downstream candidate budgets do not account for prediction uncertainty. To address these challenges, we introduce Expand-and-Prune, a test-time framework that jointly adapts candidate support and capacity without updating learned parameters. The framework expands the native candidate set using semantic driving intent and temporally propagated predictions and then uses uncertainty in intermediate trajectory scores to adaptively prune the resulting candidates. Across four heterogeneous candidate-based planners and four benchmarks, Expand-and-Prune consistently improves planning performance. It achieves state-of-the-art results on LangAuto, increasing the Driving Score from 68.2 to 75.9, while providing further improvement on Bench2Drive, Fail2Drive, and NAVSIMv2, with particularly strong improvements in out-of-distribution scenarios. Moreover, in its full instantiation, 72.5% of selected trajectories originate from the newly introduced intent-guided or temporal candidates, confirming that candidate-space expansion provides useful hypotheses beyond those generated natively by the planner. The code is available at https://github.com/iis-esslingen/Expand-and-Prune.

1. Introduction

Candidate-based planners rely on finite hypothesis sets, but sparse support and fixed downstream budgets may poorly match scene intent and prediction uncertainty. Expand-and-Prune adapts candidate support and capacity at test time, improving results across planners and benchmarks.

1. Introduction

  • Sparse candidate sets may undersupport scene-favored maneuvers, while fixed downstream budgets propagate the same number of hypotheses in confident and ambiguous scenes.These constraints may matter especially over long closed-loop horizons and under distribution shift.Source: §1 · p. 1
  • The framework adapts pretrained planners without retraining, parameter updates, or changes to their native scoring procedure.Its components operate at existing candidate-processing boundaries and are evaluated across heterogeneous candidate-based architectures.Source: §1 · p. 2
  • Expand-and-Prune adds intent-guided and temporally propagated candidates, then uses uncertainty in intermediate scores to adapt downstream candidate capacity.Expansion adapts where support is allocated; pruning adapts how many hypotheses receive further computation.Source: §1 · p. 2
  • On LangAuto, Expand-and-Prune raises LAD-Drive’s mean Driving Score from 68.2 to 75.9, achieving state-of-the-art performance, with particularly strong improvements on longer routes.Adaptations also consistently improve planning performance across three additional heterogeneous candidate-based planners and their benchmarks.Source: Figure 1 · p. 1, §1 · p. 2

2. Related Work

Prior end-to-end planners generate trajectory hypotheses through vocabularies, anchors, learned proposals, or iterative refinement, often coupled to planner-specific architectures. Expand-and-Prune instead adapts the candidate collection within a pretrained planner’s native inference pipeline.

2. Related Work

  • Candidate-based systems represent multiple plausible futures using trajectory vocabularies, clustered anchors, learned queries, or generative refinement methods.These approaches include query-decoded candidates, diffusion or flow-based refinement, and structured variations around intention anchors.Source: §2 · p. 2
  • Many scene-aware candidate-generation and refinement methods remain tightly coupled to model-specific architectures trained jointly with the planner.Expand-and-Prune modifies the candidate collection of an already trained planner at inference time.Source: §2 · p. 2
  • Test-time alternatives update planner parameters, optimize trajectories with planner scores, reweight sub-scores, or refine outputs with safety mechanisms.Examples include Centaur [25], TOAD [32], Diffusion-ES, DrivoR [13], UniAD [8], and SafetyNet [28].Source: §2 · p. 2
  • Expand-and-Prune adapts candidates inside the native inference pipeline rather than changing parameters, the final scoring objective, or adding an external iterative optimizer.It expands support using semantic intent and temporal memory, while uncertainty in intermediate scores adjusts downstream candidate capacity.Source: §2 · p. 2

3. Method

Expand-and-Prune augments native candidates with intent-guided and temporally propagated hypotheses, then adapts downstream candidate capacity using score uncertainty. It operates at existing candidate-processing boundaries while keeping learned planner parameters unchanged.

3.1. Problem Formulation and Overview

  • Candidate support may miss scene-relevant maneuvers in sparse planners, while fixed downstream budgets ignore prediction uncertainty.Source: §3.1 · p. 3

3.2. Intent-Guided Candidate Expansion

  • Intent-guided expansion allocates candidates across action-specific trajectory banks in proportion to the semantic belief over maneuvers.The banks are built offline by assigning expert trajectories to action categories and clustering within each category.Source: §3.2 · pp. 3–4

3.3. Entropy-Adaptive Temporal Memory

  • Temporal memory carries high-confidence predictions from the preceding frame, retaining more alternatives when that prediction is uncertain.The retained trajectories are transformed from the previous ego frame into the current frame to provide a scene-adaptive prior.Source: §3.3 · p. 4

3.4. Confidence-Aware Adaptive Pruning

  • Adaptive pruning uses uncertainty in intermediate trajectory scores to retain fewer candidates for confident predictions and more for uncertain ones.Only the retained highest-scoring hypotheses proceed through the remaining planner stages, reducing downstream computation.Source: §3.4 · pp. 4–5

3.5. Integration into Candidate-Based Planners

  • The framework modifies candidate collections at existing processing boundaries without changing learned parameters, including before or between diffusion steps.It can also be instantiated at native proposal-selection or refinement boundaries in coarse-to-fine and hierarchical planners.Source: §3.5 · p. 5

4. Experiments

Across four candidate-based planners, Expand-and-Prune improves planning results through test-time candidate adaptation, with LangAuto gains and transfer to other benchmarks. Candidate provenance, adaptive-capacity comparisons, and runtime analyses examine how the framework behaves and what it costs.

4.1. Experimental Setup

  • LAD-Drive evaluation uses LangAuto’s eight towns and three route-length categories, alongside Bench2Drive, Fail2Drive, and NAVSIMv2; four planners retain their pretrained weights and scoring functions.Component support varies by architecture: LAD-Drive uses intent and temporal expansion, BridgeDrive uses temporal memory, and selection-based planners use pruning.Source: §4.1 · p. 5, Table 1 · p. 5

4.2. Candidate-Space Adaptation on LAD-Drive

  • 75.9 mean DS, up from 68.2, establishes a LangAuto state of the art; RC rises from 74.5 to 81.1 and IS from 0.90 to 0.93.The gains are especially pronounced on Short and Long routes: DS improves by 12.1 and 9.1 points, respectively.Source: §4.2 · pp. 5–6, Table 2 · p. 6
  • Intent-guided expansion, entropy-adaptive temporal memory, and adaptive pruning each contribute to the selected LAD-Drive configuration; replacing native anchors with intent-guided candidates alone lowers DS.The best reported expansion configuration adds 30 candidates, raising DS from 68.2 to 72.1; entropy-adaptive memory then raises it to 73.4, and pruning to 75.9.Source: Table 3 · p. 6, §4.2 · p. 6

4.3. Transfer Across Benchmarks and Planners

  • BridgeDrive reaches 96.5 DS and 88.7 SR on Bench2Drive; on Fail2Drive, combined components raise HM from 93.6 to 95.5 on base and 76.4 to 80.5 under generalization.Adaptive pruning also improves EPDMS for DriveSuprim and SparseDriveV2 on NAVSIMv2.Source: Table 4 · p. 7, §4.3 · p. 7

4.4. Candidate-Space Analysis and Efficiency

  • 72.5% of selected trajectories come from expansion: 31.3% from intent-guided candidates and 41.2% from temporal memory, despite temporal candidates forming about 8.3% of the expanded set.Temporal selections rise from 34.9% on Tiny routes to 43.4% on Long routes.Source: §4.4 · p. 7, Figure 5 · p. 8
  • Against a matched fixed-capacity baseline, adaptive capacity achieves 75.9 DS versus 74.0, while LAD-Drive total latency rises by only 2.2 ms over baseline.Pruning reduces LAD-Drive head latency from 9.2 ms to 8.2 ms after expansion and temporal memory.Source: §4.4 · p. 8, Table 5 · p. 8

5. Conclusion

Expand-and-Prune adapts pretrained driving planners’ candidate spaces at test time without updating learned parameters. The conclusion reports cross-architecture improvements and identifies limits on the current intent and temporal components.

5. Conclusion

  • Across sparse generative and dense selection-based planners, test-time candidate-space adaptation improves planning quality, while adaptive pruning also reduces candidate-processing cost.The analyses attribute gains not merely to processing more hypotheses or choosing a favorable fixed budget, but to complementary candidates and capacity adaptation.Source: §5 · p. 8
  • Intent-guided expansion is evaluated only on LAD-Drive, and temporal memory uses only the immediately preceding frame.Future work proposed includes navigation commands or continuous goals for intent guidance and longer-term context from trajectory histories or planner-native latent representations.Source: §5 · p. 8

Supplementary Material

The supplementary material adds implementation and evaluation details, extended ablations, NAVSIMv2 metric breakdowns, and further LangAuto qualitative examples.

Supplementary Material

  • Supplementary sections cover planner-specific integration, benchmark and metric details, extended LAD-Drive ablations, NAVSIMv2 metric breakdowns, and additional LangAuto examples.The implementation details address LAD-Drive, BridgeDrive, DriveSuprim, and SparseDriveV2.Source: Supplementary Material · p. 11

6. Implementation Details

The implementation adapts candidate processing to each planner’s native inference pipeline, using entropy-based capacities where applicable. Diffusion planners add or propagate hypotheses before pruning, while selection-based planners replace fixed filtering budgets.

LAD-Drive

  • LAD-Drive adds 30 intent-guided candidates to 20 native anchors and propagates high-confidence trajectories from the preceding frame in current ego coordinates.Entropy-adaptive memory retains 1–5 candidates; pruning after the first of two denoising steps retains 4–12 for the second.Source: §6 · p. 11

BridgeDrive

  • BridgeDrive adds motion-compensated trajectories from the preceding frame and applies adaptive pruning after its first diffusion-bridge update.Memory retains 1–3 candidates and pruning retains 3–10 before the remaining diffusion updates.Source: §6 · p. 11

DriveSuprim and SparseDriveV2

  • DriveSuprim replaces fixed Top-256 selection with entropy-adaptive pruning after coarse scoring, retaining 16–64 candidates before refinement.SparseDriveV2 replaces fixed Top-K filtering at its path and velocity stages with capacities computed independently from each score distribution.Source: §6 · p. 11

7. Benchmark and Metric Details

The section defines the route-progress, safety, success, and aggregate measures used across the benchmarks. It also describes the distribution shifts evaluated by Fail2Drive and the compliance, progress, and comfort measures used by NAVSIMv2.

LangAuto

  • LangAuto reports Route Completion (RC), Infraction Score (IS), and Driving Score (DS); RC measures route progress, while IS penalizes collisions and traffic-rule violations.DS combines progress and safety; RC ranges from 0–100 and IS from 0–1, yielding DS on a 0–100 scale.Source: §7 · pp. 11–12

Bench2Drive

  • Bench2Drive evaluates 220 short routes across 44 interactive scenario types, reporting Success Rate (SR) and Driving Score (DS).A route counts as successful if the agent reaches its destination within the allotted time without a traffic-rule infraction.Source: §7 · p. 12

Success and aggregate scores

  • SR is the percentage of evaluation routes completed without failure or traffic-rule violations, while DS also accounts for partial route completion and infractions.DS averages route-level scores formed by multiplying route completion by the applicable infraction penalty factors; routes without infractions have a penalty product of one.Source: §7 · p. 12

Fail2Drive

  • Fail2Drive tests closed-loop generalization on 200 paired routes, holding road geometry, spawn locations, and traffic goals fixed while introducing targeted distribution shifts.It reports DS and SR, plus their harmonic mean (HM); RC is not reported separately because routes are intentionally short and nearly always finishable.Source: §7 · p. 12

NAVSIMv2

  • NAVSIMv2 evaluates real-world planning using EPDMS, combining compliance penalties with weighted progress, time-to-collision, lane-keeping, and comfort measures.Its compliance measures include NC, DAC, DDC, and TLC; the aggregate score is normalized to [0, 1] and reported after scaling by 100.Source: §7 · p. 12

8. Extended Results

The component ablations select probabilistic intent expansion, entropy-adaptive temporal memory, and entropy-adaptive pruning for the LAD-Drive configuration. Extended NAVSIMv2 results show adaptive pruning improves aggregate scores across the evaluated planners, alongside mixed changes in component metrics.

8.1. LAD-Drive Component Ablations

  • 72.14 mean DS and 79.50 RC result from adding 30 probabilistically allocated intent-guided candidates to LAD-Drive’s 20 native anchors.Larger expansion budgets add no gains, and using only intent-guided candidates substantially reduces performance; probabilistic allocation also beats winner-takes-all at the same budget, 72.14 vs. 70.89 DS.Source: §8.1 · pp. 12–13, Table 6 · p. 13
  • 73.39 DS and 80.94 RC are reached with entropy-adaptive temporal memory bounded by (1, 5), while increasing fixed memory to five trajectories provides no further benefit over Top-3.The narrower entropy range (1, 3) and both evaluated Top-p variants perform worse; Top-1 has the highest mean IS but lower RC and overall DS.Source: §8.1 · p. 13, Table 7 · p. 13
  • 75.92 DS, 81.13 RC, and 0.93 IS are achieved with entropy-adaptive pruning at bounds (4, 12), outperforming matched fixed-capacity Top-8 at 74.00 DS.The adaptive method forwards 8.1 candidates on average, compared with 4.6 for adaptive temporal memory; the matched fixed-capacity comparison indicates the gain is not explained solely by average capacity.Source: §8.1 · pp. 13–14, Table 8 · p. 14

8.2. DriveSuprim and SparseDriveV2 Extended Metrics

  • Adaptive pruning improves EPDMS for both DriveSuprim backbones and SparseDriveV2, while several safety and compliance metrics improve and ego progress and lane keeping decrease slightly.The largest EPDMS increase is DriveSuprim ResNet-34, from 83.069 to 83.664; history comfort remains unchanged.Source: §8.2 · p. 14

9. Qualitative Results

On LangAuto long routes, qualitative examples contrast LAD-Drive with Expand-and-Prune in lane keeping, merge completion, and handling an ambiguous highway exit. The figure shows driving and bird’s-eye views, with trajectory color indicating confidence.

Scenario 36

  • In Scenario 36, LAD-Drive leaves the intended lane and collides with an object, while Expand-and-Prune remains in the correct lane.The figure pairs driving views with bird’s-eye views; trajectory color runs from white for low confidence to red for high confidence.Source: §9 · p. 14, Figure 6 · p. 15

Scenario 37

  • In Scenario 37, LAD-Drive drifts out of its lane and collides with a traffic sign during a highway merge, whereas Expand-and-Prune safely completes the merge.The comparison uses paired driving and bird’s-eye views for each method.Source: §9 · p. 14, Figure 6 · p. 15

Scenario 43

  • In Scenario 43, Expand-and-Prune’s intent-guided and temporal candidates support continuing straight near an ambiguous exit, avoiding the guardrail reached by LAD-Drive.LAD-Drive first favors the exit maneuver, then corrects toward continuing straight; the figure encodes trajectory confidence from white (low) to red (high).Source: §9 · p. 14, Figure 6 · p. 15
Loading 2610.23262v1…