Source-linked AI summary
INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement
Aron Distelzweig, Andreas Look, Faris Janjoš, Steffen Hagedorn, Luigi Palmieri, Joschka Boedecker
TL;DR
Plan-independent forecasts and fixed proposal sets limit interactive driving planners, while jointly trained prediction and planning can interfere. INTERACT predicts reactions across map-derived maneuver intents and refines plans locally within each intent. It sets a new state of the art on nuPlan and interPlan, with the largest gains in interactive scenarios.
Problem
Plan-independent forecasts miss other agents’ responses to the ego plan, while joint training can cause task interference and fixed proposals restrict the available plans.
Source: §I · p. 1Method
INTERACT derives map-based maneuver anchors, queries a dedicated ego-conditioned predictor once per anchor, and refines each with trust-region CEM.
Source: §I · p. 2, A. Anchor Generation · p. 3Results
INTERACT sets a new state of the art on nuPlan and interPlan, including a closed-loop reactive (R) score of 89.07 on interPlan-LC, with largest gains in interactive scenarios.
Source: §I · p. 2, C. Main Results · p. 5Takeaways & Limitations
Intent-conditioned predictions allow local refinement within each maneuver intent without repeatedly querying the predictor during optimization.
Source: §I · pp. 1–2
Abstract
from arXiv · showhide
Driving in dense urban traffic is interactive: whether a merge or an unprotected turn succeeds depends on how surrounding agents respond to the ego vehicle. Conventional planners predict first and plan second and, therefore, cannot account for this dependency. Methods that integrate prediction and planning either train both jointly, which introduces task interference, or keep them separate and are restricted to a predefined set of proposals. We present INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement. Our key insight is that surrounding agents react to the intent a trajectory expresses rather than to its exact realization, so a single reactive prediction stays valid across an entire family of plans. INTERACT therefore decomposes interactive planning into prediction across driving intents and optimization within each intent. We derive a small set of diverse intents, which we call anchors, from map geometry, query a dedicated ego-conditioned prediction model once per anchor, and refine every anchor with the Cross-Entropy Method under a trust-region penalty that keeps the refined plan close enough to its anchor for the conditioned reaction to still apply. Prediction thus remains a separate model, avoiding task interference, while conditioning on anchors preserves the dependency. Because each anchor is refined continuously, the final plan is not restricted to the anchor set, yet INTERACT requires only one predictor query per anchor rather than one per candidate plan, with all anchors processed in parallel. On the nuPlan and interPlan closed-loop benchmarks, INTERACT sets a new state of the art, with the largest gains precisely in the interactive scenarios that motivate the method. The code will be released upon acceptance.
I. INTRODUCTION
INTERACT addresses the missing coupling between ego plans and other agents’ reactions while avoiding joint training and fixed-proposal constraints. It predicts across driving intents and refines plans within each intent, achieving its largest gains in interactive scenarios.
I. INTRODUCTION
- Plan-independent forecasts miss how other agents respond to the ego vehicle, limiting maneuver feasibility in dense traffic and contributing to overly conservative behavior.The passage names unprotected turns, merges, and narrow-gap passages as interactive settings.Source: §I · p. 1
- Existing integrations either jointly train prediction and planning, where task interference can degrade both, or score fixed proposals that constrain plans to a predefined set.The proposal approach also requires a separate predictor query for every plan.Source: §I · p. 1
- INTERACT assumes surrounding agents react primarily to a trajectory’s intent, so one prediction can remain valid while plans sharing that intent are locally refined.This decomposes planning into prediction across intents and optimization within each intent.Source: §I · pp. 1–2
- Map-derived anchors condition a dedicated predictor once each, and trust-region CEM refines each anchor while keeping its conditioned reaction valid.Prediction and planning remain separate specialized components, and refinement runs independently across anchors.Source: §I · p. 2
- INTERACT sets a new state of the art on nuPlan and interPlan, with its largest gains in interactive scenarios.The introduction identifies this result across both benchmarks without listing split-level scores.Source: §I · p. 2
II. RELATED WORK
Related work addresses interactions through integrated planning, conditional prediction, and test-time optimization, but faces limits in computational cost, predictor specialization, or trajectory refinement. INTERACT combines intent-conditioned prediction with local optimization as an alternative.
A. Integrated Prediction and Planning
- Bidirectional integration models interactions by conditioning prediction and planning on each other, but prior methods limit computation through restricted searches or simplified future behaviors.INTERACT instead predicts across discrete intents and optimizes within them, assuming agents primarily react to overarching intent.Source: A. Integrated Prediction and Planning · p. 2
B. Conditional Prediction
- Most trajectory predictors are unconditional, while custom conditional models can share capacity with planning and fall short of dedicated predictors.INTERACT adapts SMART into an ego-conditioned predictor without architectural changes or retraining, preserving baseline prediction performance.Source: B. Conditional Prediction · p. 2
C. Test Time Optimization
- Test-time optimization methods can incur repeated predictor queries or omit reactive multi-agent predictions; INTERACT uses local CEM refinement against fixed conditioned predictions.Its trust region keeps plans near their anchors and its continuous surrogate score avoids flat plateaus in discrete scoring.Source: C. Test Time Optimization · pp. 2–3
III. METHODOLOGY
INTERACT generates diverse maneuver anchors from route geometry, predicts surrounding-agent reactions for each anchor, and refines each against its own prediction. The resulting plans are compared for execution.
III. METHODOLOGY
- The pipeline generates map-based anchors, predicts reactions per anchor, refines each with CEM against its conditioned prediction, then selects a plan by pairwise comparison.The overview describes four stages, with prediction and refinement performed for each anchor.Source: §III · p. 3
A. Anchor Generation
- Anchors combine route-centerline or neighboring-lane paths with speed profiles, separating maneuver shape from its temporal progression.A cubic Bézier curve connects the current pose to a point on the lane, and varying that point can represent earlier or later maneuvers.Source: A. Anchor Generation · p. 3
- Farthest-point sampling selects anchors for spatial diversity, so refinement begins from distinct driving intents rather than variants of one maneuver.Sampling is initialized with the highest-scoring candidate; tracking and bicycle-model rollouts produce dynamically feasible candidates for ranking.Source: A. Anchor Generation · p. 3
B. Anchor-Conditioned Prediction
INTERACT adapts SMART to predict surrounding-agent futures conditioned on each ego anchor. It fixes the ego tokens during autoregressive sampling and reads out the non-ego agents’ predictions.
B. Anchor-Conditioned Prediction
- SMART represents agent trajectories as discrete motion tokens and factorizes joint futures autoregressively over time steps.The scene context includes the map and observed history.Source: B. Anchor-Conditioned Prediction · p. 3
- For each anchor, the model replaces the ego tokens during sampling, exactly conditioning subsequent predictions on that ego trajectory.The anchor is tokenized with the same tokenizer used by the model.Source: B. Anchor-Conditioned Prediction · p. 3
- With agent 0 indexing the ego vehicle and agents 1 through N the surrounding agents, all N ego-anchor-conditioned rollouts are computed in parallel.Source: B. Anchor-Conditioned Prediction · p. 3
C. Refinement
Each anchor is refined independently against its fixed conditioned prediction using CEM over control sequences. A continuous surrogate score evaluates planning criteria and penalizes deviation from the anchor.
C. Refinement
- CEM independently refines each anchor against its fixed conditioned prediction, iteratively sampling control sequences and refitting to the elite set.The best control sequence found during the search defines the refined trajectory.Source: C. Refinement · pp. 3–5
- Each control sequence specifies acceleration and steering rate, then rolls out through the kinematic bicycle model to produce an ego trajectory.The candidate trajectory is scored against the anchor and general planning criteria.Source: C. Refinement · pp. 3–4
- The continuous progress, TTC, and collision components provide gradients across map locations, unlike the piecewise-constant discrete PDM counterparts.The continuous collision term grades severity by footprint overlap.Source: Figure 3 · p. 4
- The continuous surrogate score evaluates comfort, collision avoidance, drivable-area compliance, and interaction-dependent criteria against predicted agent trajectories.It replaces discrete collision, drivable-area, driving-direction, and TTC terms with continuous counterparts and uses continuous progress.Source: C. Refinement · p. 4
- The search starts from the retained anchor controls, and its second score term penalizes deviations from the anchor within a local refinement region.The initial exploration range is ρ=0.15, scaled by the admissible control range.Source: C. Refinement · p. 4, Figure 2 · p. 4
- CEM samples use temporally correlated, unit-variance colored noise with β=2, concentrating the search on smooth control sequences rather than white-noise chatter.A population of P control sequences is drawn at each iteration.Source: C. Refinement · p. 4
D. Final Plan Selection
After refinement, the anchor-specific processes run independently, with synchronization needed only to select the final plan.
D. Final Plan Selection
- Each anchor’s warm start, sampling, rollout, and scoring process is fully independent and parallelizable.No re-prediction is required during the K CEM iterations because refinement stays within the validity region of its prediction.Source: D. Final Plan Selection · p. 5
- The trust region keeps each refinement within the validity region of its own prediction throughout the CEM iterations.This avoids querying the predictor again during those iterations.Source: D. Final Plan Selection · p. 5
- Only the final plan selection requires synchronization across anchors.The final plan is given by the selection described after refinement.Source: D. Final Plan Selection · p. 5
A. Experimental Setup
INTERACT leads four reported benchmarks and ranks second on two, with its strongest gains on interaction-focused scenarios. The section also describes the benchmark suite, implementation settings, and comparisons with alternative planners.
A. Experimental Setup
- The evaluation covers five nuPlan benchmarks and interPlan's 80-scenario subset, 335-scenario full set, and 30-scenario lane-change subset.The suite also includes Val14, Test14-random, Test14-hard, and Dynamic-10, a 200-scenario split spanning ten interactive scenario types.Source: A. Experimental Setup · p. 5
B. Implementation Details
- INTERACT uses five 4-second anchors and runs CEM for ten iterations per anchor with 24 candidates and six elites.The trust-region weight is 0.1, except for anchors predicted to lead to a collision, which use λ=0.Source: B. Implementation Details · p. 5
C. Main Results
- The comparison groups planners by whether they use no explicit agent prediction, constant-velocity forecasts, or reactive learned predictions.Constant-velocity forecasts are independent of the candidate ego plan; reactive learned methods model agent behavior conditioned on ego interaction.Source: C. Main Results · p. 5
- On Test14-random, gains over fixed-prediction approaches are smaller; consistent improvements on interaction-heavy splits suggest INTERACT's advantage is greatest when ego plans affect surrounding traffic.TerraZero scores higher on Val14 and interPlan, while INTERACT is ahead by +7.80 on interPlan-full; TerraZero results are not reported on all splits, limiting direct comparison.Source: C. Main Results · pp. 5–6
- INTERACT leads Test14-hard (85.28), Dynamic-10 (94.65), interPlan-LC (89.07), and interPlan-full (79.11), ranking second on Val14 (93.18) and Test14-random (92.48).Its gains over PDM-Closed and HYPE on Dynamic-10 are +3.15 and +3.38; it improves over BIBeR-CV by +2.28 on Test14-hard and +4.54 on interPlan-LC.Source: C. Main Results · p. 5, Table I · p. 6
D. Ablation Studies
The ablations examine how prediction choice, trust-region strength, anchor diversity, and refinement components affect performance. Their results emphasize the value of conditioned predictions in lane-change interactions and a balance between maneuver variety and local refinement.
Trust-region weight
- With λ=0, progress rises by +4.0 on Test14-hard, but TTC compliance falls by 6.3 there and 6.7 on interPlan-LC; λ=0.5 reduces Test14-hard progress by −2.5 without added safety gains.The default λ=0.1 achieves the highest overall score on both benchmarks, balancing local trajectory refinement against the validity range of anchor-conditioned predictions.Source: D. Ablation Studies · p. 6, Figure 4 · p. 6
Component ablation
- Removing any component decreases overall score on both benchmarks; removing CEM refinement reduces scores by −2.92 on Test14-hard and −4.95 on interPlan-LC.Without refinement, progress falls by −7.40 on Test14-hard and −6.60 on interPlan-LC, while the anchor set supplies strong, conservative driving intents.Source: D. Ablation Studies · pp. 6–7
Continuous scoring
- Removing the continuous score reduces performance by −2.19 on Test14-hard and −3.07 on interPlan-LC.The continuous formulation gives CEM a more informative ranking of candidates with different safety and progress levels, including candidates meeting the same discrete criterion.Source: D. Ablation Studies · p. 7
Prediction ablation
- On interPlan-LC, unconditioned prediction reduces overall score by −4.36, while constant-velocity prediction reduces it by −3.79 and TTC compliance by −3.33.On Test14-hard, unconditioned prediction performs nearly identically, with a +0.14 overall-score difference; constant-velocity prediction reduces overall score, TTC compliance, and comfort by −1.24, −1.47, and −1.47.Source: D. Ablation Studies · p. 7
- On interPlan-LC, unconditioned prediction reduces progress by −6.12; the results support conditioning on the anchor when surrounding-agent behavior depends on the intended ego maneuver.Constant-velocity prediction also reduces comfort on Test14-hard by −1.47.Source: D. Ablation Studies · p. 7
Anchors
- Increasing anchors from one to five improves progress and overall scores; adding more yields little benefit and increases at-fault collisions on Test14-hard.On interPlan-LC, progress and score saturate at five anchors, which cover the lane-change intents evaluated there.Source: D. Ablation Studies · p. 7
E. Runtime Analysis
Runtime is reported per anchor, with anchors evaluated independently and therefore in parallel. The section also notes the compute-performance trade-off from shortening prediction horizon and the effect of anchor count.
Runtime
- Runtime is reported per anchor averaged over ten interactive scenarios, and anchors can be evaluated in parallel.SMART inference time is reported per token, or rollout step; ten tokens cover the default five-second prediction horizon.Source: E. Runtime Analysis · pp. 7–8
Anchor-count sensitivity
- Increasing anchors from one to five improves progress, while additional anchors add at-fault collision risk on Test14-hard without improving score.On interPlan-LC, both progress and score saturate as further anchors add little benefit.Source: Figure 5 · p. 8
Runtime
- Shortening the prediction horizon reduces inference cost proportionally and causes a slight performance drop.The planned trajectory spans four seconds, with one additional second used to evaluate end-of-horizon time-to-collision.Source: E. Runtime Analysis · p. 8
V. CONCLUSION AND FUTURE WORK
INTERACT combines anchor-conditioned prediction with trust-region trajectory refinement, separating prediction across maneuver intents from optimization within each intent. The paper reports strong performance across nuPlan scenarios, with large gains on challenging interactive benchmarks, and outlines two directions for future work.
Conclusion
- INTERACT separates prediction across maneuver intents from optimization within each intent, using diverse anchors and local CEM refinement to adapt plans to predicted interactions.This avoids repeatedly querying the prediction model during optimization.Source: §V · p. 8
- The nuPlan closed-loop experiments show strong performance across varied scenarios, with large gains on challenging, highly interactive benchmarks.Source: §V · p. 8
Future Work
- Future work may adapt trust regions to prediction sensitivity or use end-to-end model trajectories as anchors for reactive-prediction refinement.Source: §V · p. 8