Source-linked AI summary
DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving
Zhiyu Huang, Peter Karkus, Boris Ivanovic, Yuxiao Chen, Marco Pavone, Chen Lv
TL;DR
Tree-policy planning requires efficient ego-conditioned predictions and cost evaluation that reflects human driving preferences. DTPP jointly trains these components with a query-centric Transformer and learned context-aware costs; it improves planning over learning-based methods and single-stage planning while closely matching a top-performing planner.
Problem
Tree-structured planning needs efficient ego-conditioned predictions across multiple branches and cost evaluation that captures intricate agent interactions difficult to quantify manually.
Source: §I · p. 1Method
Results
DTPP scores significantly better than learning-based policies and IDM, closely matches PDM(-Closed), and tree-based planning outperforms single-stage planning in closed-loop performance.
Source: C. Main Results · p. 5, D. Ablation Studies · p. 6Takeaways & Limitations
Joint training outperforms separate training, and learned costs can make node pruning effective without harming planning performance.
Source: C. Main Results · p. 5Future work includes real-world experiments, so the paper does not report results from such experiments.
Source: §V · p. 6
Abstract
from arXiv · showhide
Motion prediction and cost evaluation are vital components in the decision-making system of autonomous vehicles. However, existing methods often ignore the importance of cost learning and treat them as separate modules. In this study, we employ a tree-structured policy planner and propose a differentiable joint training framework for both ego-conditioned prediction and cost models, resulting in a direct improvement of the final planning performance. For conditional prediction, we introduce a query-centric Transformer model that performs efficient ego-conditioned motion prediction. For planning cost, we propose a learnable context-aware cost function with latent interaction features, facilitating differentiable joint learning. We validate our proposed approach using the real-world nuPlan dataset and its associated planning test platform. Our framework not only matches state-of-the-art planning methods but outperforms other learning-based methods in planning quality, while operating more efficiently in terms of runtime. We show that joint training delivers significantly better performance than separate training of the two modules. Additionally, we find that tree-structured policy planning outperforms the conventional single-stage planning approach.
I. INTRODUCTION
Tree-structured planning addresses the need to evaluate multiple possible futures, but depends on efficient ego-conditioned prediction and cost evaluation that reflects human driving preferences. DTPP brings these components together, building on prior work in conditional prediction, cost learning, and joint prediction and planning.
I. INTRODUCTION
- Tree-policy planners evaluate multiple possible futures using trajectory and scenario trees, then select an action through a discrete MDP.Each trajectory-tree branch represents a candidate ego trajectory; scenario-tree branches represent predicted outcomes for neighboring agents.Source: §I · p. 1
II. RELATED WORK
- Manually designed linear costs may miss intricate agent interactions, motivating learned context-aware cost features and weights alongside handcrafted features.The proposed cost model is trained jointly with conditional motion prediction.Source: §I · p. 1, §II · p. 2
- Existing conditional prediction approaches often encode a single ego plan or process plans repeatedly, while DTPP queries a Transformer with an entire ego trajectory tree.Joint training with the cost model also regularizes conditioning on ego plans beyond ground truth and optimizes prediction for planning.Source: §II · p. 2
- Prior joint prediction-and-planning approaches may lack explicit action outcomes or costs, and some differentiable methods cannot model bidirectional ego-agent interaction.DTPP combines a tree-structured planner with efficient conditional prediction and neural cost evaluation for joint training.Source: §II · p. 2
III. METHODOLOGY
DTPP combines a tree-policy planner with conditional prediction and learned cost evaluation, using cost-guided node pruning and expansion. It builds trajectory and scenario trees iteratively, then selects the optimal initial action by dynamic programming.
A. Planning Process
- At each planning stage, the planner expands ego trajectories, predicts other agents’ states conditioned on them, evaluates branch costs, and prunes low-scoring nodes.Expansion continues until a specified time limit or depth is reached.Source: §III · p. 2
III. METHODOLOGY
- DTPP adds learned-cost node pruning and conditional prediction to a tree-policy planner, jointly learning prediction and cost evaluation.The cost model combines features from joint agent trajectories with weights decoded from ego context, and is trained to guide test-time pruning.Source: §III · p. 2
A. Planning Process
- Dynamic programming selects the optimal initial-stage node after cost-guided expansion; experiments use two stages, though the framework supports more.The learned cost acts as a heuristic for expanding promising nodes.Source: A. Planning Process · p. 3
- Trajectory generation samples target speeds across map-based reference paths and uses a third-order polynomial for velocity before converting to Cartesian coordinates.Trajectories violating dynamic constraints are dropped.Source: A. Planning Process · p. 3
B. Conditional Prediction Model
The conditional prediction model encodes scene context once, then uses ego trajectory-tree queries to produce multi-stage predictions. Its attention masks preserve causal relationships while processing branches and timesteps in parallel.
B. Conditional Prediction Model
- The Transformer encoder processes past trajectories of nearby agents and the ego vehicle, plus nearby map elements, into scene context once per planning process.It uses LSTM trajectory encoders, MLP map encoders, and a three-layer self-attention Transformer.Source: B. Conditional Prediction Model · p. 3
- The decoder uses ego trajectory-tree information in its query to generate multi-stage predictions for all agents in a single forward pass.Queries combine agent-history, time, and ego-plan embeddings; the decoder uses two cross-attention modules and a shared MLP.Source: B. Conditional Prediction Model · pp. 3–4, Figure 3 · p. 4
- Causal masks exclude information from other branches and future timesteps, while allowing predictions across valid branches and timesteps to be parallelized.Invalid branches and timesteps are also masked during attention.Source: B. Conditional Prediction Model · p. 4
C. Cost Function
The cost function scores each candidate branch by summing ego, learned interaction, and collision features with learned context-aware weights. Its feature design combines learned behavior preferences with a handcrafted safety feature, while relying on lane-constrained trajectory generation rather than a separate lane-keeping cost.
C. Cost Function
- Each candidate branch’s cost is a weighted sum of ego, interaction, and collision features over the time horizon.The feature weights are learned and context-aware.Source: C. Cost Function · p. 4
- Ego features comprise normalized jerk, acceleration, speed, and lateral acceleration, allowing learned weights to balance comfort and progress.The weight decoder produces 10 dimensions: four ego, five interaction, and one collision feature.Source: C. Cost Function · p. 4
- Learned interaction features encode relative ego-agent position, heading, and velocity, then pool across agents and decode five latent features.The relative attributes are encoded in a 256-dimensional space before max-pooling.Source: C. Cost Function · p. 4
- The collision feature uses a handcrafted function to explicitly represent safety through distances between the ego and other agents.The function applies a Gaussian radial basis function to those distances.Source: C. Cost Function · p. 4
- Separate lane-keeping and route-following costs were unnecessary because the trajectory generator constrains the planner to a target lane.Source: C. Cost Function · p. 4
D. Model Training
Prediction training supervises other-agent trajectories for the ego plan closest to ground truth, while cost learning uses maximum entropy IRL. The approach makes prediction and planning differentiable together, with greater loss weight assigned to the first planning stage.
D. Model Training
- For conditional prediction, training selects the situation whose ego trajectory best matches ground truth and applies smooth L1 loss to other-agent predictions.The selected prediction branch is linked to the ego plan closest to ground truth.Source: D. Model Training · p. 4
- Maximum entropy IRL trains the cost function by comparing softmax-normalized plan scores with the ground-truth plan label using cross-entropy loss.Source: D. Model Training · pp. 4–5
- The prediction-planning structure is differentiable, and the first planning stage receives greater loss weight than the second.The stage weights are β1 = 1.0 and β2 = 0.2; the IRL-loss weight is α = 0.1.Source: D. Model Training · p. 5
A. Experimental setup
Evaluation uses nuPlan data and its associated simulator, with 10 dynamic scenario types represented in training and testing. The planning evaluation spans open-loop and two closed-loop tasks, alongside prediction displacement metrics.
A. Experimental setup
- The study uses 100k training scenarios from the nuPlan validation subset across 10 dynamic scenario types, each with an 8-second future horizon.Static scenario types were excluded.Source: A. Experimental setup · p. 5
- Testing uses 20 scenarios per type across 10 types, totaling 200 scenarios, each lasting 15 seconds.Source: A. Experimental setup · p. 5
- Planning evaluation covers open-loop, closed-loop non-reactive, and closed-loop reactive tasks; prediction evaluation reports non-ego ADE and FDE conditioned on the selected ego plan.The planning score reflects safety, efficiency, and comfort for closed-loop tasks, and similarity to human driving for open-loop.Source: A. Experimental setup · p. 5
B. Implementation Details
Implementation uses a two-stage, eight-second planning horizon with up to 30 branches. Training runs for 20 epochs on an RTX 3080 GPU, while inference uses the same GPU and other computations run on a CPU.
B. Implementation Details
- The eight-second horizon has two stages and at most 30 branches, retaining five nodes after the short-term stage and expanding each into six target states.The stages span 0–3 seconds and 3–8 seconds, respectively.Source: B. Implementation Details · p. 5
- Training used batch size 16 for 20 epochs on an NVIDIA RTX 3080 GPU, with AdamW and an initial learning rate of 1e-4.Source: B. Implementation Details · p. 5
- At the 10th epoch, model inference is conducted on the same GPU, while other computations run on an AMD 3900X CPU.Source: B. Implementation Details · p. 5
C. Main Results
DTPP performs strongly in closed-loop planning, closely matching PDM(-Closed) and outperforming learning-based and rule-based baselines. Its joint training and query-centric ego-conditioned prediction support effective planning with reduced runtime.
C. Main Results
- DTPP significantly outperforms learning-based policies and IDM in closed-loop tests, while closely matching the leaderboard-leading PDM(-Closed).PDM(-Closed) is highly optimized for nuPlan challenge metrics.Source: Table I · p. 5, C. Main Results · p. 5
- Joint training delivers the best planning results, while node pruning preserves performance and nearly halves runtime.With cost learning, pruning can remove less-promising nodes selected after the first stage.Source: C. Main Results · p. 5, Table II · p. 6
- At an intersection merge, predicted responses from interacting agents differ reasonably across ego plans.The authors say these conditional predictions help the cost model evaluate situations and score them more accurately.Source: C. Main Results · p. 5, Figure 5 · p. 6
- Ego-conditioned models slightly lag non-ego-conditioned models in prediction accuracy but can improve open- and closed-loop planning; query-centric prediction outperforms early fusion.Prediction metrics are calculated from the branch selected by the cost evaluation model.Source: Table IV · p. 6, Table III · p. 6, C. Main Results · p. 6
- The query-centric model predicts different branches in parallel with only a marginal runtime increase over non-ego-conditioned prediction, avoiding repeated scene encoding required by early fusion.The runtime comparison is reported in Table V.Source: Table V · p. 6, C. Main Results · p. 6
D. Ablation Studies
The ablations show that learned cost components and tree-structured planning improve closed-loop results. Better prediction accuracy alone does not ensure better closed-loop planning.
Influence of cost function learning
- Better prediction performance does not necessarily translate into better closed-loop planning, underscoring the value of co-designing and jointly training the models.Source: D. Ablation Studies · p. 6
- Learned latent interaction features are pivotal, while collision potential and context-aware cost weights improve closed-loop performance.The ablation evaluates prediction metrics and closed-loop planning scores across cost-learning settings.Source: Table VI · p. 6, D. Ablation Studies · p. 6
Influence of tree-structured planning
- Tree-based planning significantly outperforms single-stage planning in closed-loop tests at both 3 s and 8 s, despite lower prediction accuracy.Single-stage models have better prediction accuracy.Source: Table VII · p. 6, D. Ablation Studies · p. 6
V. CONCLUSIONS
DTPP jointly learns ego-conditioned prediction and planning costs for a tree policy planner. The authors report improved prediction performance and efficiency, and better planning with joint training and tree-based planning than with the stated alternatives.
V. CONCLUSIONS
- The authors report that joint training is crucial for optimal planning and that tree-based planning significantly outperforms single-stage trajectory planning.The prediction model also yields significantly better performance and efficiency.Source: §V · p. 6
- DTPP combines an efficiently ego-conditioned, query-centric Transformer predictor with a cost model using learned and handcrafted features and learned context-aware weights.The framework is differentiable and designed for a tree policy planner.Source: §V · p. 6
- Future work may extend DTPP to multi-modal probabilistic predictions and conduct real-world experiments.Source: §V · p. 6