Source-linked AI summary
Long-Horizon Consistent and Interaction-Aware World Models for Multi-Style End-to-End Driving
Yuxuan Han, Kunyuan Wu, Liyunong Yang, Zilu Wang, Cansen Jiang, Yi Xiao, Liang Hu
TL;DR
World-model driving methods face challenges in maintaining long-horizon consistency, modeling ego-relevant interactions, and adapting to diverse driving styles. Crisp-Drive combines temporal regularization, state disentanglement, and multi-style trajectory-level policy optimization. On Bench2Drive it outperforms the previous best world-model-based method, and it is deployed on a real automated guided vehicle platform.
Problem
Existing world models face long-horizon temporal inconsistency, inadequate modeling of ego-environment interactions, and limited adaptability to diverse driving styles.
Source: §I · p. 1Method
Crisp-Drive combines historical-latent temporal regularization, ego-relevant state disentanglement, and multi-style Group Relative Policy Optimization.
Source: §I · p. 2Results
84.12 driving score and 63.21% success rate, outperforming the previous best world-model-based method on closed-loop Bench2Drive evaluation.
Source: §V · p. 12Takeaways & Limitations
Deployment on a real automated guided vehicle platform demonstrates sim-to-real transfer with robust performance in dynamic traffic interactions.
Source: §V · p. 12
Abstract
from arXiv · showhide
End-to-end autonomous driving has increasingly adopted world model-based reinforcement learning frameworks to improve learning efficiency through \textit{imagined rollouts}. However, existing world models suffer from three key limitations: temporal inconsistency in long-horizon imagined rollouts, inadequate modeling of ego-environment interactions, and limited adaptability to diverse driving styles. To address these challenges, we propose \textit{StyleDrive}, a world-model-based learning framework that jointly enforces long-horizon consistency, explicitly disentangles interactive traffic states, and supports multi-style policy optimization within a unified learning paradigm. First, we introduce a temporal consistency regularization that integrates historical latent states through gated cross-attention, stabilizing long-horizon imagined rollouts and mitigating error accumulation. Second, we design an explicit state disentanglement module that separates ego-relevant from ego-irrelevant interactive states, enabling more interpretable and efficient decision-making in complex traffic scenarios. Third, we enable multi-style driving behaviors through Group Relative Policy Optimization, which replaces per-step reward optimization with trajectory-wise relative advantages, reducing reward variance and supporting diverse driving styles without retraining. We evaluate StyleDrive on the Bench2Drive closed-loop driving benchmark, achieving a driving score of 88.44 (+17.08 over the previous best world model-based method) and a success rate of 66.82 (+16.58). Furthermore, we deploy StyleDrive on a real automated guided vehicle platform and demonstrate promising sim-to-real transfer capability in dynamic driving scenarios.
I. INTRODUCTION
The introduction frames world-model reinforcement learning as a way to improve learning efficiency while identifying temporal inconsistency, weak interaction modeling, and limited style adaptability as unresolved challenges. Crisp-Drive combines temporal regularization, interaction-relevant state decomposition, and trajectory-level relative policy optimization to address them.
A. Reinforcement Learning for End-to-End Driving
- World-model RL uses learned predictive dynamics and imagined rollouts to optimize policies with less direct environmental interaction, addressing model-free RL’s costly interaction demands.Imitation learning can be vulnerable to distribution shift when demonstrations lack rare safety-critical events, while direct driving exploration can be costly and unsafe.Source: §I · p. 1, A. Reinforcement Learning for… · p. 2
I. INTRODUCTION
- Crisp-Drive combines history-conditioned temporal correction, interaction-relevant state decomposition, and trajectory-level relative policy optimization to support style-specific policies using more reliable imagined trajectories.Its contributions include gated cross-attention over historical latent states, separation of ego-relevant from ego-irrelevant states, and joint multi-style training without reward rescaling.Source: §I · pp. 1–2
B. World-Model Reinforcement Learning and Interaction-Aware Driving
- Reliable policy imagination requires both long-horizon temporal consistency and attention to decision-critical interactions, a combination prior approaches had not jointly addressed.Iso-Dream separates controllable and noncontrollable dynamics, while GraphWorld [30] models ego–agent interactions; the introduction identifies their joint integration during imagined policy optimization as unresolved.Source: B. World-Model Reinforcement Learning… · p. 2
C. Style-Conditioned Driving and Relative Policy Optimization
- Style-conditioned policies can specify desired behavior, but command conditioning alone does not compare alternative trajectories under different driving styles.Crisp-Drive couples style-conditioned trajectory-group optimization with an interaction-aware latent world model rather than training separate policies.Source: C. Style-Conditioned Driving and… · p. 2
A. Problem Setting
The method casts style-conditioned driving as partially observable control and trains a shared multi-style policy on a learned world model. Its world-model stage incorporates historical latent context and separates ego-relevant from ego-irrelevant interactive states.
A. Problem Setting
- The policy uses a learned latent state to approximate the driving belief state, with one episode-fixed style condition and shared parameters across styles.Actions are optimized for condition-dependent discounted return.Source: A. Problem Setting · p. 3
B. Architecture
- Training first learns an RSSM world model from observation reconstruction, reward and continuation prediction, and latent-state regularization, then freezes it for policy learning.Policies are PPO-pretrained and GRPO-finetuned for distinct driving styles.Source: B. Architecture · p. 3
1) World Model Representation Learning:
- The world model refines current prior predictions using the preceding m prior states through gated cross-attention.This integrates historical latent context into the prediction.Source: 1) World Model Representation Learning · p. 4
- Interactive states are split into ego-relevant and ego-irrelevant components using spatial attention and semantic relevance estimates.The relevance probability uses a boundary and sigmoid temperature, and the resulting weights aggregate relevant and irrelevant regions separately.Source: 1) World Model Representation Learning · p. 4
- World-model training combines continuation, reward, and observation losses with branchwise latent regularization and interaction-relevance supervision.Relevance targets are offline pseudo-labels, and the loss coefficients balance the component objectives.Source: 1) World Model Representation Learning · p. 4
2) Policy Learning:
Policy learning uses a frozen world model and a shared style-conditioned policy, with PPO initialization followed by GRPO refinement. Driving styles are represented by continuous reward configurations, and the reward combines route completion, smoothness, and risk penalties.
2) Policy Learning:
- The reward combines task completion, driving smoothness, and risk penalties, including collision and lane-deviation penalties.Task completion includes route checkpoints, destination arrival, and a time penalty; smoothness targets a longitudinal velocity while penalizing velocity deviations.Source: 2) Policy Learning · p. 5
- Conservative, Moderate, and Aggressive styles are representative evaluation points in a continuous reward-configuration space, not separately trained policies.A sampled configuration stays fixed throughout its rollout and conditions the shared policy.Source: 2) Policy Learning · pp. 4–5
- PPO jointly trains the shared actor, critic, and style encoder on imagined rollouts generated by the frozen world model.Each sampled reward configuration is held fixed throughout its imagined episode.Source: 2) Policy Learning · pp. 4, 6
- GRPO refines the PPO-initialized shared actor and style encoder using trajectory-wise relative advantages, while the PPO-trained critic remains fixed.The policy is refined on parallel closed-loop rollouts after PPO initialization.Source: 2) Policy Learning · pp. 5–6
1) Datasets:
Bench2Drive Base is used for world-model training and official closed-loop evaluation, while a self-collected real-world video set is reserved for open-loop rollout evaluation.
1) Datasets:
- Bench2Drive Base, built on CARLA, supplies world-model training data and closed-loop evaluation under the official 220-route protocol.The paper also identifies a separate role for its self-collected real-world video set.Source: 1) Datasets · p. 6
- A self-collected real-world video set is used only for open-loop rollout evaluation.Source: 1) Datasets · p. 6
- Dataset scale, collection conditions, splits, and each source’s precise role are documented in Supplementary Material, Sec. I-A.Source: 1) Datasets · p. 6
2) Metrics:
Evaluation reports standard Bench2Drive driving metrics alongside cross-style trajectory diversity and long-horizon visual and geometric consistency measures.
2) Metrics:
- The standard Bench2Drive metrics are Success Rate, Driving Score, Driving Efficiency, Driving Comfort, and five Multi-Ability scores.Source: 2) Metrics · p. 6
- Cross-style trajectory diversity and long-horizon consistency are additionally evaluated using FID, FVD, NTA-IoU, and NTL-IoU.Source: 2) Metrics · p. 6
- Metric definitions, the Diversity equation, normalization, and implementation details are provided in Supplementary Material, Sec. I-B.Source: 2) Metrics · p. 6
3) Training Details:
The world model is trained on Bench2Drive, then a shared style-conditioned policy is initialized with imagined-rollout PPO and refined with closed-loop GRPO. Evaluation uses common routes and five independent seeds.
3) Training Details:
- The world model is trained for 30 epochs on 8 NVIDIA A6000 GPUs with a batch size of 16.Source: 3) Training Details · p. 6
- PPO trains the shared policy for 100,000 imagined episodes with an imagination horizon of τ = 6, corresponding to 3 seconds at an effective rollout frequency of 2 Hz.Source: 3) Training Details · p. 6
- GRPO refines the PPO-initialized policy using K = 6 parallel closed-loop CARLA rollouts for each matched style–scenario group.Source: 3) Training Details · p. 6
- Crisp-Drive, its ablations, and reproduced baselines marked by † are evaluated on the same routes using five independent seeds, with results reported as mean ± standard deviation.Source: 3) Training Details · p. 6
4) Real-world Deployment:
Crisp-Drive leads the reported Bench2Drive comparisons and is deployed on a Scout automated guided vehicle equipped for real-world operation.
4) Real-world Deployment:
- The policy is deployed on a Scout automated guided vehicle with onboard inference and data processing on an NVIDIA Jetson AGX Orin.The platform includes a Bynav X1 GNSS/IMU, RoboSense Helios LiDAR, and six 1920 × 1080 cameras.Source: 4) Real-world Deployment · p. 6
1) Quantitative Results:
- 84.12 DS, 63.21% SR, and 49.51 Comf. are the highest among evaluated methods; Crisp-Drive also records DE of 192.27.It outperforms multi-modal Transfuser++ in all four metrics, despite using camera observations only.Source: 1) Quantitative Results · p. 6, Table I · p. 7
- Crisp-Drive has the best camera-only scores in Merging (45.22), Emergency Brake (80.00), Give Way (65.00), and Traffic Sign (70.00), plus the highest single-modal Mean (65.04).It surpasses Epona across all five abilities; ORION scores higher in Overtaking, while Crisp-Drive has better comfort (49.51 vs. 17.38).Source: Table II · p. 7, 1) Quantitative Results · p. 7
- Crisp-Drive provides a better balance among task completion, safety, efficiency, and driving smoothness.The Scout AGV uses an NVIDIA Jetson AGX Orin, GNSS/IMU, LiDAR, and six cameras.Source: 1) Quantitative Results · p. 7
2) Qualitative Results:
Crisp-Drive is compared with other methods in challenging CARLA interactions, while ablations examine its temporal modeling, relevance-aware state decomposition, and multi-style policy optimization.
2) Qualitative Results:
- Crisp-Drive yields to the fire truck and pedestrian, overtakes stopped traffic collision-free, and negotiates lane sharing, unlike UniAD and Iso-Dream++.The four comparisons show CARLA scenes, BEV trajectories, and third-person views.Source: 2) Qualitative Results · pp. 7–8, Figure 5 · p. 9
C. Ablation Study
- Combining distance and TTC supervision achieves 84.12 DS, 63.21% SR, 192.27 DE, and 49.51 Comf.; unsupervised attention and random labels underperform semantically supervised variants.The basic ego–environment split reaches 69.12 DS and 51.53% SR.Source: C. Ablation Study · p. 8
- Temporal regularization raises DS by 14.01, SR by 13.98%, DE by 50.14, and Comf. by 26.00 over its ablated variant.Crisp-Drive has the lowest reported FID and FVD across 1–10 seconds; beyond 5 seconds, both metrics degrade more slowly than other methods.Source: C. Ablation Study · pp. 8–9
- 84.12 DS, 63.21% SR, 69.4/428.2 10-second FID/FVD, and 0.524 Diversity are achieved by the full three-component model.The study reports consistent improvements over the baseline and single-component variants.Source: C. Ablation Study · p. 8, Table III · p. 9
- In CARLA and real-world rollouts, Crisp-Drive maintains temporal coherence, while Iso-Dream++ and Epona show inconsistent predictions and visual artifacts.Fig. 6 highlights frames with temporally inconsistent predictions using red boxes.Source: C. Ablation Study · pp. 9–10, Figure 6 · p. 10
- Full GRPO improves over PPO by 19.05 DS, 14.09 percentage points in SR, and 28.96 Comf., while increasing Diversity by 0.373.Removing KL decreases DS from 84.12 to 80.48 and Diversity from 0.524 to 0.468.Source: C. Ablation Study · p. 8, Table VI · p. 10
- Conservative, Moderate, and Aggressive styles achieve Mean scores of 60.02, 65.04, and 61.16, respectively.Conservative scores higher than Aggressive in Emergency Brake and Give Way; Aggressive scores higher in Merging and Overtaking.Source: C. Ablation Study · p. 10
- Increasing imagined-rollout τ from 3 to 6 improves DS, SR, and DE by 7.00, 3.04 percentage points, and 20.16; increasing τ from 9 to 12 lowers DS and SR.The default settings are m = 5 and τ = 6, selected to balance performance, temporal consistency, and computational cost.Source: C. Ablation Study · p. 10
D. Real-World Deployment
Crisp-Drive combines temporal regularization, state disentanglement, and GRPO for multi-style end-to-end driving. The paper reports Bench2Drive results outperforming the previous best world-model method and evaluates transfer on an AGV across safety-critical tasks.
D. Real-World Deployment
- The AGV evaluation covers oncoming vehicles and jaywalking pedestrians, static-obstacle avoidance, and left- and right-turn maneuvers.It is a closed-loop evaluation of transferability under controlled real-world conditions.Source: D. Real-World Deployment · p. 10
- 90.00%, 95.00%, and 91.67% success rates on Dynamic, Static, and Turning tasks, respectively, with seven collisions across 180 trials.Conservative driving had the largest average minimum clearance (1.41 m) and lowest jerk (0.54); Moderate had the highest overall success rate (95.00%); Aggressive had 88.33% overall success, 0.94 m clearance, and 0.82 jerk.Source: D. Real-World Deployment · p. 10
- Representative executions show scene-appropriate maneuvers while preserving distinguishable Conservative and Aggressive trajectories.Figure 7 pairs scenario views and trajectories under two styles with corresponding LiDAR BEV maps.Source: D. Real-World Deployment · p. 10, Figure 7 · p. 11
V. CONCLUSION
- Gated cross-attention over historical latent states stabilizes long-horizon dynamics; state disentanglement isolates ego-relevant states, and GRPO jointly trains diverse behaviors through intra-group relative advantages.The conclusion links these components to mitigating error accumulation, improving interpretability and risk awareness, and supporting diverse driving styles.Source: §V · pp. 11–12
- 84.12 driving score and 63.21% success rate on closed-loop Bench2Drive, outperforming the previous best world model-based method.The conclusion also reports successful sim-to-real transfer on a real AGV, with robust performance in dynamic traffic interactions.Source: §V · p. 12