PaperlayerView exact revision on arXiv

Source-linked AI summary

Model-Based Imitation Learning for Urban Driving

Anthony Hu, Gianluca Corrado, Nicolas Griffiths, Zak Murez, Corina Gurau, Hudson Yeo, Alex Kendall, Roberto Cipolla, Jamie Shotton

arXiv:2210.07729v2cs.CVcs.AIcs.RO

TL;DR

Urban driving requires models that capture scene geometry, dynamic agents, and ego behaviour from complex visual observations. MILE jointly learns a geometric world model and driving policy from offline expert demonstrations, achieving a 31% driving-score improvement in unseen CARLA town and weather conditions while generating plausible imagined futures and manoeuvres.

  • Problem

    Motion planning would benefit from an accurate model of the environment and dynamic agents, but urban driving demands geometric reasoning from high-resolution visual observations.

  • Method

    MILE jointly learns a compact latent world model and driving policy from offline expert demonstrations, using 3D geometry as an inductive bias and action-conditioned dynamics.

  • Results

    31% improvement in driving score was achieved over previous methods on CARLA in a new town and new weather conditions; MILE also predicts diverse futures and imagination-based driving plans.

  • Takeaways & Limitations

    MILE is a camera-only urban-driving model that jointly represents static scenes, dynamic scenes, and ego behaviour while supporting complex manoeuvres planned in imagination.

  • Takeaways & Limitations

    Inferring the driving reward from expert data and reducing dependence on bird’s-eye-view segmentation labels remain open problems.

Abstract

from arXiv · show

An accurate model of the environment and the dynamic agents acting in it offers great potential for improving motion planning. We present MILE: a Model-based Imitation LEarning approach to jointly learn a model of the world and a policy for autonomous driving. Our method leverages 3D geometry as an inductive bias and learns a highly compact latent space directly from high-resolution videos of expert demonstrations. Our model is trained on an offline corpus of urban driving data, without any online interaction with the environment. MILE improves upon prior state-of-the-art by 31% in driving score on the CARLA simulator when deployed in a completely new town and new weather conditions. Our model can predict diverse and plausible states and actions, that can be interpretably decoded to bird's-eye view semantic segmentation. Further, we demonstrate that it can execute complex driving manoeuvres from plans entirely predicted in imagination. Our approach is the first camera-only method that models static scene, dynamic scene, and ego-behaviour in an urban driving environment. The code and model weights are available at https://github.com/wayveai/mile.

1 Introduction

MILE jointly learns a world model and driving policy from offline expert data, using 3D geometry to handle high-resolution urban driving observations. It predicts plausible futures and imagination-based plans, achieving a 31% driving-score improvement in unseen CARLA conditions.

  • MILE jointly learns a model of the world and a driving policy from high-resolution urban driving observations.
  • 3D geometry is used as an inductive bias by lifting image features into a bird’s-eye view representation.
  • MILE models action-conditioned futures, enabling plausible long-horizon predictions and complex manoeuvres planned entirely in imagination.
  • 31% improvement in driving score was achieved against previous methods in CARLA when tested in a new town and new weather conditions.
  • MILE is trained offline on expert driving data without online environment interaction or access to a ground-truth reward.

2 Related Work

Prior work addressed imitation learning, 3D scene representation, and world modelling separately or under stronger information assumptions. MILE combines these elements in a camera-only urban-driving model that jointly represents static scenes, dynamic agents, and ego behaviour.

  • Imitation learning: Imitation-learning systems developed from modular pipelines toward end-to-end policies, with later methods using dataset aggregation, perturbations, or bird’s-eye-view supervision.
  • 3D scene representation: 3D scene-representation methods commonly lift multi-camera image features into a shared bird’s-eye-view grid or learn image-to-view mappings with transformers.
  • World models: World models have mainly been explored in reinforcement learning with rewards and online interaction, while earlier visual models often used simple 64 × 64 inputs.
  • Trajectory forecasting: Trajectory-forecasting methods predict dynamic-agent paths but generally do not control the ego-agent or learn expert behaviour from demonstrations.
  • MILE’s position: MILE jointly models static scenes, other agents, and ego behaviour without ground-truth physical states or offline HD maps, using only camera inputs.

3 MILE: Model-based Imitation LEarning

MILE jointly learns a compact latent world model and driving policy from visual observations, expert actions, and bird’s-eye-view supervision. Its probabilistic architecture combines deterministic history, stochastic latent states, geometric observation encoding, and decoders for images, segmentation, and actions.

  • 3 MILE: Model-based Imitation LEarning: MILE jointly models latent world dynamics and autonomous-vehicle behavior from video frames, expert actions, and bird’s-eye-view labels.The model uses deterministic history and stochastic latent states to represent temporal evolution.
  • 3.1 Probabilistic Generative Model: The generative model predicts observations, bird’s-eye-view segmentation, and actions from latent states while learning a prior over stochastic dynamics.Its components include a recurrent transition, prior network, image decoder, bird’s-eye-view decoder, and policy.
  • 3.2 Variational Inference: Training uses deep variational inference to infer latent variables and optimize reconstruction, action, segmentation, and posterior-prior matching objectives.The resulting observation, action, and segmentation losses correspond to mean-squared error, L1, and cross-entropy terms.
  • 3.3.1 Observation Encoder: 3D image features are lifted using predicted depth and camera geometry, then pooled into bird’s-eye view before compression into a low-dimensional vector.This geometric encoding is intended to preserve driving-relevant spatial structure while producing a compact representation.
  • 3.3.1 Observation Encoder: The observation embedding concatenates compressed image features with route-map and speed features before estimating the posterior latent state.The posterior network predicts Gaussian parameters from the current embedding, previous action, and deterministic history.

4 Experimental Setting

The experiments train and assess MILE using offline CARLA driving data generated by an expert reinforcement-learning agent. Evaluation reports challenge metrics and cumulative reward, with training using fixed simulator towns, weather conditions, and implementation settings.

  • Dataset: An expert reinforcement-learning agent generates the training data using privileged bird’s-eye-view segmentations and vehicle measurements.The agent produces more diverse runs and higher driving performance than CARLA’s built-in autopilot.
  • Dataset: The training corpus contains 2.9M frames, or 32 hours of driving data, collected at 25Hz across four towns and four weather conditions.The stored timestep tuple includes camera imagery, route information, vehicle speed, actions, and labels.
  • Training: Training runs for 50,000 iterations with batch size 64 and sequence length T = 12 on 8 V100 GPUs using AdamW.The learning rate is 10^-4 and weight decay is 0.01.
  • Metrics: CARLA evaluation reports route completion, infraction penalty, driving score, and cumulative reward to measure on-road performance.The authors note that challenge metrics can obscure competencies such as lane keeping and appropriate speed.

5 Results

MILE is evaluated on unseen CARLA conditions and compared with prior methods and ablations. It achieves strong camera-only driving performance, while experiments show benefits from 3D geometry, probabilistic modelling, recurrent inference, and multi-modal prediction.

  • 5.1 Driving Performance: 31% relative improvement in driving score over LAV is achieved on a new town and new weather conditions in CARLA.The evaluation uses an unseen town and weather conditions.
  • 5.1 Driving Performance: MILE outperforms previous works on all metrics and uses only RGB images from the front camera.Some comparison methods additionally use LiDAR.
  • 5.2 Ablation Studies: Cumulative reward drops from 7621 to 4564 when MILE is evaluated without 3D lifting, highlighting the value of 3D geometry.The corresponding single-frame model reward drops from 6084 to 1878 without 3D lifting.
  • 5.2 Ablation Studies: Cumulative reward drops from 7621 to 6084 without the prior/posterior KL-divergence loss used for probabilistic modelling.The paper motivates probabilistic modelling by the existence of multiple valid driving behaviours.
  • 5.3 Fully Recurrent Inference in Closed-Loop Driving: 43.0Hz fully recurrent inference is 7× faster than the 6.2Hz fixed-context reset-state approach while matching its driving performance.The fully recurrent state is updated with new observations rather than recomputed from the last T observations.
  • 5.4 Long Horizon, Diverse Future Predictions: MILE predicts two distinct 8-second futures, including driving through a green light or stopping when the imagined light turns red.The futures are decoded into bird’s-eye-view semantic segmentation.

6 Insights from the World Model

The world model is examined across latent-state sizes and deployment conditions. A compact 512-dimensional state supports recurrent driving, imagination, and diverse future prediction, while larger latent spaces can reduce performance.

  • 6.1 Latent State Dimension: The baseline latent state is a low-dimensional 1D vector of size 512, compared with spatial tensor alternatives of up to 160k distributions.The alternatives are evaluated in both reset-state and fully recurrent settings.
  • 6.1 Latent State Dimension: In the reset-state setting, driving performance decreases as latent-state dimensionality increases.The paper suggests that matching multivariate prior and posterior distributions becomes more difficult with increasing dimensionality.
  • 6.1 Latent State Dimension: The original 512-dimensional state shows almost no driving-performance difference between reset-state and fully recurrent deployment.Fully recurrent deployment requires discarding irrelevant information and updating the internal state from new observations.
  • 6.2 Driving in Imagination: MILE retains the same driving performance with up to 30% of the drive conducted in imagining mode without visual observations.The evaluation alternates between observing and imagining modes in the fully recurrent setting.

7 Conclusion

MILE jointly learns a driving policy and world model from offline expert demonstrations, exploits geometric inductive biases, and achieves state-of-the-art CARLA performance. It predicts diverse future states and actions well enough to drive from imagined plans, while reward inference and reduced dependence on bird’s-eye-view labels remain open problems.

  • MILE jointly learns a driving policy and world model from offline expert demonstrations alone.
  • MILE exploits geometric inductive biases, operates on high-dimensional visual inputs, and sets a new state-of-the-art on CARLA.
  • MILE predicts diverse and plausible future states and actions, enabling driving from plans entirely predicted in imagination.
  • Inferring the driving reward function from expert data remains an open problem for enabling explicit planning in the world model.

A.1 Driving in Imagination

The evaluation alternates between observing image frames and imagining future states and actions from the recurrent latent state. MILE maintains driving performance while imagining for up to 30% of the time and can negotiate a roundabout in imagination.

  • The fully recurrent deployment alternates fixed two-second windows of imagining future states and actions with observing new image frames.The model imagines without observing new images, then updates its world knowledge from newly observed frames.
  • The imagining-to-observing ratio varies from 0 to 0.6, with driving score and bird’s-eye-view segmentation intersection-over-union used for evaluation.MILE is compared with a one-frame baseline that has no memory and uses only a single image frame.
  • 30% imagining causes no significant drop in driving performance, while predicted states remain fairly accurate even with substantial imagining.After 30%, driving score decreases but remains much higher than the one-frame counterpart.
  • MILE successfully negotiates a roundabout while driving in imagination.
  • Lower image resolutions make traffic lights and pedestrians harder to distinguish, which reduces driving score and cumulative reward.The performance drop is most severe for the infraction penalty metric.

A.3 Training Town Evaluation

MILE improves driving performance over Roach in training-seen town and weather conditions. The evaluation also documents CARLA benchmark settings for comparisons with TransFuser and LAV.

  • MILE achieves a 21% relative improvement in driving score over Roach on training-seen town and weather conditions.The metrics are averaged across three runs, and Town03 is described as the most complex town.
  • The evaluation includes CARLA settings used for comparisons with TransFuser and LAV.TransFuser uses Town05 test routes in ClearNoon; LAV uses routes in Town02 and Town05 across specified weather conditions.

B Lower Bound Derivation

The derivation constructs an evidence lower bound by decomposing the KL divergence between variational and posterior latent distributions, then applying non-negativity of KL divergence. The full MILE description provides the associated graphical models and component specifications.

  • The variational distribution is defined over latent histories conditioned on observations and preceding actions, and is compared with the posterior distribution.
  • Non-negativity of the posterior KL divergence yields the evidence lower bound on log p(o1:T, y1:T, a1:T).
  • The expected log-likelihood term decomposes into observation, segmentation, and action likelihoods at each timestep.
  • The KL term is expanded using the model’s factorisations and recursively decomposed across latent timesteps.The derivation notes that one simplification follows from q(h_t|h_{t−1}, s_{t−1}) = p(h_t|h_{t−1}, s_{t−1}).
  • The full MILE description includes graphical models, parameter counts, hyperparameters, and inference and generative network specifications.

C.3 Details on the Network and on Training.

The training and network details combine geometry-aware feature lifting with latent-state regularization for long-horizon prediction. The implementation also specifies continuous vehicle controls and randomized traffic conditions.

  • Network: Image features are lifted into 3D using inverse camera intrinsics and predicted depth, then transformed into vehicle coordinates with rigid-body motion.This geometric operation supports the bird’s-eye-view representation used by the model.
  • Training: Observation dropout randomly replaces posterior states with prior states during training, forcing the model to predict states across randomly sampled temporal horizons.The number of prior unrolls follows a geometric distribution governed by the dropout probability.
  • Implementation: The action space is R2, comprising acceleration in [−1, 1] and steering in [−1, 1], with signs corresponding to braking/throttle and left/right turning.Negative acceleration denotes braking, while steering signs distinguish left from right turns.
  • Training: Training traffic is randomized by sampling vehicle and pedestrian counts from town-specific uniform intervals.The sampling intervals are summarized in Table 13.

D.2 Metrics

The evaluation reports route completion, infraction penalty, and driving score, alongside normalized cumulative reward. Performance is measured in both unseen-town/unseen-weather and training-town/training-weather settings, while the paper cautions that driving score has a comparability limitation.

  • Metrics: CARLA on-road performance is measured using route completion, infraction penalty, and driving score.These are the metrics reported for the CARLA challenge.
  • Metrics: Route completion measures the percentage of a scenario route completed, while infraction penalty multiplicatively penalizes collisions, traffic violations, and other infractions.Both quantities lie in [0, 1].
  • Metrics: Normalized cumulative reward averages timestep rewards over the deployment duration to account for simulations with different lengths.The cumulative reward is divided by the number of deployed timesteps.
  • Metric caveat: Driving score comparisons are constrained because multiplying route completion by infraction penalty conflates progress with the risk of accumulating infractions.The paper recommends timestep-level cumulative reward as an alternative metric.
  • Evaluation settings: Evaluations are repeated three times in 10 Town05 scenarios with four unseen weathers and 20 Town03 scenarios with four training weathers.The first setting tests new town and new weather conditions; the second uses training conditions.
Loading 2210.07729v2…