Source-linked AI summary
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
Mayank Bansal, Alex Krizhevsky, Abhijit Ogale
TL;DR
The paper asks how imitation learning can produce autonomous driving robust enough for a real vehicle when pure behavior cloning fails in closed loop. It combines mid-level scene and trajectory representations with synthesized trajectory perturbations and augmented losses, then reports successful simulation behavior and real-world driving. The authors conclude that constrained exploration around expert behavior can improve robustness while remaining offline and avoiding reinforcement learning.
Problem
Pure imitation learning remained insufficient for complex closed-loop driving even with 30 million expert examples and mid-level inputs and outputs.
Method
ChauffeurNet uses mid-level top-down representations, recurrent trajectory prediction, synthesized trajectory perturbations, and losses that discourage undesirable behavior and encourage progress.
Results
The final model handled complex simulated situations and successfully drove a real car through scenarios involving other agents, turns, stop signs, and traffic lights.
Takeaways & Limitations
Synthesizing difficult situations around expert behavior and augmenting losses enabled avoidance of collisions and off-road driving without those examples explicitly appearing in expert demonstrations.
Takeaways & Limitations
The model has limited visibility and does not currently handle U-turns or cul-de-sacs; highly interactive or rare situations may require simulated exploration.
Abstract
from arXiv · showhide
Our goal is to train a policy for autonomous driving via imitation learning that is robust enough to drive a real vehicle. We find that standard behavior cloning is insufficient for handling complex driving scenarios, even when we leverage a perception system for preprocessing the input and a controller for executing the output on the car: 30 million examples are still not enough. We propose exposing the learner to synthesized data in the form of perturbations to the expert's driving, which creates interesting situations such as collisions and/or going off the road. Rather than purely imitating all data, we augment the imitation loss with additional losses that penalize undesirable events and encourage progress -- the perturbations then provide an important signal for these losses and lead to robustness of the learned model. We show that the ChauffeurNet model can handle complex situations in simulation, and present ablation experiments that emphasize the importance of each of our proposed changes and show that the model is responding to the appropriate causal factors. Finally, we demonstrate the model driving a car in the real world.
1. Introduction
The paper targets imitation learning robust enough for real-world driving, where pure imitation fails under closed-loop distribution shift despite extensive expert data. ChauffeurNet addresses this by combining mid-level representations, synthesized trajectory perturbations, and losses that discourage bad behavior while encouraging progress.
- Approach: Mid-level representations reduce sample complexity by separating perception and control from trajectory prediction.A perception system renders a top-down environment and route representation, while a controller converts predicted trajectories into steering and acceleration.
- Motivation: 30 million expert driving examples still did not make pure imitation sufficient for complex closed-loop driving.The model could get stuck behind or collide with a parked vehicle when nudging and passing was viable.
- Approach: Synthesized trajectory perturbations expose the learner to nonexpert states such as collisions and off-road driving.These perturbations provide training signals for losses that discourage undesirable behavior and encourage progress.
- Evaluation: The system was evaluated in simulation and then demonstrated driving a real car through scenarios involving other agents, turns, stop signs, and traffic lights.The evaluation also tested the relative importance of loss augmentation and data augmentation.
2. Related Work
Prior work established driving from camera or laser inputs and later explored end-to-end learning, affordance prediction, and deep networks for steering.
- Prior driving systems: Earlier systems showed that shallow networks could follow roads from camera and laser range data.ALVINN is cited as an early example of direct driving prediction.
- End-to-end learning: More recent work used affordance prediction or raw-camera deep networks to support highway control and steering.The cited examples include predicted distance to a preceding car and direct steering prediction.
3. Model Architecture
ChauffeurNet represents the scene in a moving top-down view, processes it with convolutional and recurrent networks, and predicts a future trajectory for a downstream controller.
- Input-output representation: The moving top-down representation encodes the agent pose, heading, speed, roadmap, traffic lights, dynamic objects, route, and motion history.The agent remains at a fixed image location while the view moves with it; temporal inputs are sampled at fixed intervals.
- Training architecture: The training architecture includes a road-mask network and PerceptionRNN alongside the core model, with recurrent feedback across iterations.The auxiliary networks predict drivable areas and future locations of other agents using shared features.
- Input-output representation: The representation supports both real-sensor logs and simulation, enabling closed-loop testing and mixing real and simulated data.Objects are represented as oriented boxes and environmental information is rendered into input images.
- Driving pipeline: The predicted future trajectory is passed to a controls optimizer that computes vehicle-specific steering and braking commands.Intermediate trajectory outputs avoid requiring one network to produce low-level controls directly.
- Model design: FeatureNet produces contextual features, while AgentRNN recurrently predicts successive trajectory points and related agent-state outputs.The recurrent network conditions predictions on features, iteration number, and memory from prior predictions.
- Model design: At each iteration, additive memory records predicted locations and is fed into the next recurrent update.The model obtains a coarse pose from a waypoint heatmap and then predicts sub-pixel pose refinement, heading, and speed.
4. Imitating the Expert
The model is trained to imitate expert waypoints, boxes, headings, and speeds, while addressing shortcuts from expert motion history and distribution shift through dropout and trajectory perturbations.
- Imitation losses: AgentRNN training uses waypoint, agent-box, and heading predictions with corresponding losses against ground-truth targets.Waypoint prediction is represented as a spatial probability distribution, while box occupancy uses a per-pixel heatmap.
- Imitation losses: Ground-truth waypoint supervision places a single positive pixel at the target coordinate in the binary target image.The waypoint loss uses cross-entropy between the predicted spatial distribution and this target.
- Past motion dropout: The model can exploit expert motion history during training by extrapolating past motion instead of identifying environmental causes.Past history dropout forces it to use other environmental cues to explain the future motion profile.
- Trajectory perturbation: Trajectory perturbations keep the start and endpoint fixed while offsetting the midpoint and smoothing the surrounding trajectory.The procedure creates realistic deviations from the expert path for closed-loop training.
5. Beyond Pure Imitation
ChauffeurNet goes beyond expert cloning by synthesizing trajectory perturbations and adding environment-aware losses that teach the model to avoid collisions, off-road behavior, and geometric deviations.
- Trajectory perturbations: Trajectory perturbations expose the model to nonexpert situations, including collisions and off-road driving, that are absent from expert demonstrations.The perturbations help the added losses receive training signals without requiring online reinforcement-learning exploration.
- Environment losses: The collision loss measures overlap between the predicted agent box and scene-object masks, directly penalizing predictions that lead to collisions.Artificial collisions from perturbed trajectories keep this loss effective throughout training.
- Environment losses: The road-edge loss measures overlap between the predicted agent box and road/non-road masks to discourage hitting curbs, medians, or other hard boundaries.Such cases arise synthetically when trajectory perturbations push the vehicle off the road.
- Environment losses: The geometry loss constrains the predicted agent box to overlap a rendered target curve whose thickness matches the agent width, independent of speed.The target curve is fit from target waypoints in a top-down coordinate system.
- Auxiliary losses: The model jointly predicts future dynamic-object occupancy and road masks, using shared features to support both driving and auxiliary perception tasks.The PerceptionRNN iteratively predicts object heatmaps, while a convolutional head predicts the binary road mask.
- Imitation dropout: Imitation losses reproduce expert behavior, whereas environment losses discourage undesirable behavior; imitation dropout randomly sets the imitation-loss weight to 0 or 1 per example.The experiments compare imitation dropout with simply reducing the imitation-loss weight.
6. Experiments
The experiments use roughly 26 million real-world driving examples and evaluate models by adding ChauffeurNet components incrementally on top of behavior cloning.
- Dataset: Approximately 26 million examples represent about 60 days of continuous driving after stationary and highway segments were removed.The model uses an 80m × 80m field of view with an effective forward sensing range of 64m.
- Models: Models M0 through M4 introduce past-motion dropout, perturbations, environment losses, reduced imitation weighting, and imitation dropout in sequence.This staged design supports ablations of the proposed ingredients against behavior cloning.
6.3 Closed Loop Evaluation
Closed-loop simulation evaluates the models on complex driving scenarios rather than simple cases where all models perform well. The ablations show differences in collision avoidance, recovery from perturbations, interaction with slow vehicles, causal responsiveness, and real-world transfer.
- Evaluation setup: Closed-loop evaluation replays scenarios while the model drives the simulated agent forward, allowing scenario-specific metrics such as stopping and collisions to be measured.Because the agent’s new pose determines subsequent inputs, this setup evaluates behavior under accumulated prediction errors.
- Evaluation setup: All models handled simple stop-sign, red-light, and lane-following scenarios well, so evaluation focused on complex situations that separated them.The simple-case evaluation used 20 scenarios for each situation.
- Nudging around a parked car: M4 performed best around parked vehicles, while other models collided about half the time and baseline M0 sometimes became stuck.M4’s behavior was attributed to learning collision avoidance and nudging from trajectory perturbations and collision loss.
- Recovering from a trajectory perturbation: M0 recovered from none of the tested lane departures, whereas M3 and M4 handled all deviations; perturbation-trained models handled 50% of lower-speed scenarios.At higher starting speeds, M3 and M4 were believed to benefit from greater emphasis on imagination losses.
- Slowing down for a slow car: M3 handled the highest-relative-speed slow-car cases better than M4, which sometimes maneuvered too aggressively and grazed the road edge.Some models instead slowed to zero and became stuck, while extreme relative speeds caused collisions for most models.
- Input ablation tests: M4 exhibited the correct behavior in all tested scenarios when stop signs or other vehicles were rendered or omitted.These input ablations tested whether the model responded to the relevant causal factors while holding other conditions constant.
- Real-world driving: On a real car, M4 replicated simulated behaviors, including smooth long-duration closed-loop driving, stop-sign handling, and turns.The learned model replaced the existing planner module during these evaluations.
6.4 Open Loop Evaluation
Open-loop waypoint error can favor a weaker driving policy: M0 made fewer errors than M4, despite M4 driving better in closed loop. Perturbed-data training reduced the model’s tendency to veer away from the original trajectory.
- Open-loop evaluation: Open-loop evaluation computes L2 distance between predicted and ground-truth waypoints without feeding predictions back into the agent’s future inputs.The comparison used 10,000 expert-driving test examples for M0 and M4.
- Open-loop evaluation: M0 made fewer open-loop prediction errors than M4, although closed-loop testing found M4 to be the better driver.The result demonstrates that open-loop metrics can be misleading for driving models.
- Perturbed evaluation: On perturbed evaluation data, M0 had larger L2 errors than M1 because it could not recover the agent from perturbations onto its original trajectory.Trajectory examples showed a veering-off tendency for the model trained without perturbed data.
6.5 Failure Modes
The model’s driving remains bounded by limited spatial visibility and by difficult or rare situations. Reported failures include merges, U-turns, cul-de-sacs, infeasible turn geometries, low-speed nudging, and over-aggressive passing.
- Unsupported and difficult scenarios: U-turns and cul-de-sacs are not handled, while some low-speed nudging cases still cause the model to become stuck.The authors state that these situations require sufficient sampled training data.
- Unsupported and difficult scenarios: The model sometimes outputs infeasible turn geometries and can become over-aggressive in novel or rare situations, such as passing a slow vehicle.The authors suggest that highly interactive or rare situations may need adequate simulated exploration.
6.6 Sampling Speed Profiles
ChauffeurNet represents future waypoints as probability distributions, enabling either mode-based trajectory updates or sampling across different speed profiles. These distributions can also be constrained during inference to enforce speed limits or keep trajectories near a reference path.
- Distribution-based prediction: Waypoint distributions Pk(x, y) represent the model’s predicted future locations in top-down coordinates.The model uses the distribution mode to update AgentRNN memory, while sampling enables trajectories with different speed profiles.
- Speed-profile sampling: Sampling P1(x, y) produces different speed profiles as later predictions such as P5(x, y) become more constrained.Figure 10 illustrates predictions at the first and fifth iterations.
- Inference-time constraints: Constrained sampling can enforce hard restrictions such as speed limits during trajectory generation.The same mechanism could constrain trajectories to a designated region around a reference trajectory.
7. Discussion
The paper argues that robust real-world imitation learning depends on synthesizing difficult situations around expert behavior and augmenting losses to discourage undesirable actions. ChauffeurNet achieved real-car driving, while remaining less competitive with motion-planning approaches and leaving several avenues for improvement.
- Discussion: Synthesized perturbations and augmented losses helped ChauffeurNet avoid collisions and off-road driving absent from expert demonstrations.The approach uses constrained exploration around expert behavior and middle-level representations to mix real and simulated data.
- Discussion: Middle-level input and output representations supported mixing real and simulated data while reducing the burden of learning perception and control.The predicted trajectory can be passed to a controls optimizer for vehicle-specific steering and braking commands.
- Discussion: The resulting model was good enough to drive a real car but was not yet fully competitive with motion-planning approaches.The authors identify comparisons with end-to-end approaches and alternatives to imitation dropout as areas for improvement.