Source-linked AI summary
Imitation Learning for Autonomous Driving in CARLA
Jordy Kieto
TL;DR
Behavioral cloning is trained on expert-visited states, but deployment is closed loop, so offline validation alone cannot establish driving competence. This study trains and autoregressively deploys a compact policy from demonstrations, reporting hours of driving on training and held-out routes along with qualitative transfer and recovery observations.
Problem
Method
The study trains a compact policy offline on demonstrations and deploys it autoregressively at 20 Hz using five-frame observation histories.
Source: §7.1 · p. 9, §10 · p. 12Results
The policy drove autonomously for hours on training and held-out routes without collisions, and also drove in an unseen town and sometimes recovered after large deviations.
Source: §10 · p. 12Takeaways & Limitations
The report emphasizes controlling and verifying demonstration coverage and grading claims by evidence strength rather than proposing a new architecture.
Source: §10 · p. 12Closed-loop outcomes are observations rather than logged benchmark metrics, and the student receives privileged route context at test time.
Source: §8 · p. 11
Abstract
from arXiv · showhide
Behavioral cloning trains a policy offline on expert demonstrations, but deployment is closed loop: each action affects the observations the policy receives next. We study how much closed-loop driving competence a compact multimodal policy can acquire from offline demonstrations in the CARLA simulator. The policy uses five-frame histories of RGB images, LiDAR, vehicle telemetry, and lane waypoints to predict throttle, brake, and steering at 20 Hz. Demonstrations were collected in three stages, ending with a systematic route-generation procedure that enumerates spawn points and feasible maneuvers and verifies completed autopilot routes. The released 1.36 million parameter policy was trained on 236,882 windows, representing about 3.3 hours of driving from 448 captures. The resulting policy drives autonomously for hours on training and held-out routes. In our runs, it did so without collisions and also transferred qualitatively to an unseen CARLA town with different road geometry. We also observed recovery from large trajectory deviations, although we do not claim systematic recovery without controlled evaluation. We report offline metrics and distinguish measured results from qualitative closed-loop observations. We release the code, trained checkpoint, ONNX model, data sample, and an evidence audit for the reported claims.
1 Introduction
The section frames imitation learning as a mismatch between expert-state training and closed-loop deployment, then asks how demonstration coverage shapes driving competence. Its contributions center on systematic verified data generation, a reproducible released policy, and evidence-graded closed-loop observations.
1 Introduction
- The deployment question is whether offline behavioral cloning can acquire closed-loop driving competence when its own actions determine its future observations.Small errors can move a learner into states absent from expert demonstrations and compound; low validation loss is necessary but insufficient evidence of driving ability.Source: §1 · p. 1
- The project progressed from a tiny demonstration set to systematic, verified route generation, intervening on the support of the demonstration distribution.The section identifies distribution support as a fundamental limitation of behavioral cloning.Source: §1 · p. 1
- The route-generation procedure enumerates spawn points and feasible maneuvers, randomizes start poses, and accepts autopilot rollouts only after verifying the executed maneuver.It produces maneuver-balanced, reproducible, labeled captures.Source: §1 · p. 2
- The reproducible behavioral-cloning pipeline covers sensing, data handling, preprocessing, windowing, and training, and releases a policy checkpoint in Keras and ONNX.The checkpoint's provenance is tied to a logged training run.Source: §1 · p. 2
- The report gives an evidence-graded account of hours of driving, unseen-town transfer, and recovery, classifying claims as measured, observed, inferred, or unverified.The released claim audit accompanies this account.Source: §1 · p. 2
2 Related Work
The related work situates the study in end-to-end imitation driving, conditional route context, covariate shift, and privileged-teacher learning. It distinguishes the policy's multimodal temporal inputs and route-diversity collection from prior recovery-oriented approaches.
End-to-end driving from demonstrations
CARLA and conditional imitation
- Conditional imitation learning uses navigational commands to resolve junction ambiguity, while this policy uses a local lane-waypoint path that may not disambiguate junctions.The paper characterizes waypoint conditioning as weaker than.Source: §2 · p. 2
Covariate shift and data aggregation
- Behavioral cloning can incur quadratic horizon cost, while DAgger queries experts on learner-visited states; DART injects noise into expert demonstrations.Randomized start poses add limited off-nominal states here, but were designed for route diversity rather than recovery training.Source: §2 · p. 2
Privileged teachers
- Learning by Cheating distills a privileged ground-truth-state teacher into a sensorimotor student; this policy's student also receives privileged route context at test time.The paper explicitly identifies what information crosses that boundary.Source: §2 · p. 2
3 Problem Formulation
The formulation contrasts fitting expert-state actions offline with deployment under the learner's own state distribution. It defines a five-frame multimodal observation, an action-regression objective, and why closed-loop evaluation is needed.
3 Problem Formulation
- At 20 Hz, the policy predicts throttle, brake, and steering from five-frame histories of RGB, LiDAR bird’s-eye-view, telemetry, and lane waypoints.The modalities are expressed in the ego frame, and the history spans 0.25 seconds.Source: §3 · pp. 2–3
- Behavioral cloning fits the policy by empirical risk minimization under the expert state distribution, using Huber losses for throttle and brake and squared error for steering.The action comprises throttle, brake, and steering.Source: §3 · pp. 2–3
- At deployment, actions feed back into the simulator, so the learner is evaluated under its own distribution rather than the expert distribution.A single mistake can move it into states with no demonstrations, and the expected cost bound grows with horizon.Source: §3 · p. 3
- Held-out action error measures error under expert states, not driving competence, which must ultimately be assessed closed-loop, for example through route completion and infractions.At 20 Hz, a one-minute drive comprises 1,200 decisions.Source: §3 · p. 3
4 Multimodal Temporal Policy
The policy combines RGB, LiDAR BEV, telemetry, and lane-waypoint observations over time to predict driving controls. Its route context comes from the simulator’s lane graph, and its compact temporal architecture uses fixed random positional codes rather than learned ones.
4 Multimodal Temporal Policy
- The released policy uses RGB, LiDAR BEV, telemetry, and lane waypoints as inputs, while past controls are not inputs.Figure 2 depicts four frames of RGB, LiDAR BEV, and ego-frame waypoint paths entering a left turn.Source: Figure 1 · p. 3, Table 1 · p. 4, Figure 2 · p. 5
4.1 Observations
- Lane waypoints are derived from the simulator’s HD-map and ground-truth pose, and at junctions the first lane-graph successor may disagree with the expert’s maneuver.The waypoint path therefore follows the lane graph rather than necessarily indicating the expert’s chosen turn.Source: §4.1 · p. 4
4.2 Architecture
- Each modality has its own encoder with weights shared across time; fused features are position-encoded and processed by two post-norm Transformer blocks (4 heads, key dimension 64, feed-forward 512), with action decoded from the most recent timestep.The model has 1,358,211 parameters. At inference, throttle and brake are clipped to [0, 1] and steering to [−1, 1].Source: §4.2 · p. 4
- Temporal order is encoded by a fixed random positional code because the checkpoint’s positional embedding is not trainable.The report describes the model as it was actually trained.Source: §4.2 · p. 5
- The architecture is a compact, conventional late-fusion temporal policy, with the study focused on data and closed-loop behavior rather than architectural novelty.Source: §4.2 · p. 5
5 Demonstrations
The demonstrations progressed from small manual sets to systematic route generation, with labels conditioned for smooth control and captures verified by maneuver. Although captures were maneuver-balanced, straight driving still dominated the training windows.
5.1 Expert and label construction
- Throttle and steering labels were low-pass filtered and rate-limited, while brake labels were recorded raw.The expert could be a keyboard driver or CARLA Traffic Manager autopilot; sensor frames failing the alignment tolerance were dropped.Source: §5.1 · p. 5
5.2 From small demonstrations to systematic route generation
- The collection process evolved from minimal manual demonstrations through broader curated driving to systematic generation for the released policy.The broader curated stage extended coverage to hours of driving.Source: §5.2 · p. 6
- The earliest manual demonstrations reportedly produced road-following in Town01 after roughly seven minutes, with about one in ten rollouts ending in collision.These author-reported early-stage figures are not part of the released artifact.Source: §5.2 · p. 6
- Systematic route generation enumerated spawn points and feasible maneuvers, randomized start poses, and accepted autopilot captures only after verifying the executed maneuver.The procedure produced the released policy’s training data; randomized starts also introduced limited off-nominal states, not recovery training.Source: §5.2 · p. 6
- The autopilot’s stochastic intersection choices can under-represent turns, motivating a procedure that makes maneuver coverage a design variable.The generator queries feasible maneuvers from the road graph before collecting routes.Source: §5.2 · p. 6
5.3 The training set
- Each five-frame window was kept only when one direction label covered at least 60% of its frames, excluding some turn-entry transitions.The filter was used to remove ambiguous transitions and define per-direction diagnostics.Source: §5.3 · p. 6
- Straight driving made up 89% of windows despite maneuver-balanced captures, because captures spent much of their time approaching and leaving junctions.The passage identifies window-level coverage as the relevant unit because that is what the loss sees.Source: §5.3 · p. 7
6 Training and Offline Evaluation
Offline validation showed further fitting after the released checkpoint, but turn prediction remained difficult and training briefly destabilized. The temporal hold-out diagnoses fitting on seen routes, not driving competence or generalization to unseen routes or maps.
6.1 Protocol
- The 70/30 split was within each recording and direction, yielding 165,777 training and 71,108 validation windows; the released policy is the epoch-10 checkpoint.The run lasted 53 epochs, with evaluation every 50 steps.Source: §6.1 · p. 7
- The temporal hold-out measures interpolation to unseen moments of seen routes, not generalization to unseen routes or maps, so it is used only to diagnose fitting.The paper says it is not evidence of driving ability.Source: §6.1 · p. 7
6.2 Metrics
- Diagnostics included turn-detection precision, recall and F1, turn-onset delay, future-control error, and per-direction MSE.Future-control errors at one, two, and three frames were used to assess whether predictions anticipated upcoming controls or tracked current controls.Source: §6.2 · p. 7
6.3 Results
- Turn F1 rose only from 0.78 at epoch 10 to 0.82 at epochs 50 and later, although validation loss roughly halved again.Epoch 10 is the released checkpoint; the curves show raw evaluations and a 9-point moving average.Source: §6.3 · p. 7, Figure 3 · p. 8
- Right-turn validation error stayed about 2× straight-driving error, while left-turn error reached about 2.6× straight at epochs 50–53.The passage relates this pattern to turns comprising only 5–6% of windows.Source: §6.3 · p. 9
- Per-batch R2 was extremely negative, including −3.5 × 10^3 for steering at epoch 10, because contiguous validation batches often had near-zero target variance.The authors recommend dataset-level R2 in future runs.Source: §6.3 · p. 9
- Shortly after epoch 10, validation loss peaked at 5.7 × 10^-2, 37× the epoch-10 mean, and turn F1 fell to 0 before recovering around epochs 20 and 22.The cause was not investigated, and the released checkpoint precedes this episode.Source: Figure 3 · p. 8, §6.3 · p. 9
7 Closed-Loop Behavior
The policy ran autonomously for hours on training and held-out routes, and showed qualitative transfer to an unseen town and observed recovery from large deviations. These were observations rather than systematic benchmark measurements.
7.2 Sustained autoregressive driving
- For hours, the policy drove on training and held-out routes without collisions or human intervention, using only its own prior actions to determine subsequent observations.The deployment ran synchronously at 20 Hz with a five-frame history and no expert, action filter, or safety controller.Source: §7.2 · p. 9, §7.1 · p. 9
- The runs lacked fixed routes and route-completion or infraction counts, and omitted traffic, pedestrians, and traffic rules.The authors therefore characterize the result as a strong observation, not a measurement.Source: §7.2 · p. 9
7.3 Driving on unfamiliar road geometry
- With no demonstrations from the town, the policy drove on a curved multi-lane road there, a qualitative instance of zero-shot transfer to different road geometry.The training demonstrations came from Town01 and Town02, whose roads were two-lane streets with T-junctions.Source: §7.3 · p. 10, Figure 4 · p. 10
- The authors interpret transfer as possibly reflecting a local conditional driving function, but state that this mechanism is unproven and may involve privileged waypoints.Separating the explanations requires input ablations.Source: §7.3 · p. 10
7.4 Recovery from large deviations
- During closed-loop runs, the policy was observed leaving the lane or road, steering back to drivable surface, reacquiring the lane, and resuming its route.The observation was not a quantified recovery result; a proposed protocol would test controlled lateral and heading offsets and measure recovery rate and time-to-recover.Source: §7.4 · pp. 10–11
- The recoveries were not imitated from recovery-route or learner-state demonstrations; randomized capture starts introduced only small perturbations, leaving their broader recovery mechanism unverified.The authors propose that corrections from small perturbations, combined with LiDAR and waypoint geometry, may generalize to larger deviations, but identify this as a hypothesis.Source: §7.4 · p. 11
8 Limitations and Threats to Validity
The evidence is limited by qualitative closed-loop observations, privileged route information, a narrow simulation data distribution, and implementation and provenance caveats.
8 Limitations and Threats to Validity
- Closed-loop driving, unseen-town transfer, and recovery are author observations rather than logged benchmark metrics, with no multi-seed statistics or architecture and input ablations.Table 4 is a proposed protocol, not results.Source: §8 · p. 11
- At test time, the policy receives HD-map waypoints and ground-truth ego pose, while debug waypoints rendered in RGB also expose route information.The authors make no claim that the student is free of privileged information.Source: §8 · p. 11
- Validation is a temporal hold-out within the training recordings, and the first lane-graph successor can contradict the expert’s intended turn.Turn behavior is therefore not cleanly conditioned on intent.Source: §8 · p. 11
- The data are 89% straight-driving windows, lack traffic, pedestrians, and weather variation, and cover only Town01 and Town02.The autopilot ignored lights and signs during route generation, and manifest map tags cover only about 36% of windows.Source: §8 · p. 11
- Interpretation is also constrained by a fixed random positional embedding, duplicate telemetry components, LiDAR normalization dominated by near-ego returns, and unreleased evidence for early-stage quantities.The report makes no claims about real-world driving.Source: §8 · p. 11
9 Future Work
The proposed next steps focus on quantitative closed-loop evaluation, isolating the role of privileged context, and testing how demonstration coverage and recovery relate to driving behavior.
9 Future Work
- Run systematic closed-loop evaluation on fixed spawn and route sets in Town01, Town02, and unseen towns, using several seeds and reporting completion and infractions per kilometer.The proposed protocol includes collisions and off-road events.Source: §9 · p. 11
- Ablate waypoints and debug rendering, and test noisy or lagged route context to separate learned representation from privileged inputs.Source: §9 · p. 11
- Compare policies trained on 7-minute, 30-minute, 1-hour, and 3.3-hour subsets to measure closed-loop metrics against demonstration volume.Source: §9 · p. 11
- Compare recovery curves for policies trained without start jitter, with current jitter, and with explicit recovery routes using the proposed offset protocol.Source: §9 · p. 12
- Replace first-successor lane waypoints with a planned route or command to test intent conditioning.The passage cites as an example of command conditioning.Source: §9 · p. 12
10 Conclusion
The report presents a compact offline-trained policy that drove for hours in CARLA and observed qualitative transfer and recovery. Its emphasis is on verified demonstrations, artifact provenance, and evidence-graded claims, while motivating controlled study of coverage and privileged route context.
10 Conclusion
- A compact multimodal policy trained offline on about 3.3 hours of verified autopilot demonstrations drove for hours on training and held-out CARLA routes without collisions or human intervention.The conclusion also reports unseen-town driving and occasional return to the road after large deviations, despite no recovery demonstrations in training.Source: §10 · p. 12
- The report’s contribution emphasizes controlling and verifying the demonstration distribution, reconstructing released-policy provenance, and grading claims by evidence.Source: §10 · p. 12
- Earlier reinforcement-learning and less structured imitation attempts did not produce reliable closed-loop driving, but those experiments are outside this report’s evidentiary scope.Source: §10 · p. 12
A Artifact Provenance
The MLP checkpoint is traceable to a dated training run and dataset snapshot; its ONNX artifact closely matches the Keras checkpoint.
A Artifact Provenance
- Epoch 10 completed at 1,115 seconds, and the Keras checkpoint was saved eight seconds later.Source: A Artifact Provenance · p. 13
- The run used 448 recordings and 236,882 windows, while excluding post-checkpoint recovery-tagged recordings and DAgger-mode data.Source: A Artifact Provenance · p. 13
- The ONNX artifact's maximum absolute deviation from the Keras model was less than 4 × 10−7.Source: A Artifact Provenance · p. 13