PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Drive from Simulation without Real World Labels

Alex Bewley, Jessica Rigley, Yuxuan Liu, Jeffrey Hawke, Richard Shen, Vinh-Dieu Lam, Alex Kendall

arXiv:1812.03823v2cs.CV

TL;DR

The paper addresses the difficulty of transferring driving policies learned in simulation to real operation without real-world labels. It jointly learns image translation and a single-camera control policy from simulated supervision, achieving closed-loop driving on rural and urban roads. The reported system drove 3km between interventions on a real-world lane-following task.

  • Problem

    Simulation-trained driving policies face a reality gap, while obtaining real-world control labels is difficult.

    Source: §I · p. 1

  • Method

    The method jointly translates simulated and real imagery through a shared latent representation and learns control from simulation labels without real-world labels.

    Source: §I · p. 1

  • Results

    The system drove 3km between interventions on a real-world lane-following task and demonstrated closed-loop driving on public UK urban roads.

    Source: §I · p. 1, §V · p. 7

  • Takeaways & Limitations

    Simulation can provide the control supervision for transferring an end-to-end lane-following policy to real-world operation without additional human demonstrations.

    Source: §I · p. 1, §V · p. 7

Abstract

from arXiv · show

Simulation can be a powerful tool for understanding machine learning systems and designing methods to solve real-world problems. Training and evaluating methods purely in simulation is often "doomed to succeed" at the desired task in a simulated environment, but the resulting models are incapable of operation in the real world. Here we present and evaluate a method for transferring a vision-based lane following driving policy from simulation to operation on a rural road without any real-world labels. Our approach leverages recent advances in image-to-image translation to achieve domain transfer while jointly learning a single-camera control policy from simulation control labels. We assess the driving performance of this method using both open-loop regression metrics, and closed-loop performance operating an autonomous vehicle on rural and urban roads.

I. INTRODUCTION

The paper transfers a vision-based lane-following policy from simulation to real driving without real-world control labels by jointly learning domain translation and control. It reports open-loop evaluation and closed-loop driving on rural and urban roads.

I. INTRODUCTION

  • Simulation provides controllable appearance, weather, road structure, and privileged ground truth, but the reality gap limits direct transfer to camera observations.The method addresses this gap by learning a mapping between simulation and real-world features rather than only improving rendered-image realism.Source: §I · p. 1
  • The shared latent representation is learned through image translation while a control policy uses simulation expert labels to predict real-world steering.This makes driving a differentiable component of the domain-transfer architecture.Source: Figure 1 · p. 1, §I · p. 1
  • The method transfers an end-to-end driving policy from simulation to an unlabelled real-world domain without real-world control labels.It learns an invariant representation across simulated and real imagery while jointly optimizing the driving task.Source: §I · p. 1
  • The approach exploits simulation to learn manoeuvres beyond common real-world imitation-learning distributions without multiple camera rigs or data augmentation.The contribution uses controllable simulated states to broaden the learned driving experience.Source: §I · p. 1
  • The system drove a real vehicle for 3km without intervention on a rural road and also demonstrated closed-loop lane following on public UK urban roads.Evaluation included open-loop and closed-loop metrics, with urban-road operation reported as a successful demonstration.Source: §I · p. 1

II. RELATED WORK

The work builds on image translation, unsupervised domain adaptation, and end-to-end driving, extending these lines toward simulation-to-real control without real-world commands. Its method uses unpaired real images alongside simulated image-label pairs.

II. RELATED WORK

  • Image-to-image translation methods learn mappings between image distributions, and this work extends them with a supervised control decoder.Prior approaches include conditional adversarial networks and cycle-consistent translation without corresponding images.Source: §II · p. 2
  • Unsupervised domain adaptation commonly aligns source supervision with intermediate features, but feature alignment may require pairwise constraints or intermediate domains under large appearance shifts.Related work also aligns feature and pixel distributions for perception or transfers control across simulated weather conditions.Source: §II · p. 2
  • End-to-end driving has progressed from simple lane following to multi-camera imitation, conditioned simulated navigation, affordance-based control, and single-camera reinforcement learning.The paper positions its single-camera robustness as arising from noisy simulated experiences.Source: §II · p. 2
  • Unlike the closest image-translation steering work, this method uses bi-directional translation without real-world driving commands and evaluates closed-loop performance on a real system.It uses an implicit latent structure rather than an explicit representation such as semantic segmentation [9].Source: §II · p. 2

III. METHOD

  • Training uses simulated image-label pairs and an independent unlabelled real-image set, with no pairwise correspondences between the domains.This frames transfer from expert simulation demonstrations to real behavior as unsupervised domain adaptation.Source: §III · p. 2

A. Architecture

The architecture combines an image translator, domain discriminators, and a controller that operates on a shared latent representation. Encoders map simulated and real images into that space, enabling translation and control across domains.

A. Architecture

  • The model consists of an image translator, domain discriminator networks, and a controller for imitation learning.Figure 2 summarizes information flow among these modules.Source: A. Architecture · p. 2

1) Translator:

  • Translation computes a latent embedding with one domain encoder and predicts the translated image using the other domain decoder.Training adds noise to the latent embedding, while inference sets the noise to zero.Source: 1) Translator · pp. 2–3
  • Domain-specific encoders map images into a common latent space Z, which supports vehicle-control prediction and domain translation.The shared representation is learned through direct and cyclic losses.Source: 1) Translator · pp. 2–3, Figure 2 · p. 3

2) Discriminators:

  • Two-scale discriminators align each translated image’s appearance distribution with images from the corresponding target domain.The discriminators operate on translator outputs to improve domain-level appearance matching.Source: 2) Discriminators · p. 3

3) Controllers:

The controller extracts a spatially aware representation from the translated latent tensor, while reconstruction, cyclic, and control losses jointly constrain translation and driving behavior. Control supervision comes from simulated expert steering and is enforced across domains.

3) Controllers:

  • The controller passes the latent tensor through CoordConv layers, global average pooling, and fully connected layers to predict control.This creates a spatially aware embedding suited for driving from the latent translation representation.Source: 3) Controllers · p. 3

1) Image Reconstruction Loss:

  • Image reconstruction uses an L1 loss between each domain image and its reconstruction through the corresponding variational autoencoder.For domain d, the reconstruction is produced as G_d(E_d(X_d)).Source: 1) Image Reconstruction Loss · p. 3

2) Cyclic Reconstruction Loss:

  • Cyclic reconstruction translates an image to the other domain and back, then penalizes the L1 difference from the original image.This cycle-consistency constraint supports a shared latent space across domains [38], [15].Source: 2) Cyclic Reconstruction Loss · p. 3

3) Control Loss:

  • The control loss penalizes L1 error between predicted steering and autopilot ground truth from simulation.The controller prediction is ĉ = C(E_d(X_d)).Source: 3) Control Loss · p. 3
  • Cyclic control loss enforces consistent steering before and after an image is translated between domains and re-encoded.It compares the original predicted steering with the prediction from the translated image.Source: 3) Control Loss · p. 3, Figure 3 · p. 3

4) Adversarial Loss:

The method combines adversarial image translation with perceptual feature matching to preserve semantic content across simulation and real domains.

4) Adversarial Loss:

  • Adversarial losses train translated images to resemble the selected target domain.Both the image translator and discriminators use the LSGAN objective.Source: 4) Adversarial Loss · p. 3

5) Perceptual Loss:

  • Perceptual loss compares VGG16 features from original and translated images to encourage consistent semantic content.Instance normalization is applied before feature-distance computation to make the loss more domain-invariant.Source: 5) Perceptual Loss · p. 3

6) Latent Reconstruction Loss:

The model uses latent reconstruction and joint optimization of variational autoencoders and the controller, then evaluates the learned policy with offline and closed-loop tests.

6) Latent Reconstruction Loss:

  • Latent reconstruction uses an L1 loss between an image’s latent representation and the representation recovered after cross-domain decoding and re-encoding.This encourages latent features to encode semantic content independently of image domain.Source: 6) Latent Reconstruction Loss · p. 4
  • The VAEs and controller are trained jointly with a weighted sum of reconstruction, cycle-consistency, and control losses.The loss weights are denoted by λ_i.Source: 6) Latent Reconstruction Loss · p. 4

IV. EVALUATION

  • Evaluation reports in-domain and cross-domain open-loop metrics, closed-loop imitation control on a physical rural vehicle, and qualitative translation results in rural and urban settings.Tables II and III cover the open-loop and closed-loop results, respectively.Source: §IV · p. 4

A. Simulation Domain

The simulation domain procedurally varies road structure and scene content, while a privileged expert generates perturbed imitation data that cannot be collected equivalently in the real world.

1) Procedural Environment Generation:

  • Procedural generation creates rural roads with controllable curvature, randomized textures, and foliage placement, while environmental factors are held fixed for the narrow source domain.The method relies on image translation to handle higher target-domain variance.Source: 1) Procedural Environment Generation · p. 4
  • The urban simulator adds roads, buildings, trees, and parked cars while approximating real-world layout rather than visual photorealism.Image translation transfers the policy from the cartoon-like simulated world to visually rich real scenes.Source: 1) Procedural Environment Generation · p. 4

2) Simulated Expert Agent:

  • A proportional expert controller uses privileged lane-center distance to generate steering labels for simulated lane paths.The vehicle follows randomly selected lane paths offset from the generated road centerline.Source: 2) Simulated Expert Agent · p. 4
  • Ornstein-Uhlenbeck action noise perturbs vehicle states, producing robust image-label pairs throughout the drivable lane space at 10Hz.Noise is tuned to maximize perturbation while keeping the expert largely within the lane.Source: 2) Simulated Expert Agent · p. 4
  • This expert-data procedure is restricted to simulation because it requires privileged lane information and potentially dangerous real-world swerving.The limitation concerns both the expert’s inputs and the state perturbations used for data enrichment.Source: 2) Simulated Expert Agent · p. 4

1) Rural road:

  • The initial real-world test uses a 250m private one-lane rural road, with the vehicle limited to 10 kmh−1 for safety.Testing and data collection exclude other vehicles using the road.Source: 1) Rural road · p. 4

2) Urban road:

The evaluation spans opportunistic urban-road tests and domain-adaptation datasets, while steering-data balancing addresses the strong concentration of real driving near zero steering.

2) Urban road:

  • Urban tests use minor public roads in Cambridge across sun, overcast, and rain when other traffic is absent.The urban environment targets dense suburban roads rather than controlled private-road testing.Source: 2) Urban road · p. 4

C. Data

  • The dataset contains approximately 60k training frames and 20k test frames per domain, with simulation frames paired with expert control labels.Table I summarizes the datasets used for driving-policy domain adaptation.Source: C. Data · p. 5, Table I · p. 5

1) Data Balancing:

  • Naive MAE training risks predicting zero steering because the labelled data is dominated by near-zero steering angles.The steering range is parameterized as ±1.0, and the real route produces a concentrated distribution around straight driving.Source: 1) Data Balancing · p. 5, Figure 5 · p. 5
  • Uniform sampling across eight steering-angle bins upsamples underrepresented actions so every bin matches the largest bin’s sample count.A linear calibration adjusts real-vehicle steering because steering limits differ between vehicles.Source: 1) Data Balancing · p. 5

D. Transfer Learning Results

The method is compared with direct-transfer, translation, and feature-adaptation baselines using open-loop metrics and closed-loop vehicle testing. Standard offline metrics show only weak correspondence with real-world driving performance.

1) Baselines:

  • The baselines include direct simulation transfer, real-to-sim translation, sim-to-real translation, latent-feature adversarial adaptation, and a drive-straight proxy.These baselines isolate direct application, image translation, feature alignment, and curvature-related effects.Source: 1) Baselines · p. 5

2) Performance Metrics:

  • Open-loop MAE and Balanced-MAE compare controllers across simulated and real test datasets, with Balanced-MAE averaging per-bin errors to address data imbalance.The authors report that Balanced-MAE correlated with closed-loop performance in simulation.Source: 2) Performance Metrics · p. 5, Table II · p. 6
  • Closed-loop rural evaluation measures distance travelled per safety-driver intervention over 3km, while Table III compares the proposed system with baseline models.Testing ran near the camera’s 30Hz frame rate.Source: 2) Performance Metrics · p. 5, Table III · p. 6
  • In urban scenes, the model transfers its learned control policy across a large cartoon-like-to-real appearance change without real-world labels.Figure 6 illustrates the simulated and real urban domains.Source: Figure 6 · p. 6
  • Open-loop metrics showed only weak correlation with real-world performance, motivating improved offline measures that better predict closed-loop driving.The authors attribute this conclusion to differences between open- and closed-loop performance across their model and baselines.Source: 2) Performance Metrics · p. 6

3) Loss Ablation:

Ablations show that most loss terms support transfer, while urban-road tests demonstrate successful lane following but also expose dependence on accurate visual translation.

3) Loss Ablation:

  • Removing individual loss terms generally increases error, except for the cyclic control factor, whose removal leaves the other cyclic losses sufficient to maintain shared latent structure.The corresponding ablation results are reported in Table IV.Source: 3) Loss Ablation · p. 6, Table IV · p. 6
  • The L1 cyclic reconstruction and GAN losses are most critical for control transfer, while removing the control term prevents sensible learning.The ablation evaluates average error multipliers on the real-domain open-loop metrics.Source: 3) Loss Ablation · p. 6, Table IV · p. 6

4) Urban Road Results:

  • Successful urban lane following on dense Cambridge side streets depends on accurate visual translation between simulated and real domains.Figure 7b provides qualitative examples of the urban-road results.Source: 4) Urban Road Results · p. 6, Figure 7 · p. 7
  • The controller typically drives only when visual translation succeeds; mis-translating a car into a footpath requires intervention.Correct translation of road topology and cars supports the desired lane-following behavior.Source: 4) Urban Road Results · pp. 6–7

5) Visualisation:

Visualization shows how translation preserves road structure and supports steering transfer, while the conclusion reports real-world validation and identifies open-loop evaluation as an unresolved limitation.

5) Visualisation:

  • Translation preserves road curvature and vehicle offset across domains, enabling consistent transfer of the steering signal.The bidirectional translator also maps an off-road simulated state toward real-world observations, supporting corrective behavior.Source: 5) Visualisation · p. 7

V. CONCLUSIONS

  • Simulation supplies inexpensive training data, auxiliary ground truth, and difficult or dangerous driving situations, addressing prior concerns about simulation-to-reality complexity.These advantages motivate the proposed transfer approach.Source: §V · p. 7
  • The system transfers an end-to-end policy from simulation to real-world driving without additional human demonstrations, driving 3km between interventions on a rural task.The model jointly translates images and predicts controls from shared latent features using simulation labels.Source: §V · p. 7
  • The system also drives in closed loop on public urban roads in the United Kingdom.This extends the reported validation beyond the rural lane-following route.Source: §V · p. 7
  • The paper concludes that standard open-loop metrics did not prove predictive of driving performance and need improvement.The authors leave development of better metrics to future work.Source: §V · p. 7

Related papers

Loading 1812.03823v2…