Source-linked AI summary
Driving Policy Transfer via Modularity and Abstraction
Matthias Müller, Alexey Dosovitskiy, Bernard Ghanem, Vladlen Koltun
TL;DR
Autonomous-driving policies are difficult to scale and transfer from simulation because urban scenarios create high data demands and a reality gap. The paper encapsulates the learned policy between semantic perception and waypoint-based control, then evaluates direct transfer in simulation and on a physical truck. The system completes 100% of physical-world trials with data augmentation and transfers across varied conditions without real-world policy training, while the authors note that the simplified scenario omits important driving information.
Problem
End-to-end autonomous-driving policies require extensive data for realistic urban diversity, while simulation-trained policies face a reality gap when transferred to the physical world.
Method
The approach uses modularity and abstraction to encapsulate a learned driving policy between semantic segmentation from perception and waypoint-based low-level control.
Results
100% of physical-world trials were completed with data augmentation, compared with 82% without augmentation.
Takeaways & Limitations
A driving policy trained in simulation can be transferred directly to a 1/5-scale robotic truck across varied roads, weather, and environmental conditions on two continents.
Takeaways & Limitations
The simplified scenario omits lane markings, traffic signs, traffic-light states, and dynamic obstacles, and uses a simple low-level controller.
Abstract
from arXiv · showhide
End-to-end approaches to autonomous driving have high sample complexity and are difficult to scale to realistic urban driving. Simulation can help end-to-end driving systems by providing a cheap, safe, and diverse training environment. Yet training driving policies in simulation brings up the problem of transferring such policies to the real world. We present an approach to transferring driving policies from simulation to reality via modularity and abstraction. Our approach is inspired by classic driving systems and aims to combine the benefits of modular architectures and end-to-end deep learning approaches. The key idea is to encapsulate the driving policy such that it is not directly exposed to raw perceptual input or low-level vehicle dynamics. We evaluate the presented approach in simulated urban environments and in the real world. In particular, we transfer a driving policy trained in simulation to a 1/5-scale robotic truck that is deployed in a variety of conditions, with no finetuning, on two continents. The supplementary video can be viewed at https://youtu.be/BrMDJqI6H5U
1 Introduction
Autonomous driving systems face the scalability and safety limitations of both heavily engineered modular pipelines and end-to-end learning, while simulation introduces a difficult reality gap. The paper addresses this with a modular architecture that transfers a learned policy from simulation to physical vehicles.
- Motivation: Highly engineered driving systems require extensive subsystem tuning and remain limited across environmental and traffic conditions.Existing solutions often rely on HD maps and extensive sensor suites.
- Motivation: End-to-end policies reduce hand-engineering but require large amounts of data and are difficult to evaluate safely because of their black-box nature.These challenges are especially problematic for realistic urban driving.
- Sim-to-real challenge: Simulation provides abundant, safe training and testing data, but transferring policies to reality remains difficult because sensor, dynamics, and context differ.The paper identifies transfer of control policies in complex urban environments as an open problem.
- Approach: The proposed system separates perception, driving policy, and low-level control, with the policy mapping semantic segmentation to waypoint trajectories.A perception module converts raw sensor readings into semantic segmentation, while a controller actuates the vehicle toward predicted waypoints.
- Approach: The driving policy is trained purely in simulation on outputs from the actual perception system, allowing it to adapt to perception errors.Both perception and policy are learned, while the low-level controller may be learned or hand-designed.
- Benefits: The modular interfaces abstract environmental appearance and vehicle dynamics, enabling direct policy transfer without retraining or finetuning.Semantic maps and waypoints also provide interfaces that are easier to analyze and interpret.
- Evaluation: The approach is evaluated in simulated urban environments and on a 1/5-scale robotic truck across varied roads, weather, and environmental conditions on two continents.The evaluation includes simulation-to-simulation and simulation-to-reality transfer.
2 Related Work
Prior work studies sim-to-real transfer, modular driving systems, and intermediate representations, but direct transfer remains difficult for complex urban driving. This paper positions modular policy encapsulation as a complementary alternative to specialized transfer techniques and traditional pipelines.
- Transfer from simulation to reality: Synthetic data has supported perception transfer in indoor and driving scenarios, but direct sim-to-real transfer remains difficult even with high-fidelity simulation.Successful transfer examples exist for some tasks, including optical flow.
- Transfer methods: Sensorimotor policy transfer research has used domain adaptation, specialized architectures, depth maps, and domain randomization to facilitate transfer.The cited work primarily addresses manipulation, collision avoidance, or related transfer settings.
- Driving-policy transfer: Earlier driving-policy transfer demonstrated synthetic-image lane following and simulation-trained obstacle avoidance using an intermediate depth-based representation.The lane-following result was restricted to rudimentary lane following.
- Driving-policy transfer: Domain randomization enabled transfer for UAV collision avoidance, but the paper identifies outdoor urban driving as more complex in perception, planning, and control.The proposed approach is presented as complementary, using modularity to abstract nuisance factors.
- Driving-policy architectures: Classic autonomous-driving pipelines are analyzable and allow components to be developed in isolation, but they require careful engineering and can accumulate component errors.The modular design commonly contains many carefully engineered components.
- Driving-policy architectures: Deep-learning driving policies have demonstrated lane following, off-road driving, and simple urban navigation, but physical-world training is expensive and time-consuming.The passage also characterizes reinforcement learning as having high sample complexity.
- Hybrid approaches: Intermediate approaches combine learned perception or affordance prediction with standard navigation or policy execution pipelines.Related work includes self-supervised perception, affordance-based decomposition, and reinforcement learning in indoor environments.
3 Method
The system separates perception, waypoint-based driving policy, and low-level control to support simulation-to-reality transfer. The learned policy operates on segmentation maps and outputs local trajectory waypoints rather than raw images or vehicle controls.
- Architecture: The architecture maps RGB images to road segmentation, segmentation to waypoints, and waypoints to steering and throttle commands.The three stages are perception, driving policy, and low-level control.
- Perception: The perception module uses supervised encoder-decoder segmentation trained on the real-world Cityscapes dataset.It performs binary road-versus-not-road segmentation with cross-entropy loss; ERFNet provides the architectural basis.
- Driving policy: The driving policy consumes segmentation maps and predicts two local waypoints, represented by distance and oriented angle relative to the vehicle heading.The experiments fix waypoint distances at r1 = 5 and r2 = 20 meters and predict only ϕ1 and ϕ2.
- Driving policy: Conditional imitation learning trains the policy to predict actions from observations and high-level commands indicating left, straight, or right at the next intersection.The policy uses a deep branched network and mean squared error as the per-sample loss.
- Training and control: The policy is trained in simulation using noisy outputs from the actual perception system rather than perfect ground-truth segmentation.This exposes the policy to perception imperfections during training, while a PID controller converts waypoints into vehicle controls.
4 System Setup
The evaluation combines CARLA simulation with a physical 1/5-scale robotic truck. Simulation varies towns and weather, while the truck runs the modular network onboard and converts predicted waypoints into low-level controls.
- Evaluation platforms: The approach is evaluated in CARLA and on a physical 1/5-scale truck.The evaluation covers both simulated urban environments and a real robotic vehicle.
- Simulation setup: CARLA provides two towns with different layouts, sizes, and visual styles, along with configurable vehicle camera sensors.The simulator supports variation in camera field of view and position and provides sensor and privileged environmental information.
- Physical system: The physical system uses an onboard Nvidia TX2, USB camera, Pixhawk flight controller, and PID control for steering and throttle.The onboard computer predicts waypoints, and the Pixhawk converts control commands to PWM signals for the vehicle actuators.
5 Experiments
Experiments evaluate simulation generalization and real-world transfer using modular and end-to-end driving policies. The modular segmentation-to-waypoint approach performs best across challenging simulated conditions and physical-road tests, including long urban routes.
- Simulation: 25 trials per town measured goal-directed navigation after training in Town 1 and Weather 1, with evaluation across four town-weather combinations.The three unseen conditions were Town 1/Weather 2, Town 2/Weather 1, and Town 2/Weather 2.
- Simulation: Data augmentation and domain randomization were evaluated alongside image-to-control, image-to-waypoint, segmentation-to-control, and segmentation-to-waypoint policies.The + suffix denotes data augmentation, while +dr denotes domain randomization.
- Simulation: The basic image-to-control policy’s success rate dropped by a factor of 4 in Town 2 and reached zero in the unseen weather condition.Data augmentation slightly improved Town 1 performance but did not help generalization to Town 2.
- Simulation: The image-to-waypoint policy reached roughly twice the image-to-control success rate in Town 2, while domain randomization enabled some unseen-weather generalization.Even with data augmentation, image-to-waypoint did not complete an episode in Weather 2.
- Simulation: The segmentation-to-waypoint approach outperformed baselines and generalized to test weather in both towns better than domain randomization, with augmentation doubling performance in Town 2/Weather 2.The model predicts waypoints from segmentation maps produced by the perception module.
- Physical-world transfer: In 11 physical road-following trials spanning two geographic locations, the modular system succeeded in 82% without augmentation and 100% with augmentation.Driving distances varied from 10 to 50 meters, across diverse road and weather conditions.
- Physical-world transfer: On three long urban routes, the vehicle completed every track, missed only a few turns, completed all missed turns on a second attempt, and incurred one serious infraction.The serious infraction involved driving onto the curb with two wheels and becoming stuck.
6 Conclusion
The paper presents a modular deep architecture combining classic modular pipelines with end-to-end learning, while identifying extensions needed beyond its simplified driving scenario.
- The architecture combines modular pipeline structure with learned driving components, offering flexibility relative to monolithic end-to-end networks.
- Perception-module tuning supports generalization to new environments and transfer between domains, including simulation and the physical world.
- Training the policy on noisy outputs from real perception can improve robustness to error characteristics not captured by analytical uncertainty models.
- The approach remains limited by a simplified scenario and requires extension before becoming useful for real autonomous vehicles.
- Future extensions include richer perception inputs, more sophisticated low-level controllers, and reinforcement learning in simulation.
A Segmentation
The segmentation experiments examine transfer between simulation and reality across training datasets and select Cityscapes for the perception system. ERFNet-Fast trades a small mean-IoU reduction for substantially faster embedded execution and fewer parameters.
- 70 training images and 36 test images comprise a small real-world urban dataset used alongside standard public datasets for segmentation generalization.
- Simulation-trained segmentation works very well in simulation but does not generalize to the real world.
- Cityscapes generalizes to the validation data better than the other evaluated training datasets.The authors attribute this primarily to Cityscapes’ size and diversity, including more than 20K annotated images from dozens of cities.
- Qualitative segmentation results are good in typical scenes but remain imperfect, particularly under complex lighting and unusual situations.The learned driving policy is reported to adapt to these perception imperfections.
- ERFNet-Fast achieves a mean IoU of 84.6% at 25 frames per second, while original ERFNet achieves a mean IoU of 85.8% at 17 frames per second.
- ERFNet-Fast has 9 times fewer parameters than ERFNet while roughly matching its accuracy and running at 40% higher frame rate.
A.1 Network architecture
The perception module uses ERFNet building blocks, with an ERFNet-Fast architecture, while the driving-policy network follows the architecture of Codevilla et al.
- ERFNet modules are used as building blocks for the segmentation network architecture.
- The driving-policy network architecture is identical to Codevilla et al.
- The ERFNet-Fast architecture is used as the perception module in the proposed method.
A.2 Training Details
The networks use TensorFlow and Adam with fixed image resolution, while segmentation and driving-policy training use separate schedules and batch sizes. Camera variation is used to reduce overfitting to one camera configuration.
- All networks use TensorFlow and Adam, operate on 200×88 pixel images, and train with the specified network-wide implementation setup.
- Segmentation training starts at learning rate 0.001, drops to 0.0001 after 100k iterations, uses batch size 10, and ends at 200k iterations.
- Class-label balancing uses the proposed weighting method with pc defined as the dataset-average probability of class c and γ = 1.02.
- Driving-policy training starts at learning rate 0.0002, halves every 50k iterations until 250k, uses batch size 120, and runs for 500k iterations.Both waypoints receive equal loss weighting.
- Training data varies camera field of view and mounting position to prevent the driving policy from overfitting to one camera.The simulation collection uses 7 FOVs and 3 z-axis positions: 50cm, 100cm, and 150cm.
B.2 Data Augmentation
The paper regularizes training with image-level perturbations applied to RGB inputs, while the physical system uses modular perception, waypoint prediction, and PID control. Different camera configurations support lane following and complex navigation, with the lower view risking missed turns.
- Data augmentation: Image-level augmentation perturbs hue, saturation, brightness, and randomly masks input pixels to improve training variation.Augmentation is applied to RGB images rather than segmentation maps, producing more realistic variation.
- Data augmentation: Segmentation-network training uses random brightness, saturation, hue, and contrast perturbations on images scaled between 0 and 1.The stated ranges are brightness (−0.12, 0.12), saturation and contrast factors (0.5, 1.5), and hue (−0.2, 0.2).
- Data augmentation: Driving-policy training perturbs RGB images whether they enter the policy directly or first pass through a perception module.Gaussian blur and additive Gaussian noise are among the listed perturbations, each applied with 5% probability.
- Physical system: The physical system mounts nearly all components on the RC truck, while a remote control toggles autonomous driving mode.The hardware setup is illustrated in Figure 9.
- Control: At runtime, the onboard computer predicts waypoints, and a PID controller converts them into steering and throttle commands.Because the flight controller’s speed estimate was unreliable, experiments fixed throttle for approximately 3 m/s driving; steering used Kp = 0.8, Ki = 0, and Kd = 0.
- Camera configurations: A low camera mount protects electronics and supports lane following but restricts the field of view, whereas a higher mount increases visibility for complex navigation.The restricted view can lead to missed turns.
C.1 Experimental Setup
The experiments evaluate lane following from controlled starting positions in two environments, then select a model for more difficult multi-turn navigation. Additional tests examine left and right turns near an intersection, with routes documented by their turn directions.
- Lane-following evaluation: The lane-following evaluation uses four starting positions in the first environment and five in the second.Figures 10 and 11 show composite views and close-ups of the consistently marked positions.
- Lane-following evaluation: The second environment also includes runs beginning about 10 meters before an intersection, with separate left-turn and right-turn commands.These runs extend the controlled lane-following evaluation beyond the marked starting positions.
- Model selection: The basic experiments identify a model intended to follow lanes, recover from varied positions, and execute both left and right turns.The selected model is then evaluated on a more difficult navigation task.
- Complex navigation: The complex navigation evaluation requires the vehicle to complete several trajectories containing various turns.Figure 12 presents the three evaluation routes, marking right turns in yellow and left turns in green.