PaperlayerView exact revision on arXiv

Source-linked AI summary

VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control

Jingwei Zhang, Lei Tai, Peng Yun, Yufeng Xiong, Ming Liu, Joschka Boedecker, Wolfram Burgard

arXiv:1802.00265v4cs.ROcs.CVcs.LG

TL;DR

Simulation-trained visual control policies face a reality gap when deployed on real camera streams. VR-Goggles translates those streams back into the synthetic domain during deployment and uses shift loss to encourage consistent translated frames. In real-world tasks, it achieves a 100% success rate, compared with 60% for CycleGAN and 0% without adaptation.

  • Problem

    Visual control policies trained in simulation must generalize across differences in noise, texture, and lighting in real sensor readings, while consistency across subsequent frames remains an important and underconsidered adaptation aspect.

    Source: B. Domain Adaptation for Learning… · p. 2

  • Method

    VR-Goggles translates real camera streams into the synthetic domain during deployment and uses shift loss to constrain consistency between subsequent translated frames.

    Source: §I · p. 1

  • Results

    In real-world tasks, VR-Goggles achieves a 100% success rate, compared with 60% for CycleGAN and 0% without domain adaptation; it also achieves the highest temporal consistency in the video style-transfer comparison.

    Source: A. Validating Shift Loss: Artistic… · p. 4, C. Real-world Indoor & Outdoor… · p. 7

  • Takeaways & Limitations

    Policy training and adaptation are decoupled, so preparation for transferring a policy can run in parallel with its training, without retraining or fine-tuning the policy for new environments.

    Source: §I · p. 1

    The paper mainly focuses on navigation; applying the method to manipulation is left as future work.

    Source: §V · p. 8

Abstract

from arXiv · show

In this paper, we deal with the reality gap from a novel perspective, targeting transferring Deep Reinforcement Learning (DRL) policies learned in simulated environments to the real-world domain for visual control tasks. Instead of adopting the common solutions to the problem by increasing the visual fidelity of synthetic images output from simulators during the training phase, we seek to tackle the problem by translating the real-world image streams back to the synthetic domain during the deployment phase, to make the robot feel at home. We propose this as a lightweight, flexible, and efficient solution for visual control, as 1) no extra transfer steps are required during the expensive training of DRL agents in simulation; 2) the trained DRL agents will not be constrained to being deployable in only one specific real-world environment; 3) the policy training and the transfer operations are decoupled, and can be conducted in parallel. Besides this, we propose a simple yet effective shift loss that is agnostic to the downstream task, to constrain the consistency between subsequent frames which is important for consistent policy outputs. We validate the shift loss for artistic style transfer for videos and domain adaptation, and validate our visual control approach in indoor and outdoor robotics experiments.

I. INTRODUCTION

The paper addresses the reality gap that makes simulation-trained visual control policies difficult to transfer to real scenes. It translates real camera streams into the synthetic domain during deployment and adds a shift loss for consistent image streams.

I. INTRODUCTION

  • The reality gap includes differences in noise patterns, texture, and lighting between synthetic renderings and real sensor readings.DRL policies are generally trained in simulation because they typically require several million training samples.Source: §I · p. 1
  • Common sim-to-real approaches increase simulated-image fidelity, match image distributions, or adapt learned features, but may add training preprocessing or require scene-specific policy training or fine-tuning.The paper contrasts these burdens with deployment-time real-to-sim adaptation.Source: §I · p. 1
  • VR-Goggles translates real camera streams into the synthetic domain during deployment, rather than adapting synthetic images during policy training.The approach targets minimal extra computational burden in the learning pipeline.Source: §I · p. 1
  • The real-to-sim approach adds no extra processing burden to DRL policy training and lets adaptation preparation run in parallel with policy training.A separate model can be trained for each new environment using typically about 2,000 collected images; the policy need not be retrained or fine-tuned.Source: §I · p. 1
  • The paper proposes shift loss to generate consistent synthetic image streams without temporal constraints or sequential training data.It is presented as a cheap alternative to optical-flow constraints and is evaluated in video style transfer and domain adaptation.Source: §I · p. 1

A. Domain Adaptation

The section reviews unsupervised image translation and methods for bridging the visual reality gap in learning-based control. It highlights both the limitations of existing approaches and the importance of consistency across sequential frames.

A. Domain Adaptation

  • Unsupervised domain adaptation translates images from a source domain into a target domain with minimal manual effort, while CycleGAN uses cycle consistency without requiring paired data.CycleGAN can produce semantic permutations on more complex distributions; semantic constraints such as CyCADA address this by matching semantic maps.Source: A. Domain Adaptation · p. 2

B. Domain Adaptation for Learning based Visual Control

  • Sim-to-real image adaptation for manipulation and navigation adds an adaptation step before each policy-training iteration.Domain randomization,, may be costly to realize in some simulators and cannot guarantee coverage of arbitrary real scenes.Source: B. Domain Adaptation for Learning… · p. 2
  • Progressive feature adaptation still requires policy fine-tuning for each visually different real-world scenario.Other approaches use 2D lidar –[17] or depth images,, or rely on real scenes visually similar to simulation.Source: B. Domain Adaptation for Learning… · p. 2
  • Inoue et al. also adapt real images to simulation during deployment, but require paired data and address supervised perception rather than control.Source: B. Domain Adaptation for Learning… · p. 2
  • The paper focuses on visual navigation, whose larger environments make visual adaptation challenging compared with manipulators’ confined workspaces.The authors suggest that real-to-sim adaptation could potentially extend to other control domains.Source: B. Domain Adaptation for Learning… · p. 2
  • Consistency between subsequent frames has been overlooked in the cited adaptation methods, although it can be critical to DRL agents’ successful task completion.The broader domain-adaptation literature also lacks work on sequential rather than single-frame inputs.Source: B. Domain Adaptation for Learning… · p. 2

C. Artistic Style Transfer for Videos

Video style transfer seeks temporally consistent stylizations of sequential inputs, but small input shifts can produce substantially different outputs. The paper proposes shift loss to encourage consistency without the relatively expensive optical-flow constraint.

C. Artistic Style Transfer for Videos

  • A trained stylization network can change its output substantially for small input shifts that are not multiples of its total downsampling factor K, causing temporal inconsistency.Ruder et al. address this with temporal constraints that require optical flow at deployment; Huang et al. impose a training-time constraint on single-frame stylization.Source: C. Artistic Style Transfer for Videos · p. 2
  • Shift loss constrains consistency between generated subsequent frames without requiring the relatively expensive optical-flow constraint.The authors argue that small input changes should yield small output changes for a network trained to approximate a smooth function.Source: C. Artistic Style Transfer for Videos · pp. 2–3
  • In Fig. 1, the deployment path translates real sensor readings into simulated-domain images before the DRL policy produces control commands.Outdoor adaptation uses a semantic loss because CARLA supplies semantic ground truth; the indoor Gazebo setup omits it because such ground truth is unavailable.Source: Figure 1 · p. 3

A. Problem formulation

The method translates real camera streams into the simulated domain during deployment, where DRL policies were trained. It uses two mappings and cycle consistency to support this real-to-sim adaptation.

A. Problem formulation

  • Domain S contains sequential synthetic frames, while domain R contains sequential real-camera readings.The examples are simulator images for S and onboard mobile-robot camera readings for R.Source: A. Problem formulation · p. 3
  • The method targets consistent outputs for sequential inputs but does not require sequential training data.The sequential framing is used because some baseline methods require it.Source: A. Problem formulation · p. 3
  • DRL agents are trained in S and deployed in R, so the method translates real images from R to S during deployment.Source: A. Problem formulation · p. 3

B. CycleGAN Loss

C. Semantic Loss

The semantic constraint uses simulator labels to preserve semantics across image translation, while shift loss constrains output consistency for shifted inputs without optical-flow information.

C. Semantic Loss

  • Simulator-provided ground-truth semantic labels support a semantic constraint inspired by CyCADA for image translation.The paper uses “CyCADA” to mean CycleGAN plus this semantic loss, not the full CyCADA approach.Source: C. Semantic Loss · p. 3
  • When real-domain ground-truth semantics are unavailable, the synthetic-domain segmentation network supplies semi-labels for real images.The semantic losses constrain input and generated-output semantic maps.Source: C. Semantic Loss · p. 3

D. Shift Loss for Consistent Generation

  • Shift loss addresses inconsistencies that can persist despite semantic constraints, including when ground-truth semantics are unavailable in both domains.Source: D. Shift Loss for Consistent Generation · pp. 3–4
  • The shift loss compares a translated shifted input with the shifted output of the original input in both domain directions.The shifts are sampled from 1 to K−1 pixels along the image axes.Source: D. Shift Loss for Consistent Generation · p. 4
  • Assuming only limited regional movement between frames, shift loss smooths the mapping for small movements without requiring optical-flow information.The paper presents it as a cheap alternative for consistency constraints relevant to real-time robotics control.Source: D. Shift Loss for Consistent Generation · p. 4

E. Full Objective

The full VR-Goggles objective combines adversarial, cycle-consistency, and shift losses, with weights for the cycle and shift terms.

E. Full Objective

  • The full objective includes adversarial losses for both domain mappings and cycle-consistency losses in both directions.Source: E. Full Objective · p. 4
  • The objective adds λshift-weighted shift losses for both mappings.Source: E. Full Objective · p. 4
  • The paper formulates learning VR-Goggles as an optimization of the full objective over the discriminators.Source: E. Full Objective · p. 4

A. Validating Shift Loss: Artistic Style Transfer for Videos

In video style-transfer tests, shift loss improved consistency for shifted inputs and achieved lower temporal loss than the optical-flow-trained baseline, despite using no optical-flow information.

A. Validating Shift Loss: Artistic Style Transfer for Videos

  • The comparison used FF trained on single frames, FF-flow trained on sequential images with optical-flow constraints, and Ours trained on single frames with shift loss.Training footage comprised 98 HD video sequences and 2,450 frames; Sintel sequences were used for testing.Source: A. Validating Shift Loss: Artistic… · p. 4
  • For shifted single frames, FF stylizations were nearly identical only at the network’s K = 4 pixel shift, while Ours generated consistent stylizations across shifts.FF-flow improved shift invariance only by a limited amount.Source: A. Validating Shift Loss: Artistic… · p. 4
  • Ours achieved lower temporal loss than FF-flow despite using no optical-flow information, and the temporal-error maps show the highest temporal consistency.The temporal-loss comparison is reported for Sintel test sequences using ground-truth optical flow; the maps visualize errors for consecutive stylized frames.Source: A. Validating Shift Loss: Artistic… · p. 4

B. Quantitative Evaluation: Carla Benchmark

The Carla benchmark evaluates policy transfer across weather conditions using single- and multi-weather training regimes and three input-adaptation strategies. VR-Goggles generally outperforms CycleGAN, particularly on the harder navigation tasks.

B. Quantitative Evaluation: Carla Benchmark

  • The benchmark uses three training weather conditions and holds out daytime hard rain for testing because test-condition datasets were unavailable for adaptation.The original benchmark tests under cloudy daytime and soft rain at sunset.Source: B. Quantitative Evaluation: Carla… · p. 4, Figure 4 · p. 6
  • Policy training compares separate single-weather policies with a multi-weather policy trained on the combined three-condition dataset.This multi-domain setup is a limited realization of domain randomization; progressive nets are excluded because they require costly real-world policy finetuning.Source: B. Quantitative Evaluation: Carla… · pp. 4–5
  • Adaptation compares direct testing-data input, CycleGAN translation, and VR-Goggles, which adds shift loss to CycleGAN-based input translation.For both translation methods, an adaptation network maps the testing condition to each training condition.Source: B. Quantitative Evaluation: Carla… · p. 5
  • VR-Goggles outperforms CycleGAN on almost all reported metrics, especially success rate and distance-to-goal on Navigation and Nav. dynamic.Its adaptations also produce mostly lower infraction frequencies than CycleGAN.Source: B. Quantitative Evaluation: Carla… · p. 5
  • Under direct deployment without adaptation (No-Goggles), average distance between pedestrian collisions exceeds 4.6 km because the total navigation distance across all 25 episodes in this task is only 4.6 km.Source: B. Quantitative Evaluation: Carla… · p. 6

C. Real-world Indoor & Outdoor Navigation

The paper tests simulated-policy transfer in real indoor and outdoor navigation. VR-Goggles enables successful indoor deployment without policy retraining and guides a daytime-trained policy along a nighttime road in the outdoor experiment.

C. Real-world Indoor & Outdoor Navigation

  • The indoor experiment compares direct input, CycleGAN or CyCADA translation, and VR-Goggles on simulated-policy deployment to a real office.The semantic constraint is dropped indoors because Gazebo does not provide ground-truth semantics.Source: C. Real-world Indoor & Outdoor… · pp. 6–7
  • Indoor success rates were 0% for No-Goggles, 60% for CycleGAN, and 100% for VR-Goggles in real-world chair navigation.The policy was trained in Gazebo and deployed on a Turtlebot3 Waffle.Source: C. Real-world Indoor & Outdoor… · p. 7, Figure 5 · p. 7
  • The outdoor policy is trained in Carla daytime and tested on a nighttime Robotcar dataset, with the vehicle instructed to follow the road.The outdoor comparison includes No-Goggles and VR-Goggles only.Source: C. Real-world Indoor & Outdoor… · p. 7
  • In the outdoor nighttime test, VR-Goggles guides the daytime-trained policy along the road, while direct deployment fails to move along it.The authors present representative sequences because quantitative evaluation of real-world control is non-trivial.Source: C. Real-world Indoor & Outdoor… · p. 7
  • The authors report that a model for a new type of chair can be trained without finetuning the indoor control policy.This additional result is referenced as available in a demonstration video.Source: C. Real-world Indoor & Outdoor… · p. 7

V. CONCLUSIONS

VR-Goggles translates real camera images into the synthetic domain during deployment, and the paper validates the pipeline through the Carla benchmark and real-world robotics experiments. Shift loss is validated for video style transfer and domain adaptation; the authors identify manipulation and more challenging environments as future directions.

  • The VR-Goggles pipeline translates real images to the synthetic domain during deployment and is validated through the Carla benchmark and real-world robotics experiments.Source: §V · p. 8
  • Shift loss increases consistency between subsequent translated frames and is validated in artistic style transfer for videos and domain adaptation.Source: §V · p. 8
  • The paper focuses mainly on navigation; the authors identify manipulation and more challenging environments with more sophisticated control tasks as future directions.Source: §V · p. 8
Loading 1802.00265v4…