PaperlayerView exact revision on arXiv

Source-linked AI summary

RetinaGAN: An Object-aware Approach to Sim-to-Real Transfer

Daniel Ho, Kanishka Rao, Zhuo Xu, Eric Jang, Mohi Khansari, Yunfei Bai

arXiv:2011.03148v2cs.RO

TL;DR

Vision-based manipulation policies need costly real-world data, and simulation alone faces a visual gap at deployment. RetinaGAN adapts simulated images with object-detection consistency; it reports improved grasping and transfer to pushing and door opening, including settings with limited or no additional real data.

  • Problem

    Simulation can provide scalable policy-training data, but visual and physical differences from the real world impede deployment, and GAN adaptation can alter task-relevant object structure and texture.

    Source: §I · p. 1

  • Method

    RetinaGAN uses CycleGAN image translation with a frozen object detector to enforce consistent object predictions across original and translated images.

    Source: §I · p. 1

  • Results

    Across three tasks, RetinaGAN improves real-world grasping over prior sim-to-real methods by 12%, achieves 90% pushing success without additional real data, and reaches 97% door-opening success.

    Source: §I · p. 1

  • Takeaways & Limitations

    The results demonstrate reuse from grasping to a similar pushing task without extra real data and application to door opening in a drastically different environment.

    Source: §I · p. 1

    Building the initial object-detection dataset can be a significant expense, although off-the-shelf models are a proposed future direction.

    Source: A. Object Detection · p. 2

Abstract

from arXiv · show

The success of deep reinforcement learning (RL) and imitation learning (IL) in vision-based robotic manipulation typically hinges on the expense of large scale data collection. With simulation, data to train a policy can be collected efficiently at scale, but the visual gap between sim and real makes deployment in the real world difficult. We introduce RetinaGAN, a generative adversarial network (GAN) approach to adapt simulated images to realistic ones with object-detection consistency. RetinaGAN is trained in an unsupervised manner without task loss dependencies, and preserves general object structure and texture in adapted images. We evaluate our method on three real world tasks: grasping, pushing, and door opening. RetinaGAN improves upon the performance of prior sim-to-real methods for RL-based object instance grasping and continues to be effective even in the limited data regime. When applied to a pushing task in a similar visual domain, RetinaGAN demonstrates transfer with no additional real data requirements. We also show our method bridges the visual gap for a novel door opening task using imitation learning in a new visual domain. Visit the project website at https://retinagan.github.io/

I. INTRODUCTION

RetinaGAN addresses the visual reality gap that makes real-world deployment difficult despite simulation’s safe, scalable policy-training data. It combines image translation with object-detection consistency and reports transfer across grasping, pushing, and door opening.

I. INTRODUCTION

  • Real-world manipulation episodes are costly, while simulation offers a safe, controlled platform for scaling policy-training data; direct deployment still faces visual and physical differences.The authors identify visual differences as the bottleneck for their learning algorithms and focus on that gap.Source: §I · p. 1

II. RELATED WORK

  • Domain randomization depends on task-specific parameter choices, while GAN-based pixel adaptation can alter task-relevant information such as object structure and texture.RetinaGAN focuses on preserving these object-level scene features.Source: §I · p. 1, §II · p. 2

I. INTRODUCTION

  • RetinaGAN uses a CycleGAN with object-detection consistency to make simulated images more realistic while preserving object predictions across translation.A detector trained on simulated and real domains predicts on original and translated images; the predictions are constrained to remain invariant.Source: §I · p. 1
  • RetinaGAN improves real-world grasping over prior sim-to-real methods by 12%, loses only 14% with 5–10% of the data, and reaches 90% pushing success without additional real data.The pushing adapter is reused from grasping data.Source: §I · p. 1
  • RetinaGAN reaches a 97% success rate on door opening in a drastically different environment using imitation learning.The method is trained for this task in the new visual domain.Source: §I · p. 1

II. RELATED WORK

  • RetinaGAN uses object detection as a task-decoupled alternative to task-loss approaches such as RL-CycleGAN, which requires task-specific real-world episodes.The detector can be reused across robotics tasks, but RetinaGAN requires real-world bounding-box labels.Source: §II · p. 2

III. PRELIMINARIES

The perception model is trained on labeled simulated and real images and then reused as a task-agnostic source of object awareness. The authors note both the cost of building detection data and successful transfer to a substantially different door-opening domain.

III. PRELIMINARIES

  • The detector is trained on mixed simulated and real datasets with ground-truth bounding boxes, including 44,000 recycling-station and 37,000 desk images and 625,000 simulated images.The real images cover disposable objects and objects on desks; simulated data are generated with PyBullet.Source: A. Object Detection · p. 2, Figure 2 · p. 2
  • Object detection is task-agnostic, allowing one multi-domain detector to be used across tasks, and the authors report successful transfer to a door-opening domain unlike its perception-training data.They suspect in-domain detection data are not crucial to the method’s success.Source: A. Object Detection · p. 2
  • Creating the initial object-detection dataset can be a significant expense, while off-the-shelf models are proposed as a possible way to reduce that cost.The authors point to door-opening results as motivating this future direction.Source: A. Object Detection · p. 2
  • The detector uses EfficientDet-D1 with RetinaNet losses, an EfficientNet backbone, multiscale feature fusion, and heads for class logits and box regression.The feature pyramid network fuses features at multiple scales.Source: A. Object Detection · p. 2

B. CycleGAN

RetinaGAN builds on CycleGAN’s bidirectional image translation and cycle-consistency training for unpaired domains. It adds a frozen object detector whose predictions are constrained to remain consistent across original, translated, and cycled images.

B. CycleGAN

  • CycleGAN learns bidirectional mappings between unpaired image domains using adversarial discriminators and cycle consistency.The generators map between domains X and Y, while the cycle loss encourages each image to map back to its source.Source: B. CycleGAN · p. 2

IV. RETINAGAN

  • The training pipeline translates simulated and real images in both directions, cycles them back, and computes detector-consistency loss between image pairs alongside CycleGAN losses.The algorithm applies the detector loss to pairs drawn from each domain’s original, translated, and cycled images.Source: B. CycleGAN · p. 3, Figure 3 · p. 3
  • RetinaGAN adds a frozen, pre-trained EfficientDet detector to provide object-consistency loss during training, then adapts simulated images for task-policy training.The framework uses unpaired data without labels.Source: §IV · p. 3
  • A frozen EfficientDet evaluates the six original, translated, and cycled images so predictions remain invariant across the three images in each domain branch.The perception consistency loss is computed for each such pair.Source: §IV · p. 3
  • The detector predicts boxes and classes, and the perception loss captures consistency between image predictions with losses similar to those used in object-detection training.The accompanying figure diagrams this loss computation.Source: Figure 4 · p. 3

A. Perception Consistency Loss

Perception consistency loss constrains object detections across translated images, using detector outputs on image pairs and a batch formulation spanning simulated and real images.

A. Perception Consistency Loss

  • The perception consistency loss penalizes discrepancies in object detections between translations, using anchor-based box regressions and class logits at multiple feature-pyramid levels.It is computed for image pairs using a pre-trained EfficientDet network.Source: A. Perception Consistency Loss · p. 3
  • The loss uses box and class losses like RetinaNet and EfficientDet, but replaces one-hot-only Focal Loss with a variation compatible with probability labels.Huber loss supplies the box-regression loss.Source: A. Perception Consistency Loss · p. 3
  • The batch formulation applies perception consistency to simulated images x and real images y using the sim-to-real generator G and real-to-sim generator F.Source: A. Perception Consistency Loss · p. 3
  • Losses on cycled images F(G(x)) and G(F(y)) are halved because those images are compared twice, although this weighting has little practical effect.Source: A. Perception Consistency Loss · p. 3
  • The section then combines these terms into the overall RetinaGAN loss.Source: A. Perception Consistency Loss · p. 3

B. Focal Consistency Loss (FCL)

Focal Consistency Loss extends Focal Loss to probability-valued targets, preserving its one-hot behavior while providing an interpolated loss and a probability-based normalization.

B. Focal Consistency Loss (FCL)

C. Hyperparameters

The experiments evaluate RetinaGAN for grasping across data regimes and compare it with other adaptation methods; hyperparameters are selected to preserve objects while retaining image realism.

C. Hyperparameters

  • The cycle-consistency weight is set to 10, while the perception-consistency weight is set to 0.1 to preserve objects; relative weights between perception-loss terms were not important.The authors report that perception-consistency weights from 0.1 to 1.0 were stable.Source: C. Hyperparameters · p. 4

V. TASK POLICY MODELS AND EXPERIMENTS

  • The experiments compare RetinaGAN against no-RetinaGAN policies at different data sizes, GAN-based adaptation methods, and transfer to other tasks.The grasping sequence includes limited-data mixing, larger real-data training, and sim-only comparisons; the study then applies the model to pushing and retrains it for door opening.Source: §V · p. 4, A. Reinforcement Learning: Grasping · p. 4

A. Reinforcement Learning: Grasping

  • The grasping setup uses Q2-Opt with RGB images and a binary target mask, evaluating cups, cans, and bottles across six robots and sorting stations.The evaluation uses 90 total attempts; simulation-only Q2-Opt reaches 92% success in simulation but 18.9% on real object equivalents.Source: A. Reinforcement Learning: Grasping · pp. 4–5
  • 80% success with RetinaGAN on translated-only data outperforms other methods by over two standard deviations and is on par with RetinaGAN+Real.With 10K real episodes, RetinaGAN+Real achieves 65.6% and outperforms Sim+Real trained with 135K+ episodes, showing more than 10X data efficiency.Source: Table I · p. 4, A. Reinforcement Learning: Grasping · p. 5

B. Reinforcement Learning: 3D Object Pushing

The grasping-trained RetinaGAN is reused for a visually similar pushing task without fine-tuning or additional real data. It enables strong real-world transfer where a simulation-only policy fails.

B. Reinforcement Learning: 3D Object Pushing

C. Imitation Learning: Door Opening

In a new visual domain, RetinaGAN is tested for imitation-learned door opening using a detector trained on recycling objects. Ensemble-RetinaGAN improves reported success, including in the no-real-data training condition.

C. Imitation Learning: Door Opening

  • The detector is reused from recycling-object training, requiring no further human labeling, although confident door-domain detections are limited to the robot arm.The authors hypothesize that consistency in low-probability prediction regimes may maintain structures such as door frames.Source: C. Imitation Learning: Door Opening · p. 6
  • Baselines without consistency loss lose room-structure and door-location consistency, creating a stated safety risk that led the authors not to evaluate them.RetinaGAN strongly preserves those structures relative to the baselines.Source: C. Imitation Learning: Door Opening · p. 6
  • The simulation-trained IL policy reaches 98% success in simulation but has no successes in real-world evaluation.Evaluation uses three conference rooms represented in the training demonstrations.Source: C. Imitation Learning: Door Opening · p. 6, Table III · p. 6
  • RetinaGAN achieves 76.7% success on trained conference-room doors, comparable to 75% when real-world demonstrations are mixed into IL training.These are doors in conference rooms seen during training.Source: C. Imitation Learning: Door Opening · p. 6, Table III · p. 6
  • Ensemble-RetinaGAN reaches 93.3% success, while its no-real-data IL variant reaches 96.6%, within the margin of error of Ensemble-RetinaGAN+Real.The ensemble combines data from three RetinaGAN models trained with different random seeds and consistency-loss weights; the table covers 30 trials.Source: C. Imitation Learning: Door Opening · p. 6, Table III · p. 6

VI. CONCLUSIONS

RetinaGAN transfers across environments and tasks, including with limited real data. The paper reports instance-grasping results and identifies extensions for future work.

VI. CONCLUSIONS

  • 80% success on instance grasping, a 12 percentagepoint improvement upon baselines.The conclusion also describes RetinaGAN as robust across environments and tasks, including with limited real data.Source: §VI · p. 6
  • RetinaGAN is presented as an object-aware sim-to-real adaptation technique that transfers across environments and tasks, even with limited real data.Source: §VI · p. 6
  • Future work could investigate pixel-level perception consistency, depth, and transfer to visual environments unseen during training.Source: §VI · p. 6

APPENDIX

The appendix discusses alternative perception losses, a CycleGAN failure case, and training configurations for the detector and RetinaGAN.

A. Alternative Perception Losses

  • Separate perception networks add complexity and require synthetic and real object sets to be close to bijective for consistent predictions on paired images.Source: A. Alternative Perception Losses · p. 7
  • Off-the-shelf pretrained perception models could avoid model-creation costs, but imbalanced sim/real predictions may destabilize training.Source: A. Alternative Perception Losses · p. 7
  • Segmentation offers dense pixel-level object supervision, while detection data is easier and more efficient to label.Segmentation may provide stronger supervision and consistency for background objects and structures.Source: A. Alternative Perception Losses · p. 7

B. Door Opening Figure

C. Perception Model Training

  • EfficientDet training used default augmentation parameters, including a 0.8–1.2 scale range, across 59 classes.Frequently used classes included robot, bottle, bowl, can, cup, bag/wrapper, and plate; hyperparameters are listed in Table IV.Source: C. Perception Model Training · p. 7, Table IV · p. 7

D. RetinaGAN Model Training

  • RetinaGAN followed Appendix A of hyperparameters; training batches contained 256 simulation and 256 real images, and Lprcp was primarily searched between 0.1 and 1, without CycleGAN hyperparameter tuning.Source: D. RetinaGAN Model Training · p. 7

E. Q2-Opt RL Model Training

The appendix describes model training for grasping, pushing, and door-opening policies, along with the real-world evaluation setups.

E. Q2-Opt RL Model Training

  • Q2R-Opt was used for grasping and pushing with the prior work’s hyperparameters and the same simulated object set as RetinaGAN.Training batches sampled full-dataset episodes evenly from simulation and real data; for the restricted 10K dataset, weights were 20% simulation and 80% real.Source: E. Q2-Opt RL Model Training · pp. 7–8

F. ResNet-FiLM IL Model Training

  • The IL policy used ResNet-FiLM with a ResNet-18 architecture and mixed real, simulated, and RetinaGAN-adapted demonstrations.Real demonstrations received 20% weight, with simulated and adapted demonstrations weighted evenly.Source: F. ResNet-FiLM IL Model Training · p. 8

G. Evaluation

  • Grasping evaluation targeted cups, cans, and bottles across replicated station setups, totaling 90 grasps.The station setups display the object classes in each bin.Source: G. Evaluation · p. 8, Figure 9 · p. 8
  • Pushing evaluation used a single Ito En Green Tea Bottle filled 25% with water.Source: G. Evaluation · p. 8
  • Door-opening evaluation used three conference-room doors; success required the robot to push a door open and enter the room.Two doors swung right and one swung left.Source: G. Evaluation · p. 8

Related papers

Loading 2011.03148v2…