Adapting Deep Visuomotor Representations with Weak Pairwise Constraints

1 Introduction

• The work addresses domain transfer because perception systems trained offline, in laboratories, or in simulation may fail when deployed in real-world settings.Accurate state annotations are often available only in simulated or instrumented environments, whose appearance can differ substantially from the real world.Source: §1 · p. 1
• The framework combines pairwise and distribution-level alignment and finds source-target image pairs without target-domain labels.It is applied to pose estimation from synthetic to real images and to pretraining visuomotor policies.Source: §1 · p. 2
• Prior deep adaptation methods primarily reduce domain discrepancy or maximize domain confusion, missing instance-level constraints available from synthetic-real image pairs.The paper notes that earlier work had not learned such pairs without explicit annotations or combined them with contemporary discrepancy or confusion models.Source: §1 · p. 2
• The evaluation uses corresponding synthetic and real-world images for pose estimation, with the method finding pairs without real-world supervision.The figure depicts a synthetic image on the left and its real-world counterpart on the right.Source: Figure 1 · p. 2
• The method improves pose estimation adaptation and supports real-world manipulation policies without pose-annotated real images.The reported PR2 experiments also transfer annotations from source data when target-domain pose annotations are unavailable.Source: §1 · pp. 2–3

2 Related work

• Earlier synthetic-data work spans robotic classification and grasping, simulated driving with ALVINN, and rendered-scene pose estimation.Fixed feature representations limited performance in pose estimation, and state-of-the-art methods generally trained exclusively on real imagery.Source: §2 · p. 3
• Prior domain adaptation evolved from transformations or target-model regularization to methods combining both, then to deep approaches minimizing distribution discrepancy.The deep methods cited include maximum mean discrepancy and a-distance objectives.Source: §2 · p. 3
• Earlier synthetic-to-real research studied rendered object training and deep representation invariances [25], while noting a domain shift when applying synthetic-trained models to real data.Source: §2 · p. 4
• Deep reinforcement learning has shown broad task performance but often requires many samples, making many methods impractical on physical systems [28].The paper presents adaptation of simulated visual systems as an initial step toward real-world visuomotor policies.Source: §2 · p. 4
• Taylor et al. combined synthetic and real data but found performance suffered when simulated data overwhelmed real data; this paper instead learns from synthetic data and refines with few real examples.Source: §2 · p. 4
• Visuomotor policies learned from imagery require real-world visual data for each new task, while direct transfer from simulation typically fails because of systematic discrepancies [28].The paper explores adapting simulated imagery for robotic tasks in the real world.Source: §2 · p. 4

3 Preliminaries

• The task is to learn image features that support visuomotor policies from many labeled source images and a small set of unlabeled target images.The source may be simulation, while the target is the real world.Source: §3 · p. 5
• Regression training learns an image representation and a regressor to predict labels from input images.The representation maps each image to a feature vector.Source: §3 · p. 5
• Real-world ground-truth labels can require expensive instrumented setups, whereas simulation can generate large amounts of fully annotated training data.Synthetic examples can be generated by changing the environment configuration, recording labels, and rendering views.Source: §3 · p. 5
• Synthetic and real images differ in appearance, and this domain shift generally reduces performance when source-trained models transfer directly to the target.Source: §3 · p. 5
• The proposed model augments the task loss with two adaptation losses to align synthetic-source and real-target domains in feature space.Source: §3 · p. 5

• The model combines task loss, distribution-level domain confusion, and pairwise alignment to learn task-relevant features robust to domain shift.Source: §4 · p. 6, Figure 2 · p. 6
• Domain confusion trains a classifier to distinguish domains while encouraging representations that make its predictions match a uniform distribution.Source: §4 · pp. 6–7
• Pairwise loss aligns source and target images with similar task labels in representation space, focusing features on task-relevant rather than domain-specific information.Source: §4 · p. 7
• A source-trained task feature matches each target image to its nearest source neighbor, forming weak pairs and transferring source labels to target images.Source: §4 · pp. 7–8
• The joint objective uses λ and ν to trade off the strength of domain confusion and weak pairwise constraints, and its loss combination is architecture-agnostic.Source: §4 · p. 8
• After pairing, the method fixes P and minimizes pairwise loss with respect to the representation so paired images lie close in feature space.Source: §4 · pp. 7–8

5 Adapting visuomotor control policies

• The approach combines pairwise and distribution-level alignment, then uses adapted visual features in guided policy search to train a controller that acts from vision at test time.The representation is pretrained for pose estimation; the policy is then trained using feature points and robot state, without backpropagating policy gradients through the representation.Source: §5 · pp. 8–9

5.1 Supervised robotic pose estimation evaluation

• 20% (relative) improvement in PR2 gripper pose-estimation performance came from combining domain-confusion and pair-alignment losses with five labeled, paired real examples.The reported metric is average prediction error in centimeters, averaged over three trials.Source: Table 1 · p. 10, §5.1 · p. 11

5.2 Unsupervised synthetic-real alignment evaluation

• Weak synthetic-real alignment produced generally high-quality pairings, with the visual examples matching hook positions while largely ignoring arm position.The alignment evaluation compared task-loss-only features and features trained with an additional domain-confusion loss; both approaches had lower pairing error than random, approaching Oracle in simpler settings.Source: Table 2 · p. 11, Figure 4 · p. 12, §5.2 · p. 12

5.3 Visuomotor policies for manipulation tasks

• The final policy trained with the learned features was much more accurate than methods not using labeled real images, using 100 unlabeled real images for adaptation.Features adapted from synthetic renderings and unlabeled real images performed equally well to features trained on 500 labeled real images in the Oracle comparison.Source: §5.3 · pp. 13–14

6 Conclusion

• The model combines pairwise and distribution alignment losses and optimizes over weak label assignments for weakly paired source–target examples.The authors describe the model as applicable to a wide variety of deep adaptation architectures and tasks.Source: §6 · p. 14
• Pose estimation experiments validate the importance of image pairs for strong adaptation, including when real-world pose annotations are unavailable.Source: §6 · p. 14
• The work addresses visual inputs for robotic state estimation; fully simulated control policies still require physical adaptation to account for simulated-to-real physics mismatch.The authors identify physical adaptation as future work.Source: §6 · p. 14