OSOP: A Multi-Stage One Shot Object Pose Estimation Framework

1. Introduction

• Object-specific 6 DoF pose annotation is time-consuming, and synthetic rendering and training must be repeated for each new target.These requirements limit practical applicability; labels can also be imperfect.Source: §1 · p. 1
• Prior one-shot pose methods had limited reach, often restricting objects by category or geometry, using target-object training, or estimating viewpoint alone.OSOP is presented as a one-shot method for full 6 DoF pose that generalizes to new objects without target-object training or data preparation.Source: §1 · p. 1
• The pipeline uses a textured 3D model to localize and segment an object, match its viewpoint, predict dense image-template correspondences, then estimate pose with PnP or Kabsch and RANSAC.The correspondences transfer from the matched template, whose pose is known, to the input image and 3D model.Source: §1 · p. 2, Figure 2 · p. 2
• Evaluation spans LineMOD, Occlusion, Homebrewed, YCB-V, and TLESS; the paper presents the method as generalizing to unseen objects and scenes.The contributions also include a one-shot segmentation architecture and dense matching architecture for transferring 2D-3D correspondences.Source: §1 · p. 2

2. Related Work

• Many deep-learning pose methods predict sparse 2D keypoints or dense pixel-wise 2D-3D correspondences, then estimate pose with PnP; CosyPose instead regresses pose and refines across views.The cited methods include IPose, YOLO6D, PVNet, HybridPose, DPOD, CDPN, Pix2Pose, EPOS, and CosyPose.Source: §2 · p. 2
• PPF approximates local geometry with oriented point pairs and matches scene and object features, but depends on depth and per-point normals and can run slowly.PPF methods led the BOP challenge until deep-learning methods surpassed them more recently.Source: §2 · p. 2
• Category-focused methods generalize according to within-category similarity, while geometry-similarity methods require strong resemblance between training and test objects.NOCS and follow-ups [31] target narrow categories; CorNet uses corners, and predicts dense local shape descriptors.Source: §2 · p. 2
• Template-matching rotation estimation can generalize across datasets, but the cited method trains a detector on target classes; PfS [57] assumes perfect 2D localization and estimates rotation only.OSOP is described as fully one-shot and as using the 3D object model during inference.Source: §2 · pp. 2–3
• One-shot 2D detectors match template and image features to localize novel objects, while OSOP uses the object's full 3D information.OS2D applies sliding-window correlation to template and image features.Source: §2 · p. 3

3.1. One-Shot Segmentation

• Rendered viewpoints are pooled into one model descriptor, correlated against image pixels, and combined with cross-level attention to predict the visible-object mask.Attention from the last feature level conditions noisier, higher-resolution attention; values below the average are zeroed to produce cleaner maps.Source: §3.1 · pp. 3–4

3.2. Template Matching

• The detected object patch is matched to rendered templates using deep features, and the highest-similarity template supplies an initial viewpoint estimate.Training uses a modified triplet loss to increase similarity for close rotations and penalize similarity for distant rotations.Source: §3.2 · pp. 4–5

3.3. One-Shot Dense Correspondence Estimation

• Dense image-to-template correspondences establish image-to-model matches, which can be used to estimate pose with PnP+RANSAC or Kabsch+RANSAC.The network predicts a foreground/background mask and a template coordinate for each pixel, with outlier-aware rejection for correspondences having large 3D spatial discrepancy.Source: §3.3 · p. 5

3.4. Pose Hypothesis Verification

• An optional verification stage generates pose hypotheses from top matched templates to reduce imprecision from incorrect initial viewpoint estimates.The method filters matches using a 15-degree threshold and picks the top 25; hypotheses are ranked using depth fit when depth is available, or correspondence error in RGB.Source: §3.4 · p. 5

4. Experiments

• The experiments use synthetic PBR images and separate training and test objects for each target dataset; LineMOD uses ADD, while other datasets use BOP Average Recall.The reported datasets include LineMOD, Occlusion, Homebrewed, and YCB-V; TLESS results are in supplementary material.Source: §4 · pp. 5–6

4.1. 2D Object Localization

• OSOP’s detector runs about 800× faster than OS2D (25 ms versus 20 seconds per object), while YOLO achieves higher precision and recall.OSOP’s recall is close to YOLO’s when YOLO uses the confidence threshold maximizing F1.Source: §4.1 · p. 6, Table 1 · p. 6

4.2. 6 DoF Pose Results

• On Linemod, using ADD scores, OSOP significantly outperforms SSD6D and outperforms AAE and SSD6D with deep learning-based refinement, despite not being trained on Linemod objects.The comparison is among methods using only synthetic training data; all methods except ours, PPF and PfS require prior training on RGB target objects.Source: §4.2 · p. 6, Table 2 · p. 6
• On LineMOD, hypothesis verification raises ADD from 39.3 to 43.6; using depth with Kabsch nearly doubles the ADD score and further verification puts it just above PPF.The RGB-only pipeline runs at approximately 10 FPS; using Kabsch runs at 16 FPS.Source: §4.2 · p. 7
• On Occlusion and Homebrewed, OSOP performs similarly to some fully supervised methods and beats Multi-Path AAE; on YCB-V, Multi-Path AAE leads Our+PnP, but OSOP beats PPF.The authors note that real- or mixed-data methods perform considerably better than synthetic-only methods on YCB-V.Source: §4.2 · p. 7, Table 3 · p. 7, Table 4 · p. 7

4.3. Ablation Studies

• Using feature correlation alone yields 54% recall and 51% IoU; combining correlation with pixel-wise attention significantly increases localization performance.The study also evaluates localization architectures, pose-estimation stages, template counts, and angular mismatch between the matched and ground-truth templates.Source: §4.3 · p. 7, Table 6 · p. 8, Figure 6 · p. 8

• The method relies on a pre-trained feature extractor producing distinctive features for synthetic object renderings and input images.Source: §5 · p. 8
• Its matching premise is that templates and images depicting similar poses have higher feature correlations.Source: §5 · p. 8
• The authors identify the real-to-synthetic domain gap as affecting this assumption and suggest unsupervised domain adaptation as a potential remedy.Source: §5 · p. 8

• On LineMOD, Occlusion, Homebrewed, YCB and TLESS, the method achieves results similar to methods trained on synthetic data.Source: §6 · p. 8
• The method generalizes to unseen objects without imposing specific requirements on the objects.Source: §6 · p. 8
• Its one-shot localization network performs significantly better and faster than an alternative 2D one-shot detector.Source: §6 · p. 8
• For new unseen objects, the pipeline reduces preparation and training time because those steps are not needed.Source: §6 · p. 8