Wide-Depth-Range 6D Object Pose Estimation in Space

1. Introduction

• Space maneuvers can cause large changes in target scale and aspect, while limited depth variation in common benchmarks leaves this challenge underrepresented.Space imagery also presents harsh contrast, under- and over-exposure, and specular reflections.Source: §1 · p. 1
• Prior methods handle large depth variation with a detector that resizes a target crop before pose estimation, separating the two stages and duplicating feature computation.Their reprojection-error objective is also sensitive to keypoint distance and target position, degrading performance across depth ranges.Source: §1 · p. 1, Figure 2 · p. 2
• A single hierarchical network predicts correspondences across pyramid levels, fuses them with RANSAC-based PnP, and trains on 3D positions rather than 2D projections.Training instances are assigned to multiple pyramid levels to promote joint use of multi-scale information.Source: §1 · pp. 1–2
• The method outperforms state of the art on SPEED and Occluded-LINEMOD, while also delivering the same benefits on the more challenging SwissCube scenario.The authors also report that their method is much faster on SPEED.Source: §1 · p. 2
• The authors introduce a larger-scale satellite pose dataset with more realistic and complex images than SPEED.They describe the renderings as including realistic satellite geometry, celestial backdrops, and illumination effects.Source: §1 · p. 2

2. Related Work

• Standard pose methods establish 3D-to-2D correspondences and use PnP, but were designed for minimal scale changes and perform poorly across dramatic depth variation.Source: §2 · p. 2
• Prior scale-handling approaches use object detection as preprocessing, while this work uses a single network with shared weights across hierarchical levels to handle scale robustly and efficiently.Source: §2 · p. 2
• Unlike assigning each pyramid level a predefined object scale, dynamic sampling lets each training instance use all levels with different weights, whose predictions are fused at inference.Source: §2 · pp. 2–3
• SPEED has no provided satellite 3D model, uses non-physics-based rendering, and contains few distant-object images; SwissCube instead uses physics-based rendering and an accurate reference model.Source: §2 · p. 3, Figure 4 · p. 3

3. Approach

• The overall approach regresses keypoint projections at multiple scales and fuses the resulting estimates into a robust pose prediction.The stated goal is estimating a known rigid object's 3D rotation and translation from an RGB image.Source: §3 · p. 3

3.1. Pyramid Network Architecture

• A Feature Pyramid Network built on Darknet-53 predicts keypoint projections and objectness at each of five feature-map levels.Each cell predicts eight 2D keypoint offsets and an objectness score for each object class.Source: §3.1 · p. 3

3.2. Ensemble-Aware Sampling

• The training strategy assigns each instance to multiple pyramid levels, so predictions from levels beyond the conventional size-matched level can be jointly used.The standard single-level assignment leaves most levels poorly trained for a given scale; the proposed sampling uses size-dependent reference levels, with α setting the maximum active cells and λ controlling their distribution.Source: §3.2 · p. 4, Figure 5 · p. 4

3.3. Loss Function in 3D Space

• The regression loss maps keypoint reprojection error into 3D space, making it invariant to keypoint depth and less influenced by the observed object's relative position.The paper describes this error as consistent across 3D keypoints and computable with simple algebraic operations for end-to-end learning.Source: §3.3 · pp. 4–5

3.4. Inference via Multi-Scale Fusion

• The network estimates pose from correspondences selected across pyramid levels and combines them with RANSAC-based PnP to obtain a robust 6D pose.At inference, it estimates object size from the highest-objectness prediction, selects high-objectness cells at each level, and forms 3D-to-2D correspondences.Source: §3.4 · p. 5

4.1. Evaluation on the SPEED Dataset

• At 960× input resolution, our method outperforms the two top-performing SPEED methods; at 640×, it remains on par with DLR while running 20+ times faster.The 960× network is also described as much smaller and faster than the competing methods.Source: Table 1 · p. 6, §4.1 · p. 6

4.2. Evaluation on the SwissCube Dataset

• On SwissCube, our method outperforms SegDriven, SegDriven-Z, and DLR, and is 20+ times faster than DLR.The dataset includes accurate modeling of the satellite’s mechanical parts and physically rendered space imagery; evaluation uses 50K images across three depth ranges.Source: §4.2 · p. 6, Table 2 · p. 7, §4.2.1 · p. 7

4.2.1 Effect of our Ensemble-Aware Sampling

• Sampling with λ = 1 approximately balances overlapping scale ranges across pyramid levels; λ > 10 yields lower accuracy, while assigning every instance to every level (λ = 0) is not best.The experiments use λ = 1 in subsequent evaluations.Source: §4.2.1 · p. 7, Figure 9 · p. 7

4.2.2 Effect of our Multi-Scale Fusion

• Multi-scale fusion outperforms every individual pyramid level, while L4 and L5 perform worse, which the authors attribute to their 8×8 and 4×4 feature maps.L1–L3 follow the expected depth-range pattern: larger receptive fields suit close objects, while smaller ones suit farther objects.Source: Table 3 · p. 7, §4.2.2 · p. 7

4.2.3 Effect of the 3D Loss

• The 3D loss outperforms the 2D reprojection loss at every depth range, with a larger advantage for farther objects.The 2D loss degrades near the image center, while the 3D loss remains stable across most positions; truncation near image borders degrades both.Source: §4.2.3 · pp. 7–8, Table 4 · p. 8, Figure 10 · p. 8

4.3. Results on Real Images

• Fine-tuning the synthetic-data model on 20 annotated lab images produces pose estimates on real images.The images show a mock-up satellite, with ground-truth poses obtained using motion capture.Source: §4.3 · p. 8, Figure 11 · p. 8

4.4. Evaluation on Occluded-LINEMOD

• On Occluded-LINEMOD, the method outperforms the state of the art, using separate models per object and only pyramid levels F1, F2, and F3.The evaluation trains on LINEMOD and tests on Occluded-LINEMOD without overlapping data.Source: §4.4 · p. 8, Table 5 · p. 8

• Training pyramid levels for different object scales and fusing their predictions improves accuracy and robustness.Source: §5 · p. 8
• The introduced SwissCube dataset combines an accurate satellite model with physically based rendering and simulations of the Sun, Earth, and stars.Source: §5 · p. 8
• The approach outperforms the state of the art in both wide-depth-range pose estimation and the more classical Occluded-LINEMOD setting.Source: §5 · p. 8