Source-linked AI summary
Vision Foundation Models with Synthetic-Only Training for Monocular Spacecraft Pose Estimation
John Church, Vazghen Nikolian
TL;DR
Monocular pose estimation for uncooperative spacecraft remains difficult under harsh lighting and limited real training data. The paper adapts DINOv3 in a heatmap-based architecture using synthetic-only training, and its best model reports 1.17° lightbox and 1.56° sunlamp mean rotation errors. The authors also measure inference of the 840M model on a Jetson Orin NX, while noting hardware scope limits.
Problem
Accurately estimating the 6D pose of an uncooperative spacecraft is needed for rendezvous, but real imagery is difficult to obtain and harsh lighting complicates monocular vision.
Source: §I · pp. 1–2Method
The paper adapts self-supervised DINOv3 encoders in a heatmap-based keypoint-and-PnP architecture and trains on synthetic imagery.
Source: §II · p. 2Results
1.17° lightbox and 1.56° sunlamp mean rotation error are reported for the rank-64 840M three-seed ensemble with four-rotation test-time augmentation.
Source: §V · p. 5Takeaways & Limitations
For missions with sufficient onboard compute, the adapted large encoders provide a more accurate option than the previous state of the art described by the authors.
Source: §VI · p. 6
Abstract
from arXiv · showhide
We present an improvement on previous spacecraft pose estimation architectures that results in the lowest published mean rotation errors we know of on the SPEED+ lightbox and sunlamp test sets for a known, non-cooperative spacecraft. By using a previously established heatmap-based pose estimation architecture and adapting a large self-supervised ViT foundation model (DINOv3) in place of the smaller convolutional and ViT encoders of previous work, we show that pose estimation accuracy improves from 300M to 840M parameters with no saturation yet observed. We also evaluate our 840M model on a Jetson Orin NX 16GB, measuring single-pass network inference at 133.8 ms per crop with a board draw of 32.0 W. These measurements demonstrate embedded inference feasibility on a processor family with orbital flight heritage. Our resulting model outperforms previous models across lightbox and sunlamp domains while training only on synthetic data. Our best model, using DINOv3 840M adapted with LoRA as the encoder (rank 64, three-seed ensemble with four-rotation test-time augmentation), results in $1.56^\circ$ mean rotation error on sunlamp and $1.17^\circ$ on lightbox, compared to the previous best mean rotation errors we know of on these test sets, $2.66^\circ$ and $1.75^\circ$ by EagerNet.
I. INTRODUCTION
Monocular pose estimation is needed to approach non-cooperative spacecraft, but harsh orbital lighting and limited real training imagery make the task difficult. This work adapts a large self-supervised vision model and reports improved SPEED+ results while training only on synthetic imagery.
I. INTRODUCTION
- The method targets 6D pose estimation for spacecraft that provide no assistance, while assuming their geometry and selected keypoint locations are known.Harsh sunlight creates black shadows and overexposed surfaces, and active sensors add mass, power, and complexity.Source: §I · p. 1
- SPEED+ pairs synthetic Tango renders with 9,531 real HIL test images staged under sunlamp and lightbox illumination.Its roughly 60,000 synthetic images form the training and validation data.Source: §I · p. 1
- On the same DINOv3-L encoder, the synthetic-only single-pass model reaches 1.90°/2.26° mean rotation error on lightbox/sunlamp, versus Bergia et al.’s self-trained 3.27°/4.06°.The best configuration reaches 1.17°/1.56°, versus their self-trained 2.57°/3.25°.Source: §I · p. 2
- The paper asks whether scaling improves pose estimation and measures the resulting model’s embedded-hardware cost, reporting gains over prior published results known to the authors.Recent keypoint-based studies do not report separate results on the full SPEED+ lightbox and sunlamp test sets.Source: §I · p. 2
II. METHOD
The method holds the crop-and-pose-solving stages largely fixed to focus on keypoint detection. It replaces the smaller encoder with DINOv3 and trains heatmap-based models on synthetic SPEED+ imagery using parameter-efficient adaptation.
II. METHOD
- SPNv3’s pipeline crops the target, detects keypoints, then solves 6D pose with PnP; this study uses ground-truth-derived crops to isolate keypoint detection.The comparison therefore evaluates the keypoint-and-PnP stages, not an end-to-end detector under a common protocol.Source: §II · p. 2
- DINOv3 replaces the ViT-S encoder in a K-heatmap keypoint head; the 300M variant uses LoRA or DoRA and the 840M variant uses LoRA.The 840M rank-64 three-model ensemble with four-rotation test-time augmentation reaches 1.17° lightbox and 1.56° sunlamp mean rotation error, using twelve passes per frame.Source: §II · p. 2
- Training uses the official synthetic SPEED+ split, 448×448 crops, and a heatmap head trained from scratch; no real images are used for training.LoRA and DoRA leave the original encoder weights frozen and add trainable parameters equivalent to roughly 0.5–3% of encoder weights.Source: §II · pp. 2–3
- Table I reports selected prior SPEED+ HIL results under each source’s own training and crop/detection protocol, not a controlled common-pipeline comparison.Prior rotation-error values may use the HIL precision floor, unlike this paper’s unthresholded Table II means.Source: Table I · p. 3, §II · p. 3
III. CONTROLS
Control runs compare encoder scale, frozen pretrained features, and adaptation across model families. They indicate that gains do not follow from using a modern encoder or fine-tuning method alone.
III. CONTROLS
- The 22M DeiT3 ViT-S baseline matches the range of SPNv3, while frozen DINOv3 and SigLIP2-L encoders produce catastrophic flips on 18%/29% and 54%/70% of lightbox/sunlamp frames.These controls test whether pretrained representations can support pose estimation without encoder fine-tuning.Source: §III · p. 3
- LoRA-tuned EUPE-S performs significantly worse than the fully trained 22M DeiT3 baseline, so modern encoders and fine-tuning alone do not explain the gains.The controls were designed to distinguish encoder-scale gains from architectural or general base-encoder improvements.Source: §III · p. 3
- LoRA-tuned DINOv3-L reaches 1.90°/2.26° lightbox/sunlamp mean rotation error, compared with 2.47°/2.74° for similarly sized LoRA-tuned SigLIP2-L.SigLIP2-L remains 18–23% behind DINOv3-L.Source: §III · p. 3
IV. HARDWARE TESTING
The 840M model was benchmarked on a Jetson Orin NX as a preliminary embedded-inference test. The measured platform is relatively power- and thermal-heavy, and the chip was not tested under radiation or flight-representative thermal conditions.
IV. HARDWARE TESTING
- 133.8 ms per network pass at 32.0 W board draw, and 419.8 ms at 10.8 W, were measured for the 840M model on Jetson Orin NX 16GB.Four-view batches took 501.9 ms and 1.63 s at those respective operating points.Source: §IV · p. 4
- The Orin’s 10–40 W rated envelope and 10.8–32 W measured draw make it heavier in power and thermal terms than many flown systems; radiation and flight-representative thermal behavior remain untested.The measurements inform viability when a mission already has sufficient onboard compute; more testing is needed for flight readiness and smaller, energy-constrained craft.Source: §IV · p. 4
- Figure 2 shows failure examples under near-black lightbox conditions and glare-saturated sunlamp conditions for DINOv3-L seeds and the 22M baseline.The caption identifies examples from two training seeds of the 300M DINOv3-L + LoRA model and the baseline.Source: Figure 2 · p. 4
V. RESULTS
The 840M rank-64 ensemble with four-rotation test-time augmentation achieves the paper’s strongest reported full-test-set rotation errors. The results also examine scaling, frame abstention, failures, and embedded benchmark scope.
V. RESULTS
- The rank-16 840M single-pass model achieves 1.34° / 1.84° mean rotation error and exceeds previously published full-test-set results on both domains.The passage does not specify which value corresponds to which domain.Source: §V · pp. 4–5
- Table III defines Mean° over all frames, Bulk° over frames with error at or below 30°, and Flip% as in Table II; its controls use seed 2026 and one pass per crop.The LoRA controls use rank 16.Source: Table III · p. 5
- Table IV specifies Jetson Orin NX 16GB benchmark conditions, but its caption alone does not provide latency or power values.Timing excludes acquisition, crop preparation, transfers, TTA aggregation, and pose solving.Source: Table IV · p. 5
- 1.17° mean rotation error on lightbox and 1.56° on sunlamp for the rank-64 840M three-seed ensemble with four-rotation test-time augmentation.These results use the full test sets without abstention and require twelve network passes per frame.Source: §V · p. 5
- The rank-64 840M ensemble fails on 13 of 9,531 real frames; one frame in each domain fails for every training seed, often under near-black or glare-saturated conditions.A solved-distance range check catches only one flip per domain; other flips yield plausible distances.Source: §V · p. 5
- Withholding the 2% least-confident frames reduces the 300M model’s flip rate to 0.09% / 0.07% and mean rotation error to 1.46° / 1.81°.Withholding 10% removes every flip in both domains; the results tables do not use abstention to remain comparable with prior methods.Source: §V · p. 5
VI. CONCLUSION
The conclusion presents large, adapted DINOv3 encoders as a route to improved spacecraft pose estimation, while noting the evaluation’s narrow scope.
VI. CONCLUSION
- Adapting large self-supervised ViT encoders improves on smaller-encoder methods, and LoRA or DoRA adapts a small share of encoder weights while frozen weights remain unchanged.Scaling from 300M to 840M cuts mean rotation error by roughly 15–30% and roughly halves catastrophic flips.Source: §VI · p. 6
- The authors conclude that onboard pose estimation has a more accurate option when the available compute supports a model of this size.On the Jetson Orin NX, one network pass takes 134–420 ms at 11–32 W of board draw.Source: §VI · p. 6
- The study is limited to single-frame predictions for one target satellite with a well-defined 3D model and predefined keypoints.This uses a stronger geometric prior than approximate-model methods such as EagerNet require.Source: §VI · p. 6