Source-linked AI summary
Pose Estimation for Non-Cooperative Spacecraft Rendezvous Using Convolutional Neural Networks
Sumant Sharma, Connor Beierle, Simone D'Amico
TL;DR
Reliable monocular pose estimation for noncooperative spacecraft is hindered by illumination sensitivity and scarce labeled space imagery. The paper addresses these issues with a CNN that classifies discretized pose labels using synthetically generated spacecraft images, and reports robustness and scalability across its experiments. The study also identifies the need for validation on real imagery and broader spacecraft and orbit conditions.
Problem
Monocular pose estimation for noncooperative spacecraft lacks robustness to illumination conditions and sufficient image datasets for training and benchmarking.
Method
The paper trains a CNN on synthetic spacecraft imagery generated by discretizing a four-dimensional view-space into pose labels.
Results
The architecture showed desirable robustness and scalable properties, while net8 provided high-confidence solutions on 68% of Imitation-25 images compared with 20% for SVD.
Takeaways & Limitations
Synthetic-data CNN pose determination is presented as a building block for future formation-flying navigation systems.
Takeaways & Limitations
The networks require testing on actual space imagery, and their accuracy and robustness with other spacecraft and orbit regimes remain to be evaluated.
Abstract
from arXiv · showhide
On-board estimation of the pose of an uncooperative target spacecraft is an essential task for future on-orbit servicing and close-proximity formation flying missions. However, two issues hinder reliable on-board monocular vision based pose estimation: robustness to illumination conditions due to a lack of reliable visual features and scarcity of image datasets required for training and benchmarking. To address these two issues, this work details the design and validation of a monocular vision based pose determination architecture for spaceborne applications. The primary contribution to the state-of-the-art of this work is the introduction of a novel pose determination method based on Convolutional Neural Networks (CNN) to provide an initial guess of the pose in real-time on-board. The method involves discretizing the pose space and training the CNN with images corresponding to the resulting pose labels. Since reliable training of the CNN requires massive image datasets and computational resources, the parameters of the CNN must be determined prior to the mission with synthetic imagery. Moreover, reliable training of the CNN requires datasets that appropriately account for noise, color, and illumination characteristics expected in orbit. Therefore, the secondary contribution of this work is the introduction of an image synthesis pipeline, which is tailored to generate high fidelity images of any spacecraft 3D model. The proposed technique is scalable to spacecraft of different structural and physical properties as well as robust to the dynamic illumination conditions of space. Through metrics measuring classification and pose accuracy, it is shown that the presented architecture has desirable robustness and scalable properties.
1. INTRODUCTION
Monocular vision-based pose determination supports autonomous operations around noncooperative spacecraft, but conventional spaceborne methods face illumination and computational challenges. The paper proposes a CNN architecture trained with synthetic imagery to provide real-time pose estimates and examines its scalability and robustness.
- Motivation: Monocular pose estimation provides relative position and attitude information needed for on-board trajectory generation and control during noncooperative spacecraft operations.It is especially relevant when the target’s kinematic characteristics are largely unknown before the mission.
- Motivation: Monocular navigation offers lower power and mass requirements than LiDAR- or stereo-camera-based systems.
- Limitations of prior methods: Conventional spaceborne methods detect and match hand-engineered features against spacecraft models, but adverse illumination and many pose hypotheses reduce robustness and increase computational complexity.
- Related work: Terrestrial pose determination is shifting toward deep learning as large image datasets and inexpensive computation become available.
- Paper contribution: The paper introduces a CNN-based spaceborne pose method using transfer learning, synthetic imagery, and discretized pose labels, with experiments varying dataset size, sensor noise, and pose-space resolution.It also compares the approach with state-of-the-art spaceborne pose determination methods.
2. METHODS
The method formulates spacecraft pose determination as a CNN classification problem over a discretized four-dimensional view-space and trains it using synthetically generated, labeled imagery. The pipeline samples camera positions, boresight rotations, and distances, then uses an AlexNet-based network with transfer learning to predict pose labels.
- Pose problem: The problem is to determine the target spacecraft’s relative position and attitude from the camera frame.The relative position is represented by t_BC, while attitude is represented by quaternion q(R_BC).
- Synthetic data: Synthetic imagery addresses the difficulty of obtaining the large labeled space-image datasets required for CNN training.The approach combines an automated synthetic-image generation and labeling pipeline with transfer learning from ImageNet.
- Pose-space discretization: The pipeline discretizes a four-dimensional view-space containing three attitude degrees of freedom and one camera-distance degree of freedom.Camera locations are distributed on spheres, boresight rotation is discretized into m − 1 intervals, and varying sphere radii represent distance.
- Pose-space discretization: Camera locations on each sphere are generated using a Thomson-problem formulation and iteratively refined toward a locally optimal distribution.The procedure minimizes an energy function associated with inverse-square repulsion, then successively subdivides the resulting mesh.
- Image labeling: Each rendered image receives the closest pose label by matching camera distance and minimizing quaternion angular difference.The labeled images are grouped into datasets for different discretization levels and used to train the CNN.
- CNN architecture: The CNN adopts AlexNet, uses transfer learning, and predicts a distribution over pose labels through a final softmax classifier.AlexNet was selected for its relatively low inference-operation count, while transfer learning reuses low-level features learned from terrestrial imagery.
3. EXPERIMENTS
The experiments evaluate CNN classification and pose estimation across dataset sizes, noise conditions, pose-space discretizations, and competing architectures. Results indicate that training-data scale and modeled noise improve robustness, while finer pose discretization improves pose accuracy.
- Type 1 experiments: Classification accuracy declines as sensor noise increases, while noise-augmented training improves robustness when the noise is known and modeled.Net5 performed comparatively well because noisy images were included during training.
- Type 1 experiments: Classification accuracy correlates with training-set size, with net1 outperforming nets 2–4 on Clean-6 and the three Uncentered datasets.The comparison covers five separately trained networks evaluated on multiple datasets.
- Type 1 experiments: 99.4% classification accuracy was achieved by net6 on Clean-18 without data augmentation.t-SNE visualization further indicates separation of images from different inter-satellite ranges in the learned feature space.
- Type 2 experiments: Net8 achieved higher pose estimation accuracy than net7 because Clean-3k used finer pose-space discretization, although net7 had higher classification accuracy.The comparison uses ER for relative attitude error and ET for relative position error.
- Type 2 experiments: Net8 was more accurate than EPnP-RANSAC but less accurate than SVD, while providing high-confidence solutions on 68% of Imitation-25 images versus 20% for SVD.The authors suggest CNN-based estimates as coarse initial guesses for feature-based SVD-like architectures.
4. CONCLUSIONS
The CNN pose-determination framework was tested across datasets, with results linking accuracy to training-set size and modeled sensor noise. The study identifies deployment potential while emphasizing validation needs for real imagery, broader spacecraft and orbit conditions, and onboard resource constraints.
- The CNN pose-determination framework was successfully established and exhaustively tested against different datasets.
- Larger training sets correlated with higher accuracy, motivating synthetic datasets with more variations in target location and orientation.
- Networks trained with small Gaussian white noise performed much better on high-noise test images than networks trained on noise-free images.
- The networks still require testing on actual space imagery and larger datasets for comprehensive comparison with conventional architectures.
- Applying the architecture to other spacecraft and orbit regimes, and validating synthetic illumination, texture, and reflectance assumptions, remain open requirements.Larger networks and more pose labels may improve accuracy but increase onboard servicer memory requirements.