Source-linked AI summary
In-Situ Reconstruction of the International Space Station Using 3D Gaussian Splatting and Astrobee
Hudson Kim, Ryan Soussan, Brian Coltin, Jordan Kam
TL;DR
The paper addresses the need for current, photorealistic ISS interior maps despite changing station contents and limited imagery. It combines global SfM with 3DGS on Astrobee NavCam data, producing a JEM reconstruction that outperforms compared methods in reported quality and rendering speed and can be built from roughly 500 images. The authors also identify limitations from grayscale, low-resolution imagery, scene changes, and reconstruction of only one ISS module.
Problem
Changing ISS interiors are represented by a static CAD model, while existing Astrobee mapping approaches have struggled to produce hyper-photorealistic representations from limited imagery.
Method
The pipeline recomputes camera poses with SfM, uses the resulting poses and point cloud as priors, and optimizes 3DGS with adaptive density control and learned appearance embeddings for flight-to-flight lighting differences.
Results
31.19 dB PSNR and 75.2 fps are achieved, outperforming existing radiance-field methods in reported quality and speed; a sufficient reconstruction takes 1.3 hours and approximately 500 images.
Takeaways & Limitations
The reconstruction provides a photorealistic, freely explorable JEM model for inspecting cargo stowage, panel configurations, and cable routing in situ.
Takeaways & Limitations
The reconstruction is limited by grayscale, low-resolution NavCam imagery, assumes a static scene despite changes between sessions, and covers only one ISS module.
Abstract
from arXiv · showhide
This article presents a novel 3D reconstruction and mapping of the interior of the International Space Station (ISS) using 3D Gaussian Splatting (3DGS). Using existing grayscale images from the Astrobee free-flying robot dataset, we construct a full 3D splat of the ISS' Kibō or Japanese Experiment Module (JEM). 3DGS has in recent years shown promise in providing novel view synthesis of scenes captured from many images or videos, this article applies this approach to human spaceflight systems. We compare our 3DGS architecture to existing methods such as Nerfacto and TensoRF and show that reconstruction improves the state-of-the-art in both scene quality and rendering speed. We show that with as little as 500 in-situ images, a high-fidelity map can be constructed using Astrobee's Navigation Camera (NavCam) during free-flight in the JEM. These reconstructions could enable free-flyers to rapidly create and update interior maps for intra-vehicular habitats like the ISS.
I. INTRODUCTION
The paper addresses the difficulty of maintaining current, photorealistic ISS interior maps by applying 3D Gaussian Splatting to Astrobee imagery from the JEM. It presents a pipeline that uses limited in-situ images to create rapidly updatable reconstructions.
- Motivation: The ISS interior changes with cargo, payloads, and crew equipment use, while its static CAD model cannot capture these updates.This limits interior mapping and complicates ground-based mission-control decision-making.
- Motivation: Existing Astrobee-based maps support localization and denser reconstruction but have struggled to produce hyper-photorealistic representations from limited imagery.
- Contribution: The pipeline uses as little as 500 Astrobee images to create a 1.2 million 3D Gaussian Splat model of the JEM for real-time novel-view rendering.
- Contribution: The work presents the first reported 3DGS reconstruction of the ISS and a scalable approach for combining thousands of images from one or more trajectories.
- Pipeline: Five Astrobee survey flights are filtered to 5,000 frames, processed by COLMAP and GLOMAP, and used as priors for 3DGS optimization with adaptive density control and learned appearance embeddings.
II. METHODOLOGY
The methodology filters Astrobee survey imagery, estimates camera poses and a point cloud with Structure from Motion, and optimizes a 3DGS model using those estimates as priors.
- Image preparation: The pipeline first filters raw Astrobee images into a smaller pool of usable images.
- Structure from Motion: Classical Structure from Motion then recovers camera poses and a point cloud representing the surrounding environment.
- 3DGS optimization: The recovered point cloud and camera poses initialize 3DGS optimization until rendered views match the training photographs.
- Data collection: The selected dataset contains five April 2021 flights traversing the JEM with Astrobee’s NavCam pointed forward, up, down, left, and right.
A. Camera Poses from Structure from Motion
Because Astrobee’s navigation poses lack reconstruction precision and the flights are disconnected by lighting and timing differences, the paper recomputes poses with global SfM and aligns them metrically.
- Pose limitations: Astrobee’s visual-localizer poses are suitable for navigation but lack the precision required for high-fidelity 3D reconstruction.
- Cross-flight matching: The method proposes matching image pairs across flights when localization poses indicate visual overlap, including under different lighting conditions or ISS changes.
- Global SfM: GLOMAP jointly solves rotations, positions, and points, avoiding incremental SfM’s fragile image-by-image growth while averaging errors across the image network.
- Global SfM: Global SfM is reported as 10-100 times faster than COLMAP for this comparison.
- Metric alignment: A similarity transform sets metric scale by aligning SfM camera centers with Astrobee localization poses, yielding a median disagreement of 6.7 cm.
B. 3D Gaussian Splatting Optimization
The reconstruction represents the ISS scene with colored, semi-transparent anisotropic Gaussians, initializes them from SfM points, and optimizes them by differentiable rendering against training images.
- Each Gaussian stores a center, opacity, view-dependent color, and covariance describing its ellipsoidal shape.The covariance is factored into rotation and three axis lengths to remain positive semi-definite.
- The model is warm-started by placing initial Gaussians at 1.5 million SfM points.
- Novel views are rendered by projecting visible Gaussians, sorting them by depth, and alpha-blending them front to back.Each pixel becomes a weighted mixture in which nearer, more opaque Gaussians occlude those behind them.
- The reconstruction is optimized by rendering training viewpoints, comparing them with ground-truth images, and minimizing a differentiable loss.Gradient descent updates the Gaussian parameters through this rendering-and-comparison loop.
- The loss uses λ = 0.2 to combine per-pixel error and SSIM, while adaptive density control adds detail and removes ineffective Gaussians.Densification splits Gaussians with persistently large image-space gradients, while pruning removes nearly transparent or oversized Gaussians.
- The model assigns each flight a learned appearance embedding to account for exposure differences between flights.
III. RESULTS
The evaluation measures reconstruction quality and efficiency across held-out views, image counts, training iterations, and competing methods. The method achieves strong quality and speed, with diminishing gains beyond 500 images and 60,000 iterations.
- 31.19 dB PSNR, 0.918 SSIM, and 0.208 LPIPS are achieved on 625 held-out views, while 1.21M Gaussians render at 75.2 fps.Evaluation uses 4,375 training and 625 evaluation views at full resolution on an RTX 3090 Ti.
- Beyond 500 images, reconstruction quality improves only marginally rather than scaling linearly.
- Beyond 60,000 training iterations, reconstruction performance begins to plateau.
- Compared with Splatfacto 3DGS, Nerfacto, Instant NGP, and TensoRF, the method achieves higher PSNR at 31.19 and render speed at 75.2 fps.
IV. CONCLUSION & DISCUSSION
The pipeline produces a rapidly updatable, photorealistic 3DGS reconstruction of the ISS JEM from existing Astrobee imagery, while its fidelity is constrained by grayscale, low-resolution inputs and static-scene assumptions.
- 31.19 dB PSNR and 75.2 fps demonstrate high-quality, real-time rendering from five disconnected JEM survey flights.The reconstruction took 1.3 hours and approximately 500 images on a consumer GPU.
- Approximately 500 images are sufficient for a reconstruction that makes routinely updated interior maps practical with existing perception hardware.
- The resulting model supports inspection of cargo stowage, panel configurations, and cable routing, as well as crew procedure development and spaceflight training.Its arbitrary-view renders provide a freely explorable model for these tasks.
- The model lacks precise RGB color and fine texture because Astrobee’s NavCam is grayscale and low-resolution.A color science camera is identified as a future route to higher-fidelity RGB reconstruction.
- 3DGS assumes a static scene, so changes between survey sessions appear as blurs in regions that moved.The current reconstruction covers only one ISS module, with whole-ISS validation left for future work.