PaperlayerView exact revision on arXiv

Source-linked AI summary

Illumination-Aware Active Perception for Spacecraft Inspection

Sagarika Rao Valluri, Benjamin Riviere

arXiv:2610.22335v1cs.RO

TL;DR

Geometric inspection checks can miss whether orbital viewpoints produce informative camera measurements under challenging illumination. The paper replaces those checks with an image-based learned inspection score and evaluates it in a photorealistic closed-loop simulator. The learned approach yields brighter observations than the geometric baselines in the reported comparison.

  • Problem

    Geometric checks cannot distinguish orbital viewpoints that produce photometrically informative measurements from those that do not, and may admit false positives or negatives.

    Source: §I · p. 2

  • Method

    A classifier VAE learns component classes and illumination distributions from rendered patches, using image-derived latent variance as the inspection score in a closed-loop GNC evaluation.

    Source: §I · p. 2, 1) Architecture · p. 5

  • Results

    Mean brightness was 0.57 for the learned approach, versus 0.04 for geometric planning and 0.02 for geometric planning with an illumination band; 81.6% of its observations exceeded 0.25, versus 6.0% and 3.0%.

    Source: §I · p. 2

  • Takeaways & Limitations

    The framework is presented as an image-based alternative to analytical inspection models under harsh lighting for complex orbital structures.

    Source: §VII · p. 6

    High uncertainty may reflect genuine image ambiguity or limitations of the model architecture or training; physical free-flyer validation remains future work.

    Source: §VII · pp. 6–7

Abstract

from arXiv · show

Harsh and rapidly varying orbital illumination introduces failure modes during spacecraft inspection. Existing planners rely on line-of-sight inspection metrics which perform occlusion checks against a known target and conflate geometric visibility with task-relevant information. These metrics overlook illumination effects including eclipse, self-shadowing, and specular reflection from spacecraft materials and tend to declare components as inspected despite the corresponding images lacking sufficient information. We propose an inspection-planning framework that replaces geometric visibility with a learned measure of camera information derived from a classifier variational autoencoder (VAE) that operates directly on the captured image. The encoder model is trained to classify spacecraft surface components from rendered patches, and its total latent variance is used to learn the varying illumination of said components, replacing binary visibility checks with a continuous, image-grounded quantity. We evaluate the approach by implementing a Guidance, Navigation, and Control architecture in a photorealistic physics simulator, and vary lighting and viewing geometries with multiple orbits. We show that, although all approaches successfully maximize their own objective, our approach results in better illumination: a mean brightness of $0.57$, with $81.6\%$ of observations above $0.25$, compared with $0.02$ ($3.0\%$) for geometric planning with an illumination band and $0.04$ ($6.0\%$) for purely geometric planning.

I. INTRODUCTION

Autonomous inspection supports space situational awareness and proximity operations, but geometric checks can confuse visibility with useful image information. The paper motivates an image-based inspection metric against prior trajectory, learning-based, and safety-constrained approaches.

I. INTRODUCTION

  • Autonomous resident-space-object inspection supports space situational awareness and constrained proximity operations, including debris removal, repair, and in-orbit assembly.The introduction cites recent inspection and close-range debris-imaging missions.Source: §I · p. 1
  • Camera-free geometric checks cannot distinguish informative orbital viewpoints from uninformative ones, leading to false positives and negatives during inspection.The inspection goal includes identifying components and assessing whether their condition is nominal or degraded.Source: §I · p. 2

II. RELATED WORK

  • Trajectory-based methods optimize inspection metrics under fuel and safety constraints, often using analytical checks such as line-of-sight or sensor-distance criteria.Prior methods include passive-relative-orbit planning, finite-thrust optimization, and projected trajectory patterns.Source: §II · p. 2
  • Learning-based inspection has used geometric field-of-view and occlusion criteria, with some approaches adding illumination gates for uniformly shaped targets.The cited work includes multi-agent and six-degree-of-freedom settings.Source: §II · p. 2
  • Safety-constrained inspection uses control barrier functions to enforce operational constraints, while illumination may still be handled through analytical shadow and band checks.The cited approaches extend safety filtering to multi-spacecraft proximity inspection and robust policies.Source: §II · p. 2
  • Analytical inspection metrics remain limited to line-of-sight checks, which can be inadequate for surfaces that are persistently shadowed or dimly lit in sun-synchronous orbits.Source: §II · p. 2

A. Inspection models

The paper frames inspection as an MDP in which a detection probability determines whether each point of interest is observed. Its state combines spacecraft motion with inspection status, while physical and inspection-state transitions are factorized.

A. Inspection models

  • Inspection planners use camera data for reconstruction or rendering, but the cited robotics approaches do not use it to represent target-inspection measurement requirements under orbital illumination.The section identifies a gap in combining a known target model with camera-informed inspection metrics in that setting.Source: A. Inspection models · p. 3

III. PROBLEM STATEMENT

  • The MDP state combines a 24-dimensional physical state for chaser and target motion with binary inspection indicators for the assigned points of interest.The physical state includes both bodies’ positions, attitudes, and velocities.Source: §III · p. 3
  • The action consists of chaser force and torque, and the model factorizes physical-state evolution from inspection-state evolution.The physical transition uses linearized two-body relative motion and rigid-body attitude dynamics.Source: §III · p. 3
  • Each POI has a detection probability ν_j(x), and the reward is the expected change in detection across the POIs.The inspection-state transition uses that probability when a POI has not yet been observed.Source: §III · p. 3

IV. GUIDANCE, NAVIGATION, AND CONTROL (GN&C) SYSTEM ARCHITECTURE

The closed-loop GN&C architecture separates discrete viewpoint selection from continuous trajectory generation and control. The inspection metric supplies detection probabilities to the decision layer while the remaining planning and control machinery executes its selected targets.

Navigation

  • The navigation model reads the chaser and target physical states directly from the simulator with zero measurement noise.The paper describes the resulting planner as hierarchical.Source: §IV · p. 3

Discrete Planning

  • The decision layer selects a passive relative orbit and a camera-pointing POI by maximizing expected newly inspected POIs over predicted rollouts.At each predicted position, the camera is pointed greedily at a POI, and the selected orbit maximizes expected reward.Source: §IV · pp. 3–4

Continuous Planning

  • Two convex programs convert the selected orbit and POI into references, including a minimum-fuel impulse sequence for the orbit transfer.The thrust plan is subject to thrust limits and a linearized spherical keep-out constraint.Source: §IV · p. 4

Attitude Planning and Control

  • The attitude planner tracks a SLERP to the desired pointing subject to linearized dynamics, terminal conditions, and torque limits.A robust nonlinear attitude controller tracks the resulting reference, while the thrust controller tracks the planned impulse sequence using model-predictive control.Source: §IV · p. 4

V. METHODS

The three inspection models share the same simulator-based GN&C stack and differ only in their detection probability function.

V. METHODS

  • All three inspection probabilities use the same Isaac Sim environment, planner, controllers, chaser, and target, differing only in ν_k(x).The chaser camera renders 256 × 256 RGB frames through a photorealistic ray-traced pipeline.Source: §V · p. 4

A. Geometric baseline ν1 [6]

The geometric baseline uses an inverse-square detection probability gated by binary visibility. A POI is eligible only in states meeting range, field-of-view, incidence, and occlusion conditions.

A. Geometric baseline ν1 [6]

  • The baseline uses an inverse-square detection probability gated by a binary visibility test.The supplied figure caption describes a VAE architecture, not this baseline, so it does not support a baseline claim.Source: A. Geometric baseline ν1 [6] · p. 4
  • A POI is visible only when it is in range, in the field of view, favorably incident, and unoccluded.Source: A. Geometric baseline ν1 [6] · p. 4

B. Illumination-aware analytic baseline ν2 [33]

The analytic baseline detects a POI only when it is visible, Sun-facing, and within an illumination band. It uses geometry and modeled illumination rather than rendered image patches.

B. Illumination-aware analytic baseline ν2 [33]

1) Architecture.:

The learned model uses a classifier variational autoencoder on local image patches. Its posterior variance measures encoding confidence and informs the detection probability.

1) Architecture.:

  • The model is a classifier variational autoencoder operating on local image patches of size 3 × 64 × 64.Source: 1) Architecture · p. 5
  • The latent code has dimension 32 and a standard normal prior, while the generative model reconstructs patches conditioned on the code and component class.Source: 1) Architecture · p. 5
  • At inference, posterior variance measures encoding confidence and determines the detection probability, so accumulating confident observations increases the reward.Source: 1) Architecture · p. 5

2) Training:

Training minimizes a weighted negative evidence lower bound augmented with classification loss. The rendered-patch dataset uses component labels derived from the Lunar Gateway USD hierarchy and a PRO-index split.

2) Training:

  • The networks are trained by minimizing a weighted negative evidence lower bound augmented with a classification term.Source: 2) Training · p. 5
  • The objective combines reconstruction loss, class-conditional KL regularization, and label-smoothed cross-entropy for the classifier.The reconstruction term is the negative Gaussian log-likelihood up to a constant.Source: 2) Training · p. 5
  • Patch classes follow the Lunar Gateway USD hierarchy, labels use a 3 × 3 raycast majority vote, and the split is 80/20 by PRO index.The passage characterizes this as a stricter generalization test than a patch-level split.Source: 2) Training · p. 5

VI. EXPERIMENTS

The experiments use both quantitative and qualitative analyses to compare three inspection-probability functions.

VI. EXPERIMENTS

  • The study tests three different inspection-probability functions.The evaluation includes quantitative and qualitative analyses.Source: §VI · p. 5
  • Quantitative analysis is one part of the comparison.Source: §VI · p. 5
  • Qualitative analysis is also used to compare the functions.Source: §VI · p. 5

A. Experimental Setup

The evaluation uses a photorealistic simulator and compares inspection probabilities across quantitative and qualitative measures. The learned method performs strongly across reward objectives and orbital regimes, while geometric criteria can be adequate in favorable, simple regions.

A. Experimental Setup

  • The simulator uses NASA Lunar Gateway models for the target, an adapted CloudSat satellite model for the chaser, and Earth, Moon, and Sun modules.Source: A. Experimental Setup · p. 5

B. Quantitative

  • ν3 reaches 190 POIs under J3, while ν1 and ν2 reach 42 and 100, respectively, over two orbits.Under J1, all three policies reach 200 POIs; under J2, ν3 reaches 193 versus 164 for ν1 and 197 for ν2 on its own reward.Source: B. Quantitative · p. 6
  • On 12 well-lit POIs in geometrically simple regions, ν1’s mean brightness is 0.66, exceeding ν3’s 0.50 on the same POIs.This result applies under favorable Sun geometry.Source: B. Quantitative · p. 6
  • Across MEO and GEO, ν3 retains 99.8% of ν1’s J1 performance (199.5 vs. 200) and 98.3% of ν2’s J2 performance (196.5 vs. 200); under ν3’s reward, ν1 achieves 61.0% of J3 (94.5 vs. 155) and ν2 achieves 14–15% (22–23).Source: B. Quantitative · p. 6
  • Fig. 4 compares ν1, ν2, and ν3 across camera views at 5, 100, and 800 seconds.The rows correspond to the three inspection probabilities and the columns to the viewing times.Source: Figure 4 · p. 6, B. Quantitative · p. 6

C. Qualitative analysis

The qualitative comparison shows different inspection decisions under darkness, illumination-band constraints, and learned image-based scoring. The conclusion presents the classifier-VAE framework and notes uncertainty attribution as a limitation.

C. Qualitative analysis

  • At 5 seconds, ν1 marks POIs observed on a fully dark module; ν2 rejects some POIs under its illumination band and misses self-shadows, while ν3 observes well-lit and some dim POIs.Source: C. Qualitative analysis · p. 6

VII. CONCLUSION AND FUTURE WORK

  • The framework replaces geometric visibility with a learned, image-based detection score from a classifier VAE in a complete GNC system.It evaluates information directly from captured images under harsh lighting and on complex orbital structures.Source: §VII · p. 6
  • A limitation is that high classifier uncertainty may reflect image ambiguity or model architecture and training limitations.Source: §VII · pp. 6–7
  • Fig. 5’s graph 2 plots POIs at first observation against direct camera pixel brightness and combines results for the three reward-maximizing functions.Source: Figure 5 · p. 7

Related papers

Loading 2610.22335v1…