PaperlayerView exact revision on arXiv

Source-linked AI summary

Hardware-Accelerated Instance Segmentation for Resource-Constrained Space Robotics with Criticality Analysis

Siddhant Shete, Hilmi Dogu Kücüker, Udo Frese, Frank Kirchner

arXiv:2609.02219v1cs.ROcs.ARcs.CVcs.LG

TL;DR

Autonomous lunar robotics requires real-time perception despite low light, limited compute, and radiation-induced faults, while in-flight retraining, manual intervention, and hardware repair are infeasible. This paper integrates AVIS calibration, hardware-aware DPU deployment, and criticality analysis; on a lunar micro-rover, it recovers 69.8% of quantization-induced accuracy loss, achieves 309 ms inference at 5.7 W, and reduces global criticality by 31.7%.

  • Problem

    Autonomous lunar missions need deterministic, fault-tolerant perception under low-light, compute, and radiation constraints, with no feasible in-flight retraining, manual intervention, or hardware repair.

    Source: §I · p. 1

  • Method

    The framework combines label-free AVIS calibration, hardware-aware architectural adaptation for statically compiled DPU execution, and software-level criticality analysis with targeted mitigation.

    Source: §I · p. 1, B. Hardware-Aware Model Adaptation for… · p. 3, C. Activation Variance Informative… · p. 4

  • Results

    69.8% of quantization-induced accuracy loss is recovered, with 309 ms inference at 5.7 W and a 31.7% reduction in global criticality on a lunar micro-rover.

    Source: §I · p. 2, A. Segmentation Performance · p. 5, §VI · p. 7

  • Takeaways & Limitations

    The integrated approach demonstrates a deployable instance segmentation system for lunar robotics under quantization, deterministic execution, and radiation-induced fault constraints.

    Source: §I · p. 2, §VI · p. 7

    Criticality analysis uses analytical fault modeling without physical irradiation validation, AVIS assumes a representative offline dataset, and 309 ms latency may require optimization for faster traversal or temporal fusion.

    Source: §VI · p. 7

Abstract

from arXiv · show

Autonomous lunar missions require real-time per- ception under three coupled constraints: extreme low-light conditions, limited onboard compute, and radiation-induced hardware faults that can silently corrupt inference. We present a deployment-oriented instance segmentation framework for resource-constrained lunar robotics that jointly addresses quan- tization calibration and system-level fault exposure under strict compute constraints. First, we introduce Activation Variance Informative Sampling (AVIS), a label-free calibration strategy that deterministically selects calibration samples based on activation variance statistics. Second, we deploy a YOLO-based segmentation model on a Deep Learning Processor Unit (DPU) with architectural modifications that reduce CPU fallback paths and enable statically compiled execution with bounded latency in low-lighting conditions. We further introduce a software-level criticality analysis to estimate fault exposure and guide mitigation under radiation-constrained operation. On a lunar micro-rover platform, AVIS with bias correction recovers 69.8% of quantization-induced accuracy loss while achieving 309 ms inference latency and 5.7 W power consumption. Targeted mitigation reduces global criticality by 31.7%. The results demonstrate an integrated approach and a blueprint for a reliable and safe AI perception framework under space deployment constraints.

I. INTRODUCTION

Lunar autonomy requires reliable onboard perception despite low light, severe compute limits, and radiation-induced faults that can silently corrupt inference. The framework combines deterministic calibration, DPU deployment, and software criticality analysis, achieving measured accuracy, latency, power, and risk improvements.

  • Several-second communication latencies, harsh extremes, and no in-situ repair make real-time onboard autonomy necessary for mission success.Source: §I · p. 1
  • Perception failures can cause mission loss, motivating deterministic execution, autonomous fault tolerance, and graceful degradation without retraining or manual intervention.Source: §I · p. 1
  • Three coupled constraints shape deployment: extreme low-light illumination, limited onboard computing, and radiation-induced hardware defects.Source: §I · p. 1
  • AVIS selects label-free calibration inputs using activation variance statistics to improve INT8 quantization stability.Source: §I · p. 1
  • The hardware-aware pipeline uses statically compiled DPU execution to reduce runtime variability and CPU fallback paths.Source: §I · p. 1
  • Software criticality analysis ranks functional blocks by memory footprint and execution time to guide radiation-informed mitigation.Source: §I · p. 1
  • 69.8% of quantization-induced accuracy loss is recovered, with 309 ms inference at 5.7 W and 31.7% lower radiation-induced criticality through targeted mitigation.Source: §I · p. 2

II. RELATED WORK

Prior work addresses planetary perception, quantization, or radiation tolerance largely in isolation and often omits deployment-level determinism and fault prioritization. This framework positions deterministic label-free calibration, DPU execution, and function-wise criticality analysis as a unified contribution.

Planetary Perception

  • Planetary perception studies commonly use offline datasets without considering deterministic execution, bounded latency, or radiation-informed hardware deployment.Source: §II · p. 2

Post-Training Quantization and Calibration

  • PTQ reduces memory footprint and latency but depends on calibration-data selection, with poor inputs causing activation-range misestimation and unpredictable accuracy degradation.Source: §II · p. 2
  • AVIS deterministically ranks calibration samples by aggregated layer-wise activation variance without labels, stochastic sampling, or retraining.Source: §II · p. 2

Radiation-Tolerant AI Acceleration

  • Hardware mitigations such as TMR, EDAC, and memory scrubbing improve accelerator resilience but do not address software-level fault propagation across inference pipelines.Source: §II · p. 2

Gap and Positioning

  • The authors state that no prior work simultaneously integrates deterministic label-free PTQ calibration, DPU segmentation execution, and function-wise radiation-informed criticality analysis.Source: §II · p. 2

III. SYSTEM DESIGN AND PROBLEM FORMULATION

The system targets autonomous lunar hazard detection on a micro-rover operating without ground-in-the-loop support and under strict power, latency, determinism, and fault-resilience requirements. Its formulation treats quantization fidelity and fixed execution bounds as deployment requirements.

III. SYSTEM DESIGN AND PROBLEM FORMULATION

  • The framework targets LuNiS, a micro-class rover for semi-autonomous navigation under extreme low-angle illumination and below-10 W onboard-computation power.Source: §III · p. 2
  • The platform exemplifies a perception system expected to be accurate, deterministic, energy-efficient, and resilient to radiation-induced faults.Source: §III · p. 2
  • The deployable pipeline performs fully onboard instance segmentation of rocks and craters, with model weights frozen after offline training.Source: §III · p. 2

A. Problem Formulation

  • The deployment objective is a high-accuracy DPU model designed for deterministic fixed execution graphs and bounded latency.Source: A. Problem Formulation · pp. 2–3
  • Quantization fidelity requires INT8 segmentation quality to remain close to the FP32 baseline with bounded and characterizable degradation.Source: A. Problem Formulation · p. 3
  • Deterministic real-time execution requires inference to complete within a fixed latency bound dictated by the rover’s motion profile.Source: A. Problem Formulation · p. 3

IV. METHODOLOGY

The methodology combines a YOLOv8m segmentation model with hardware adaptations for full DPU compilation and deterministic execution. It also uses frozen weights and replaces unsupported or dynamic operations to meet deployment constraints.

  • The framework comprises segmentation architecture, hardware-deterministic deployment, activation-variance calibration, and radiation-aware criticality analysis.Source: §IV · p. 3

Segmentation Architecture

Hardware-Aware Model Adaptation for DPU Deployment

C. Activation Variance Informative Sampling (AVIS)

AVIS deterministically selects label-free calibration inputs using activation variance statistics, supporting stable INT8 scale estimation and reduced calibration-data requirements.

C. Activation Variance Informative Sampling (AVIS)

  • AVIS evaluates activation variance across network layers to select informative calibration inputs without labels or retraining.Higher activation variance is interpreted as broader learned-representation activation and more informative quantization scale estimation.Source: C. Activation Variance Informative… · pp. 3–4
  • The offline pipeline applies AVIS calibration selection before INT8 quantization with bias correction and XMODEL compilation.The system architecture separates these offline steps from onboard inference execution.Source: Figure 1 · p. 4
  • Calibration images are selected by deterministic Top-K ranking from the frozen network’s activation statistics.The positivity constraint excludes near-uniform or under-exposed inputs, while fixed model parameters ensure reproducible selection.Source: C. Activation Variance Informative… · p. 4
  • AVIS reduces calibration data requirements by approximately 2× relative to random sampling while improving INT8 accuracy.The method operates independently of training or architectural modifications.Source: C. Activation Variance Informative… · p. 4

D. Quantized Deployment and CPU–DPU Partitioning

The quantized model is compiled as a single DPU graph, while deterministic CPU post-processing completes the instance-segmentation pipeline without uncontrolled fallback paths.

D. Quantized Deployment and CPU–DPU Partitioning

1) Scope and Assumptions:

The radiation analysis estimates software-level exposure of memory-resident components during onboard execution using simplified spatial and temporal assumptions.

1) Scope and Assumptions:

  • The analysis evaluates SEE susceptibility in RAM and associated runtime buffers during onboard execution, while neglecting flash effects.Mission-level particle flux and device cross-sections are not explicitly modeled.Source: 1) Scope and Assumptions · p. 4
  • Memory-level SEE events are approximated as spatially uniform, with the 1.2 s inference cycle supporting a time-invariant approximation within each execution window.These assumptions define the occurrence model over the memory layout and execution interval.Source: 2) Occurrence Model · p. 4
  • The occurrence expression models memory residency and execution time as independent uniform distributions over RM and RT, yielding a uniform distribution over their intersection.RM denotes the memory region and RT denotes the temporal execution region.Source: 2) Occurrence Model · p. 5
  • Operations are aggregated into CPU-static, CPU-dynamic, and DPU functional blocks using pmap, massif, and vaitrace profiling, respectively.Temporal proportions are extracted through callgrind.Source: 3) Functional Block Decomposition · p. 5
  • Figure 2 presents the risk-assessment pipeline following ECSS methodology.Source: Figure 2 · p. 5

4) Criticality Classification:

Criticality follows ECSS methodology by combining occurrence derived from measured resource exposure with severity assigned from functional fault impact and propagation potential.

4) Criticality Classification:

  • Criticality is computed as occurrence multiplied by severity, with occurrence based on measured memory footprint and execution time.Severity reflects functional impact, detectability, and propagation potential under fault conditions.Source: 4) Criticality Classification · p. 5
  • Dynamic heap allocations receive conservatively high severity because silent corruption and deterministic recovery mechanisms are concerns.After mitigation, severity is reassigned according to software-stack hierarchy and propagation impact.Source: 4) Criticality Classification · p. 5
  • The criticality formulation is a first-order risk approximation consistent with standard FMEA practice, not a physical failure-probability model.Source: 4) Criticality Classification · p. 5
  • The evaluation covers segmentation accuracy under quantization, deployment efficiency, and radiation-informed robustness.Source: §V · p. 5

A. Segmentation Performance

Segmentation performance is evaluated across platforms and quantization configurations on a held-out deployment-representative test set. AVIS with bias correction provides the strongest INT8 accuracy and improves low-light mask structure toward the FP32 reference.

  • FP32 segmentation accuracy varies negligibly between CPU and GPU, indicating quantization is the primary source of sensitivity.Source: A. Segmentation Performance · p. 5
  • 6.6% relative mAP reduction occurs with INT8 random calibration, falling from 0.802 to 0.749 through boundary and small-object degradation.The degradation is consistent with activation-range misestimation under uninformative calibration.Source: A. Segmentation Performance · p. 5
  • mAP reaches 0.768 with bias correction, a +2.5% improvement that partially compensates systematic layer-wise activation shifts.Source: A. Segmentation Performance · p. 5
  • 69.8% of quantization-induced accuracy loss is recovered by AVIS with bias correction, reaching mAP = 0.786 and IoU = 0.767.AVIS improves activation-range estimation through informative sample selection, while bias correction compensates residual layer-wise shifts.Source: A. Segmentation Performance · p. 5
  • AVIS with bias correction improves mask completeness and geometric fidelity under representative low-light conditions, producing outputs closer to FP32-level structure.Random calibration instead produces mask fragmentation and boundary distortion from activation clipping in fine-detail layers.Source: B. Qualitative Analysis · p. 5

C. Inference Efficiency

The hybrid CPU-DPU deployment balances latency, power, deterministic execution, and radiation-aware mitigation. Function-wise analysis identifies the DPU kernel as the dominant criticality contributor, while staged protections reduce global criticality.

Inference Efficiency

  • 309 ms at 5.7 W is achieved by the hybrid CPU-DPU configuration, corresponding to 1.76 J/frame and 60% lower energy than CPU-only.Source: C. Inference Efficiency · pp. 5–6
  • More than 16× temporal margin is provided for hazard detection under the 0.1 m/s, 5 s capture-interval operational profile.Reported latency includes full on-device DPU inference and CPU post-processing without offloading.Source: C. Inference Efficiency · p. 6

Criticality Analysis

  • The DPU kernel contributes most to estimated system-level criticality because of its memory footprint and modeled propagation sensitivity.The DPU is modeled as a single monolithic functional entity and assigned maximum static severity, S = 4.Source: D. Criticality Analysis · p. 6, Figure 4 · p. 6
  • 31.7% total global criticality reduction is obtained through static allocation, selective TMR, EDAC, and memory scrubbing, lowering criticality from 0.3389 to 0.2316.Static allocation contributes 6.7%, selective TMR 17.4%, and EDAC with memory scrubbing an additional 11.3%.Source: D. Criticality Analysis · p. 7
  • AVIS and criticality-driven mitigation address complementary robustness dimensions by targeting quantization perturbations and radiation-induced system faults.Source: D. Criticality Analysis · p. 7

VI. CONCLUSION

The paper presents a deployment-oriented lunar instance-segmentation framework combining calibration-aware quantization, deterministic DPU execution, and function-wise radiation fault analysis. It reports improved quantization fidelity and reduced criticality, while identifying irradiation validation, distributional-shift evaluation, and faster-traversal optimization as remaining needs.

  • The framework integrates calibration-aware quantization, deterministic hardware execution, and function-wise fault analysis for resource-constrained lunar platforms.Source: §VI · p. 7
  • 69.8% of quantization-induced accuracy loss is recovered, while deterministic execution achieves 309 ms at 5.7 W and global criticality falls by 31.7%.Source: §VI · p. 7
  • Analytical criticality modeling still requires empirical validation under proton or heavy-ion irradiation.Source: §VI · p. 7
  • AVIS performance under distributional shifts, such as unfamiliar terrain absent from calibration, remains unexplored.Source: §VI · p. 7
  • The 309 ms latency is sufficient for the current 0.1 m/s profile but requires further optimization for faster traversal or temporal fusion.Source: §VI · p. 7

Related papers

Loading 2609.02219v1…