Hardware-Accelerated Instance Segmentation for Resource-Constrained Space Robotics with Criticality Analysis

Siddhant Shete; Hilmi Dogu Kücüker; Udo Frese; Frank Kirchner

Source-linked AI summary

Hardware-Accelerated Instance Segmentation for Resource-Constrained Space Robotics with Criticality Analysis

Siddhant Shete, Hilmi Dogu Kücüker, Udo Frese, Frank Kirchner

arXiv:2609.02219v1Sep 2, 2026cs.ROcs.ARcs.CVcs.LG

TL;DR

Autonomous lunar robotics requires real-time perception despite low light, limited compute, and radiation-induced faults, while in-flight retraining, manual intervention, and hardware repair are infeasible. This paper integrates AVIS calibration, hardware-aware DPU deployment, and criticality analysis; on a lunar micro-rover, it recovers 69.8% of quantization-induced accuracy loss, achieves 309 ms inference at 5.7 W, and reduces global criticality by 31.7%.

Problem
Autonomous lunar missions need deterministic, fault-tolerant perception under low-light, compute, and radiation constraints, with no feasible in-flight retraining, manual intervention, or hardware repair.
Method
The framework combines label-free AVIS calibration, hardware-aware architectural adaptation for statically compiled DPU execution, and software-level criticality analysis with targeted mitigation.
Results
69.8% of quantization-induced accuracy loss is recovered, with 309 ms inference at 5.7 W and a 31.7% reduction in global criticality on a lunar micro-rover.
Takeaways & Limitations
The integrated approach demonstrates a deployable instance segmentation system for lunar robotics under quantization, deterministic execution, and radiation-induced fault constraints.
Takeaways & Limitations
Criticality analysis uses analytical fault modeling without physical irradiation validation, AVIS assumes a representative offline dataset, and 309 ms latency may require optimization for faster traversal or temporal fusion.

Abstract

from arXiv · show

Autonomous lunar missions require real-time per- ception under three coupled constraints: extreme low-light conditions, limited onboard compute, and radiation-induced hardware faults that can silently corrupt inference. We present a deployment-oriented instance segmentation framework for resource-constrained lunar robotics that jointly addresses quan- tization calibration and system-level fault exposure under strict compute constraints. First, we introduce Activation Variance Informative Sampling (AVIS), a label-free calibration strategy that deterministically selects calibration samples based on activation variance statistics. Second, we deploy a YOLO-based segmentation model on a Deep Learning Processor Unit (DPU) with architectural modifications that reduce CPU fallback paths and enable statically compiled execution with bounded latency in low-lighting conditions. We further introduce a software-level criticality analysis to estimate fault exposure and guide mitigation under radiation-constrained operation. On a lunar micro-rover platform, AVIS with bias correction recovers 69.8% of quantization-induced accuracy loss while achieving 309 ms inference latency and 5.7 W power consumption. Targeted mitigation reduces global criticality by 31.7%. The results demonstrate an integrated approach and a blueprint for a reliable and safe AI perception framework under space deployment constraints.

I. INTRODUCTION

Lunar autonomy requires reliable onboard perception despite low light, severe compute limits, and radiation-induced faults that can silently corrupt inference. The framework combines deterministic calibration, DPU deployment, and software criticality analysis, achieving measured accuracy, latency, power, and risk improvements.

Several-second communication latencies, harsh extremes, and no in-situ repair make real-time onboard autonomy necessary for mission success.
Perception failures can cause mission loss, motivating deterministic execution, autonomous fault tolerance, and graceful degradation without retraining or manual intervention.
Three coupled constraints shape deployment: extreme low-light illumination, limited onboard computing, and radiation-induced hardware defects.
AVIS selects label-free calibration inputs using activation variance statistics to improve INT8 quantization stability.
The hardware-aware pipeline uses statically compiled DPU execution to reduce runtime variability and CPU fallback paths.
Software criticality analysis ranks functional blocks by memory footprint and execution time to guide radiation-informed mitigation.
69.8% of quantization-induced accuracy loss is recovered, with 309 ms inference at 5.7 W and 31.7% lower radiation-induced criticality through targeted mitigation.

II. RELATED WORK

Prior work addresses planetary perception, quantization, or radiation tolerance largely in isolation and often omits deployment-level determinism and fault prioritization. This framework positions deterministic label-free calibration, DPU execution, and function-wise criticality analysis as a unified contribution.

Planetary Perception: Planetary perception studies commonly use offline datasets without considering deterministic execution, bounded latency, or radiation-informed hardware deployment.
Post-Training Quantization and Calibration: PTQ reduces memory footprint and latency but depends on calibration-data selection, with poor inputs causing activation-range misestimation and unpredictable accuracy degradation.
Post-Training Quantization and Calibration: AVIS deterministically ranks calibration samples by aggregated layer-wise activation variance without labels, stochastic sampling, or retraining.
Radiation-Tolerant AI Acceleration: Hardware mitigations such as TMR, EDAC, and memory scrubbing improve accelerator resilience but do not address software-level fault propagation across inference pipelines.
Gap and Positioning: The authors state that no prior work simultaneously integrates deterministic label-free PTQ calibration, DPU segmentation execution, and function-wise radiation-informed criticality analysis.

III. SYSTEM DESIGN AND PROBLEM FORMULATION

The system targets autonomous lunar hazard detection on a micro-rover operating without ground-in-the-loop support and under strict power, latency, determinism, and fault-resilience requirements. Its formulation treats quantization fidelity and fixed execution bounds as deployment requirements.

III. SYSTEM DESIGN AND PROBLEM FORMULATION: The framework targets LuNiS, a micro-class rover for semi-autonomous navigation under extreme low-angle illumination and below-10 W onboard-computation power.
III. SYSTEM DESIGN AND PROBLEM FORMULATION: The platform exemplifies a perception system expected to be accurate, deterministic, energy-efficient, and resilient to radiation-induced faults.
III. SYSTEM DESIGN AND PROBLEM FORMULATION: The deployable pipeline performs fully onboard instance segmentation of rocks and craters, with model weights frozen after offline training.
A. Problem Formulation: The deployment objective is a high-accuracy DPU model designed for deterministic fixed execution graphs and bounded latency.
A. Problem Formulation: Quantization fidelity requires INT8 segmentation quality to remain close to the FP32 baseline with bounded and characterizable degradation.
A. Problem Formulation: Deterministic real-time execution requires inference to complete within a fixed latency bound dictated by the rover’s motion profile.

IV. METHODOLOGY

The methodology combines a YOLOv8m segmentation model with hardware adaptations for full DPU compilation and deterministic execution. It also uses frozen weights and replaces unsupported or dynamic operations to meet deployment constraints.

The framework comprises segmentation architecture, hardware-deterministic deployment, activation-variance calibration, and radiation-aware criticality analysis.
Segmentation Architecture: YOLOv8m instance segmentation detects and delineates rocks and craters, using pixel-level boundaries needed for trajectory planning.
Segmentation Architecture: All model weights are frozen after training to ensure deterministic and reproducible inference throughout the mission lifetime.
Hardware-Aware Model Adaptation for DPU Deployment: Direct YOLOv8m deployment is infeasible because unsupported components cause compilation failures or CPU fallbacks that violate latency bounds.
Hardware-Aware Model Adaptation for DPU Deployment: The model replaces SiLU and Swish with natively supported HardSwish, avoiding unsupported or poorly quantized sigmoid-based operations.
Hardware-Aware Model Adaptation for DPU Deployment: Unsupported attention multiplication is redesigned as a DPU-compatible additive projection that preserves cross-channel feature reweighting.
Hardware-Aware Model Adaptation for DPU Deployment: Dynamic shape operations are replaced with fixed dimensions, enabling full-graph XMODEL compilation and bounded execution latency.

C. Activation Variance Informative Sampling (AVIS)

AVIS deterministically selects label-free calibration inputs using activation variance statistics, supporting stable INT8 scale estimation and reduced calibration-data requirements.

C. Activation Variance Informative Sampling (AVIS): AVIS evaluates activation variance across network layers to select informative calibration inputs without labels or retraining.Higher activation variance is interpreted as broader learned-representation activation and more informative quantization scale estimation.
C. Activation Variance Informative Sampling (AVIS): The offline pipeline applies AVIS calibration selection before INT8 quantization with bias correction and XMODEL compilation.The system architecture separates these offline steps from onboard inference execution.
C. Activation Variance Informative Sampling (AVIS): Calibration images are selected by deterministic Top-K ranking from the frozen network’s activation statistics.The positivity constraint excludes near-uniform or under-exposed inputs, while fixed model parameters ensure reproducible selection.
C. Activation Variance Informative Sampling (AVIS): AVIS reduces calibration data requirements by approximately 2× relative to random sampling while improving INT8 accuracy.The method operates independently of training or architectural modifications.

D. Quantized Deployment and CPU–DPU Partitioning

The quantized model is compiled as a single DPU graph, while deterministic CPU post-processing completes the instance-segmentation pipeline without uncontrolled fallback paths.

D. Quantized Deployment and CPU–DPU Partitioning: Following AVIS calibration, INT8 post-training quantization with bias correction produces an XMODEL for DPU deployment.Bias correction compensates for systematic layer-wise activation shifts that can degrade mask boundary precision.
D. Quantized Deployment and CPU–DPU Partitioning: The DPU executes the backbone, neck, and segmentation head as one INT8 XMODEL graph with bounded latency.This concentrates the compute-intensive neural-network stages in the accelerator.
D. Quantized Deployment and CPU–DPU Partitioning: The CPU performs confidence decoding, NMS, and mask reconstruction using static memory without dynamic control flow or heap allocation.Eliminating CPU fallbacks and uncontrolled execution paths supports deterministic, bounded-latency system execution.
D. Quantized Deployment and CPU–DPU Partitioning: The partitioning enables structured isolation of compute-intensive kernels for downstream radiation-aware analysis.The cited deployment design connects execution partitioning with the later software-level criticality assessment.

1) Scope and Assumptions:

The radiation analysis estimates software-level exposure of memory-resident components during onboard execution using simplified spatial and temporal assumptions.

1) Scope and Assumptions:: The analysis evaluates SEE susceptibility in RAM and associated runtime buffers during onboard execution, while neglecting flash effects.Mission-level particle flux and device cross-sections are not explicitly modeled.
1) Scope and Assumptions:: Memory-level SEE events are approximated as spatially uniform, with the 1.2 s inference cycle supporting a time-invariant approximation within each execution window.These assumptions define the occurrence model over the memory layout and execution interval.
1) Scope and Assumptions:: The occurrence expression models memory residency and execution time as independent uniform distributions over RM and RT, yielding a uniform distribution over their intersection.RM denotes the memory region and RT denotes the temporal execution region.
1) Scope and Assumptions:: Operations are aggregated into CPU-static, CPU-dynamic, and DPU functional blocks using pmap, massif, and vaitrace profiling, respectively.Temporal proportions are extracted through callgrind.
1) Scope and Assumptions:: Figure 2 presents the risk-assessment pipeline following ECSS methodology.

4) Criticality Classification:

Criticality follows ECSS methodology by combining occurrence derived from measured resource exposure with severity assigned from functional fault impact and propagation potential.

4) Criticality Classification:: Criticality is computed as occurrence multiplied by severity, with occurrence based on measured memory footprint and execution time.Severity reflects functional impact, detectability, and propagation potential under fault conditions.
4) Criticality Classification:: Dynamic heap allocations receive conservatively high severity because silent corruption and deterministic recovery mechanisms are concerns.After mitigation, severity is reassigned according to software-stack hierarchy and propagation impact.
4) Criticality Classification:: The criticality formulation is a first-order risk approximation consistent with standard FMEA practice, not a physical failure-probability model.
4) Criticality Classification:: The evaluation covers segmentation accuracy under quantization, deployment efficiency, and radiation-informed robustness.

A. Segmentation Performance

Segmentation performance is evaluated across platforms and quantization configurations on a held-out deployment-representative test set. AVIS with bias correction provides the strongest INT8 accuracy and improves low-light mask structure toward the FP32 reference.

FP32 segmentation accuracy varies negligibly between CPU and GPU, indicating quantization is the primary source of sensitivity.
6.6% relative mAP reduction occurs with INT8 random calibration, falling from 0.802 to 0.749 through boundary and small-object degradation.The degradation is consistent with activation-range misestimation under uninformative calibration.
mAP reaches 0.768 with bias correction, a +2.5% improvement that partially compensates systematic layer-wise activation shifts.
69.8% of quantization-induced accuracy loss is recovered by AVIS with bias correction, reaching mAP = 0.786 and IoU = 0.767.AVIS improves activation-range estimation through informative sample selection, while bias correction compensates residual layer-wise shifts.
AVIS with bias correction improves mask completeness and geometric fidelity under representative low-light conditions, producing outputs closer to FP32-level structure.Random calibration instead produces mask fragmentation and boundary distortion from activation clipping in fine-detail layers.

C. Inference Efficiency

The hybrid CPU-DPU deployment balances latency, power, deterministic execution, and radiation-aware mitigation. Function-wise analysis identifies the DPU kernel as the dominant criticality contributor, while staged protections reduce global criticality.

Inference Efficiency: 309 ms at 5.7 W is achieved by the hybrid CPU-DPU configuration, corresponding to 1.76 J/frame and 60% lower energy than CPU-only.
Inference Efficiency: More than 16× temporal margin is provided for hazard detection under the 0.1 m/s, 5 s capture-interval operational profile.Reported latency includes full on-device DPU inference and CPU post-processing without offloading.
Criticality Analysis: The DPU kernel contributes most to estimated system-level criticality because of its memory footprint and modeled propagation sensitivity.The DPU is modeled as a single monolithic functional entity and assigned maximum static severity, S = 4.
Criticality Analysis: 31.7% total global criticality reduction is obtained through static allocation, selective TMR, EDAC, and memory scrubbing, lowering criticality from 0.3389 to 0.2316.Static allocation contributes 6.7%, selective TMR 17.4%, and EDAC with memory scrubbing an additional 11.3%.
Criticality Analysis: AVIS and criticality-driven mitigation address complementary robustness dimensions by targeting quantization perturbations and radiation-induced system faults.

VI. CONCLUSION

The paper presents a deployment-oriented lunar instance-segmentation framework combining calibration-aware quantization, deterministic DPU execution, and function-wise radiation fault analysis. It reports improved quantization fidelity and reduced criticality, while identifying irradiation validation, distributional-shift evaluation, and faster-traversal optimization as remaining needs.

The framework integrates calibration-aware quantization, deterministic hardware execution, and function-wise fault analysis for resource-constrained lunar platforms.
69.8% of quantization-induced accuracy loss is recovered, while deterministic execution achieves 309 ms at 5.7 W and global criticality falls by 31.7%.
Analytical criticality modeling still requires empirical validation under proton or heavy-ion irradiation.
AVIS performance under distributional shifts, such as unfamiliar terrain absent from calibration, remains unexplored.
The 309 ms latency is sufficient for the current 0.1 m/s profile but requires further optimization for faster traversal or temporal fusion.