Evaluating Spiking MoveNet Deployability for Near-Future Onboard Spiking Satellite Pose Estimation

I. INTRODUCTION

• Spacecraft pose-estimation datasets and solutions remain scarce, especially for event-based approaches.SEENIC has limited scenario variety and a synthetic-to-real domain gap; the authors choose realistic RAPTOR because their focus is operational cost and no satisfactory event-based dataset is available.Source: §I · pp. 1–2
• Neuromorphic devices remain in the research phase, with architecture support, deployment workflows, hardware limits, and asynchronous operation complicating use.Asynchronous operation is specifically noted as a potential complication for real-time, critical applications such as satellite pose estimation.Source: §I · p. 2
• The work targets architectures aligned with neuromorphic processing paradigms to pursue computational efficiency and lower power consumption.The introduction frames this design alignment as necessary for effective solutions.Source: §I · p. 2
• The paper presents a method to build and train an SNN counterpart to a non-spiking architecture for neuromorphic deployment.It also introduces the first SNN solution for RAPTOR keypoint prediction with heatmaps and evaluates its computational and activity costs against formal approaches.Source: §I · p. 2

A. CNNs for Pose Estimation

• CNN pose estimation includes direct end-to-end prediction, indirect methods with pose refinement, and hybrid approaches integrating PnP computation into learning.Indirect methods generally use Perspective-n-Point refinement.Source: A. CNNs for Pose Estimation · p. 2
• Radiation-hardened platforms impose stringent computational-resource and deployment-chain constraints, prioritizing computational efficiency.These constraints shape the pose-estimation community's deployment priorities.Source: A. CNNs for Pose Estimation · p. 2
• Existing footprint-reduction techniques include pruning, quantisation, low-rank factorisation, and knowledge distillation.The paper introduces SNNs as an alternative approach to reducing neural-network computational footprint.Source: A. CNNs for Pose Estimation · p. 2

B. Spiking Neural Networks

• SNN neurons communicate through discrete binary spikes over time, accumulating membrane charge and firing when a threshold is reached.Integrate-and-Fire neurons lack leakage, while Leaky Integrate-and-Fire and Parametric Leaky Integrate-and-Fire neurons add leakage, which can be manually set or learned.Source: B. Spiking Neural Networks · p. 2

C. Event-based Processing

• Rate coding can require many spikes across hundreds of timesteps, whereas time coding uses spike latency but can prolong processing and complicate implementation.Time coding reduces the number of timesteps but its temporal sparsity can increase processing time.Source: C. Event-based Processing · pp. 2–3
• Direct encoding keeps dense valued input at the first spiking layer, which learns to convert it into spikes for solutions with few timesteps.This approach is described as enabling energy-efficient processing.Source: C. Event-based Processing · p. 3
• Thresholding is non-differentiable, and timesteps can cause latency and vanishing-gradient problems, especially in large networks or networks with many timesteps.Surrogate Gradient Learning and Truncated Backpropagation Through Time help address training issues, particularly with direct encoding.Source: C. Event-based Processing · p. 3
• Sparse spike activity can reduce multiplication operations, but membrane-potential updates raise memory-access costs and energy gains depend on timestep count and activity.Dedicated neuromorphic hardware has been reported to achieve approximately six times greater energy efficiency than conventional networks.Source: C. Event-based Processing · p. 3

D. Neuromorphic hardware

• Neuromorphic hardware is defined here as sparse event-based computation that computes a neuron's output only when an input stimulus occurs.The paper's reasoning assumes a maximum of one spike per timestep per neuron.Source: D. Neuromorphic hardware · p. 3
• Sparse spike feature maps let layers sum weights associated with incoming spikes rather than perform weighted sums over dense inputs.The paper says layers must avoid dense computation or manipulation to support neuromorphic deployment; sparse inputs can also be processed as address event representations.Source: D. Neuromorphic hardware · p. 3
• The closest cited neuromorphic architecture, SPLEAT, supports upsampling routing only experimentally, limiting this work to an exploratory study.The authors suggest designing around such constraints could reduce deployment complexity in future developments.Source: D. Neuromorphic hardware · p. 3

III. EXPERIMENTS

• The comparison targets embedded deployment of MoveNet-Lightning and its SNN counterpart, with operational cost as the main objective and performance used for comparison.The SNN was created with SpikingJelly; the paper also aims to clarify factors influencing the computational-cost ratio between SNN and FNN approaches.Source: §III · p. 3
• The benchmark uses RAPTOR’s D1 subset and is limited to the approach phase, from 100 to 20 m, and one target-satellite 3D model.RAPTOR contains 120,000 synthetic samples; robustness to its domain gap is outside this study’s scope.Source: §III · p. 3

A. MoveNet Architecture

• Figure 2 depicts the complete architecture and the CNN and SNN implementations of each block.Source: Figure 2 · p. 4

III. EXPERIMENTS

• The evaluation compares operational cost using metrics in, torch summary outputs, and Pose Score from the Kelvin Satellite Pose Estimation Challenge 2021.Source: §III · p. 4

A. MoveNet Architecture

• MoveNet uses a MobileNetV2 feature extractor with an FPN and CenterNet-style prediction heads, adapted to predict 26 Sentinel-3 keypoints.The Lightning input resolution is 192 × 192 pixels; pose performance in Table II uses SQPnP refinement.Source: A. MoveNet Architecture · p. 4

B. S-MoveNet-Lightning Architecture

• The spiking design converts the computationally dominant backbone while keeping the heatmap head formal, targeting deployment across a neuromorphic accelerator and CPU.The formal head has negligible computational cost in this configuration and supports regression performance.Source: B. S-MoveNet-Lightning Architecture · p. 5
• ReLU activations are replaced with PLIF neurons, and added spike activations and batch-normalization layers preserve spiking behavior through the network’s blocks.Nearest-neighbor upsampling routes spikes, while addition merges residual and main-path spikes; depthwise convolution is supported by SPLEAT but may not be supported on other platforms.Source: B. S-MoveNet-Lightning Architecture · p. 5
• The spiking backbone processes each static input over five timesteps, and the formal head receives the average backbone output across those timesteps.Unrolling timesteps can improve performance at the cost of increased latency and computational demand.Source: B. S-MoveNet-Lightning Architecture · p. 5
• An activity-loss term regulates network activity; the target was arbitrarily set to 30%, and Table I lists hyperparameters for the MoveNet versions.The loss weights squared error between processing-spike activity and the target by 10^-9.Source: B. S-MoveNet-Lightning Architecture · p. 5, Table I · p. 5

D. Quantization

• Quantization converts FP32 weights and activations to lower precision to reduce model size and computational cost, generally at a potential accuracy cost.Source: C. Quantization · p. 5
• Post-training quantization uses a calibration dataset to determine scale and offset for an 8-bit encoding, but may incur significant accuracy loss.Source: C. Quantization · p. 5
• The reported naive conversion uses four integer bits and four fractional bits, quantizing weights, biases, membrane potentials, thresholds, decay, and activations.The passage refers to Table II for results but does not provide its values.Source: C. Quantization · p. 5

D. Comparison Tool

• Torch summary reports parameters, MACs, and memory footprint, while the operational-cost comparison uses MACC and ACC counts.The approach combines torch summary with the cost evaluation in [18].Source: D. Comparison Tool · p. 6
• The convolution-cost equation is adapted for depthwise convolutions, which are used in MoveNet-Lightning but omitted from the referenced evaluation method.The accompanying depthwise-convolution expression defines ACC for the FNN case.Source: D. Comparison Tool · p. 6
• The equation’s symbols include incoming and output spike counts, output-channel and feature-map dimensions, kernel dimensions, stride, and timesteps.Source: D. Comparison Tool · p. 6
• Network activity sums processed spikes across layers and divides by total network dimensions, accounting for upscaling and addition paths that can merge spikes.Merged feature-map values can range from 0 to 2 in this network, so counting generated spikes would obscure additional processing cost.Source: D. Comparison Tool · p. 6

• Both networks show a significant performance drop when quantised to 8-bit integer, highlighting the need for more advanced techniques such as QAT.Source: §IV · p. 6
• MoveNet-Lightning shows significantly lower pose-estimation performance than SPNv2-B0, while reducing MACs by 3.461G for the FNN and 3.59G for the SNN.Source: §IV · p. 6, Table II · p. 6
• 2.2 times fewer MACs are required by S-MoveNet-Lightning than FP32 MoveNet-Lightning, including the five timesteps.Source: §IV · p. 6
• Figure 1 shows the SNN focusing on the satellite despite overlap with Earth and ignoring dropout-augmentation squares.Source: §IV · p. 6
• The results are intended for relative comparison: MoveNet-Lightning does not meet the mission's required performance, and event-driven input sequences still need evaluation.Source: §IV · p. 6
• Fig. 3 details MAC operations by layer, with S-MoveNet-Lightning represented at mean, minimum, and maximum activity over the D1 testing subset.Source: §IV · p. 6, Figure 3 · p. 7
• 995.03 million mult-add operations are counted per inference densely, while mean activity of 26.68% yields an estimate of 108 million MAC operations on dedicated hardware.Source: §IV · p. 7

• MoveNet's original other-head post-processing was not integrated or validated; the benchmark used raw heatmaps at a single measurement point.Source: §V · p. 7
• Both SNN and CNN MoveNet versions overfit without sufficient data augmentation; the proposed augmentation regularises training, while the SNN takes longer to train.Source: §V · p. 7
• With video samples, unrolling five timesteps over consecutive images could exploit temporal correlation, make the network equivalent to one timestep, and reduce MAC operations.Source: §V · p. 7
• 16.46%–30.90% activity corresponds to 78M–125M MACs, while the 35.59% impulse-response case reaches 141M MACs versus 239M for the formal version.Source: §V · p. 7
• The SNN has 22M MACs and 86M ACCs; membrane-potential updates incur significant memory costs, so energy estimation cannot be extrapolated solely from MAC operations.Source: §V · p. 8

VI. CONCLUSION

• Approximately 2.2 times fewer MACs are required by S-MoveNet-Lightning than its formal counterpart, reducing computation from 239M to 108M MACs.The comparison is reported for the RAPTOR dataset.Source: §VI · p. 8
• Network activity ranges from 16.46% to 30.90%, and remains bounded at 35.59% under maximum input stimulation, limiting computation to 141M MACs.Even this worst-case estimate remains below the formal version.Source: §VI · p. 8
• Post-training quantisation caused significant performance degradation, motivating future exploration of Quantisation-Aware Training.Source: §VI · p. 8
• Evaluation on event-based datasets is identified as future work for better insight into SNN temporal processing.Source: §VI · p. 8
• Actual neuromorphic-hardware deployment is needed to measure energy consumption beyond the study's MAC-based estimates.Source: §VI · p. 8