Source-linked AI summary
ORBITALIF: An Efficient Spiking Federated Learning Framework for Onboard Cloud Removal
Bohan Zhang, Chenyu Xu, Yijie Mao, Yuanming Shi
TL;DR
Cloud coverage obscures Earth-observation imagery, while ground processing is constrained by bandwidth, contact windows, and latency, and onboard inference must meet tight energy budgets. OrbitALIF combines decentralized satellite federated learning with a compact SNN backbone for onboard training and inference. It delivers competitive cloud-removal quality with substantially lower inference energy than an equivalent ANN.
Problem
Cloud coverage obscures Earth-observation imagery, while ground processing faces constrained satellite-to-ground bandwidth and high latency, and onboard models can be too computationally and energy intensive.
Method
OrbitALIF combines decentralized satellite federated learning over inter-satellite links with a 2.30 M-parameter SNN backbone using AGFM and SHAM for onboard cloud removal.
Results
72.3× lower inference energy than an equivalent ANN is achieved while maintaining comparable cloud-removal performance.
Takeaways & Limitations
The results indicate that spiking computation and lightweight design are promising for resource-constrained onboard cloud removal.
Abstract
from arXiv · showhide
Low-earth-orbit (LEO) satellites enable high-resolution, large-scale Earth observation for applications such as disaster monitoring and environmental surveillance. However, cloud coverage often obscures the Earth's surface, and conventional cloud-removal pipelines that download cloudy images to ground stations for processing suffer from limited contact windows, constrained satellite-to-ground bandwidth, and high latency. In this work, we propose a novel satellite federated learning framework for cloud removal across LEO constellations, named orbital attention leaky integrate-and-fire (OrbitALIF). OrbitALIF performs both onboard training and inference using a compact 2.30,M-parameter spiking neural network (SNN) backbone with an adaptive gated fusion module (AGFM) and a spectral-spatial hybrid attention module (SHAM), combined with a decentralized federated learning strategy that shares model weights via inter-satellite links. Our experiments show that OrbitALIF achieves competitive cloud removal quality while consuming only 0.287,mJ per inference on neuromorphic hardware, a 72.3 times (98.6%) energy reduction versus an equivalent artificial neural network (ANN).
I. INTRODUCTION
Cloud coverage degrades Earth-observation imagery, while ground processing and existing onboard-learning approaches face communication, latency, energy, and task-complexity constraints. ORBITALIF addresses these challenges with a decentralized spiking federated framework for onboard cloud removal.
- Cloud removal is a key preprocessing step because heavy cloud coverage severely degrades captured Earth-observation imagery.
- Ground-side cloud removal requires transmitting cloudy images over bandwidth-constrained links, causing high latency.
- Dense pixel-level cloud removal remains largely unexplored in satellite federated learning, which previously focused on classification and detection under non-IID conditions.
- SNNs provide event-driven inference for energy efficiency, but conventional binary spikes may lack expressiveness for dense per-pixel regression.
- ORBITALIF jointly supports onboard training and inference for cloud removal across LEO constellations using decentralized SFL and an efficient SNN backbone.
- 2.30 M parameters and a 72.3× inference-energy reduction make the proposed approach suitable for resource-constrained satellite deployment while retaining comparable cloud-removal performance.
II. SYSTEM MODEL
The system models a LEO constellation whose satellites communicate through inter-satellite links. Link credibility requires both geometric line-of-sight visibility and Doppler reliability.
- The constellation contains N orbital planes with K satellites per plane, and satellites communicate through inter-satellite links.
- A satellite pair is mutually line-of-sight visible when its geocentric angle satisfies φ ≤ ρ1 + ρ2.
- Doppler reliability requires the inter-satellite Doppler shift to satisfy fd ≤ fmax.
- A satellite pair becomes credible for information transmission only after satisfying both line-of-sight and Doppler-reliability metrics.
B. Satellite Federated Learning Model
The federated-learning model combines reliable intra-plane ring aggregation with intermittent inter-plane gossip averaging. This topology-aware procedure distributes, locally trains, and progressively aggregates model updates across the constellation.
- Step 1 — Model Distribution: Each global iteration begins by disseminating an orbital plane’s model copy to its K satellites through a reliable intra-plane ring.
- Step 2 — Local Training: Each satellite performs local mini-batch AdamW training while sharing the learning-rate schedule and retaining optimizer moments locally.
- Step 3 — Intra-Plane Model Aggregation: Ring all-reduce aggregates local updates within each orbital plane, yielding an exact intra-plane average after K communication rounds.
- Step 4 — Inter-Plane Model Aggregation: Inter-plane gossip averaging exchanges models only with neighboring orbital planes because inter-plane links are intermittent.
- Step 4 — Inter-Plane Model Aggregation: Repeated gossip rounds gradually diffuse model information across the constellation and reach consensus without the overhead of global aggregation.
A. Motivation and Architectural Overview
ORBITALIF uses a lightweight U-shaped SNN architecture tailored to onboard cloud removal under constrained satellite computation. AGFM and SHAM adapt feature fusion and attention across spatial, temporal, and frequency dimensions.
- Architectural Overview: The framework uses a U-shaped Encoder–Decoder architecture for onboard cloud removal in resource-constrained satellite deployment.A cloudy RGB image is embedded and replicated across synthetic time steps to process spatial and temporal information.
- Architectural Overview: AGFM replaces rigid U-Net skip connections with learnable gating to adaptively control encoder–decoder feature fusion.The gate determines how much information to fuse at each spatial location.
- Architectural Overview: SHAM jointly attends to temporal steps and frequency components to emphasize informative features for cloud removal.Its temporal branch weights time-step importance, while its spectral branch reweights frequency components using a 2-D FFT.
- Architectural Overview: The complete model contains only 2.30 M parameters, supporting deployment on resource-constrained satellites.A global residual formulation makes the network learn the cloud correction rather than the entire restored image.
B. Adaptive Gated Fusion Module (AGFM)
AGFM addresses semantic discrepancies between encoder and decoder features, which become more pronounced under heterogeneous satellite data. Its per-pixel, per-channel gate selectively suppresses contaminated features while preserving useful information.
- Motivation: Under non-IID satellite data, encoder and decoder features can exhibit semantic discrepancies that make rigid skip connections less suitable.Encoder features capture increasingly abstract representations, whereas the decoder reconstructs fine-grained spatial details.
- Mechanism: AGFM inserts a lightweight learnable gate between each encoder–decoder skip connection to control feature fusion.For matching-resolution encoder feature e and decoder feature d, fusion is computed through the gated formulation.
- Mechanism: The gate is per-pixel and per-channel, suppressing contaminated encoder features at heavily clouded pixels while preserving them at clear pixels.It is produced by a 1 × 1 convolution over concatenated features followed by a sigmoid activation.
- Deployment Benefits: AGFM provides minimal parameter overhead, improved representational capacity with negligible inference cost, and better generalization under heterogeneous orbital data.
C. Spectral-Spatial Hybrid Attention Module (SHAM)
SHAM models temporal and frequency information because time steps contribute unequally and cloud artifacts have characteristic spectral signatures. It combines temporal weighting with amplitude-based spectral attention while preserving phase and using a residual formulation.
- Motivation: SHAM jointly models temporal and frequency domains because spiking time steps contribute unequally and cloud artifacts have characteristic spectral signatures.
- Temporal Attention: The temporal attention branch produces a vector a ∈ (0, 1)^T that rescales each simulation time step.Spatial pooling and a fully connected layer operate along the time dimension before sigmoid activation.
- Spectral Attention: The spectral attention branch applies a 2-D real FFT to temporally averaged features, reweights amplitude through a lightweight 1 × 1 MLP, and preserves phase.An inverse FFT and 7 × 7 convolution produce a spatial attention map.
- Fusion: SHAM combines temporal and spatial attention in a residual output, with a learnable scalar initialized to zero to support training stability.Temporal attention is broadcast across channels and spatial dimensions, while spatial attention is broadcast across time and channels.
- Deployment Benefits: SHAM adds low latency and negligible parameter overhead while preserving the full model’s 2.30 M-parameter budget.The paper attributes the low latency partly to optimized 2-D FFT operations.
D. Training Objective
ORBITALIF is trained with a combined Charbonnier–SSIM objective that balances pixel-wise reconstruction accuracy with local luminance and structural consistency.
- Training Objective: The training objective combines Charbonnier loss with SSIM loss for cloud-removal reconstruction.The Charbonnier term penalizes pixel-wise errors, while SSIM encourages local luminance and spatial-structure consistency.
- Training Objective: The combined loss provides complementary constraints intended to preserve fidelity and structural detail while mitigating over-smoothed textures.The SSIM term is balanced with λ = 0.1.
A. Centralized Evaluation and Ablation
Centralized evaluation shows that ORBITALIF combines competitive cloud-removal quality with a compact model and low operation count, while decentralized experiments expose a clear accuracy–energy trade-off. Its spiking design substantially reduces inference energy while preserving practical visual recovery under federated training.
- Centralized evaluation: 25.374 dB PSNR is achieved with 2.30 M parameters and 3.7 G SOPs, the lowest operation count among the compared methods.CVAE uses 6.8× more parameters and 10.0× more operations, while MemoryNet reaches 26.073 dB at over 148× the computational cost.
- Ablation: Removing SHAM reduces PSNR to 24.854 dB, while removing AGFM reduces it to 24.774 dB.Both ablations fall below 25 dB; AGFM adds negligible parameter and operation overhead through lightweight 1×1 gating convolutions.
- Energy evaluation: 0.287 mJ per inference is consumed by ORBITALIF, a 72.3× (98.6%) reduction from ORBITANN at 20.75 mJ.ORBITALIF also achieves an approximately 11× energy reduction compared with ORBITUNET.
- Visual quality: ORBITALIF recovers cloud-free surface details under decentralized training with only 2.30 M parameters.The visual results support practical cloud removal performance for onboard satellite deployment.
V. CONCLUSION
ORBITALIF is presented as a framework for onboard cloud-removal training and inference across LEO constellations. It combines a lightweight spiking architecture with federated operation, achieving substantial energy savings while retaining competitive cloud-removal quality.
- Conclusion: ORBITALIF jointly supports onboard training and inference for optical cloud removal on a LEO satellite constellation.The framework is designed around Walker-Star operational constraints rather than compressing a ground-side model.
- Conclusion: 72.3× lower energy consumption than ORBITANN is obtained at the cost of 0.793 dB PSNR degradation.The conclusion identifies this as the principal energy–accuracy trade-off of the spiking architecture.
- Conclusion: 25.374 dB PSNR is achieved with 2.30 M parameters and 3.7 G operations under centralized evaluation.The authors interpret these requirements as suitable for resource-constrained satellite deployment.
- Future work: Future work includes multispectral and SAR-assisted inputs and hardware-measured energy validation on Loihi-2, Speck, or Akida substrates.The current energy bracket is identified for validation with direct hardware measurements.