Source-linked AI summary
ORBITALIF: An Efficient Spiking Federated Learning Framework for Onboard Cloud Removal
Bohan Zhang, Chenyu Xu, Yijie Mao, Yuanming Shi
TL;DR
Cloud coverage obscures Earth-observation imagery, while ground processing is constrained by bandwidth, contact windows, and latency, and onboard inference must meet tight energy budgets. OrbitALIF combines decentralized satellite federated learning with a compact SNN backbone for onboard training and inference. It delivers competitive cloud-removal quality with substantially lower inference energy than an equivalent ANN.
Problem
Cloud coverage obscures Earth-observation imagery, while ground processing faces constrained satellite-to-ground bandwidth and high latency, and onboard models can be too computationally and energy intensive.
Source: §I · p. 1Method
OrbitALIF combines decentralized satellite federated learning over inter-satellite links with a 2.30 M-parameter SNN backbone using AGFM and SHAM for onboard cloud removal.
Source: §I · pp. 1–2, A. Motivation and Architectural Overview · p. 3, B. Adaptive Gated Fusion Module (AGFM) · p. 4Results
72.3× lower inference energy than an equivalent ANN is achieved while maintaining comparable cloud-removal performance.
Source: §I · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
Low-earth-orbit (LEO) satellites enable high-resolution, large-scale Earth observation for applications such as disaster monitoring and environmental surveillance. However, cloud coverage often obscures the Earth's surface, and conventional cloud-removal pipelines that download cloudy images to ground stations for processing suffer from limited contact windows, constrained satellite-to-ground bandwidth, and high latency. In this work, we propose a novel satellite federated learning framework for cloud removal across LEO constellations, named orbital attention leaky integrate-and-fire (OrbitALIF). OrbitALIF performs both onboard training and inference using a compact 2.30,M-parameter spiking neural network (SNN) backbone with an adaptive gated fusion module (AGFM) and a spectral-spatial hybrid attention module (SHAM), combined with a decentralized federated learning strategy that shares model weights via inter-satellite links. Our experiments show that OrbitALIF achieves competitive cloud removal quality while consuming only 0.287,mJ per inference on neuromorphic hardware, a 72.3 times (98.6%) energy reduction versus an equivalent artificial neural network (ANN).
I. INTRODUCTION
Cloud coverage degrades Earth-observation imagery, while ground processing and existing onboard-learning approaches face communication, latency, energy, and task-complexity constraints. ORBITALIF addresses these challenges with a decentralized spiking federated framework for onboard cloud removal.
- Cloud removal is a key preprocessing step because heavy cloud coverage severely degrades captured Earth-observation imagery.Source: §I · p. 1
- Ground-side cloud removal requires transmitting cloudy images over bandwidth-constrained links, causing high latency.Source: §I · p. 1
- Dense pixel-level cloud removal remains largely unexplored in satellite federated learning, which previously focused on classification and detection under non-IID conditions.Source: §I · p. 1
- SNNs provide event-driven inference for energy efficiency, but conventional binary spikes may lack expressiveness for dense per-pixel regression.Source: §I · p. 1
- ORBITALIF jointly supports onboard training and inference for cloud removal across LEO constellations using decentralized SFL and an efficient SNN backbone.Source: §I · pp. 1–2
- 2.30 M parameters and a 72.3× inference-energy reduction make the proposed approach suitable for resource-constrained satellite deployment while retaining comparable cloud-removal performance.Source: §I · p. 2
II. SYSTEM MODEL
The system models a LEO constellation whose satellites communicate through inter-satellite links. Link credibility requires both geometric line-of-sight visibility and Doppler reliability.
- The constellation contains N orbital planes with K satellites per plane, and satellites communicate through inter-satellite links.Source: A. Satellite Communication Model · p. 2
- A satellite pair is mutually line-of-sight visible when its geocentric angle satisfies φ ≤ ρ1 + ρ2.Source: A. Satellite Communication Model · p. 2
- Doppler reliability requires the inter-satellite Doppler shift to satisfy fd ≤ fmax.Source: A. Satellite Communication Model · p. 2
- A satellite pair becomes credible for information transmission only after satisfying both line-of-sight and Doppler-reliability metrics.Source: A. Satellite Communication Model · p. 2
B. Satellite Federated Learning Model
The federated-learning model combines reliable intra-plane ring aggregation with intermittent inter-plane gossip averaging. This topology-aware procedure distributes, locally trains, and progressively aggregates model updates across the constellation.
Step 1 — Model Distribution
- Each global iteration begins by disseminating an orbital plane’s model copy to its K satellites through a reliable intra-plane ring.Source: B. Satellite Federated Learning Model · p. 2
Step 2 — Local Training
- Each satellite performs local mini-batch AdamW training while sharing the learning-rate schedule and retaining optimizer moments locally.Source: B. Satellite Federated Learning Model · pp. 2–3
Step 3 — Intra-Plane Model Aggregation
- Ring all-reduce aggregates local updates within each orbital plane, yielding an exact intra-plane average after K communication rounds.Source: B. Satellite Federated Learning Model · p. 3
Step 4 — Inter-Plane Model Aggregation
- Inter-plane gossip averaging exchanges models only with neighboring orbital planes because inter-plane links are intermittent.Source: B. Satellite Federated Learning Model · p. 3
- Repeated gossip rounds gradually diffuse model information across the constellation and reach consensus without the overhead of global aggregation.Source: B. Satellite Federated Learning Model · p. 3
A. Motivation and Architectural Overview
ORBITALIF uses a lightweight U-shaped SNN architecture tailored to onboard cloud removal under constrained satellite computation. AGFM and SHAM adapt feature fusion and attention across spatial, temporal, and frequency dimensions.
Architectural Overview
- The framework uses a U-shaped Encoder–Decoder architecture for onboard cloud removal in resource-constrained satellite deployment.A cloudy RGB image is embedded and replicated across synthetic time steps to process spatial and temporal information.Source: A. Motivation and Architectural Overview · p. 3
- AGFM replaces rigid U-Net skip connections with learnable gating to adaptively control encoder–decoder feature fusion.The gate determines how much information to fuse at each spatial location.Source: A. Motivation and Architectural Overview · p. 3
- SHAM jointly attends to temporal steps and frequency components to emphasize informative features for cloud removal.Its temporal branch weights time-step importance, while its spectral branch reweights frequency components using a 2-D FFT.Source: A. Motivation and Architectural Overview · p. 3
- The complete model contains only 2.30 M parameters, supporting deployment on resource-constrained satellites.A global residual formulation makes the network learn the cloud correction rather than the entire restored image.Source: A. Motivation and Architectural Overview · p. 3
B. Adaptive Gated Fusion Module (AGFM)
AGFM addresses semantic discrepancies between encoder and decoder features, which become more pronounced under heterogeneous satellite data. Its per-pixel, per-channel gate selectively suppresses contaminated features while preserving useful information.
Motivation
- Under non-IID satellite data, encoder and decoder features can exhibit semantic discrepancies that make rigid skip connections less suitable.Encoder features capture increasingly abstract representations, whereas the decoder reconstructs fine-grained spatial details.Source: B. Adaptive Gated Fusion Module (AGFM) · pp. 3–4
Mechanism
- AGFM inserts a lightweight learnable gate between each encoder–decoder skip connection to control feature fusion.For matching-resolution encoder feature e and decoder feature d, fusion is computed through the gated formulation.Source: B. Adaptive Gated Fusion Module (AGFM) · p. 4
- The gate is per-pixel and per-channel, suppressing contaminated encoder features at heavily clouded pixels while preserving them at clear pixels.It is produced by a 1 × 1 convolution over concatenated features followed by a sigmoid activation.Source: B. Adaptive Gated Fusion Module (AGFM) · p. 4
Deployment Benefits
- AGFM provides minimal parameter overhead, improved representational capacity with negligible inference cost, and better generalization under heterogeneous orbital data.Source: B. Adaptive Gated Fusion Module (AGFM) · p. 4
C. Spectral-Spatial Hybrid Attention Module (SHAM)
SHAM models temporal and frequency information because time steps contribute unequally and cloud artifacts have characteristic spectral signatures. It combines temporal weighting with amplitude-based spectral attention while preserving phase and using a residual formulation.
Motivation
- SHAM jointly models temporal and frequency domains because spiking time steps contribute unequally and cloud artifacts have characteristic spectral signatures.Source: C. Spectral-Spatial Hybrid Attention… · p. 4
Temporal Attention
- The temporal attention branch produces a vector a ∈ (0, 1)^T that rescales each simulation time step.Spatial pooling and a fully connected layer operate along the time dimension before sigmoid activation.Source: C. Spectral-Spatial Hybrid Attention… · p. 4
Spectral Attention
- The spectral attention branch applies a 2-D real FFT to temporally averaged features, reweights amplitude through a lightweight 1 × 1 MLP, and preserves phase.An inverse FFT and 7 × 7 convolution produce a spatial attention map.Source: C. Spectral-Spatial Hybrid Attention… · p. 4
Fusion
- SHAM combines temporal and spatial attention in a residual output, with a learnable scalar initialized to zero to support training stability.Temporal attention is broadcast across channels and spatial dimensions, while spatial attention is broadcast across time and channels.Source: C. Spectral-Spatial Hybrid Attention… · p. 4
Deployment Benefits
- SHAM adds low latency and negligible parameter overhead while preserving the full model’s 2.30 M-parameter budget.The paper attributes the low latency partly to optimized 2-D FFT operations.Source: C. Spectral-Spatial Hybrid Attention… · p. 4
D. Training Objective
ORBITALIF is trained with a combined Charbonnier–SSIM objective that balances pixel-wise reconstruction accuracy with local luminance and structural consistency.
Training Objective
- The training objective combines Charbonnier loss with SSIM loss for cloud-removal reconstruction.The Charbonnier term penalizes pixel-wise errors, while SSIM encourages local luminance and spatial-structure consistency.Source: D. Training Objective · p. 4
- The combined loss provides complementary constraints intended to preserve fidelity and structural detail while mitigating over-smoothed textures.The SSIM term is balanced with λ = 0.1.Source: D. Training Objective · p. 4
A. Centralized Evaluation and Ablation
Centralized evaluation shows that ORBITALIF combines competitive cloud-removal quality with a compact model and low operation count, while decentralized experiments expose a clear accuracy–energy trade-off. Its spiking design substantially reduces inference energy while preserving practical visual recovery under federated training.
Centralized evaluation
- 25.374 dB PSNR is achieved with 2.30 M parameters and 3.7 G SOPs, the lowest operation count among the compared methods.CVAE uses 6.8× more parameters and 10.0× more operations, while MemoryNet reaches 26.073 dB at over 148× the computational cost.Source: A. Centralized Evaluation and Ablation · p. 5
Ablation
- Removing SHAM reduces PSNR to 24.854 dB, while removing AGFM reduces it to 24.774 dB.Both ablations fall below 25 dB; AGFM adds negligible parameter and operation overhead through lightweight 1×1 gating convolutions.Source: A. Centralized Evaluation and Ablation · p. 5
Energy evaluation
- 0.287 mJ per inference is consumed by ORBITALIF, a 72.3× (98.6%) reduction from ORBITANN at 20.75 mJ.ORBITALIF also achieves an approximately 11× energy reduction compared with ORBITUNET.Source: B. Decentralized Federated Learning… · pp. 5–6
Visual quality
- ORBITALIF recovers cloud-free surface details under decentralized training with only 2.30 M parameters.The visual results support practical cloud removal performance for onboard satellite deployment.Source: B. Decentralized Federated Learning… · p. 6
V. CONCLUSION
ORBITALIF is presented as a framework for onboard cloud-removal training and inference across LEO constellations. It combines a lightweight spiking architecture with federated operation, achieving substantial energy savings while retaining competitive cloud-removal quality.
Conclusion
- ORBITALIF jointly supports onboard training and inference for optical cloud removal on a LEO satellite constellation.The framework is designed around Walker-Star operational constraints rather than compressing a ground-side model.Source: §V · p. 6
- 72.3× lower energy consumption than ORBITANN is obtained at the cost of 0.793 dB PSNR degradation.The conclusion identifies this as the principal energy–accuracy trade-off of the spiking architecture.Source: §V · p. 6
- 25.374 dB PSNR is achieved with 2.30 M parameters and 3.7 G operations under centralized evaluation.The authors interpret these requirements as suitable for resource-constrained satellite deployment.Source: §V · p. 6
Future work
- Future work includes multispectral and SAR-assisted inputs and hardware-measured energy validation on Loihi-2, Speck, or Akida substrates.The current energy bracket is identified for validation with direct hardware measurements.Source: §V · p. 6