PaperlayerView exact revision on arXiv

Source-linked AI summary

Ground-Assisted Federated Learning in LEO Satellite Constellations

Nasrin Razmi, Bho Matthiesen, Armin Dekorsy, Petar Popovski

arXiv:2109.01348v2eess.SPcs.LG

TL;DR

Satellite FL must accommodate predictable but sparse ground-station connectivity and local datasets that are not shared. The paper adapts FedAvg to this setting with an asynchronous procedure that exploits deterministic worker availability. Across the reported experiments, the proposed method outperforms FedAsync in convergence time and test accuracy and is more robust to heterogeneity.

  • Problem

    Satellite FL combines sparse, predictable communication with heterogeneous local datasets, unlike the random availability and low-delay parallel communication assumed by conventional FL algorithms.

    Source: §I · p. 1, §III · p. 2

  • Method

    The paper adapts FedAvg to ground-assisted satellite FL by exploiting deterministic worker availability to make the procedure asynchronous.

    Source: §VI · p. 5

  • Results

    The proposed algorithm outperforms FedAsync in convergence time and test accuracy, and shows greater robustness to heterogeneity across the reported experiments.

    Source: §VI · p. 5, §V · p. 5

  • Takeaways & Limitations

    In the evaluated satellite setting, unrolling FedAvg reduces its training time by several hours and requires fewer hyperparameters to tune than FedAsync.

    Source: §VI · p. 5

    FedAsync’s oscillatory behavior on non-IID data in Fig. 5 could not be avoided by tuning α and η; a staleness function and decaying learning rate might help dampen it.

    Source: §VI · p. 5

Abstract

from arXiv · show

In Low Earth Orbit (LEO) mega constellations, there are relevant use cases, such as inference based on satellite imaging, in which a large number of satellites collaboratively train a machine learning model without sharing their local datasets. To address this problem, we propose a new set of algorithms based on Federated learning (FL), including a novel asynchronous FL procedure based on FedAvg that exhibits better robustness against heterogeneous scenarios than the state-of-the-art. Extensive numerical evaluations based on MNIST and CIFAR-10 datasets highlight the fast convergence speed and excellent asymptotic test accuracy of the proposed method.

I. INTRODUCTION

LEO constellations offer a setting for satellite-based machine learning, but centralized training can incur high transmission costs and delays, while data sharing may be restricted. The paper proposes federated-learning algorithms for ground-assisted satellite training, including an asynchronous FedAvg variant designed for predictable satellite availability.

I. INTRODUCTION

  • LEO constellations are presented as a cost-efficient, versatile alternative to traditional satellites in medium Earth and geostationary orbits.The introduction also notes their use for ubiquitous connectivity and low-latency Internet service.Source: §I · p. 1
  • Centralized training can incur high transmission costs and delays, and satellite data may be restricted by privacy or ownership concerns.Federated learning instead trains locally and shares model updates rather than local datasets.Source: §I · p. 1
  • Raw Earth-observation data can remain onboard after preprocessing, so improving models with new observations requires on-board retraining.The paper cites ESA’s PhiSat-1 as an example.Source: §I · p. 1
  • The proposed asynchronous FedAvg variant exploits predictable satellite availability and is reported as more robust to heterogeneous scenarios than FedAsync.The paper also introduces a satellite FL scenario and a communication protocol.Source: §I · p. 1

II. SYSTEM MODEL AND BACKGROUND ON FL

The system models satellites with distinct, potentially non-IID datasets and ground-station contact windows that are brief relative to the time between contacts. Training is orchestrated by a ground station, with satellites computing locally between contacts and exchanging model parameters when connected.

II. SYSTEM MODEL AND BACKGROUND ON FL

  • Only a subset of satellites can connect to the ground station at once, and contact intervals are much longer than online periods.A link is feasible when a satellite is visible above a minimum elevation angle.Source: §II · p. 1
  • Different satellite orbits and positions produce disjoint datasets that may be non-IID.Source: §II · p. 1
  • The ground station orchestrates iterative training without sharing satellite datasets; satellites compute between visits and exchange model parameters during contact.The satellites are assumed to have very limited computational resources.Source: §II · p. 2

A. Federated Learning Background

The background describes FedAvg as a parameter-server procedure that selects workers, sends them the global model, and waits for their local results before aggregation. FedAsync instead incorporates client updates as they arrive, with a mixing weight that can depend on update staleness.

A. Federated Learning Background

  • FedAvg selects available workers each global iteration, sends them the current global model, and waits for their results before starting the next epoch.The parameter server aggregates returned local models into a new global model.Source: A. Federated Learning Background · p. 2
  • Workers perform minibatch SGD on local datasets and transmit updated model parameters to the parameter server.Each local epoch partitions the dataset into batches and applies SGD steps with learning rate η.Source: A. Federated Learning Background · p. 2
  • FedAsync incorporates client updates as they arrive, using a mixing factor that may reduce the weight of updates based on staleness.The staleness function depends on how far the global model has advanced since the worker received it.Source: A. Federated Learning Background · p. 2

III. FEDERATED LEARNING ON SATELLITES

Satellite FL differs from conventional FL because computation is available continuously while communication occurs in short, predictable windows and reaches few workers at a time. The paper considers FedAvg as a baseline and motivates asynchronous methods that exploit predictable connectivity.

III. FEDERATED LEARNING ON SATELLITES

  • Satellite communication is limited to small, highly predictable windows, and only a small fraction of workers is in range at any instant.The contrasted FL algorithms assume random device availability and parallel communication without significant delay.Source: §III · p. 2
  • The section treats asynchronous FL as best suited to this scenario while retaining synchronous FedAvg as a baseline.Source: §III · p. 2
  • The proposed approach uses predictable satellite connectivity to implement FedAvg asynchronously without unnecessary delays.Source: §III · p. 2

A. Communication Protocol and Satellite Operation

Satellites initiate contact with the ground station (GS), exchanging available local updates and receiving updated global parameters when computation continues. Satellite computation is bounded to finish before the GS’s next visit.

A. Communication Protocol and Satellite Operation

  • On contact, a satellite sends an unsent local update if available; the GS updates the global model and may return the new parameters to continue computation.If computation is not continued, the connection ends and the satellite does not reconnect during that pass.Source: A. Communication Protocol and… · p. 2
  • Satellite computation is designed to avoid large deviations from the global model during asynchronous operation and long gaps between GS contacts.The computation task is described in Algorithm 1.Source: A. Communication Protocol and… · p. 2
  • The fixed iteration stopping criterion should finish computation before the GS is visited again.The model uses L2 regularization, with its regularization term defined in equation (2).Source: A. Communication Protocol and… · p. 3

B. Synchronous Ground Station Operation

The synchronous ground-station adaptation of FedAvg schedules workers predictively and waits for scheduled workers to receive the model and return updates. This blocking loop can lengthen epochs when satellites require multiple passes.

B. Synchronous Ground Station Operation

  • A naïve satellite adaptation of FedAvg waits until every scheduled worker has received the current model and returned its update before the inner loop ends.The GS tracks workers yet to receive the model in Si and workers yet to return updates in Ri.Source: B. Synchronous Ground Station Operation · p. 3
  • With multiple scheduled satellites, an epoch takes at least one orbital period on average if a satellite cannot finish computation in one pass, and the time increases further.The blocking loop waits for all scheduled satellites to connect twice to the GS.Source: B. Synchronous Ground Station Operation · p. 3
  • From an optimization-theoretic perspective, Algorithm 2 is equivalent to FedProx, and its convergence follows from FedProx’s Theorem 4.The stated equivalence uses the regularization in equation (5).Source: B. Synchronous Ground Station Operation · p. 3

C. Asynchronous Ground Station Operation

Asynchronous ground-station operation lets satellites work on different global-model versions to reduce delay. The GS incorporates received updates, advances the epoch, and may send the updated model for further computation.

C. Asynchronous Ground Station Operation

1) FedAsync:

For FedAsync, the ground station weights incoming updates using a staleness function. A hinged function accounts for the orbital-period constraint and can inform whether to schedule a satellite.

1) FedAsync:

  • FedAsync’s SERVERUPDATE computes the mixing factor from a staleness function before applying the update.The staleness function is hinged because no client update can be fresher than one orbital period.Source: 1) FedAsync · p. 3
  • The staleness function uses elapsed time between GS-processed epochs, with its hinged form parameterized by ε, a, and the constellation’s maximum orbital period.Source: 1) FedAsync · p. 3
  • The scheduler can skip a satellite when its predicted update weight falls below a threshold, conserving energy and computational resources.The GS can calculate the staleness value at that satellite’s next pass.Source: 1) FedAsync · p. 3

IV. UNROLLED FEDERATED AVERAGING ALGORITHM

Synchronous satellite FL can incur high latency, while FedAsync avoids waiting but FedAvg has stronger convergence properties under full participation and suitable hyperparameters. The proposed unrolled procedure uses predictable satellite contacts to apply FedAvg incrementally, with convergence depending on revisit patterns.

IV. UNROLLED FEDERATED AVERAGING ALGORITHM

  • The construction uses a near-polar, single-shell Walker Delta constellation whose satellites visit a North Pole ground station in a repeating order.Each satellite visits once per orbital period, and the contact sequence repeats each period.Source: §IV · p. 4
  • With full client participation, FedAvg can be implemented incrementally without synchronizing the update phase.The procedure leverages predictable satellite connectivity to address the high latency of synchronous FL.Source: §IV · pp. 3–4
  • After each orbital period, the K incremental updates should produce a model close to Algorithm 2; with approximately equal revisit periods, the procedure converges as established in [17, §2].The update weighting accounts for different dataset sizes.Source: §IV · p. 4
  • In multiple-shell constellations, unequal revisit rates may bias the solution, although this can also affect other state-of-the-art FL methods.The reported numerical results indicate this effect is less pronounced than with FedAsync.Source: §IV · p. 4

V. EMPIRICAL RESULTS

Across the experiments, FedSat performs well while FedAvg incurs delays and FedAsync struggles with satellite-scenario heterogeneity. FedAsync has a slight edge over FedSat in the homogeneous North Pole case, but performs worse with a Bremen ground station.

  • The experiments used five satellites in each of two Walker Delta shells and evaluated logistic regression on MNIST and ResNet-18 on CIFAR-10.Source: §V · p. 4
  • For Bremen non-IID MNIST and CIFAR, FedSat led in convergence speed and final test accuracy; the staleness function helped MNIST stability but hurt CIFAR training.Source: §V · p. 4, Figure 1 · p. 4, Figure 2 · p. 4
  • With IID CIFAR, FedAsync edged FedSat at the North Pole with tuned α = 0.3, but was strictly worse at Bremen, highlighting FedSat’s heterogeneity robustness.Source: §V · p. 5, Figure 3 · p. 5, Figure 4 · p. 5
  • Across experiments, FedAvg incurred tremendous delays, FedAsync struggled with satellite-scenario heterogeneity, and FedSat performed excellently.Source: §V · p. 5

VI. CONCLUSIONS

The paper adapts FedAvg to exploit predictable satellite availability, converting it to asynchronous learning without sacrificing training performance. It reports shorter training time and better convergence time and test accuracy than FedAsync, while leaving several scheduling and ground-station extensions for future work.

VI. CONCLUSIONS

  • FedAsync oscillated on non-IID CIFAR data despite tuning α and η; a better staleness function and decaying learning rate might dampen the behavior.Fig. 5 shows top-1 accuracy for uniform client sampling on IID and non-IID CIFAR data, using FedAsync without a staleness function.Source: §VI · p. 5, Figure 5 · p. 5
  • FedAvg can be unrolled into asynchronous learning without sacrificing training performance, reducing training time by several hours and outperforming FedAsync in convergence time and test accuracy.The proposed algorithm also has fewer hyperparameters to tune than FedAsync.Source: §VI · p. 5
  • Worker scheduling, multiple data exchanges per ground-station pass, and using multiple ground stations remain open topics that could lead to considerably faster training.Source: §VI · p. 5

Related papers

Loading 2109.01348v2…