PaperlayerView exact revision on arXiv

Source-linked AI summary

SatDL: Jointly Optimizing Data Redistribution and Training for Satellite-Based Distributed Learning

Hao Wu, Kin Whye Chew, Yizhan Han, Han Li, Jingxian Wang

arXiv:2608.24516v1cs.DCcs.LG

TL;DR

Satellite-based distributed learning faces prolonged training when data are non-IID across satellites. SatDL jointly optimizes redistribution and training time through selective partial redistribution, reducing end-to-end learning time and onboard energy consumption.

  • Problem

    Satellite-based distributed learning must train on data produced onboard or collected remotely, while non-IID data can prolong training and increase energy use.

  • Method

    SatDL uses a Distributor–Critic framework to jointly optimize data-transfer delay and wall-clock training time through selective partial data redistribution.

  • Results

    SatDL achieves significant reductions in end-to-end learning time and onboard energy consumption compared with state-of-the-art baselines across satellite simulations and hardware emulations.

  • Takeaways & Limitations

    Balancing communication overhead against convergence speed enables SatDL to minimize total end-to-end learning time under non-IID conditions.

Abstract

from arXiv · show

Satellite-based distributed learning promises to train machine-learning models directly in orbit using massive, globally dispersed sensor data, thereby avoiding large-scale data downloads to ground servers. However, training convergence is significantly slowed by severe non-IID data, specifically label imbalance, as each satellite observes different geographic regions with distinct labels. This imbalance extends training duration and increases energy consumption for solar-powered satellites. Existing approaches either fully redistribute data to enforce IID conditions - accelerating convergence but incurring substantial communication delays - or avoid redistribution entirely by modifying local learning algorithms to mitigate the impact of label imbalance, which, however, still prolong training and increase energy use. Both extremes result in excessive total end-to-end learning time (data-transfer delay plus training time) and thus elevated onboard energy consumption. We present SatDL, a data-redistribution framework designed to minimize total end-to-end learning time. At its core, SatDL develops a Distributor-Critic framework that jointly models and optimizes data-transfer delay and training time. Evaluations through trace-driven simulations of a 1,584-satellite Starlink constellation and hardware emulations using NVIDIA Jetson and A100 GPUs across five datasets show SatDL reduces total end-to-end learning time by up to 18.6% and onboard energy consumption by 12.23-88.00%, while maintaining inference accuracy within a few percentage points of state-of-the-art baselines.

I. INTRODUCTION

Satellite-based distributed learning keeps raw data in orbit and exchanges model updates, but geographic label imbalance slows convergence and increases energy use. SatDL jointly optimizes redistribution and training time through a Distributor–Critic framework.

  • Satellite distributed learning keeps raw data onboard while satellites exchange gradient updates to learn a shared global model.
  • Geographic label imbalance creates conflicting local updates that weaken aggregation and slow global-model convergence.Satellites observing urban and rural regions can collect substantially different labels.
  • IID redistribution accelerates training but requires extensive inter-satellite transfers, whereas non-IID training avoids movement but prolongs convergence.
  • SatDL jointly models data-transfer delay and wall-clock training time, selecting partial transfers that reduce gradient diversity per transmitted bit.Its objective is total end-to-end learning time, defined as data-transfer delay plus wall-clock training time.
  • 14.3% on CIFAR-10 and 18.6% on CIFAR-100 are reported reductions in total end-to-end learning time.The evaluation uses a simulated Starlink constellation, satellite hardware emulation, and five datasets.

II. RELATED WORK

Prior work mitigates satellite data heterogeneity either by changing learning algorithms or by moving data across clients. Motivating experiments show that IID balancing reduces training time but increases transfer delay as client count grows.

  • II. RELATED WORK: Non-IID client datasets can cause model divergence, slower convergence, and degraded overall performance.
  • Federated Learning Algorithms: Federated-learning methods such as FedProx, SCAFFOLD, and MOON modify local training to constrain or correct divergent updates without explicit data movement.
  • Data Movement Strategies: Data-movement strategies reduce statistical divergence by manipulating client distributions, including random IID partitioning or sharing balanced reference data.
  • C. Impact of Non-IID Data on Wall-clock Training Time: Under IID movement, increasing client count reduces training time through increased parallelism.
  • C. Impact of Non-IID Data on Wall-clock Training Time: With 10 clients, non-IID movement requires over 2× as many iterations as IID movement because increasing client count amplifies heterogeneity and conflicting updates.

D. Impact of Data Balancing on Data-Transfer Delay

IID balancing reduces wall-clock training time but incurs costly inter-satellite redistribution, while non-IID movement minimizes transfer delay but prolongs training. SatDL therefore selectively redistributes data to balance communication and convergence.

  • D. Impact of Data Balancing on Data-Transfer Delay: Increasing client count reduces non-IID data-transfer time because users are assigned to geographically closer clients.
  • D. Impact of Data Balancing on Data-Transfer Delay: IID balancing requires samples to cross clients, and low-capacity inter-satellite links cause prolonged multi-hop transfers that delay training.The passage contrasts roughly 1 Gbps ISLs with terrestrial fiber operating at tens of Gbps.
  • E. Key Insight: Balancing Communication and Convergence: The trade-off is between faster convergence with IID data and lower communication cost with non-IID data.
  • E. Key Insight: Balancing Communication and Convergence: SatDL’s workflow uploads data to access satellites, selectively redistributes samples to clients, and aggregates their local models into a global model.Clients can include some access satellites, and aggregation is performed by a designated satellite.
  • A. SatDL’s Design: The Strategy Critic estimates convergence time while the Distributor uses network costs and critic feedback to propose data-movement strategies.

B. Adaptation to Satellite Dynamics

SatDL adapts to orbital dynamics by precomputing user access schedules and continuously available inter-satellite links. Its optimization operates under resource constraints allocated by a centralized ground-based orchestrator.

  • B. Adaptation to Satellite Dynamics: Satellite motion changes user links and inter-satellite connectivity, requiring periodic reoptimization.
  • B. Adaptation to Satellite Dynamics: Precomputed access-satellite schedules identify satellites visible during each user’s expected upload period.The schedule uses predictable trajectories together with dataset size and uplink speed.
  • B. Adaptation to Satellite Dynamics: SatDL retains only inter-satellite links continuously available during the estimated transfer interval, such as 30 minutes.
  • C. Optimization under Orchestrator-Allocated Resource: The optimization takes user and dataset information, satellite topology and positions, available clients, and orchestrator-provided resources as inputs.
  • 1) Optimization Variables: Data-transfer variables represent samples uploaded from users to access satellites and redistributed from access satellites to client satellites by label.

2) Data Uploading:

SatDL models data collection and redistribution alongside training, minimizing total end-to-end learning time under movement constraints. Upload and redistribution delays are determined by the slowest relevant communication link.

  • Data Uploading: Each user uploads its entire dataset only to visible access satellites.The visibility indicator ψu,s determines whether satellite s can receive user u’s data.
  • Data Uploading: The collected satellite distribution is formed by summing uploads from connected users.
  • Data Redistribution: Redistribution transfers data among satellites subject to a constraint requiring non-client access satellites to send their datasets to client satellites.
  • Data Redistribution: The final client distribution subtracts transmitted samples from each client and adds samples received from other satellites.
  • Data Redistribution: Redistribution time is determined by the slowest inter-satellite link, using transferred volume divided by link bandwidth.ϕsk,e indicates whether a source-to-client path traverses link e, and Ce is that link’s bandwidth.
  • Objective Function: Data movement variables x and y are optimized jointly with training time under constraints (1), (2), and (5).The resulting client distribution determines ttrain, while tcollect and trdis represent data movement time.

B. Strategy Critic

The Strategy Critic estimates training time for a proposed data distribution by combining theoretical convergence analysis with empirically calibrated, device-dependent parameters. It provides both the estimated training time and its gradient for redistribution optimization.

  • Strategy Critic: The Critic estimates ttrain and its gradient for a particular client data distribution.These outputs enable the Task & Data Distributor to optimize redistribution for minimized total learning time.
  • Strategy Critic: SatDL combines theoretical convergence analysis with one-time empirical calibration of task-specific and hardware-dependent parameters.Calibration uses satellite emulation platforms and includes quantities such as smoothness, bounded local variance, gradient diversity, and device capabilities.
  • Convergence Bound: Gradient Diversity λ quantifies data heterogeneity among satellite clients.Higher λ indicates more pronounced non-IID conditions and bounds the learning rate.
  • Convergence Bound: The convergence guarantee relates the learning rate and average-squared gradient to environmental constants, local iterations, client count, and distributed rounds.The global objective f, its lower bound f∗, and minimum achievable average-squared gradient G define the bound’s optimization quantities.

2) Modifications of Theoretical Bounds:

SatDL replaces dataset-level gradient-diversity evaluation with a class-distribution formulation to make candidate redistribution plans tractable. The reformulation uses class gradient prototypes and a class-assignment matrix.

  • Original Formulation: The original client gradient is defined as the empirical risk gradient over the client’s local dataset.This dataset-based definition is suitable for static evaluation of a fixed client partition.
  • Challenge: The dataset-based Critic formulation is prohibitive because it requires per-sample gradients, O(Nd) communication, O(N) evaluation per plan, and a combinatorial K^N search space.Discrete assignment variables also prevent direct optimization of ttrain by gradient descent.
  • Class-Based Reformulation: SatDL approximates each client gradient using class gradient prototypes and the client’s class probability distribution.The approximation assumes class imbalance is the dominant source of heterogeneity across clients.
  • Class-Based Reformulation: The reformulation computes client gradients from class prototypes rather than per-sample gradients for each candidate assignment plan.This supports efficient evaluation of simulated client–data assignments proposed by the Distributor.

3) Convergence Time Estimation:

SatDL estimates convergence rounds from a theoretical gradient threshold and converts them into wall-clock training time using the slowest device’s computation time. A one-time calibration supports the bound parameters, and MNIST experiments validate the estimates.

  • Convergence Time Estimation: SatDL estimates convergence rounds using a threshold ϵg and learning rate η, then converts rounds into training time.The conversion multiplies the required rounds by the computation time per round of the slowest device.
  • Convergence Time Estimation: Device computation time depends on FLOPSk and the operations F required per local iteration.
  • Calibration: Parameters L and C1 are empirically estimated through a one-time calibration.The calibration is described in Appendix B.
  • Validation: Predicted convergence closely aligns with observed MNIST iterations across Dirichlet non-IID parameters α from 0.1 to 1.0.Observed convergence uses a loss threshold ϵℓ, while prediction uses a gradient threshold ϵg.

VI. IMPLEMENTATION, EVALUATION, RESULTS

SatDL is evaluated through Starlink-scale simulation and GPU hardware emulation across five non-IID datasets. It achieves the shortest total learning time by balancing redistribution delay against training duration while maintaining comparable test accuracy.

  • Implementation and Evaluation: Evaluations use a 1,584-satellite Starlink simulation and NVIDIA A100 and Jetson GPU emulations across five datasets.The datasets are CIFAR-10, CIFAR-100, Flickr Mammals, Satellite Land Cover, and Traffic Signs.
  • Results: SatDL maintains comparable or slightly superior test accuracy compared with the baselines.Test performance is summarized using average class accuracy.
  • Results: IID achieves the shortest training time but incurs substantial redistribution delay, whereas Non-IID and FedProx retain high total runtime because training remains prolonged.A method that minimizes data movement alone can produce the longest training times under heterogeneous data.
  • Implementation and Evaluation: SatDL jointly minimizes data-transfer and training time rather than optimizing either component in isolation.Its Distributor-Critic approach models redistribution delay and convergence duration together.
  • Results: 14.3%, 18.6%, 16.7%, 3.2%, and 5.7% are SatDL’s total end-to-end learning-time reductions on CIFAR-10, CIFAR-100, Flickr Mammals, Satellite, and Traffic Signs, respectively.These reductions are reported against the best baselines, with the shortest total time across all datasets.
  • Results: SatDL’s data-transfer time is 61.8% lower than IID and its training time is 35.3% lower than Non-IID.These component-level results explain how SatDL reduces total end-to-end learning time.

C. Ablation Results

The ablation results show that SatDL adapts redistribution to data heterogeneity and datapoint size. This adaptive policy consistently produces the lowest end-to-end learning time across the tested configurations and reduces energy use in Jetson emulations.

  • Ablation Summary: SatDL consistently achieves the lowest end-to-end learning time across the ablation configurations.Unlike IID and Non-IID, it dynamically changes its redistribution policy in response to input conditions.
  • Alpha: As Dirichlet Alpha decreases, SatDL moves more data while tolerating a less IID distribution.Lower Alpha represents increasingly non-IID user data; IID instead redistributes extensively as skew worsens.
  • Size of Datapoint: As datapoint size increases, SatDL tolerates more heterogeneity to save communication cost.Moving and redistributing larger datapoints increases cost proportionally, while IID maintains fixed training time through extensive redistribution.
  • Energy Use: 16.7%, 6.98%, 7.04%, and 64.1% are SatDL’s total-learning-time reductions versus IID, Non-IID, FedProx, and Hybrid-FL, respectively.These results come from Jetson-emulated satellite clients performing satellite-image land-cover classification.
  • Energy Use: 47.75%, 12.23%, 12.31%, and 88.00% are SatDL’s corresponding onboard-energy reductions versus IID, Non-IID, FedProx, and Hybrid-FL.Energy is calculated from redistribution transmission power and Jetson GPU runtime power.

APPENDIX

The convergence analysis adopts smoothness, lower-boundedness, bounded-variance, and communication-matrix spectral assumptions. These conditions constrain local objectives, stochastic gradients, and network mixing.

  • Assumptions: The analysis assumes each local objective is differentiable and L-smooth, with gradients changing at most proportionally to parameter distance.The global objective is also assumed to have a lower bound f ∗.
  • Assumptions: Each local data shard supports independent mini-batch sampling and an unbiased stochastic-gradient estimate.The mini-batch has size B, and its variance is bounded using constants C1 and σ.
  • Assumptions: The client communication matrix has one largest eigenvalue equal to one, while all other eigenvalues have magnitude strictly below one.The second-largest eigenvalue magnitude is denoted ζ and satisfies ζ < 1.

B. Estimation of parameters in pre-experiment

The pre-experiment estimates convergence parameters empirically and reformulates gradient diversity using clients’ class-probability distributions. These procedures target smoothness, stochastic-gradient variance, and label-imbalance effects.

  • Estimation of L: The smoothness constant L is estimated from the maximum observed ratio of gradient-distance to parameter-distance across random model pairs.The procedure repeats measurements over multiple random samples to obtain a worst-case estimate.
  • Estimation of C1: The variance parameters C1 and σ are obtained by fitting an upper-bounding line to mini-batch gradient variances versus unbiased-gradient magnitude.Measurements use randomly initialized models and repeated samples.
  • Estimation of λ: Gradient diversity is reformulated as a function of each client’s class-probability distribution pk rather than only its current dataset distribution.The reformulation uses class-specific gradients and probabilities pk(c) = nk,c / nk.
  • Estimation of λ: The approximation computes class-average gradients, assigns class gradients to data points, and estimates each client gradient from pk.It assumes class imbalance is the leading source of gradient diversity under non-IID data.
Loading 2608.24516v1…