PaperlayerView exact revision on arXiv

Source-linked AI summary

Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregations

Frank Po-Chen Lin, Seyyedali Hosseinalipour, Sheikh Shams Azam, Christopher G. Brinton, Nicolo Michelusi

arXiv:2103.10481v3cs.LGcs.DCstat.ML

TL;DR

Heterogeneous local data can bias federated models and degrade global accuracy, while communication efficiency remains a challenge. TT-HF combines local SGD with cooperative D2D consensus and adaptive control. Experiments report at least 75% lower total cost than baselines in each evaluated case.

Abstract

from arXiv · show

Federated learning has emerged as a popular technique for distributing machine learning (ML) model training across the wireless edge. In this paper, we propose two timescale hybrid federated learning (TT-HF), a semi-decentralized learning architecture that combines the conventional device-to-server communication paradigm for federated learning with device-to-device (D2D) communications for model training. In TT-HF, during each global aggregation interval, devices (i) perform multiple stochastic gradient descent iterations on their individual datasets, and (ii) aperiodically engage in consensus procedure of their model parameters through cooperative, distributed D2D communications within local clusters. With a new general definition of gradient diversity, we formally study the convergence behavior of TT-HF, resulting in new convergence bounds for distributed ML. We leverage our convergence bounds to develop an adaptive control algorithm that tunes the step size, D2D communication rounds, and global aggregation period of TT-HF over time to target a sublinear convergence rate of O(1/t) while minimizing network resource utilization. Our subsequent experiments demonstrate that TT-HF significantly outperforms the current art in federated learning in terms of model accuracy and/or network energy consumption in different scenarios where local device datasets exhibit statistical heterogeneity. Finally, our numerical evaluations demonstrate robustness against outages caused by fading channels, as well favorable performance with non-convex loss functions.

I. INTRODUCTION … C. TT-HF: Two Timescale Hybrid Federated Learning

TT-HF combines local D2D consensus with server-based federated learning to address communication costs and heterogeneous device data. The paper develops convergence analysis and adaptive control for this architecture, and reports improved resource consumption and/or training time over federated learning with infrequent global aggregations.

I. INTRODUCTION

  • Edge-generated data makes dataset uploads bandwidth-intensive and time-consuming for latency-sensitive applications, while privacy concerns may discourage sharing data with servers.Examples include autonomous-vehicle object detection and smartphone next-word prediction.Source: §I · p. 1

A. Federated Learning at the Wireless Edge

  • Conventional federated learning faces heterogeneous device resources and server distances, while infrequent global aggregation can leave local models biased by statistically heterogeneous datasets.This motivates resource-efficient learning that limits model divergence across local datasets.Source: A. Federated Learning at the Wireless… · p. 1

B. Related Work

  • Unlike [37], this work allows multiple local SGD iterations, aperiodic global aggregation, and aperiodic within-cluster consensus, and introduces a new gradient-diversity metric for its analysis.The architecture occupies a middle ground between conventional server-centered and fully decentralized learning, without requiring a well-connected network-wide graph.Source: B. Related Work · pp. 2–3

C. Outline and Summary of Contributions •

  • The paper contributes convergence bounds for TT-HF and an adaptive controller that tunes aggregation intervals, D2D rounds, and learning rate to target O(1/t) convergence while balancing energy, delay, and accuracy.The bounds quantify effects of model, dataset, consensus, and aggregation properties; experiments also test popular learning tasks.Source: C. Outline and Summary of… · p. 3
  • Experiments report that TT-HF substantially outperforms federated learning with infrequent global aggregations in resource consumption and/or training time.The adaptive controller adjusts local and global aggregation periods to address resource limitations and data heterogeneity.Source: C. Outline and Summary of… · p. 3

A. Edge Network System Model

  • The system model partitions edge devices into clusters able to communicate through low-energy D2D links, with cluster topology evolving slowly relative to global-aggregation intervals.Devices within each cluster conduct consensus between server aggregations.Source: A. Edge Network System Model · p. 3

B. Machine Learning Task Model

  • The learning task defines global loss as an average across clusters with equal device contributions, and introduces gradient diversity to characterize statistical heterogeneity across local datasets.The analysis assumes a strongly convex global loss, but not strong convexity of every local loss; experiments demonstrate effectiveness with non-convex neural-network losses.Source: B. Machine Learning Task Model · p. 4

C. TT-HF: Two Timescale Hybrid Federated Learning

1) Overview and rationale: · 2) TT-HF procedure:

TT-HF combines local SGD, aperiodic within-cluster D2D consensus, and server-based global aggregation. The design aims to reduce local-data bias and communication energy while using consensus and global updates on distinct timescales.

1) Overview and rationale:

  • Local D2D consensus is motivated as a way to suppress local-data bias under non-i.i.d. data, with short-range communication expected to use less power than global uplinks.The paper also notes D2D's growing prevalence in 5G-and-beyond networks.Source: 1) Overview and rationale · p. 4

2) TT-HF procedure:

  • During each local training interval, devices perform successive SGD updates using mini-batches sampled from their own datasets and a positive step size.The local gradient estimate is calculated from the sampled mini-batch before the intermediate model update.Source: 2) TT-HF procedure · p. 5
  • Clusters may skip consensus or run multiple neighbor-to-neighbor D2D rounds; more rounds reduce consensus error but incur energy and delay costs.The number of rounds is selected through an adaptive control algorithm to manage this trade-off.Source: 2) TT-HF procedure · pp. 5–6
  • Global aggregation intervals are aperiodic, while local iterations use time t and consensus rounds use a second timescale, t′.Interval k spans Tk from tk−1 + 1 through tk, with duration τk = tk − tk−1.Source: 2) TT-HF procedure · pp. 4–5
  • The analysis assumes fixed cluster membership during each global aggregation interval and error-free transmissions, while numerical evaluation examines outages from fast fading and limited CSI.The fixed-membership assumption is described as practical when devices move slowly and remain in their clusters during local training.Source: 2) TT-HF procedure · p. 5, aggregation) are received with no… · p. 7
  • At each global aggregation, the server uniformly samples one device per cluster to upload its model, then aggregates and broadcasts the new global model to all devices.Sampling is enabled by within-cluster consensus and reduces uplink communication cost by a factor of the cluster sizes.Source: 2) TT-HF procedure · p. 6

III. CONVERGENCE ANALYSIS OF TT-HF · A. Definitions and Bounding Model Dispersion · B. General Convergence Behavior of ˆw(t)

The section analyzes TT-HF convergence by bounding dispersion among cluster models and relating it to the global model’s progress toward the optimum. It identifies how consensus error, SGD noise, and dataset heterogeneity affect convergence, and the conditions needed to target the O(1/t) rate of centralized SGD.

III. CONVERGENCE ANALYSIS OF TT-HF

  • Section III develops a theoretical analysis of TT-HF convergence, introducing definitions and a key proposition before presenting its main results.Source: §III · p. 7

A. Definitions and Bounding Model Dispersion

  • The consensus-error condition can be satisfied by tuning the number of consensus steps, and the analysis derives sufficient conditions for convergence.The stated conditions on the consensus-error bound are given in Proposition 1.Source: A. Definitions and Bounding Model… · p. 7
  • Model dispersion A(t) measures how far cluster models deviate from their average across clusters during training.The average is the global average of the local models at time t.Source: A. Definitions and Bounding Model… · p. 7
  • Proposition 1 upper-bounds expected model dispersion, which grows with initial consensus error, gradient-estimation noise, and local-data heterogeneity.The analysis uses a diminishing step size and a non-increasing consensus-error bound as conditions for the proposition.Source: A. Definitions and Bounding Model… · p. 7

B. General Convergence Behavior of ˆw(t)

  • Theorem 1 bounds the global model’s one-step loss gap to the optimum, with additional degradation from cluster dispersion, consensus errors, and SGD noise.Its bound applies when η_t ≤ 1/β; the theorem’s dispersion term is bounded using Proposition 1.Source: B. General Convergence Behavior of ˆw(t) · p. 7, A. Definitions and Bounding Model… · p. 7
  • The theorem’s centralized-gradient-descent progress term is offset by penalties from model dispersion, consensus errors, and SGD noise; without suitable control, the expected loss gap may diverge.The theorem applies throughout each update period, including at global aggregation.Source: B. General Convergence Behavior of ˆw(t) · pp. 7–8
  • To match centralized SGD’s O(1/t) asymptotic rate, TT-HF must keep both model dispersion A(t) and consensus error ε(t) of order O(η_t).The target follows from making the additional terms in the one-step bound the same order as degradation from SGD noise.Source: B. General Convergence Behavior of ˆw(t) · p. 8

C. Sublinear Convergence Rate of ˆw(t) · IV. ADAPTIVE CONTROL ALGORITHM FOR TT-HF · A. Learning-Related Parameters (α, γ, φ, ηt)

TT-HF’s convergence analysis identifies conditions for O(1/t) convergence, and its adaptive control algorithm tunes learning and communication parameters to balance model performance with network resource use.

C. Sublinear Convergence Rate of ˆw(t)

  • Under Theorem 2’s parameter conditions, TT-HF achieves O(1/t) convergence when ηt, ϵ(t), and gradient diversity satisfy the stated constraints.The conditions include γ > 1/µ, φ > 0, α ≥ αmin, ω < ωmax(α), and ϵ(t) = ηtφ.Source: C. Sublinear Convergence Rate of ˆw(t) · p. 8
  • Larger gradient diversity requires a smaller step size, since increasing α raises the tolerable ω but can slow convergence of the suboptimality gap.If ω exceeds ωmax(α), error propagation may cause divergence.Source: C. Sublinear Convergence Rate of ˆw(t) · p. 8
  • Longer local training intervals sharply increase ν, while consensus error has a quadratic impact; therefore, larger τ requires more accurate consensus to preserve a target ν.Consensus rounds reduce global aggregation frequency, saving uplink bandwidth and power.Source: C. Sublinear Convergence Rate of ˆw(t) · p. 8

IV. ADAPTIVE CONTROL ALGORITHM FOR TT-HF

  • The adaptive controller tunes local training intervals τk, step size ηt, and D2D rounds Γ(t), with the server setting learning and interval parameters and clusters controlling D2D rounds.The controller is based on Theorem 2 and operates across two timescales: at global aggregations and within device clusters.Source: §IV · p. 8

A. Learning-Related Parameters (α, γ, φ, ηt)

  • The learning-parameter procedure chooses γ > 1/µ and the smallest α meeting α ≥ αmin and ω < ωmax(α), since smaller α is associated with faster convergence.The procedure uses model, dataset, and SGD-noise properties; application latency determines a tolerable interval τ between global aggregations.Source: A. Learning-Related Parameters (α, γ,… · p. 9
  • A desired end-of-training loss ξ at duration T sets νmax = ξ(T + α); if the feasibility check fails, τ, ξ, and/or T must be loosened.Once α, γ, and τ are chosen, the method selects the largest feasible φ to reduce required D2D rounds.Source: A. Learning-Related Parameters (α, γ,… · p. 9
  • The server’s optimization balances average energy consumption, D2D delay, and expected model performance, while clusters trigger synchronization when model divergence exceeds a threshold.Future divergence values are not causally available, so the optimization estimates them with a time-series predictor.Source: A. Learning-Related Parameters (α, γ,… · pp. 9–10
  • The resource optimization is non-convex and integer-valued; it uses historical observations for linear regression and a line search over feasible integer values of τk.The stated complexities are O(τk−1) for regression and O(τmax) for the line search.Source: A. Learning-Related Parameters (α, γ,… · p. 10

1) Estimation of δ, ζ, σ2: … A. Experimental Setup

The sections explain how TT-HF estimates quantities for adaptive control, describe its server-and-device control loop, and specify the numerical evaluation setup. The reported comparisons indicate that more D2D consensus can improve performance and offset less frequent global aggregation.

1) Estimation of δ, ζ, σ2:

  • Because w* is unknown, the server estimates δ and ζ using δ′ ≥ δ + ζ∥w*∥, sets ζ < 2β, and selects the smallest δ′ satisfying the gradient-diversity bound.For σ², devices compare gradients from two independent mini-batches, estimate the noise locally, and report it; the server chooses the maximum reported value.Source: 1) Estimation of δ, ζ, σ2 · p. 10

2) Estimation of Υ(t) c

  • Devices estimate Υ_c(t) by distributed scalar message passing: sharing local norms and updating maxima and minima for more rounds than the graph diameter yields the estimate.The server obtains the estimates for t ∈ T_k from the sampled node in each cluster; the approximation uses a lower bound on vector differences that the authors report gives better estimates experimentally.Source: 2) Estimation of Υ(t) c · pp. 10–11

D. TT-HF with Adaptive Parameter Control

  • During local intervals, devices perform SGD, estimate cluster divergence, select D2D consensus rounds, and communicate with neighbors; at global aggregation, the server updates control parameters and broadcasts them with the model.The server uses device reports to estimate ζ, δ′, and σ², fit linear-model coefficients, and choose the next local-training interval length.Source: D. TT-HF with Adaptive Parameter Control · p. 11

A. Experimental Setup

  • The simulated network has 125 devices in 25 clusters of five, uniformly placed in separate 50 m × 50 m fields; D2D communication uses OFDMA and a fading channel model.The channel model includes large- and small-scale fading, with Rayleigh fading varying independently over time and assumed channel reciprocity.Source: A. Experimental Setup · p. 11
  • Clusters use outage-probability channel reliability at R_c = 14 Mbps, yielding connected within-cluster graphs with numerically observed average degree 2; fast-fading outages lose consensus packets, while topology may change between global aggregations.With d_c = 1/8, nodes can conduct D2D communication and local averaging without global coordination after the server broadcasts at each global aggregation.Source: A. Experimental Setup · p. 12

V. NUMERICAL EVALUATIONS

  • At fixed local-training interval τ, increasing D2D consensus rounds Γ considerably improves model accuracy/loss over time for non-i.i.d. data versus current art [16], [58]; with larger τ, TT-HF can still outperform baseline federated learning if Γ increases.Source: Figure 5 · p. 12, Figure 4 · p. 12

A. Experimental Setup

  • The evaluation uses MNIST and Fashion-MNIST with extreme non-i.i.d., moderate non-i.i.d., and i.i.d. device data distributions; the reported MNIST results are representative of qualitatively similar Fashion-MNIST results.Each dataset has 70,000 images across 10 labels; results use regularized squared SVMs and fully connected neural networks, with further results and non-convex-control details in Appendix G.Source: A. Experimental Setup · pp. 12–13

B. TT-HF Model Training Performance and Convergence

The experiments test whether cooperative cluster-level consensus during local training can preserve model performance while reducing global aggregation frequency, and/or improve training accuracy under statistical data heterogeneity.

B. TT-HF Model Training Performance and Convergence

  • TT-HF experiments test whether cluster-level consensus preserves model performance with fewer global aggregations or improves training accuracy under data heterogeneity.These are stated premises to be validated, not reported experimental outcomes.Source: B. TT-HF Model Training Performance… · p. 13

1) Local consensus reducing global aggregation frequency: … PRELIMINARIES AND NOTATIONS USED IN THE PROOFS

TT-HF combines global federated aggregation with cooperative D2D consensus to reduce reliance on server communication while addressing heterogeneous device data. The supplied results show accuracy, convergence, and resource tradeoffs, alongside proof notation and future research directions.

1) Local consensus reducing global aggregation frequency:

  • Increasing consensus rounds Γ counteracts longer local training intervals, allowing TT-HF to outperform τ = 20 federated learning with less frequent global aggregation despite fading-related packet loss.The baselines use full device participation and are 5x more uplink-resource-intensive per aggregation; the τ = 1 baseline replicates centralized training.Source: 1) Local consensus reducing global… · p. 13

2) D2D enhancing ML model performance:

  • D2D consensus improves accuracy or loss for moderate and extreme non-i.i.d. data, with diminishing returns as performance approaches τ = 1 federated learning; gains are absent for i.i.d. data.The results also demonstrate applicability to non-convex neural-network classifiers.Source: 2) D2D enhancing ML model performance · p. 13

3) Convergence behavior:

  • Excessively long local descent periods can cause poor convergence under extreme non-i.i.d. data, as increased τ and reduced Γ raise model dispersion and consensus error.The behavior is similar to baseline federated learning with τ = 50 and motivates tuning these parameters using the convergence result.Source: 3) Convergence behavior · p. 13, Figure 6 · p. 13

C. TT-HF with Adaptive Parameter Control / 1) Improved resource efficiency compared with baselines:

  • TT-HF reduces total cost by at least 75% versus both baselines, with energy savings and delay gains depending on D2D-to-global communication ratios.The comparison measures cost, energy, and delay up to 75% of peak accuracy; TT-HF no longer saves energy above a D2D energy ratio threshold, or improves delay at a delay ratio of about 0.1.Source: 1) Improved resource efficiency… · pp. 13–14, Figure 7 · p. 14

2) Impact of design choices on local model training interval:

D. Main Takeaways / VI. CONCLUSION AND FUTURE WORK

  • TT-HF reduces local-model bias through cluster consensus, and its analysis bounds convergence using gradient diversity, consensus error, and global aggregation periods, enabling adaptive tuning for O(1/t) convergence.The method can also benefit approaches beyond vanilla SGD, including preliminary experiments with FedProx.Source: §VI · p. 14, D. Main Takeaways · p. 14

VI. CONCLUSION AND FUTURE WORK

  • Future work includes modeling device computation differences, cluster-to-server delay variation, and D2D interference, and extending convergence analysis to non-convex settings and stationary-point guarantees.The paper identifies conditions and rates for approaching a stationary point as targets for that analysis.Source: §VI · pp. 14–15

PRELIMINARIES AND NOTATIONS USED IN THE PROOFS

  • The proof preliminaries define scaled quantities for strong convexity, gradient diversity, step size, SGD noise variance, and consensus error within clusters and across the network.They also introduce an instantaneous global model to track global-model variation.Source: PRELIMINARIES AND NOTATIONS USED IN… · p. 17

APPENDIX A PROOF OF PROPOSITION 1

The appendix proves an upper bound on expected model dispersion across clusters under stated assumptions and a non-increasing error sequence, using a three-part analysis of the coupled dynamics.

APPENDIX A PROOF OF PROPOSITION 1

  • Under Assumptions 1 and 3, the proposition gives an upper bound on expected model dispersion when η_t = γ/(t+α) and ϵ(t+1)/ϵ(t) ≤ 1.The supplied passage states the bound's existence but does not include its expression.Source: Appendix A · p. 17
  • The proof derives relationships for the mean-model error, solves the resulting coupled dynamic system, and connects it to A(t) to obtain the dispersion bound.The coupled system is solved using an eigen-decomposition of its matrix.Source: Appendix A · pp. 17, 21
  • For an intermediate bound, the proof assumes ˜η_0 ≤ ˜μ, which implies α ≥ ˜γ/˜μ.It also uses ˜η_t ≤ ˜η_0.Source: Appendix A · p. 18

APPENDIX B PROOF OF THEOREM 1

Appendix B proves Theorem 1’s one-step behavior bound for TT-HF under Assumptions 1–3 and step sizes η_t ≤ 1/β. The proof derives the bound by combining global-average model dynamics with smoothness, expectation, sampling, and strong-convexity arguments.

APPENDIX B PROOF OF THEOREM 1

  • Under Assumptions 1–3 and η_t ≤ 1/β for all t, Theorem 1 characterizes the one-step behavior of the TT-HF model.The supplied passage does not include the bound’s formula.Source: Appendix B · p. 23
  • The proof starts from the global average of local models’ dynamics and uses β-smoothness to bound the resulting change in the global function.It then takes conditional and unconditional expectations and invokes Lemma 2 to further bound the expression.Source: Appendix B · pp. 23–24
  • The derivation also uses uniform device sampling and strong convexity, then combines the resulting inequalities to complete the proof.The supplied passages do not expose the final inequality’s formula.Source: Appendix B · p. 25

APPENDIX C PROOF OF THEOREM 2

Appendix C proves Theorem 2 by induction, using bounds derived from Theorem 1 and verifying the required condition for the prescribed step-size and error schedules.

APPENDIX C PROOF OF THEOREM 2

  • The proof applies TT-HF under Assumptions 1–3 with η_t = γ/(t + α) and ϵ(t) = η_t φ for all t.The opening fragment also defines ω_max using Z1, though its full expression is not recoverable from the supplied passage.Source: Appendix C · p. 25
  • The proof establishes Theorem 2 by induction over global aggregations and the iterations within each aggregation, then concludes that the induction is complete.It starts at the first global aggregation, where the condition holds at t = 0, and uses the induction hypothesis to extend it through each aggregation interval.Source: Appendix C · pp. 25, 26, 29
  • To bound the error terms, the proof separately bounds two instances of [Σ_+,t]^2 and combines those bounds before completing the induction.It bounds the first instance by treating terms (i) and (ii), then bounds the second instance separately.Source: Appendix C · pp. 26–28
  • For the condition to hold for all t ≥ 0, the proof uses convexity to reduce verification to t = 0 and the limit t → ∞.The argument applies to the left-hand side of an inequality after expressing it using the prescribed schedule.Source: Appendix C · p. 28

APPENDIX D PROOF OF LEMMA 1

The proof establishes Lemma 1 by describing device-parameter evolution and bounding the consensus error using Assumption 2.

APPENDIX D PROOF OF LEMMA 1

  • The proof begins by expressing the evolution of device parameters through equation (12).This evolution is considered for c consensus rounds in cluster S_c with consensus matrix V_c.Source: Appendix D · p. 29
  • A matrix represents the average model parameters across the cluster, with its rows corresponding to those averages.The passage also identifies E(t)_{i,:} as the ith row of E(t).Source: Appendix D · p. 29
  • Using Assumption 2, the proof bounds the consensus error and concludes that Lemma 1 follows.Source: Appendix D · p. 30

APPENDIX E PROOF OF LEMMA 2

The appendix proves Lemma 2 under Assumption 1 using a vector norm identity and inequalities from convexity, smoothness, and strong convexity.

APPENDIX E PROOF OF LEMMA 2

  • The proof applies a vector identity to rewrite −2a⊤b in terms of squared norms, then uses Jensen’s inequality for the convex squared-norm function.The identity is stated for any two real-element vectors.Source: Appendix E · p. 30
  • It then invokes β-smoothness of Fj and µ-strong convexity of F to derive the result and conclude the proof.The supplied passages do not include the complete intermediate inequalities or the lemma’s conclusion.Source: Appendix E · p. 30

APPENDIX F PROOF OF FACT 1

Fact 1 is stated for an arbitrary set of n random variables, and its proof uses algebraic manipulations including the Cauchy–Schwarz inequality.

APPENDIX F PROOF OF FACT 1

  • Fact 1 is stated for an arbitrary set of n random variables.The supplied passage does not include the fact’s formula.Source: Appendix F · p. 31
  • The proof proceeds through algebraic manipulations and invokes Cauchy–Schwarz, producing an E[X2]E[Y 2] term.Source: Appendix F · p. 31

APPENDIX G ADDITIONAL EXPERIMENTAL RESULTS · A. Complimentary Experiments of the Main Text · B. Extension to Other Federated Learning Methods

Additional experiments show that TT-HF’s D2D consensus benefits persist across datasets, models, and FedProx, while very long local-training intervals can impair convergence. In the non-convex FCN setting, a fixed-step-size workaround still yields comparable total-cost reductions to federated-learning baselines.

A. Complimentary Experiments of the Main Text

  • The additional experiments use FMNIST and fully connected networks, and their results are qualitatively similar to the main-text SVM/MNIST cases.FMNIST has 60,000 training and 10,000 test images, each a 28x28 grayscale image from one of 10 classes.Source: A. Complimentary Experiments of the… · p. 32
  • Because Algorithm 2 is not directly applicable to non-convex FCN loss, these experiments skip its control steps and use a fixed step size, yet achieve comparable total-cost reductions to federated-learning baselines.A constant φ is used to calculate the Γ values using (30).Source: A. Complimentary Experiments of the… · p. 32

B. Extension to Other Federated Learning Methods

  • Across FMNIST/SVM, MNIST/neural-network, and FMNIST/neural-network cases, increasing Γ improves accuracy/loss at fixed τ, and increasing Γ can offset a larger τ.Source: Figure 9 · p. 33, Figure 10 · p. 34, Figure 14 · p. 36
  • When Γ is small in extreme non-i.i.d. settings, substantially increasing τ causes poor convergence because of model dispersion.Source: Figure 11 · p. 34, Figure 16 · p. 37, Figure 20 · p. 39
  • Upon reaching 75% of peak accuracy, TT-HF obtains significantly lower total cost than baselines, with energy savings and delay gains in some configurations.Source: Figure 12 · p. 35, Figure 17 · p. 38, Figure 21 · p. 40
  • Higher energy and delay weights prolong the second local-training interval, while a higher global-model-loss weight shortens it and prompts more rapid global aggregation.The default weights are c1 = 10^-3, c2 = 10^2, and c3 = 10^4.Source: Figure 13 · p. 35
  • With the same local-training period, increasing Γ considerably improves TT-HF accuracy/loss over FedProx on non-i.i.d. MNIST with a neural network [65].The comparison varies the number of D2D consensus rounds.Source: Figure 22 · p. 40
  • With a larger τ, TT-HF can still outperform FedProx if Γ increases, letting local D2D communications offset less frequent global aggregations [65].Source: Figure 23 · p. 41
Loading 2103.10481v3…