PaperlayerView exact revision on arXiv

Source-linked AI summary

Resilient Decentralized Wireless Federated Learning via Gradient Tracking with AdamW

Nguyen Van Thieu, Ti Ti Nguyen, Ons Aouedi, Vu Nguyen Ha, Symeon Chatzinotas

arXiv:2608.25535v1cs.LGcs.DCcs.NI

TL;DR

Wireless DecL must handle heterogeneous data and unreliable, resource-constrained links without excessive communication overhead. QEF-GT-AdamW combines gradient tracking, AdamW, error-feedback quantization, and packet-loss fallback, consistently improving convergence and robustness over representative baselines on MNIST and CIFAR-10.

  • Problem

    Wireless DecL faces coupled challenges from non-IID data, limited bandwidth, strict airtime budgets, fading, and packet losses.

  • Method

    QEF-GT-AdamW combines gradient tracking, AdamW updates, dual-stream biased quantization with error feedback, and local fallback under unreliable wireless communication.

  • Results

    QEF-GT-AdamW outperforms representative baselines in convergence and robustness, approaching full-precision MNIST accuracy at approximately 30% Top-K sparsification versus around 90% for QGT-AdamW.

  • Takeaways & Limitations

    The method provides favorable accuracy-communication trade-offs and particularly strong convergence and robustness in resource-limited wireless environments.

Abstract

from arXiv · show

Wireless Internet-of-Things (IoT) edge networks require decentralized learning (DecL) methods that can operate reliably under both heterogeneous local data and communication-constrained wireless links. However, existing decentralized optimization schemes often incur substantial communication overhead and degraded performance when transmissions are constrained by strict airtime budgets, fading channels, and packet losses. This paper proposes QEF-GT-AdamW, a communication-efficient and outage-resilient algorithm for DecL over wireless communication (WCom) networks. The proposed method combines gradient tracking to mitigate the effect of non-IID data, AdamW-based adaptive optimization to improve training stability, and dual-stream biased quantization with error feedback to reduce communication payloads for both model and tracking exchanges. To address unreliable broadcast communication, the proposed framework further employs a local fallback strategy when scheduled packets are not successfully received. We explicitly model the effect of bandwidth, transmit power, airtime constraints, and fading channels on DecL performance, and establish convergence guarantees for the proposed algorithm under compressed and unreliable wireless communication. Experimental results on heterogeneous MNIST and CIFAR-10 settings show that QEF-GT-AdamW consistently improves robustness and convergence performance over representative DecL baselines while achieving favorable accuracy-communication trade-offs under limited wireless resources.

I. INTRODUCTION … 1) Time-Slot based Learning and Information Exchange Framework:

The paper frames decentralized learning over wireless IoT networks as a solution for distributed model training, while highlighting challenges from heterogeneous data and unreliable, resource-constrained communication. It then introduces QEF-GT-AdamW and its time-slot communication framework, combining gradient tracking, AdamW, quantization with error feedback, and outage fallback.

  • I. INTRODUCTION: Wireless DecL trains models across distributed IoT devices without a central coordinator but faces non-IID data, limited bandwidth, strict airtime budgets, fading, and packet losses.These wireless constraints are coupled because communication limitations directly affect decentralized training.
  • I. INTRODUCTION: Gradient tracking mitigates client drift under heterogeneous data by enabling nodes to asymptotically follow the network-wide gradient.Classical gradient tracking requires repeated exchanges of model and tracking variables, creating substantial wireless communication overhead.
  • I. INTRODUCTION: QEF-GT-AdamW combines gradient tracking, AdamW updates, dual-stream quantization with error feedback, and local fallback to reduce payloads and improve resilience to failed packet reception.The quantization targets exchanges of both model and tracking variables, while fallback supports operation when neighbor packets are not successfully received.
  • II. SYSTEM MODEL: The system model uses N nodes that exchange information only with neighbors over wireless channels while minimizing average empirical loss under non-IID data distributions.Each node holds a local dataset and participates in decentralized optimization without a central coordinator.
  • 1) Decentralized GT Learning:: Each node maintains a local model and uses a fixed doubly-stochastic mixing matrix in classical gradient-tracking updates to support convergence toward the global objective.The tracking variables are designed to follow the network-wide gradient.
  • 2) Adaptive Optimization and AdamW:: AdamW decouples weight decay from gradient-based updates, addressing Adam’s inconsistent regularization and supporting more stable optimization in decentralized settings.Adam uses exponential moving averages of first- and second-order gradients to accelerate convergence, but coupling weight decay with adaptive updates can reduce generalization.
  • 1) Time-Slot based Learning and Information Exchange Framework:: A synchronized time-slot protocol schedules local computation and transmission over orthogonal subcarriers under limited airtime, unreliable links, and a strict deadline ∆t(iter)max.The protocol governs training and communication in each time slot.
  • 1) Time-Slot based Learning and Information Exchange Framework:: Slower-computing devices have less remaining transmission time, increasing their risk of outage in the wireless time-slot framework.The model also incorporates pathloss based on node distance and small-scale fading.

2) Channel Model and Outage Probability:

The channel model uses Rician fading, while outage probability depends on transmission power, bandwidth, noise, and an iteration-specific SNR threshold. The threshold is determined by exchanged information and available airtime, with payload size depending on model dimension and compression.

  • Channel model: The channel gain h_ij follows a Rician distribution with factor κ ≥0.
  • Channel model: The average SNR from node i to node j is determined by transmission power, bandwidth, channel-related ν_ij, and noise spectral density N_0.
  • Outage probability: Outage probability at iteration k is estimated using the first-order Marcum Q-function and the node’s SNR threshold γ_th,i,k.
  • Outage probability: The SNR threshold γ_th,i,k is determined by the exchanged information amount and available airtime through the transmission-rate expression.
  • Outage probability: The transmitted information amount D_i depends on model dimension d and the compression scheme.

III. PROPOSED ALGORITHM: QEF-GT-ADAMW

QEF-GT-AdamW combines robust consensus mixing, AdamW adaptive updates, gradient tracking, and dual-stream quantization with error feedback for model and tracking exchanges under packet losses and heterogeneous data.

  • III. Proposed Algorithm: QEF-GT-AdamW: QEF-GT-AdamW integrates packet-loss-robust consensus mixing, AdamW-based adaptive updates, gradient tracking for heterogeneous data, and dual-stream quantization with error feedback.The quantization and error-feedback mechanisms apply to both model and tracking exchanges.

A. Initialization and Robust Mixing

The algorithm warm-starts each node’s model, gradient-tracking state, AdamW buffers, and error-feedback residuals before compressing and broadcasting initial variables. During communication, sequence filtering and self-loop reassignment preserve robust stochastic mixing under packet losses.

  • Packet Handling: Node i accepts neighbor j’s decoded packet only when its sequence number exceeds LastSeqi[j]; otherwise, the packet is treated as missing.This sequence-number check prevents duplicate processing of packets from potential in-neighbors.
  • Initialization: Each node warm-starts its local model and tracking variables, initializes AdamW moment buffers and pre-initialization error-feedback residuals to zero, then compresses both streams for broadcast.The warm start begins from x0_i, computes g0_i, sets y0_i, and applies Qx and Qy before the first communication round.
  • Robust Mixing: When packets are lost, missing neighbor weights are reassigned to the self-loop so the effective mixing matrix remains row-stochastic.The outage indicator determines successful packet acceptance, while self-loop reassignment preserves stochastic mixing.
  • Robust Mixing: The resulting robust mixed X and Y variables provide consensus states for the subsequent local adaptive update and gradient-tracking steps.The compact matrix form describes the robust mixing operation for the stacked model and tracking variables, with an analogous expression for Y.

B. AdamW Update, GT, and Dual Quantization with EF

This section defines the per-round QEF-GT-AdamW routine: nodes mix received states with local fallback, perform AdamW and gradient-tracking updates, then compress model and tracking variables with separate error-feedback compressors before broadcasting. Sequence counters and packet reception logic ensure that only newly received packets are decoded, while missing packets are handled locally.

  • AdamW Update and Gradient Tracking: Nodes use the mixed tracking variable Yk_i as a robust estimate of the global gradient before updating local AdamW moments and applying bias correction.The moment updates use momentum parameters βm and βv, with the primal model subsequently updated using AdamW.
  • Reception and Local Fallback: For each neighbor, nodes decode only packets with successful reception and newer sequence numbers; missing packets are treated as unavailable and handled through robust mixing with local fallback.The reception procedure checks ak_j→i and compares received Seqj against LastSeqi[j] before decoding payloads.
  • Broadcast Routine: After compression, each node increments its local sequence counter and broadcasts the sequence number together with the compressed update to neighbors.The complete per-round routine includes reception, mixing, AdamW, gradient tracking, error-feedback compression, and broadcast.
  • AdamW Update and Gradient Tracking: After updating the primal model, each node computes its new local gradient and updates the tracking variable accordingly.The routine explicitly separates the AdamW model update from the subsequent gradient-tracking update.
  • Dual Quantization with Error Feedback: To reduce wireless communication overhead, nodes compress both model and tracking variables using separate compressors Qx and Qy with error feedback.The compression routine applies the error-feedback recursion separately for z ∈ {x, y} before transmission.

C. Convergence Analysis

The convergence analysis accounts for row-stochastic, packet-dependent mixing through a Perron-weighted objective rather than the uniform network average. Under stated regularity, compression, mixing, independence, and bounded-state assumptions, QEF-GT-AdamW achieves an O(1/K) convex rate and stronger PL-condition guarantees depending on perturbation decay.

  • Reweighted Objective and Notation: Packet-dependent fallback makes B_k row-stochastic rather than generally doubly stochastic, so convergence is governed by Perron weighting.The uniform network average is not preserved; in the doubly-stochastic case, the Perron-weighted objective reduces to the original objective.
  • Assumptions: The analysis assumes convex L-smooth local objectives, a minimizable Perron-weighted aggregate, biased compression conditions, primitive random mixing, delivery-state independence, and bounded states.These assumptions support the communication-error bounds and convergence results for compressed, unreliable wireless communication.
  • Corollary 1: Linear rate under the PL condition: Under the PL condition and 0 < α ≤ 1/(4L), bounded perturbations yield lim sup K→∞ Δ_K ≤ 5 Ē/(4μ_PL), while vanishing perturbations give exact convergence.If E_k = O(γ^k), convergence is linear with rate max{ρ, γ} for γ ≠ ρ, and Δ_K = O(Kρ^K) when γ = ρ.
  • Remarks: Model-stream compression enters through C_k^x, whereas tracking-stream compression is captured by T_k; the PL condition is not required for the general convex result.Theorem 1 applies directly to smooth convex objectives, including the logistic regression setting used in the experiments.

IV. EXPERIMENT RESULTS AND DISCUSSION

The experiments simulate a 15-node decentralized wireless network with specified spatial, connectivity, and channel settings. Evaluation includes logistic regression on a smooth convex problem with Top-K biased compression and non-IID MNIST/CIFAR10 partitions.

  • Experimental Setup: The simulated network contains N = 15 nodes deployed across a 2000 × 2000 m2 area using Poisson-disk sampling with dmin = 250 m.Nodes are placed under a minimum-distance constraint.
  • Experimental Setup: Directed wireless links connect nodes within rmax = 750 m, with default transmit power Pi = 0.2 W and channel parameters following.These settings define the communication environment for the experiments.
  • Experimental Setup: The smooth convex evaluation uses logistic regression with Top-K biased compression on non-IID MNIST and CIFAR10 dataset partitions.The figure reports the non-IID partitioning of both datasets.

A. Datasets and Model

The study evaluates decentralized learning on heterogeneous MNIST and CIFAR-10 datasets over 1000 epochs, using distinct non-IID partitioning schemes. It compares QEF-GT-AdamW with CHOCO-SGD, GT-AdamW, and QGT-AdamW under specified AdamW and CHOCO-SGD training settings.

  • Datasets: Experiments use 50,000 training samples from MNIST and CIFAR-10, together with standard test sets, over 1000 epochs.The samples are distributed across N local datasets.
  • Datasets: MNIST uses pathological label-skew partitions with 2–5 labels per node, whereas CIFAR-10 uses Dirichlet partitioning with concentration α = 0.5.Both datasets are distributed non-IID across local nodes.
  • Baselines: The comparison includes CHOCO-SGD, GT-AdamW, and QGT-AdamW as representative decentralized learning baselines.CHOCO-SGD uses quantized communication; GT-AdamW combines classical gradient tracking with AdamW; QGT-AdamW uses quantized gradient tracking with AdamW.
  • Training settings: AdamW-based methods use η = 0.005 for MNIST and η = 0.001 for CIFAR10, with βm = 0.9 and βv = 0.999.The supplied training configuration specifies dataset-dependent learning rates for AdamW-based methods.
  • Evaluation dimensions: The study examines Top-K density through test-accuracy and training-loss convergence curves for MNIST and CIFAR-10.MNIST appears in the first row and CIFAR-10 in the second row of Fig. 3.
  • Wireless evaluation: Wireless evaluations report final average accuracy and average packet-drop rate versus bandwidth and transmission power under deadlines and outage-based drops.These measurements are shown separately against bandwidth and transmission power.

B. Results and Discussion

Under perfect communication, QEF-GT-AdamW outperforms QGT-AdamW and approaches full-precision accuracy with much less sparsification. Under wireless constraints, compressing both model and tracking streams substantially reduces packet drops, while error feedback improves loss and optimization quality.

  • Quantization and Error Feedback: QEF-GT-AdamW consistently outperforms QGT-AdamW across all Top-K sparsifications under perfect communication.GT-AdamW provides the full-precision reference because it incurs no quantization error.
  • Quantization and Error Feedback: 30% for MNIST and 35% for CIFAR-10 are the approximate Top-K sparsifications where QEF-GT-AdamW approaches full-precision accuracy.QGT-AdamW requires around 90% Top-K sparsification for comparable accuracy.
  • Quantization and Error Feedback: QEF-GT-AdamW yields uniformly lower average training loss than QGT-AdamW at the same Top-K sparsification.The results attribute this improvement to error feedback compensating quantization bias.
  • Wireless Packet Drops: 67% at 0.05 MHz, roughly 10% at 0.1 MHz, and essentially zero from 0.2 MHz onward are the packet drop rates reported for both quantized GT variants.At power levels, the rates are about 8% at 0.005 W, around 1% at 0.01 W, and nearly zero for 0.02 W and above; both model and tracking streams are compressed before transmission.

V. CONCLUSION

The paper proposes QEF-GT-AdamW, a resilient and communication-efficient decentralized learning algorithm for unreliable wireless edge networks. Experiments on MNIST and CIFAR-10 show improved convergence speed and robustness over representative baselines, especially under resource constraints.

  • V. CONCLUSION: QEF-GT-AdamW is proposed for resilient, communication-efficient decentralized learning in unreliable wireless edge networks.The method combines AdamW, gradient tracking, and dual biased quantization-based error feedback.
  • V. CONCLUSION: AdamW and gradient tracking stabilize optimization against non-IID data, while dual biased quantization-based error feedback minimizes communication payloads.
  • V. CONCLUSION: Experiments on MNIST and CIFAR-10 show that QEF-GT-AdamW outperforms representative baselines in convergence speed and robustness, particularly in resource-limited environments.
Loading 2608.25535v1…