PaperlayerView exact revision on arXiv

Source-linked AI summary

A Joint Power-Privacy Control Framework for Decentralized Learning over Heterogeneous Wireless Multicasting Networks

Amir Ziaeddini, Yauhen Yakimenka, Joerg Kliewer

arXiv:2609.03245v1cs.ITmath.IT

TL;DR

Exchanging model information in decentralized learning can expose private data, while wireless networks add asymmetric channels and limited transmit power. This paper combines row-stochastic wireless aggregation with power allocation between model updates and Gaussian privacy noise. Its CIFAR-10 comparisons report higher accuracy than PED2FL under matched privacy constraints, and the analysis establishes an O(log T) regret bound.

  • Problem

    Exchanging model coefficients in decentralized learning can expose private data, creating a need to protect privacy during collaborative training.

    Source: §I · p. 1, B. Differential Privacy · p. 3

  • Method

    The framework uses row-stochastic wireless aggregation and jointly allocates transmit power between model updates and Gaussian differential-privacy noise.

    Source: §I · p. 2, §VII · p. 10

  • Results

    In fully connected 4-client tests, the method consistently achieved higher test accuracy than PED2FL at ϵmax=1 and ϵmax=0.5 under IID and non-IID data; theory establishes O(log T) regret.

    Source: §I · p. 2, B. Results and Discussion · p. 9

  • Takeaways & Limitations

    CIFAR-10 experiments show robust convergence across client counts, data distributions, privacy budgets, and network topologies, with connectivity affecting the impact of DP noise.

    Source: §VII · p. 10

Abstract

from arXiv · show

In this paper, we propose a decentralized learning framework that incorporates both power control and privacy guarantees. Specifically, we enable a set of clients in a wireless multicast network to jointly train a common model while maintaining a prescribed per-iteration maximum privacy leakage level. The communication network is represented by a rowstochastic adjacency matrix, allowing us to capture asymmetric channel gains as well as heterogeneous maximum transmit power levels. Differential privacy is enforced through an explicit powersplitting strategy that allocates each node's limited maximum transmit power between model coefficients and injected Gaussian noise, thereby jointly controlling learning performance and privacy leakage. We further prove that the proposed algorithm achieves a cumulative regret bound of O(logT), whereTdenotes the time horizon. To evaluate the practical performance of our approach, we perform comprehensive experiments on the CIFAR-10 dataset under both IID and non-IID data distributions, considering different privacy levels, diverse numbers of clients, and various graph topologies. The results demonstrate strong performance across the considered settings and improved performance over existing methods, highlighting the effectiveness of the proposed algorithm under realistic wireless communication constraints.

I. INTRODUCTION

The introduction motivates decentralized learning over asymmetric wireless multicast networks, where privacy and power constraints complicate communication and learning. It proposes wireless-aware row-stochastic mixing and joint power-privacy control, with theoretical guarantees and CIFAR-10 evaluation.

I. INTRODUCTION

  • Exchanged model coefficients can expose sensitive data, motivating differential privacy as a way to protect users during decentralized learning.Source: §I · p. 1
  • Existing DP-DL approaches often assume balanced graphs or omit wireless-aware power allocation, leaving privacy-accuracy control under asymmetric channel conditions unresolved.Wireless links have heterogeneous gains and power limits, while earlier row-stochastic approaches do not model these constraints.Source: §I · pp. 1–2
  • The framework derives row-stochastic aggregation weights from heterogeneous channel gains and transmit-power allocation, then optimizes power to balance learning performance and privacy leakage.Clients use the limited power budget for model coefficients and privacy noise while maintaining a predefined DP level.Source: §I · p. 2
  • O(log T) cumulative regret is established alongside differential-privacy guarantees for the effects of power control, noise injection, and network characteristics.Source: §I · p. 2
  • CIFAR-10 experiments span fully connected, ring, and random topologies, different client counts, IID and non-IID data, and privacy levels.The evaluation also compares performance against existing methods and analyzes cumulative privacy leakage.Source: §I · p. 2

A. Wireless DL via OAC

  • In wireless decentralized SGD, row-stochastic mixing biases the standard update; scaling each node’s gradient by 1/π_i removes this bias and targets the global objective.Over-the-air computation uses wireless signal superposition to aggregate model updates.Source: A. Wireless DL via OAC · p. 3

B. Differential Privacy

Exchanging model information can expose private data, so the section introduces differential privacy and calibrated noise as protections for decentralized learning. It describes privacy parameters, sensitivity, and the Gaussian mechanism used to limit leakage.

B. Differential Privacy

  • Exchanged model coefficients may reveal sensitive details about local datasets, motivating privacy protection in decentralized learning.Source: B. Differential Privacy · p. 3
  • A smaller ϵ means stronger privacy, while δ denotes the probability that the privacy guarantee may fail.Source: B. Differential Privacy · p. 3
  • Sensitivity measures the maximum output change from modifying one dataset element and determines the noise required for privacy.The Gaussian mechanism adds noise with variance proportional to sensitivity.Source: B. Differential Privacy · p. 3
  • Differential privacy adds calibrated randomness so outputs from neighboring datasets remain statistically indistinguishable within specified privacy parameters.The paper provides guarantees for each transmitted model update and communication round.Source: B. Differential Privacy · p. 3

III. SYSTEM MODEL

The system models decentralized learning over a strongly connected, unbalanced directed wireless graph, with clients exchanging model updates while keeping raw data local. Each client divides its limited transmit power between model coefficients and Gaussian DP noise, and wireless superposition combines their transmissions.

III. SYSTEM MODEL

  • Each node keeps its dataset local and exchanges model coefficients with neighbors over a strongly connected, time-invariant, potentially unbalanced directed graph.The analysis assumes bidirectional links, but their directed weights need not be equal.Source: §III · p. 4
  • The channel model uses real-valued gain magnitudes, assuming perfect channel state information and phase pre-compensation to simplify analysis.The channel coefficient includes a phase shift, but only its magnitude is retained in the model.Source: §III · p. 4
  • Each node splits its power budget between model coefficients and zero-mean Gaussian noise, using αi and βi = 1 − αi to balance learning performance and privacy.The split is subject to a maximum average transmit power per coordinate.Source: §III · p. 4
  • Neighbors multicast simultaneously, and each receiver obtains their superposed signals through over-the-air computation.The received model and noise components are scaled by heterogeneous channel gains and transmit powers; additive thermal noise is omitted for simplicity.Source: §III · p. 5

IV. PROPOSED ALGORITHM

The algorithm selects power fractions subject to a per-link privacy threshold, then uses wireless aggregation and decentralized updates to train the model. Local normalization yields row-stochastic mixing weights, while an auxiliary update supports convergence analysis.

IV. PROPOSED ALGORITHM

  • The algorithm chooses the largest feasible model-power fractions under a prescribed privacy threshold, balancing stronger model signals against increased leakage.With noise standard deviation proportional to the learning rate, the privacy constraint becomes time-independent and the power fractions are fixed throughout training.Source: §IV · p. 5
  • Each node multicasts its privacy-protected model-and-noise signal, then updates its coefficients using the received wireless aggregate and a stochastic-gradient adjustment.The update also uses the auxiliary variable zii,t; γt is the time-decreasing learning rate.Source: §IV · pp. 5–6
  • Local channel-based normalization makes the neighboring-model weights and self-weight sum to one, producing row-stochastic mixing without centralized computation.Nodes estimate channel magnitudes locally, while transmit powers and scaling coefficients are configured offline or exchanged during initialization.Source: §IV · p. 6
  • The auxiliary vectors are updated linearly from one-hot initializations, and each zii,t converges to the corresponding left Perron eigenvector entry.The update procedure repeats until convergence.Source: §IV · p. 6
  • The convergence analysis assumes strongly convex local costs, a nonempty closed bounded convex constraint set, and uniformly bounded stochastic gradients.These are the stated conditions for analyzing the decentralized update rule.Source: §IV · p. 6
  • Cumulative regret measures accumulated optimization error over the training horizon, even though the objective is formulated as a static offline problem.The algorithm’s inputs include the horizon, power budgets, and maximum privacy threshold.Source: §IV · p. 7

V. THEORETICAL ANALYSIS

The section analyzes the proposed algorithm’s privacy guarantees and convergence under Gaussian DP noise and heterogeneous network settings.

V. THEORETICAL ANALYSIS

  • The theoretical analysis addresses the proposed algorithm’s privacy guarantees.The section introduces analysis of privacy and convergence under Gaussian DP noise and heterogeneous networks.Source: §V · p. 7
  • The section also analyzes convergence of the proposed algorithm under the stated network and noise setting.Source: §V · p. 7
  • The analysis is scoped to Gaussian DP noise and heterogeneous network settings.Source: §V · p. 7

A. Privacy

The privacy analysis gives a per-iteration (ϵij, δ)-DP guarantee for each node relative to its neighbor. Under the specified noise scaling, its privacy constraints become linear, yielding a linear-program formulation for selecting power fractions.

A. Privacy

  • The proposed algorithm achieves (ϵij, δ)-DP for node i with respect to neighbor j at each iteration.The leakage term quantifies the change in node i’s received signal when one sample in node j’s dataset changes.Source: A. Privacy · p. 7
  • At receiver i, independently injected Gaussian noises from neighboring transmitters combine through the wireless channels to mask their contributions.Their aggregate noise is scaled by each transmitter’s noise-power fraction, power budget, and channel magnitude.Source: A. Privacy · p. 7
  • When σk,t = ρkγt and βk = 1 − αk, the privacy constraints become linear inequalities in the αk, so the power-allocation problem is a linear program.The common learning-rate factor cancels from the privacy expression.Source: A. Privacy · p. 7

B. Privacy Composition and Accounting

The privacy-accounting discussion gives a per-iteration local DP guarantee and describes subsampled RDP accounting. The convergence analysis establishes O(log T) expected regret and vanishing average regret, with privacy noise and network conditions affecting the bound's constants.

B. Privacy Composition and Accounting

  • The privacy analysis states a per-iteration local DP guarantee and uses a subsampled RDP accountant for cumulative privacy accounting.Poisson subsampling with probability q converts an (ϵij, δ)-DP mechanism to (log (1 + q(eϵij −1)), qδ)-DP.Source: B. Privacy Composition and Accounting · pp. 7–8

C. Convergence Rate

  • The proposed algorithm achieves expected regret of order O(log T) under Assumptions 1 to 3 with a diminishing step size.The theorem's bound uses the row-stochastic mixing lemma.Source: C. Convergence Rate · p. 8
  • The O(log T) rate follows from strong convexity and a diminishing learning rate, while privacy noise, power-splitting coefficients, transmit powers, and channel gains affect the bound's constants.The constants use worst-case system parameters and are conservative, but this does not change the regret rate.Source: C. Convergence Rate · p. 8
  • The algorithm achieves vanishing average regret, establishing convergence in the time-averaged objective (ergodic) sense.Source: C. Convergence Rate · p. 8

VI. SIMULATION RESULTS

The simulations evaluate the algorithm across channel, privacy, data-distribution, client-count, and topology settings, and compare it with existing methods.

  • The simulations compare the algorithm with existing methods.Source: §VI · p. 9
  • The experiments examine IID and non-IID data distributions and different numbers of clients.Source: §VI · p. 9
  • The evaluation explores heterogeneous channel gains, injected DP noise, and communication topology.Source: §VI · p. 9

A. Simulation Setup

The simulations evaluate CIFAR-10 learning across client counts, graph and data regimes, and privacy budgets, comparing the proposed method with PED2FL and examining accuracy alongside cumulative RDP leakage. The paper also reports an O(log T) regret bound and robust convergence across the tested settings.

A. Simulation Setup

  • CIFAR-10 experiments use ResNet-20, 80/20 training-test splits, IID or Dirichlet(α=1) client partitions, and privacy budgets εmax ∈ {0.5, 1, ∞}.The tested client counts are K = 4, 10, and 20; the topologies are fully connected and random, with a ring used for K = 4.Source: A. Simulation Setup · p. 9

B. Results and Discussion

  • PED2FL’s per-iteration leakage sets the shared εmax constraint before the proposed method’s power-allocation factors are optimized for comparison.The comparison adopts PED2FL’s channel-coefficient setting.Source: B. Results and Discussion · p. 9
  • Under both IID and non-IID data, the proposed method achieves higher test accuracy than PED2FL at εmax = 1 and εmax = 0.5 in a fully connected four-client network.The comparison uses the same maximum privacy leakage constraint.Source: B. Results and Discussion · p. 9, Figure 2 · p. 9, Figure 3 · p. 9
  • Across K = 4, 10, and 20, IID generally yields higher accuracy than non-IID, stronger privacy lowers accuracy, and fully connected graphs generally outperform ring and random topologies.Cumulative RDP leakage generally rises with client count; at a fixed client count, it mainly reflects the number of training iterations.Source: B. Results and Discussion · p. 10, Figure 4 · p. 10, Figure 5 · p. 10

VII. CONCLUSION

  • The framework uses row-stochastic mixing and jointly allocates transmit power to model updates and Gaussian DP noise, with a proved O(log T) regret bound.Experiments report robust convergence across client counts, data distributions, privacy budgets, and topologies.Source: §VII · p. 10

APPENDIX A PROOF OF CONVERGENCE

Appendix A bounds consensus error and the noise-related terms in expected regret. It combines these bounds to establish the upper bound stated in Theorem 2.

APPENDIX A PROOF OF CONVERGENCE

  • The proof obtains an expected consensus-error bound by combining inequalities and bounding terms using zero-mean Gaussian noise and stated assumptions.It multiplies the resulting inequality by π_i and sums over nodes.Source: Appendix A · p. 11
  • Lipschitz continuity and bounded subgradients relate local losses to node regret, which is summed over T iterations to bound expected regret.Source: Appendix A · p. 12
  • The proof bounds noise-related terms, substitutes the bounds into expected regret, and collects the result into constants U1 and U2 for Theorem 2.The displayed intermediate bounds include (1 + log T) terms.Source: Appendix A · p. 12
Loading 2609.03245v1…