PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaboration

Xiao Ma, Hong Shen, Hui Tian, Wenqi Lyu, Wei Ke

arXiv:2609.07312v1cs.LGcs.CRcs.DC

TL;DR

Decentralized personalized federated learning must preserve client-specific optimization while coping with Byzantine neighbors whose perturbations can be hard to distinguish from legitimate update diversity. R-DPFL combines robust neighborhood aggregation, history-based prediction, and adaptive clipping to make collaboration a controlled correction to local updates. The paper reports bounded personalized stationarity in theory and robust, stable personalized performance in heterogeneous decentralized experiments.

  • Problem

    Heterogeneous honest clients can produce substantially different updates, making it difficult to distinguish Byzantine perturbations from legitimate personalized differences.

    Source: C. Robust Neighborhood Direction… · p. 5

  • Method

    R-DPFL combines robust neighborhood aggregation, historical update prediction, and adaptive clipping to use neighbor information as a bounded correction to each client’s local descent direction.

    Source: §I · p. 2

  • Results

    The analysis establishes bounded personalized stationarity, and experiments under heterogeneous decentralized settings and multiple Byzantine attacks show strong robustness and stable personalized performance.

    Source: §VII · p. 14

  • Takeaways & Limitations

    Neighbor information can serve as a controlled correction to personalized optimization without requiring neighboring clients to converge to the same model.

    Source: §I · p. 2

Abstract

from arXiv · show

This paper proposes a robust decentralized personalized federated learning method R-DPFL, that enables clients to reduce the impact of Byzantine attacks via robust neighborhood direction estimation and history-based update trend prediction, rather than purely aggregating client models as in the existing work. In R-DPFL, each client first computes the current-round model update by aggregating the received neighborhood update vectors. It then predicts what this update should be based on its historical values and local model changes. Finally, R-DPFL computes the difference between these two quantities, adaptively clips this difference, and adds it to the local update. We prove convergence of the learning process through rigorous analysis and show that honest clients maintain stable personalized descent dynamics under Byzantine neighbor perturbations without requiring consensus among neighboring models. Extensive experiments on CIFAR-10 demonstrate that RDPFL consistently outperforms state-of-the-art decentralized and personalized federated learning baselines under heterogeneous and adversarial settings.

I. INTRODUCTION

The paper addresses the tension between personalized learning and Byzantine robustness in decentralized collaboration: honest clients can produce diverse updates, making malicious perturbations difficult to distinguish. R-DPFL uses neighborhood information as a controlled correction rather than enforcing consensus, and its analysis and experiments support stable personalized learning under attack.

B. Decentralized Federated Learning

  • Decentralized federated learning removes the central coordinator and enables peer-to-peer exchange, but existing decentralized personalized methods largely assume reliable neighbors.The related work emphasizes communication efficiency, model heterogeneity, and benign-setting optimization rather than adversarial peer behavior.Source: §I · p. 1, B. Decentralized Federated Learning · p. 3

I. INTRODUCTION

  • Honest clients’ diverse updates make malicious perturbations difficult to distinguish, while consensus-oriented defenses can suppress useful heterogeneous information or allow attacks to propagate.The challenge is to preserve beneficial collaboration without forcing clients toward a common model.Source: §I · pp. 1–2
  • R-DPFL combines robust aggregation, history-based update prediction, and adaptive deviation clipping to use neighbor information as a controlled correction without requiring model consensus.The correction is added to each client’s local descent direction, so abrupt or malicious deviations cannot dominate the update.Source: §I · p. 2
  • The analysis establishes bounded personalized stationarity: honest clients maintain stable descent on their own objectives, with Byzantine influence appearing as a bounded residual.The stated guarantee assumes standard smoothness and bounded error.Source: §I · p. 2

A. Personalized Federated Learning

  • Personalized federated learning accommodates heterogeneous clients by learning client-specific models; existing approaches include partial sharing, multi-task optimization, distillation, and clustering.The cited PFL methods are largely developed for centralized or benign settings, limiting direct application to adversarial decentralized collaboration.Source: A. Personalized Federated Learning · pp. 2–3

C. Byzantine-Robust Decentralized Learning

  • R-DPFL studies personalization and Byzantine robustness jointly: each client retains its own model and uses neighbor information only as a controlled local-update correction.Unlike consensus-oriented robust methods, the paper does not require neighboring clients to learn the same model.Source: C. Byzantine-Robust Decentralized… · p. 3

A. Decentralized Personalized Learning System

  • Because heterogeneous clients generally have different optimal models, decentralized personalized learning seeks client-specific adaptation while retaining neighborhood collaboration rather than a shared solution.The paper presents soft neighborhood collaboration as motivation; its method instead implements collaboration through robust direction-level correction.Source: A. Decentralized Personalized Learning… · p. 3

B. Robust Personalized Collaboration under Byzantine Neighbors

Robust personalized collaboration must tolerate Byzantine messages without mistaking legitimate differences in honest clients’ objectives for attacks. R-DPFL combines each client’s local descent direction with a robust neighborhood correction, targeting stable descent on each honest client’s objective rather than consensus.

B. Robust Personalized Collaboration under Byzantine Neighbors

  • Legitimate update disagreement in personalized learning weakens consensus-based anomaly detection, creating a tension between suppressing harmful perturbations and retaining useful heterogeneous collaboration.The paper frames neighborhood interaction as a controlled correction, not a consensus constraint.Source: B. Robust Personalized Collaboration… · p. 4
  • The threat model allows Byzantine neighbors to send arbitrary corrupted messages and bounds their fraction in each honest client’s neighborhood.The goal is not to identify every malicious neighbor or recover every clean update, but to keep neighborhood influence useful under heterogeneity and bounded under attack.Source: B. Robust Personalized Collaboration… · p. 4
  • The desired guarantee is stable descent toward each honest client’s own objective, with adversarial neighbor influence causing only bounded distortion rather than requiring asymptotic consensus.The paper characterizes this goal through a bounded personalized stationarity measure.Source: B. Robust Personalized Collaboration… · p. 4

A. Framework Overview

  • R-DPFL’s round-level framework performs local SGD, constructs a local descent direction, estimates a robust neighborhood direction, and updates the personalized model using both directions.The collaboration direction is estimated from received neighbor messages together with historical neighborhood dynamics.Source: A. Framework Overview · pp. 4–5

B. Local Descent Direction Construction

R-DPFL constructs a client-specific descent direction from the cumulative effect of local optimization, using the intermediate locally optimized model to form that direction rather than as the round-level model. It then estimates a robust neighborhood direction by combining robust aggregation, temporal prediction, and adaptive deviation clipping.

B. Local Descent Direction Construction

  • The client’s local descent direction summarizes the cumulative effect of its local optimization steps, and the intermediate locally optimized model is used to construct that direction rather than directly update the round-level model.The direction is interpreted as the average local stochastic descent direction accumulated over K steps.Source: B. Local Descent Direction Construction · p. 5

C. Robust Neighborhood Direction Estimation

  • The robust neighborhood direction combines robust aggregation, temporal prediction, and adaptive deviation clipping to preserve benign collaborative trends while bounding malicious-neighbor influence.The method targets a stable, informative neighborhood signal without forcing agreement among neighbors.Source: C. Robust Neighborhood Direction… · p. 5

1) Neighbor Message Collection:

R-DPFL estimates a robust neighborhood collaboration direction without identifying malicious neighbors or requiring exact recovery of every clean message. It combines robust aggregation with a history-based prediction of neighborhood dynamics.

1) Neighbor Message Collection:

  • The goal is to estimate a neighborhood-level collaboration direction that captures the dominant benign trend while remaining robust to outliers and adversarial distortions.The method does not aim to identify malicious neighbors or recover every clean neighbor direction.Source: 1) Neighbor Message Collection · p. 6

2) Robust Neighborhood Aggregation:

  • Coordinate-wise median, trimmed mean, or geometric median can aggregate received directions to suppress large instantaneous outliers.Robust aggregation alone may overreact to natural heterogeneity or be misled by temporally coordinated attacks.Source: 2) Robust Neighborhood Aggregation · p. 6

3) Historical Prediction of Neighborhood Dynamics:

  • A Barzilai--Borwein-style coefficient uses the local model trajectory to predict the expected neighborhood direction from historical information before incorporating the current observation.The prediction is intended to track smooth benign evolution and provide a temporally consistent expected direction.Source: 3) Historical Prediction of… · p. 6

4) Adaptive Deviation Clipping:

R-DPFL compares the aggregated neighborhood direction with a history-based prediction, then clips deviations to limit the collaboration correction. The local direction remains the dominant personalized optimization component, without requiring neighboring models to agree.

4) Adaptive Deviation Clipping:

  • The deviation measures how much the observed neighborhood trend differs from the expected benign evolution predicted from history.Because benign deviations can arise from heterogeneity and non-stationary optimization, the clipping threshold adapts using historical deviation statistics.Source: 4) Adaptive Deviation Clipping · p. 6
  • Adaptive clipping preserves deviations consistent with recent history while bounding large deviations and the norm of the final neighborhood collaboration direction.The method uses a maximum allowed norm for the final direction.Source: 4) Adaptive Deviation Clipping · p. 6

D. Personalized Descent with Robust Neighborhood Collaboration

  • The personalized update combines the client’s local descent direction with a bounded neighborhood correction, whose influence is controlled by the collaboration weight.At zero collaboration weight, training reduces to independent local optimization; increasing the weight increases neighborhood influence without requiring a common solution.Source: D. Personalized Descent with Robust… · p. 7
  • The local direction determines the dominant personalized trajectory, while Byzantine perturbations can affect optimization only through the bounded neighborhood contribution.This design retains useful heterogeneous collaborative information without directly averaging neighboring models or enforcing asymptotic agreement.Source: D. Personalized Descent with Robust… · p. 7

V. THEORETICAL ANALYSIS

The analysis establishes bounded personalized stationarity for honest clients under smoothness and stochastic-gradient assumptions. It treats Byzantine influence as a bounded residual rather than requiring honest clients to converge to a shared model.

V. THEORETICAL ANALYSIS

  • The analysis targets stable descent on each honest client’s local objective, with Byzantine neighbor perturbations entering the convergence bound only as a bounded residual.The guarantee is bounded personalized stationarity, not convergence to a common model.Source: §V · p. 7, B. Proof Sketch · p. 8

A. Assumptions

  • The analysis assumes smooth local objectives and unbiased stochastic gradients with bounded norm for each honest client.The stated gradient condition also implies bounded stochastic-gradient variance.Source: A. Assumptions · p. 7

B. Proof Sketch

  • The proof separately bounds local descent and the corrected neighborhood contribution, then combines them to show that personalized descent dominates the bounded collaboration residual.Because Byzantine messages may be arbitrary, the proof bounds their transformed neighborhood effect rather than each malicious message.Source: B. Proof Sketch · pp. 7–8

C. Bounded Local Descent Direction

D. Bounded Neighborhood Collaboration Direction

The analysis bounds the final neighborhood collaboration direction, allowing its effect on personalized optimization to be treated as a controlled residual.

D. Bounded Neighborhood Collaboration Direction

  • Lemma 3 establishes that the final neighborhood collaboration direction is bounded, so neighborhood influence enters the personalized update as a controlled residual.The clipping operator limits a vector’s norm to its clipping radius.Source: D. Bounded Neighborhood Collaboration… · p. 9

E. Stable Personalized Descent

The analysis establishes bounded personalized stationarity for each honest client under stated smoothness, sampling, step-size, and neighborhood-direction conditions. It treats neighborhood collaboration as a bounded residual in the descent process.

E. Stable Personalized Descent

  • The convergence argument models corrected neighborhood collaboration as a bounded residual in each client’s personalized descent process.The proof decomposes one-step descent into local direction, neighborhood collaboration, and a quadratic term, then averages and telescopes over T rounds.Source: E. Stable Personalized Descent · p. 9
  • Under Assumption 1 and bounded neighborhood directions, each honest client’s average personalized descent over T rounds is bounded.The step size satisfies γ ≤ 1/(8L).Source: E. Stable Personalized Descent · p. 9
  • The theorem’s bound depends on constants defined in Lemma 2, including B, η, K, and σ²_u.The supplied passage identifies these quantities as constants but does not provide their definitions.Source: E. Stable Personalized Descent · p. 9

A. Experimental Objectives

The experiments assess R-DPFL’s clean-setting personalized performance, robustness to Byzantine neighbors, and the contribution of its design components.

A. Experimental Objectives

B. Datasets and Models

Experiments use CIFAR-10 image classification with a lightweight convolutional network and a decentralized, heterogeneous client setup.

B. Datasets and Models

  • CIFAR-10 supplies 32 × 32 RGB images across 10 categories, modeled with a lightweight convolutional network.The network uses convolutional layers for feature extraction and fully connected layers for classification.Source: B. Datasets and Models · p. 11

C. Experimental Setup

  • The setup uses 10 clients in a fully connected topology, Dirichlet heterogeneity with α = 0.3, and 500 communication rounds.The main robustness and ablation experiments use λ = 0.01; clients perform one local SGD epoch per round with batch size 64 and learning rate 0.01.Source: C. Experimental Setup · p. 11

D. Attack Settings and Evaluation Metrics

R-DPFL remains stable under the considered Byzantine attacks, with honest-client performance close to the clean reference. The section also examines attack settings, evaluation, ablations, and the effect of neighborhood collaboration strength.

Attack Models.:

  • The tested attacks include noise, sign-flip, scaling, min-max, min-sum, and adaptive poisoning, with 3 of 10 clients Byzantine by default.The adaptive adversary poisons messages around the predicted neighborhood trend; Byzantine clients independently corrupt directions sent to neighbors.Source: Attack Models · p. 11

Evaluation protocol and metrics.:

  • The evaluation reports average test accuracy for honest clients, excluding attackers whose arbitrary or corrupted parameters could obscure system performance.The metric measures whether the defense protects honest clients’ utility against adversarial influence.Source: Evaluation protocol and metrics · pp. 11–12

1) Clean Performance:

2) Robustness under Byzantine Attacks:

F. Ablation and Sensitivity Analysis

VII. CONCLUSIONS

  • R-DPFL predicts the aggregated neighborhood update from historical local model changes and neighborhood updates, clips the prediction residual, and adds it to each client’s local descent direction.The analysis establishes bounded personalized stationarity and stable descent on each honest client’s local objective without enforcing consensus among neighboring models.Source: §VII · p. 14

Related papers

Loading 2609.07312v1…