PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentially private federated learning with Byzantine-robust aggregation: A cross-domain framework for secure model training in banking and healthcare systems

Srikumar Nayak

arXiv:2609.03064v1cs.CRcs.LG

TL;DR

Federated updates can expose information about local records, and ordinary averaging does not protect the aggregate from adversarial client updates. DP-BR-FedAvg combines client-level differential privacy with coordinate-wise trimmed-mean Byzantine-robust aggregation. In final-round performance (mean of last 5 of 60 rounds), DP-BR-FedAvg reaches F1 = 0.119, while FedAvg and DP-FedAvg fall below F1 0.03.

  • Problem

    Federated updates can expose information about local records, and ordinary averaging does not protect the aggregate from adversarial client updates.

    Source: §I · pp. 1–2

  • Method

    DP-BR-FedAvg combines client-level differential privacy with coordinate-wise trimmed-mean Byzantine-robust aggregation.

    Source: §I · p. 2

  • Results

    In final-round performance (mean of last 5 of 60 rounds), DP-BR-FedAvg reaches F1 = 0.119, while FedAvg and DP-FedAvg fall below F1 0.03.

    Source: B. Minority-Class Performance · p. 6

  • Takeaways & Limitations

    The robustness advantage narrows as noise increases, showing that privacy and robust aggregation do not simply add without trade-offs.

    Source: C.Privacy-Utility Trade-off · p. 7, §VII · p. 8

    The reported evaluation is a simulation of a fraud/clinical-risk task with 20 clients and 25% Byzantine participants.

    Source: §VII · p. 8

Abstract

from arXiv · show

Federated learning allows banks, hospitals, and other regulated organizations to train a shared model without moving raw records off their own servers, which is attractive wherever data protection law or competitive sensitivity rules out pooling data centrally. Two problems limit how far this promise can be trusted in practice. First, the parameter updates that clients exchange still leak information about local records through gradient inversion and membership inference attacks. Second, an honest averaging rule such as FedAvg has no defense against a subset of clients that submit corrupted or adversarial updates, so a small number of malicious or compromised participants can quietly steer the shared model off course. This paper presents a federated learning framework, DP-BR-FedAvg, that combines a Gaussian-mechanism differential privacy layer with a coordinate-wise trimmed-mean Byzantine-robust aggregation rule, evaluated on a simulated cross-institutional classification task resembling fraud and clinical-risk scoring. Across sixty communication rounds with twenty clients, a quarter of them Byzantine, plain FedAvg collapses on the minority class (F1-score 0.030) while the proposed framework recovers substantially more of the signal (F1-score 0.119) while bounding the privacy loss of any single client's contribution. A Byzantine-robust aggregator with no privacy layer performs best in raw accuracy, quantifying the cost privacy imposes on robustness. The results show that privacy and robustness mechanisms interact rather than simply add, and that system design for regulated, adversarial, cross-institutional settings needs to budget for that interaction.

I. INTRODUCTION

Federated learning keeps raw records local, but shared updates can still expose information and ordinary averaging can be steered by Byzantine clients. This paper combines client-level differential privacy and Byzantine-robust aggregation, studying their interaction in a simulated cross-institutional task.

I. INTRODUCTION

  • Shared model updates can reveal information about local records, while ordinary averaging offers no mechanism to discard or down-weight adversarial client updates.Gradient inversion can reconstruct training information; a modest fraction of malicious participants can dominate an ordinary aggregate.Source: §I · pp. 1–2
  • Privacy and Byzantine robustness interact: clipping and noise alter the update statistics used for robust aggregation, while aggregation changes the sensitivity calculation underlying privacy guarantees.The paper studies this interaction rather than assuming the mechanisms compose without cost.Source: §I · p. 2
  • DP-BR-FedAvg combines per-client gradient clipping and Gaussian noise with coordinate-wise trimmed-mean aggregation and provides accounting for a combined (epsilon, delta) privacy guarantee.The framework is evaluated in a controlled simulation of class-imbalanced cross-institutional classification under a sign-flipping Byzantine attack.Source: §I · p. 2

II. RELATED WORK

Related work covers federated learning across heterogeneous institutions, privacy mechanisms for protecting records and updates, Byzantine-robust aggregation, and banking and healthcare applications.

A. Federated Learning Foundations

  • FedAvg lets clients perform local gradient steps before the server averages updates, while non-IID institutional data makes statistical heterogeneity a central cross-institutional concern.Structured and sketched updates have also been studied to improve communication efficiency.Source: A. Federated Learning Foundations · p. 2

B. Privacy-Preserving Mechanisms

  • Federated privacy work includes client-level differential privacy, while secure aggregation cryptographically hides individual updates from the server without adding statistical noise.Gradient inversion work demonstrates that shared gradients can expose training images and labels.Source: B. Privacy-Preserving Mechanisms · p. 3

C.Byzantine-Robust Aggregation

  • Krum, coordinate-wise trimmed mean, and median are Byzantine-robust aggregation approaches; FLTrust instead derives trust from a clean server-held root dataset.Prior work also shows that a single carefully constructed malicious client can implant a backdoor that survives federated averaging.Source: C.Byzantine-Robust Aggregation · p. 3

D.Domain Applications in Banking and Healthcare

  • Hospital federated-learning deployments and financial-sector analyses motivate cross-institutional training where patient records or transaction data remain local.Security and privacy surveys catalog poisoning, inference, and free-riding threats motivating combined defenses.Source: D.Domain Applications in Banking and… · p. 3

III. PROPOSED FRAMEWORK

The framework models a coordinating server that cannot access raw records, an honest-but-curious server or eavesdropper, and a fraction of Byzantine clients. It clips client updates and uses coordinate-wise trimmed-mean aggregation.

A. System Architecture and Threat Model

  • The threat model combines an honest-but-curious server or eavesdropper seeking inferences from updates with a fraction of clients able to submit arbitrarily corrupted updates.The two threat classes are treated as simultaneous and independent.Source: A. System Architecture and Threat Model · pp. 3–4

B. Local Training and Gradient Clipping

  • Clients train locally from the current global model, then an L2-norm clip bounds each accumulated update before it leaves the client.The shared clipping norm bounds sensitivity for privacy-noise calibration and caps how far a Byzantine raw update can deviate before aggregation.Source: B. Local Training and Gradient Clipping · p. 4
  • The server sorts client values coordinate by coordinate, discards the largest and smallest beta-fraction, and averages the remainder; beta is set at least as high as the assumed Byzantine fraction.Trimmed mean is the default because it is coordinate-wise, cheap, and described as interacting more predictably with Gaussian noise than Krum's pairwise selection.Source: B. Local Training and Gradient Clipping · p. 4

D.Differential Privacy Mechanism

The privacy mechanism clips client updates and adds Gaussian noise to them before transmission, yielding a client-level (epsilon, delta) guarantee. Privacy loss is composed across rounds, though the displayed closed-form approximation is reserved for interpretability.

D.Differential Privacy Mechanism

  • After clipping, each client adds independent Gaussian noise scaled by the clipping norm C and noise multiplier sigma before transmitting its update.The paper applies the Gaussian mechanism used in DP-SGD at the client-update level.Source: D.Differential Privacy Mechanism · p. 4
  • The mechanism is presented as providing a client-level (epsilon, delta) privacy guarantee.The passage frames neighboring populations as differing by a client.Source: D.Differential Privacy Mechanism · p. 4
  • Privacy loss is composed over T rounds using a moments-accountant-style approximation with per-round sampling probability q = 1/K.Equation (6) is for interpretability; the paper says a deployed system should use a tight numerical accountant instead.Source: D.Differential Privacy Mechanism · p. 4

E. Global Aggregation and Update Rule

DP-BR-FedAvg applies a robust aggregation rule to clipped, noised client updates and uses the result to update the global model. The algorithm specifies the client and server steps across rounds.

E. Global Aggregation and Update Rule

  • The server applies trimmed-mean aggregation to the clipped, noised updates, then adds the aggregate to the global model.The paper names the clip, noise, and trimmed-mean composition DP-BR-FedAvg.Source: E. Global Aggregation and Update Rule · p. 4

F. Algorithm

  • Each client trains locally, forms an update, clips its norm, adds Gaussian noise, and sends the update to the server.The algorithm repeats this client process within each communication round.Source: F. Algorithm · p. 5
  • The algorithm initializes the global model and, for each round, aggregates client updates before returning the final model.Its listed inputs include the initial model, clients, rounds, local epochs, learning rate, clipping norm, noise, and trim parameter.Source: F. Algorithm · p. 5

IV. EXPERIMENTAL SETUP

The evaluation uses a synthetic, class-imbalanced classification task split across simulated institutional clients. A quarter of the clients submit sign-flipped updates.

A. Simulated Cross-Institutional Dataset

  • The synthetic task has 12,000 samples, 24 features, an approximately 70:30 class ratio, and three clusters per class.Fourteen features are informative; the clustered data are intended to avoid trivial linear separability.Source: A. Simulated Cross-Institutional Dataset · p. 5
  • Training data are split across 20 simulated clients using Dirichlet allocation with alpha = 0.5, producing non-IID class distributions.The setup represents differences in fraud or disease prevalence across institutions.Source: A. Simulated Cross-Institutional Dataset · p. 5

B. Attack and Defense Configuration

  • Five clients, or 25%, are Byzantine and negate and scale their legitimate updates by 2 before submission.The paper describes this sign-flipping attack as a standard, strong robustness test.Source: B. Attack and Defense Configuration · p. 5

C.Baselines and Evaluation Metrics

The evaluation compares standard averaging, privacy-only and robust aggregation variants, and the proposed combination under attack. Accuracy, minority-class F1-score, and binary cross-entropy loss are reported.

C.Baselines and Evaluation Metrics

  • FedAvg uses standard averaging without privacy noise or robust aggregation, while the no-attack FedAvg reference assumes all clients are honest.The no-attack reference is identified as an upper-bound reference.Source: C.Baselines and Evaluation Metrics · p. 5
  • DP-FedAvg adds clipping and Gaussian noise to simple averaging, whereas Byzantine-Robust FL uses trimmed-mean aggregation without privacy noise.Both configurations are evaluated under attack.Source: C.Baselines and Evaluation Metrics · p. 5
  • DP-BR-FedAvg combines clipping, noise, and trimmed-mean aggregation; all methods train for 60 rounds and are evaluated each round.Reported metrics are accuracy, minority-class F1-score, and binary cross-entropy loss; F1 is emphasized because accuracy alone is misleading on the imbalanced task.Source: C.Baselines and Evaluation Metrics · pp. 5–6

A. Convergence Under Attack

Under Byzantine attack, DP-BR-FedAvg improves minority-class F1 over FedAvg, while Byzantine-Robust FL reaches the highest accuracy among attacked configurations. As privacy noise increases, the proposed method's advantage over DP-FedAvg narrows.

A. Convergence Under Attack

  • Accuracy reaches 0.767 for Byzantine-Robust FL, the highest among attacked configurations; DP-BR-FedAvg drifts to about 0.65.Plain FedAvg and DP-FedAvg plateau around 0.65–0.68, below the 0.84 clean reference.Source: Figure 1 · p. 6, A. Convergence Under Attack · p. 6
  • Binary cross-entropy test loss is markedly lower for Byzantine-Robust FL and DP-BR-FedAvg than for FedAvg or DP-FedAvg.The paper says the lower loss indicates better-calibrated probability estimates.Source: Figure 2 · p. 6, A. Convergence Under Attack · p. 6

B. Minority-Class Performance

C.Privacy-Utility Trade-off

  • As sigma increases and approximate epsilon tightens below about 10, DP-BR-FedAvg's accuracy advantage over DP-FedAvg narrows.The paper attributes the crossover to large noise interfering with the coordinate ordering used by trimmed mean.Source: Figure 4 · p. 7, C.Privacy-Utility Trade-off · p. 7

D.Robustness to Increasing Byzantine Fraction

Accuracy declines as the Byzantine share approaches the trimmed-mean tolerance, and added privacy noise also weakens the robustness bound. The stated protections leave residual risks involving malicious servers and some record-level inference.

D.Robustness to Increasing Byzantine Fraction

VI. SECURITY AND PRIVACY ANALYSIS

  • The framework provides client-level (epsilon, delta)-DP and bounds the influence of subsets smaller than beta, but added noise degrades that bound.Source: §VI · p. 7
  • The framework assumes correct server execution, and client-level privacy alone may not protect record-level inferences from intermediate local checkpoints.For especially sensitive data, local record-level DP-SGD is suggested in addition.Source: §VI · pp. 7–8

VII. CONCLUSION

DP-BR-FedAvg combines client-level differential privacy with trimmed-mean Byzantine-robust aggregation. In simulation, it improves minority-class F1 over plain or privacy-only FedAvg, while the results support co-tuning privacy noise and trim fraction and identify several directions for further work.

VII. CONCLUSION

  • On the simulated task, DP-BR-FedAvg roughly quadruples minority-class F1 versus plain or privacy-only FedAvg, while robust aggregation without privacy achieves the best raw performance.Source: §VII · p. 8

VIII. FUTURE RESEARCH DIRECTIONS

  • Future work proposes adaptive trim and noise schedules and tighter accounting for how trimmed-mean aggregation changes privacy sensitivity.Source: §VIII · p. 8
  • Further evaluation should use de-identified institutional data and examine adaptive or colluding adversaries, alongside secure aggregation to address malicious-server risk.Source: §VIII · p. 8
  • Future work also calls for mapping guarantees to sector regulations and reducing communication costs for robust aggregation.Source: §VIII · p. 8
Loading 2609.03064v1…