Source-linked AI summary
Learning to Detect Malicious Clients for Robust Federated Learning
Suyi Li, Yong Cheng, Wei Wang, Yang Liu, Tianjian Chen
TL;DR
Malicious clients can degrade or redirect federated learning while remaining outside the server’s control. The paper uses server-side spectral anomaly detection to identify and remove harmful updates, and reports robustness across Byzantine and targeted attacks. Its scope relies on centralized unbiased updates and efficiently sampled surrogate representations.
Problem
Malicious clients can send modified updates that degrade global performance or create targeted backdoor behavior.
Method
The server trains a spectral anomaly detector using low-dimensional representations and removes detected malicious updates before aggregation.
Results
The method accurately detects untargeted and targeted attacks and outperforms existing defense-based methods in model accuracy across the reported evaluations.
Takeaways & Limitations
Targeted detection preserves distinctions among benign updates from heterogeneous client datasets while excluding malicious updates from aggregation.
Takeaways & Limitations
The detector relies on centralized training updates and random-sampled surrogate vectors, while optimal update representations remain outside the paper’s scope.
Abstract
from arXiv · showhide
Federated learning systems are vulnerable to attacks from malicious clients. As the central server in the system cannot govern the behaviors of the clients, a rogue client may initiate an attack by sending malicious model updates to the server, so as to degrade the learning performance or enforce targeted model poisoning attacks (a.k.a. backdoor attacks). Therefore, timely detecting these malicious model updates and the underlying attackers becomes critically important. In this work, we propose a new framework for robust federated learning where the central server learns to detect and remove the malicious model updates using a powerful detection model, leading to targeted defense. We evaluate our solution in both image classification and sentiment analysis tasks with a variety of machine learning models. Experimental results show that our solution ensures robust federated learning that is resilient to both the Byzantine attacks and the targeted model poisoning attacks.
1 Introduction
Federated learning is exposed to malicious client updates because the server cannot control client behavior or access private data. The paper proposes spectral anomaly detection to identify and remove such updates, supporting targeted defense across untargeted and targeted attacks.
- Threat model: Malicious clients can send harmful model updates that degrade overall performance or alter predictions on attacker-chosen instances.The paper considers untargeted Byzantine attacks and targeted backdoor attacks.
- Prior limitations: Existing Byzantine-tolerant methods do not differentiate malicious updates from normal ones, limiting their performance in federated learning.They tolerate attacks through modified aggregation mechanisms rather than identifying attackers.
- Proposed approach: The proposed framework detects abnormal updates through low-dimensional embeddings that remove noisy features while retaining essential ones.In the latent space, malicious updates have features that differ substantially from normal updates.
- Proposed approach: The framework supports unsupervised and semi-supervised detection, uses VAE-based dynamic thresholding, and removes malicious updates at the server.The threshold is determined after all client updates arrive, preventing attackers from learning it beforehand.
- Evaluation: Across image classification and sentiment analysis with LR, CNN, and RNN models, the method accurately detects untargeted and targeted attacks and nearly eliminates their negative impacts.The evaluation uses heterogeneous federated learning settings.
2 Prior Arts
Prior robust federated learning methods mainly estimate a center of client updates without identifying malicious clients. Their assumptions and designs limit coverage of heterogeneous data and targeted or untargeted attacks, motivating anomaly detection.
- Robust aggregation: Many robust aggregation methods estimate a center of local updates instead of using a weighted average and often assume IID client data.Under this assumption, benign updates can approximate true gradients or model weights.
- Robust aggregation: GeoMed and Trimmed Mean mitigate malicious attacks to some degree but do not distinguish malicious updates from normal ones.Consequently, their protection does not fully eliminate attack impacts.
- Attack coverage: Several Byzantine-robust approaches cannot defend against targeted attacks.The cited methods combine regularization, distributed SAGA, or geometric-median optimization for Byzantine settings.
- Attack coverage: Existing robust federated learning solutions are mostly defense-based and limited to targeted attacks, while some detection methods rely on distributions absent in federated learning.The cited mask-feature assumption does not hold in the federated setting.
- Spectral anomaly detection: Spectral anomaly detection embeds normal and abnormal instances in a low-dimensional space where their representations differ significantly.It removes noisy features and projects important features to facilitate abnormal-instance identification.
3 Spectral Anomaly Detection for Robust FL
The paper models malicious-update impact and uses spectral anomaly detection to separate malicious from benign updates in a compact latent space. Dynamic thresholding then excludes detected malicious updates from aggregation, although training relies on centralized updates and sampled surrogate vectors.
- 3.2 Impact of Malicious Model Updates: Theorem 1 identifies attacker noise ψ and the malicious clients’ total weight fraction f_a as determinants of malicious-update impact.The analysis uses a linear regression model and quantifies deviation under additive noise.
- 3.2 Impact of Malicious Model Updates: Sign-flipping attacks cause more model-performance damage than random additive noise at the same malicious weight, and degradation increases as malicious clients rise from 0–50%.These observations are confirmed by simulations in Figure 1.
- 3.2 Impact of Malicious Model Updates: The analysis motivates excluding malicious updates entirely by setting f_a to 0, while large attacks become more detectable because they require drastically different updates.The resulting attacker tradeoff is either ineffective damage or exposure.
- 3.3 Malicious Clients Detection: The framework detects malicious updates in low-dimensional embeddings that retain essential variability while removing noisy and redundant features.Benign client updates may be locally biased, but their shift is described as smaller than the malicious-to-unbiased difference.
- 3.3 Malicious Clients Detection: The detection model is trained using centralized unbiased updates and represents each update with an efficiently sampled surrogate vector.The paper states that learning optimal update representations is future work.
- 3.4 Remove the Malicious Updates: Figure 2 visualizes centralized, benign, and malicious updates as green, blue, and red latent vectors for additive-noise and sign-flipping attacks.The paper reports that benign and malicious representations can be separated, especially for sign-flipping attacks.
- 3.4 Remove the Malicious Updates: Each client update receives a reconstruction error, with malicious updates producing much larger errors than benign ones.The reconstruction error is the key signal used for malicious-update detection.
- 3.4 Remove the Malicious Updates: Each round sets the threshold to the mean reconstruction error; updates above it are excluded, and aggregation uses only benign updates weighted by local dataset size.This implements dynamic thresholding before aggregation.
4 Performance Evaluation
The evaluation tests spectral anomaly detection across heterogeneous image-classification and sentiment-analysis FL tasks under untargeted and targeted attacks. The method achieves the best performance in untargeted settings, mitigates backdoor effects, separates malicious updates, and introduces little bias when all clients are benign.
- Experiment Setup: Each round randomly selects 100 clients, with 30% or 50% attackers for untargeted attacks and dataset-specific attacker counts for backdoor attacks.Targeted attacks use 30 attackers for FEMNIST and Sentiment140 and 20 for MNIST.
- Datasets and ML Models: The evaluation uses MNIST, FEMNIST, and Sentiment140 with LR, CNN, and RNN models in heterogeneous FL settings.MNIST uses label-based shards, FEMNIST assigns writers as clients, and Sentiment140 assigns users as clients.
- Anomaly Detection Model: The detection model is trained from centralized model-update weights and uses a VAE to produce low-dimensional latent representations for anomaly detection.The VAE uses two dense hidden layers in both encoder and decoder, with a 100-dimensional latent vector.
- Untargeted-Attack Results: Our method achieves the best performance across sign-flipping and additive-noise attacks, while GeoMed fails under sign flipping and Krum remains unchanged across attack settings.Krum’s limitation is attributed to biased benign updates in the non-IID setting; GeoMed is robust to additive noise.
- Targeted-Attack Results: Under backdoor attacks, the proposed method mitigates attack impact, Krum is robust, and GeoMed fails on MNIST.Excluding malicious clients yields the best theoretical performance because their local examples are not learned.
- Discussion: The anomaly detector separates benign and malicious updates in latent space, supporting targeted defense and very little bias when all clients are benign.The benign and malicious representations are especially separable for sign-flipping attacks, while defense methods introduce negligible performance loss relative to FedAvg without attack defense.
5 Conclusion
The paper proposes server-side spectral anomaly detection to identify and remove malicious federated-learning updates, reporting improved robustness and accuracy over existing defenses.
- The proposed framework performs spectral anomaly detection at the server to detect and remove malicious updates from adversarial clients.
- The method accurately detects malicious model updates and eliminates their impact across the evaluated attacks.
- The numerical results show higher model accuracy than existing defense-based methods.