PaperlayerView exact revision on arXiv

Source-linked AI summary

BaFFLe: Backdoor detection via Feedback-based Federated Learning

Sebastien Andreina, Giorgia Azzurra Marson, Helen Möllering, Ghassan Karame

arXiv:2011.02167v2cs.CRcs.LG

TL;DR

Federated learning needs defenses against poisoning attacks that inject backdoors while secure aggregation limits inspection of individual updates. BAFFLE uses diverse clients to validate global models through a feedback loop, achieving strong detection on CIFAR-10 and FEMNIST, including adaptive attacks.

  • Problem

    Secure aggregation hides individual updates, while backdoor accuracy cannot be directly measured because the attacker alone knows the backdoor set.

  • Method

    BAFFLE adds a feedback loop in which diverse, randomly selected clients validate the global model on private data and vote on updates.

  • Results

    BAFFLE achieved detection accuracy of up to 100% with a false-positive rate below 5% and detected 95%–100% of adaptive injections.

  • Takeaways & Limitations

    Decentralized client data helps BAFFLE detect poisoning even when attackers adapt to the defense or validating clients lack backdoor data.

Abstract

from arXiv · show

Recent studies have shown that federated learning (FL) is vulnerable to poisoning attacks that inject a backdoor into the global model. These attacks are effective even when performed by a single client, and undetectable by most existing defensive techniques. In this paper, we propose Backdoor detection via Feedback-based Federated Learning (BAFFLE), a novel defense to secure FL against backdoor attacks. The core idea behind BAFFLE is to leverage data of multiple clients not only for training but also for uncovering model poisoning. We exploit the availability of diverse datasets at the various clients by incorporating a feedback loop into the FL process, to integrate the views of those clients when deciding whether a given model update is genuine or not. We show that this powerful construct can achieve very high detection rates against state-of-the-art backdoor attacks, even when relying on straightforward methods to validate the model. Through empirical evaluation using the CIFAR-10 and FEMNIST datasets, we show that by combining the feedback loop with a method that suspects poisoning attempts by assessing the per-class classification performance of the updated model, BAFFLE reliably detects state-of-the-art backdoor attacks with a detection accuracy of 100% and a false-positive rate below 5%. Moreover, we show that our solution can detect adaptive attacks aimed at bypassing the defense.

I. INTRODUCTION

Federated learning improves collaborative model training but secure aggregation can hide individual contributions, enabling model poisoning. BAFFLE adds client feedback to validate global updates and reports strong detection against standard and adaptive attacks.

  • Secure aggregation protects client privacy but prevents accountability for individual updates, leaving malicious clients able to tamper with training.
  • Model poisoning injects backdoor behavior that misclassifies attacker-chosen inputs while preserving normal behavior on other inputs.
  • BAFFLE uses diverse client data for both training and validation, with randomly refreshed clients voting through a feedback loop on whether to accept each global update.
  • Malicious clients may misreport validation results, but appropriate use of client judgments can detect backdoors using straightforward classification-performance checks.
  • BAFFLE retained high detection accuracy against adaptive attacks designed to bypass the misclassification-based validation method.

II. BACKGROUND

The paper introduces supervised classification and federated learning as the technical foundations for its defense. FL trains a shared model through repeated local updates and server aggregation, optionally using secure aggregation.

  • A. Machine Learning: A classification model maps inputs to labels, and its quality is measured by accuracy on naturally occurring or test data.
  • B. Federated Learning: Federated learning trains a shared model across clients while keeping their data on-device and aggregating locally computed updates.
  • B. Federated Learning: Each FL round selects clients, distributes the current global model, and collects locally trained model updates for aggregation.
  • B. Federated Learning: Secure aggregation can prevent the server from learning information from individual updates, while the completed round produces the next global model.

III. SECURITY MODEL

The security model formalizes backdoor attacks as targeted misclassification on attacker-chosen inputs while requiring correct predictions elsewhere. Defenders cannot directly measure the attacker’s backdoor accuracy because the backdoor set is private.

  • A backdoor attack makes a classifier assign an attacker-selected target label to specified inputs while preserving correct predictions on other instances.
  • Backdoor accuracy measures the portion of backdoor instances assigned the attacker’s target label.
  • Defenders cannot measure backdoor accuracy because only the attacker knows the backdoor instance set.
  • Backdoor attacks rely on poisoning part of the training data with labeled backdoor samples.

B. Backdoor Attacks in Federated Learning

The paper frames FL backdoor defense as detecting compromised global models without inspecting individual updates, preserving secure aggregation and privacy. BAFFLE is proposed as the corresponding defense under an adversarial client model.

  • B. Backdoor Attacks in Federated Learning: FL is vulnerable to model replacement, in which poisoned local training can replace the global model with one containing a backdoor.
  • B. Backdoor Attacks in Federated Learning: The defense detects whether the global model has been backdoored without inspecting individual updates, maintaining compatibility with secure aggregation.
  • B. Backdoor Attacks in Federated Learning: The attacker controls Byzantine clients that may deviate arbitrarily and coordinate across rounds to backdoor the global model.
  • B. Backdoor Attacks in Federated Learning: BAFFLE is the paper’s proposed defense and feedback-loop design for detecting backdoor attacks in federated learning.

A. Intuition and overview

BAFFLE uses clients’ diverse private datasets to validate global models through a feedback loop, helping detect poisoning while accommodating malicious votes and adaptive attacks.

  • In the illustrated two-round design, a detected poisoned update leaves the global model unchanged, whereas an accepted update becomes the next global model.The figure shows Gr = G after rejection and Gr+1 = G′ after acceptance.
  • BAFFLE adds a validation phase in which randomly selected clients use private data to provide feedback on the global model.The feedback loop implicitly enlarges and enriches the validation set through clients’ diverse data.
  • Diverse validation data limits an adaptive attacker’s ability to optimize poisoned models to evade detection.The attacker does not know or control substantial federated data held by clients.
  • BAFFLE bootstraps trust by comparing each model’s prediction behavior with recent accepted models, while early backdoors tend to disappear as training stabilizes.The defense can remain effective when started after maturation or following earlier poisoning attempts.
  • The server accepts or rejects an updated model according to validating clients’ verdicts and a quorum threshold.A model is rejected when sufficiently many clients report poisoning; otherwise it is accepted.
  • Malicious validating clients can misreport votes, so the quorum must balance false rejection of clean models against acceptance of poisoned models.The paper derives conditions on q based on the tolerated number of malicious clients and honest-client judgment errors.

V. MODEL VALIDATION

BAFFLE’s validation procedure detects suspicious updates by comparing per-class error-rate changes between consecutive models and using historical variation patterns to identify outliers.

  • The validation method monitors source-focused and target-focused per-class error-rate variations between the current and updated models.These variations measure wrong-prediction gaps on the same validation dataset for each label.
  • A poisoned update is considered suspicious when its error-variation pattern differs substantially from the gradual changes expected during benign federated training.The method compares the updated model with the previous model for every label.
  • The method represents source and target error variations as a point in Z2|Y | and applies Local Outlier Factor to measure its relative outlierness.LOFk(x; N) greater than 1 indicates that a point is less clustered than neighboring points.
  • The validation routine is modular: any entity holding labelled data can validate a model when sufficiently many previously accepted models are available.The evaluation considers configurations in which the server, clients, or both run VALIDATE.
  • The current update is judged against error-variation values computed from the latest ℓ + 1 accepted models, with ℓ controlling the look-back window.The window should provide a stable reference while remaining responsive to recent model behavior.

VI. IMPLEMENTATION & EVALUATION RESULTS

The evaluation studies BAFFLE’s detection accuracy under varied system, data, timing, and adaptive-attack conditions in image-classification federated learning.

  • The experiments vary the look-back window, quorum threshold, data splits, and poisoning timing to evaluate BAFFLE under multiple configurations.The study compares late and early poisoning and also tests an adaptive attack.

A. Implementation Setup

The implementation evaluates BAFFLE on CIFAR-10 and FEMNIST image-classification tasks using ResNet18, with randomly selected validating clients applying the validation algorithm.

  • The evaluation uses two image-classification tasks, CIFAR-10 and FEMNIST, and trains a ResNet18 CNN model.CIFAR-10 contains 50,000 training and 10,000 test samples; FEMNIST contains 731,668 training and 82,587 test samples.
  • An old model may have substantially higher error variations than the current model, potentially causing a genuine model to be rejected.This is identified as a limitation of the validation approach.
  • For BAFFLE, the server selects 10 validating clients uniformly at random and supplies the current model plus the last ℓ + 1 accepted models.Each client validates locally and sends a vote; the server applies the quorum threshold, including its own vote.

B. Evaluation Methodology

The evaluation varies BAFFLE’s look-back window, quorum threshold, client-server data split, and poisoning time across CIFAR-10 and FEMNIST, including stabilized and early-training settings.

  • BAFFLE is evaluated for look-back windows ℓ=10, 20, 30 and quorum thresholds 3 ≤ q ≤ 9.
  • Client-server data splits are 90%-10%, 95%-5%, and 99%-1% for CIFAR-10, and 99%-1%, 99.5%-0.5%, and 99.9%-0.01% for FEMNIST.
  • The study compares poisoning after model stabilization with poisoning during early rounds when the model is not yet close to convergence.

C. Results

BAFFLE generally detects poisoning accurately across parameter settings, datasets, poisoning times, and adaptive attacks, while feedback-based configurations reduce false positives relative to server-only validation.

  • Look-back window: For CIFAR-10, feedback-loop configurations achieve false-negative rates of 0.0–0.1 for ℓ=10, 20 across data splits, while BAFFLE-C reaches 0.6 at ℓ=30 under the 90%-10% split.
  • Look-back window: For FEMNIST, ℓ=10 yields a false-negative rate of 1.0, whereas ℓ=20, 30 reduce false negatives to 0 in most cases and at most 0.1 in all cases.
  • Quorum threshold: For CIFAR-10, q ≤ 7 sharply approaches zero false negatives, while 5 ≤ q ≤ 7 provides high detection accuracy and nearly equal error rates.
  • Early poisoning: Early backdoors injected at rounds 100 and 300 are not durable, while BAFFLE detects nearly all poisoning attempts after round 530, missing one FEMNIST injection.
  • Adaptive attacks: Adaptive injections are detected at rates between 95% and 100%; BAFFLE records 0 false negatives across CIFAR-10 data splits, unlike BAFFLE-S’s 33.3% rate in two splits.

D. Communication Overhead

BAFFLE’s validation uses recent accepted-model history at validating clients, while the evaluated figures examine poisoning effects and vote requirements across configurations.

  • Validating clients assess each round’s update using the latest ℓ+1 accepted models to derive a rejection threshold.
  • Figure 4 compares main-task and backdoor accuracy without defense and with BAFFLE activated at round 500 under early poisoning.
  • Figure 5 reports the votes required to reject adaptive poisoning under 90%-10%, 95%-5%, and 99%-1% client-server data splits.
  • Model compression could reduce the reported communication overhead by a factor of 10 to around 20MB per selected client.

VII. RELATED WORK

Prior FL defenses inspect client updates or rely on assumptions that do not hold in realistic, non-uniform settings. BAFFLE instead uses client feedback and reports strong detection across datasets, including adaptive attacks.

  • Attacks: Model-poisoning studies include targeted semantic backdoors injected by a single malicious client and distributed backdoor attacks using multiple clients.BAFFLE’s evaluation does not require honest clients to possess correctly labelled backdoor data.
  • Defenses: FoolsGold filters suspicious local updates but assumes class coverage by honest clients and multiple-client attackers, allowing adaptive single-client circumvention.
  • Defenses: Spectral anomaly detection and layered defenses also inspect local updates, whereas BAFFLE engages clients in validating the global model through a round-based feedback loop.
  • BAFFLE: 100% detection accuracy with a false-positive rate below 5% was achieved on both CIFAR-10 and FEMNIST.The reported results also cover late activation, prior poisoning, and adaptive attacks.
Loading 2011.02167v2…