PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Art Chase: Privacy Leakage and Data Poisoning in Federated Fine-Tuning

Vasileios Perifanis, Nikolaos Pavlidis

arXiv:2610.21547v1cs.LGcs.CR

TL;DR

This report asks how privacy and integrity risks apply when federated learning is the final, head-only adaptation stage of a pretrained model. It studies those risks in a reproducible pipeline combining centralized fine-tuning, federated adaptation, attacks, and a defense. A single client update exposes the full private label set, while aggregation weakens label-count recovery and backdoor poisoning can persist with little clean-accuracy cost.

  • Problem

    How strongly do privacy and poisoning risks documented for full-model federated training apply when federated learning is the final, head-only stage of a pretrained pipeline?

    Source: §1 · p. 3

  • Method

    The report builds a documented pipeline with centralized fine-tuning followed by head-only federated adaptation, then evaluates privacy attacks, poisoning, and FLTrust.

    Source: §1 · p. 3

  • Results

    A single update reveals a client's full label set, but pixel inversion yields class-plausible sketches and aggregation across five or more clients reduces label-count recovery to an uninformed-guess level.

    Source: §6.4 · pp. 15–16

  • Takeaways & Limitations

    Head-only federated training can still expose client labels and accept backdoors that clean accuracy alone does not reveal.

    Source: §11 · p. 23

    Federated global accuracies are comparable with one another but not with Stage 1 test accuracy, and each configuration was run once with a fixed seed, without confidence intervals.

    Source: §11 · p. 24

Abstract

from arXiv · show

Federated learning (FL) is increasingly used to adapt a pretrained, centrally fine-tuned model to data that remain on user devices, beyond training a model from scratch. In this report, prepared for the 4th Federated Machine Learning International Summer School, we study the privacy and integrity of this approach with a reproducible pipeline. An ImageNet-pretrained MobileNetV4-Conv-Small model is fine-tuned centrally on ten Quick, Draw! classes (97.50% test accuracy) and then adapted with head-only FedAvg over ten non-IID clients, which raises client training accuracy from 90.34% to 91.49% while preserving global accuracy. We evaluate four adversarial scenarios. First, from a single one-step client update, an honest-but-curious server recovers the complete private label set, but its reconstructions follow class prototypes rather than the private drawings (mean SSIM 0.130). Second, from a one-round FedAvg aggregate, label-count recovery falls from 87.5% with one client to 75.0% with five and ten clients, below the 80.0% of an uninformed uniform guess. Third, under FedAvg, a backdoor injected by 30%, 50%, and 100% of the clients reaches an attack success rate (ASR) of 21.9%, 31.3%, and 54.7% at a cost of at most 0.8 percentage points of clean accuracy, whereas label flipping is effective only when most of the source-class data are relabeled. Fourth, FLTrust with a root set retained from centralized fine-tuning lowers the backdoor ASR to 1.8% and 4.0% at 30% and 50% malicious clients and the label-flipping ASR from 12.6% to 0.8% without measurable clean-accuracy loss, but provides no protection when every client is malicious. The report closes with a live session in which participants draw, fine-tune, and poison the model through Indigma's FedMaestro orchestrator. Code and executed notebooks, runnable on Google Colab, are at https://github.com/Indigma-Innovations/fl-summer-school.

1 Introduction

The report examines how privacy and integrity risks apply when federated learning is the final, head-only adaptation stage of a pretrained pipeline. It presents a reproducible tutorial pipeline and evaluates privacy attacks, poisoning, and FLTrust, alongside a live demonstration.

Contributions

  • A deployment-oriented pipeline documents data provenance across pretraining, centralized fine-tuning, and federated fine-tuning, with private data disjoint from Stage 1 and adversary-public data.The report also documents overlap between the federated global evaluation set and the held-out Stage 1 test set.Source: §1 · p. 3

Privacy

  • Under one-step local SGD, a server recovers the exact classifier-head gradient and a single client's full private label set, while pixel inversion yields low-fidelity class-plausible sketches and aggregate label-count recovery falls below an uninformed baseline.The report notes that full-label-set recovery holds only within a narrow range of a heuristic weight.Source: §1 · p. 3

Poisoning

  • The report evaluates label flipping and pixel-trigger backdoors under FedAvg across five malicious-client fractions, finding attack strength is explained by poisoned data reaching the aggregate rather than nominal malicious fraction.The supplied contribution passage states the explanatory relation but does not give the five-fraction results.Source: §1 · p. 3

FLTrust defense

  • FLTrust is evaluated with a root set retained from centralized fine-tuning, using per-round trust diagnostics to compare malicious-client aggregation weight with FedAvg.The contribution describes the evaluation and diagnostic, not its numerical outcomes.Source: §1 · p. 3

Live session

  • The tutorial closes with a live FedMaestro session where participants observe inference, federated fine-tuning, and participant-driven poisoning on a running system.The pipeline uses an ImageNet-pretrained MobileNetV4-Conv-Small centrally fine-tuned on ten Quick, Draw! classes before head-only FedAvg over ten non-IID clients.Source: §1 · p. 3

2 Related Work

Related work covers federated learning from pretrained models, information leakage from model updates, and poisoning with robust aggregation. The report combines a centrally adapted backbone with a federated classifier head and examines these security issues in that setting.

Federated learning from pretrained models

  • Pretrained initialization can reduce harm from client heterogeneity, and lightweight task-specific heads support federated personalization; this report centrally adapts and then freezes the backbone while FL updates the final linear classifier [4] [5].FedAvg averages client models weighted by local sample counts [1, 2].Source: §2 · p. 4

Privacy leakage from model updates

  • Model updates can reveal training data; prior work studies gradient-based image and label recovery, while secure aggregation and differential privacy [29] are privacy techniques not applied in this report.Reported attack strength can depend on assumptions including known labels, batch-normalization statistics, and batch size [27].Source: §2 · p. 4

Data poisoning and robust aggregation

  • Prior defenses include distance-based and coordinate-wise robust aggregation [13, 14, 33], while FLTrust scores client updates against a clean-root-set server update and rescales accepted updates [15].The report describes FLTrust as well-matched to its pipeline because the provider can retain a root set from centralized fine-tuning.Source: §2 · p. 4

3 Experimental Pipeline

The pipeline uses one pretrained-model checkpoint for privacy attacks, clean federated training, poisoning, defense, and a live session. It specifies the task and data provenance, model preprocessing, and head-gradient notation underlying these experiments.

3.1 Overview

  • Stage 1 centrally fine-tunes the complete pretrained network, whose checkpoint initializes the privacy attacks, clean FedAvg baseline, poisoning experiments, FLTrust defense, and live session.Unless stated otherwise, federated experiments update only the classifier head and include all clients in every round.Source: §3.1 · p. 4, Figure 1 · p. 5, Table 1 · p. 5

3.2 Task and data

  • The task is ten-class Quick, Draw! sketch recognition; Stage 1, client, and privacy-attack data use separated index ranges, while the global federated evaluation set is a different split from the Stage 1 test set.The global evaluation set is disjoint from client data and identical across federated runs, so it supports comparisons among federated configurations.Source: §3.2 · pp. 4–5

3.3 Model and preprocessing

  • The 0.5-width MobileNetV4-Small uses a 1280-dimensional backbone embedding and a 12,810-parameter classifier head, representing 1.32% of the 970,762-parameter network.Inputs are resized to 256 pixels, center-cropped to 224×224, and normalized with mean and standard deviation 0.5 per channel.Source: §3.3 · p. 5

3.4 Notation and head gradients

  • For head-only training, the mean cross-entropy gradient aggregates each sample's class probabilities and one-hot label contributions; a negative bias-gradient entry implies that class occurs in the batch.The converse does not hold: a present class can still have a positive bias-gradient entry.Source: §3.4 · pp. 6–7

4 Stage 1: Centralized Fine-Tuning

The report fine-tunes the complete pretrained network centrally, selecting the checkpoint by validation accuracy and using test performance only for monitoring. The selected epoch-15 checkpoint reaches 97.50% test accuracy.

4.1 Protocol

  • The complete network is fine-tuned for up to 15 epochs with AdamW and cosine learning-rate scheduling; the highest-validation-accuracy checkpoint is retained, and the test split is used only for monitoring.Training metrics are computed on augmented training views.Source: §4.1 · p. 7

4.2 Results

  • 97.50% test accuracy and 0.0969 test loss are achieved by the selected epoch-15 checkpoint after validation accuracy peaks at 97.48%.The early-stopping counter never reaches its patience of four; validation and test curves have a maximum gap of 0.58 percentage points.Source: §4.2 · p. 7

5 Stage 2: Clean Federated Fine-Tuning

Stage 2 establishes an attack-free federated baseline for adapting the Stage 1 model to user drawings that remain on-device. Head-only FedAvg modestly improves client training accuracy while preserving global accuracy.

5.1 Client population

  • Clients have 7 to 10 classes and 1,015 to 1,890 samples, with class counts varying across the non-IID population.Client data are selected from high-loss candidates, making them harder than the Stage 1 test set; the Stage 1 model reaches 90.34% accuracy on pooled client training data versus 97.66% on the global evaluation set.Source: §5.1 · p. 8, Figure 4 · p. 9

5.2 Head-only FedAvg

  • Head-only FedAvg freezes the Stage 1 backbone, trains each client’s copied head for one local epoch per round, and aggregates heads by local training-sample count.The run uses 10 clients and 50 rounds; client and global evaluation metrics track adaptation and preservation of Stage 1 behavior, respectively.Source: §5.2 · p. 8

5.3 Results

  • Client training accuracy rises from 90.34% to 91.49% after 50 FedAvg rounds, while global accuracy ends at 97.64%, one image below its initial value.Client training loss falls from 0.3876 to 0.3703; global accuracy stays between 97.60% and 97.74% in every round.Source: §5.3 · pp. 8–9, Table 4 · p. 9
  • Client validation accuracy changes from 89.67% to 89.89%, while client validation loss bottoms at 0.4089 in round 10 and later rises to 0.4364.Training loss reaches its minimum of 0.3605 at round 17 before rising; the report describes the adaptation as modest and notes that only a linear head over fixed features is trained.Source: §5.3 · p. 9

6 Privacy: Reconstructing Private Data from Updates

A single head-only update reveals labels more reliably than drawing details, while aggregation across clients weakens label-count recovery. The attacks rely on one-step local SGD under an honest-but-curious server setting.

6.1 Threat model and gradient recovery

  • With one shared-rate SGD step, the server can recover the client gradient from its update; this equivalence does not hold for multiple steps, momentum, adaptive optimizers, or heterogeneous learning rates.The threat model is an honest-but-curious server that follows the protocol but analyzes received information.Source: §6.1 · pp. 9–10

6.2 Attack 1: a single client update

  • The true five-label set ranks first only for 0.00079 ≤ λ ≤ 0.0188; above that range, the rule recovers four of five labels.The result depends on λ, and the margin over the second-ranked set is small.Source: §6.2 · pp. 11–12, Table 5 · p. 12
  • Mean SSIM is 0.130: reconstructions are class-plausible prototypes, not copies of the private drawings.By MSE and PSNR, every reconstruction is worse than an empty canvas; the oracle-label diagnostic produces practically the same images and metrics.Source: Figure 6 · p. 11, §6.2 · p. 12, Table 6 · p. 13

6.3 Attack 2: a FedAvg aggregate update

  • For m = 5 and m = 10, label-count recovery accuracy is 75.0%, below the 80.0% uninformed uniform-allocation baseline, so the estimator provides no information about label counts beyond knowledge of N.For m ≥ 2, the solution places five classes at the bound and omits classes that occur several times.Source: §6.3 · p. 13, Figure 8 · p. 14

6.4 Summary of the privacy results

  • A single head-only update leaks its label set almost completely, but pixel reconstructions resemble class prototypes; with five or more clients, label-count recovery falls below an uninformed guess.These conclusions are specific to head-only updates, one SGD step, and a server that does not tamper with the model.Source: §6.4 · pp. 15–16

7 Integrity: Data Poisoning under FedAvg

The report compares label flipping and pixel-trigger backdoors across malicious-client fractions under FedAvg, measuring attack success against clean accuracy. Their effectiveness depends on poisoned-data mass, with label flipping requiring most source-class data to be relabeled and backdoor ASR growing with triggered-data mass.

7.1 Threat model

  • Each malicious fraction f = 0, 0.1, 0.3, 0.5, or 1.0 is tested in an independent 50-round run starting from the Stage 1 head.Malicious clients are selected randomly, so the sets for different fractions are not nested.Source: §7.1 · p. 16

7.2 Attacks

  • Label flipping relabels every malicious client's circle samples as donut, while the backdoor triggers and relabels a random 20% of its non-donut samples.The backdoor uses a fixed 32-pixel corner pattern; because the backbone is frozen, only features it already computes can support the attack.Source: §7.2 · p. 16

7.3 Metrics

  • Attack success rate measures donut predictions on clean circle images for label flipping and on triggered non-donut images for the backdoor; clean accuracy is measured on the global evaluation set.The global evaluation set has 5,000 images; label-flipping ASR uses 500 circle images, and backdoor ASR uses 4,500 non-donut images.Source: §7.3 · p. 16

7.4 Results

  • 12.60% label-flipping ASR at f = 0.3, where malicious clients hold 64.3% of circle data, contrasts with ineffective attacks when less circle data are relabeled.At f = 1.0, relabeling every circle sample yields 97.80% ASR and reduces clean accuracy to 87.92%.Source: §7.4 · pp. 16–18, Figure 11 · p. 17
  • Final ASR is plotted against poisoned-data mass: relabeled circle-training samples for label flipping and triggered training samples for the backdoor.The label-flipping plot places f = 0.5 left of f = 0.3 because the malicious sets are not nested.Source: Figure 12 · p. 18
  • 54.71% backdoor ASR at f = 1.0 and 31.27% at f = 0.5, while clean accuracy falls by at most 0.80 pp; no run reaches 80% ASR within 50 rounds.At f = 0.3, ASR is 21.87%; the attack strengthens with triggered-data mass and remains incomplete over the evaluated rounds.Source: §7.4 · pp. 18–19, Figure 13 · p. 18

8 Defense: FLTrust with a Retained Root Set

FLTrust uses a retained clean root set to generate a trusted update and weight norm-equalized client updates by their alignment with it. It suppresses partial label-flipping and backdoor attacks with little clean-accuracy change, but cannot protect against an all-malicious client population.

8.1 Method

  • The server trains a copy of the current head on 1,000 retained Stage 1 images each round to obtain a trusted update.Each client update is rescaled to the trusted update's norm, then the server aggregates norm-equalized updates using trust scores; if all scores are zero, it applies the trusted update.Source: §8.1 · p. 19

8.2 Results

  • With no malicious clients, FLTrust reaches 97.66% and 97.62% clean accuracy in the label-flipping and backdoor controls, respectively, with no measurable utility cost relative to FedAvg.The corresponding FedAvg control accuracies are 97.62% and 97.64%.Source: §8.2 · p. 19
  • At f = 0.3 and 0.5, FLTrust reduces backdoor ASR from 21.87% to 1.76% and from 31.27% to 4.02%, respectively, while suppressing label-flipping ASR at f = 0.3 from 12.60% to 0.80%.At f = 1.0, label-flipping ASR matches FedAvg at 97.80%; backdoor ASR is 50.20% versus 54.71% under FedAvg.Source: §8.2 · pp. 19–20
  • FLTrust gives less-aligned malicious updates lower trust scores, reducing their aggregate share, while norm equalization limits their magnitude.All clients still receive positive scores; when every client is malicious, the defense can only reweight poisoned updates among themselves.Source: §8.2 · p. 20

9 Computational Footprint

The tutorial's federated stage is inexpensive because it trains heads on precomputed embeddings, whereas centralized full-network fine-tuning dominates runtime. FLTrust adds modest overhead to the 50-round federated runs.

Computational footprint

  • Peak PyTorch GPU memory remains below 600 MiB in every notebook, and federated runtimes exclude one-time embedding precomputation.The reported runtime and peak-memory measurements were recorded on a Google Colab GPU runtime.Source: §9 · p. 20, Table 11 · p. 21
  • Fifty rounds of head-only FedAvg over ten clients take 12.55 s for local training and aggregation, with embeddings precomputed once.The uploaded 12,810-parameter head is 50 KiB in 32-bit floating point, 76 times smaller than the full model.Source: §9 · p. 20
  • Centralized full-network fine-tuning averages 227.7 s per epoch and takes 56.9 min for 15 epochs.This is the dominant cost of the tutorial.Source: §9 · p. 20
  • FLTrust raises average 50-round runtime from 11.57 s to 12.31 s, a 6.4% increase, by training the trusted head each round.A single head-only SGD step on five images takes 0.086 s and adds 15.3 MiB of allocated GPU memory.Source: §9 · p. 20

10 Live Session: Federated Learning with FedMaestro

The live session uses FedMaestro to let participants experience inference, federated fine-tuning, model changes, and poisoning from their own devices. It is a demonstration rather than a quantitative evaluation because participant numbers, drawings, and devices are uncontrolled.

Live session

  • The session follows inference, federated fine-tuning, and poisoning, with FedMaestro managing model distribution, participation, round execution, update collection, aggregation, and monitoring.Participants join as clients, starting from the Stage 1 checkpoint.Source: §10 · p. 21

Step 1: direct inference

  • Participants draw one of ten classes and receive the deployed model's five most probable classes with confidences.Drawings are converted to 28 × 28 grayscale QuickDraw-format bitmaps before processing.Source: §10 · p. 22, Figure 15 · p. 22

Step 2: federated fine-tuning on participant drawings

  • Participants' labeled drawings remain on-device as local training data, and the dashboard lists each federated round with its status and received updates.The session expects visible fine-tuning changes mainly for contributed classes and drawing styles; a single head update can reveal which classes its sender drew, while aggregation weakens label leakage.Source: §10 · pp. 22–23, Figure 16 · p. 23

Step 3: observing whether the model learns

  • After training rounds, participants compare updated-model predictions with their Step 1 drawings; the clearest changes are expected on contributed styles and previously confused classes.Predictions on canonical drawings are expected to remain stable, consistent with preserved global accuracy in the simulations.Source: §10 · p. 23

Step 4: participant-driven poisoning

  • In the poisoning step, participants deliberately relabel a class and compare aggregation rules; FLTrust is expected to suppress partial attacks but not an attack where every participant relabels the same class.The live session reports no quantitative results because participant counts, drawing styles, devices, and drawing counts are uncontrolled and several differ from the simulations.Source: §10 · pp. 21, 23

11 Discussion

Post-training federated personalization creates label-privacy and poisoning risks that convergence alone does not capture. The report emphasizes update-level assessment, operational safeguards, and limitations in the experiments.

Lessons for post-training federated personalization

  • A 12,810-parameter head-only update can reveal a small client batch’s label composition while carrying little pixel-level information.The report therefore identifies label privacy, rather than image privacy, as the primary concern in this regime.Source: §11 · p. 23
  • Poisoning impact depends on how much poisoned data reaches aggregation and how it is distributed across clients.In non-IID populations, a minority holding most of a class can cause more damage than a majority holding little of it.Source: §11 · p. 23

From convergence to production-ready FL

  • FL evaluation must extend beyond convergence because the same head-only FedAvg procedure can expose labels and accept a backdoor that clean accuracy does not reveal.The discussion lists trust-based aggregation, update filtering, secure aggregation, differential privacy, client authentication, and round monitoring with rollback as system components, each with its own requirements and costs.Source: §11 · p. 23

Limitations

  • FLTrust uses a clean root set and offers no protection when every client is malicious, while outlier filters can exclude honest clients with unusual data.The experiments also use a root set drawn from the global evaluation distribution, which favors the defense; different distributions may yield noisier trust scores.Source: §11 · pp. 23–24
  • The experiments are single-run, use high-loss private samples, and assume one-step SGD for privacy attacks, limiting confidence and reconstruction interpretation.The attackers are non-adaptive, and the FLTrust root set matches the global evaluation distribution, which favors the defense.Source: §11 · p. 24

12 Conclusions and Future Work

The report presents a reproducible tutorial pipeline for studying privacy and integrity risks in post-training federated personalization. Its experiments show label leakage, backdoor vulnerability, and partial-attack defenses, while identifying single-run, head-only, and non-adaptive-attack scope limits.

12 Conclusions and Future Work

  • The pipeline fine-tunes pretrained MobileNetV4 on QuickDraw-10 and applies head-only FedAvg over ten non-IID clients; FLTrust suppresses partial attacks but not an all-malicious federation.The server recovered a single client’s private label set, backdoor ASR reached 54.7% at a clean-accuracy cost below one percentage point, and label flipping succeeded only when most source-class data were relabeled.Source: §12 · p. 24
  • The results are limited to single runs, head-only updates, and attackers that neither tamper with the model nor adapt to the aggregation rule.Planned work includes differential privacy and secure aggregation, partial-backbone fine-tuning, comparisons with other robust aggregators, adaptive attackers, and repeated runs across seeds and client populations.Source: §12 · p. 24

Code and Data Availability

The report makes its six executed notebooks and outputs available under Apache License 2.0. They run in Google Colab, and the Quick, Draw! data and pretrained weights remain subject to their respective licenses.

Code and Data Availability

  • Six executed notebooks, including the outputs underlying the reported results, are available under Apache License 2.0.The notebooks are hosted in the report’s GitHub repository.Source: Code and Data Availability · p. 24
  • All notebooks run in Google Colab, allowing browser-based reproduction without local installation or high-end hardware.Source: Code and Data Availability · p. 24
  • Quick, Draw! data are subject to CC BY 4.0, while pretrained weights follow their distributor’s license.Source: Code and Data Availability · p. 24

Related papers

Loading 2610.21547v1…