PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning

Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, Hairong Qi

arXiv:1812.00535v3cs.LGcs.CRcs.CV

TL;DR

Federated learning can conceal private data from the server, yet existing GAN attacks mainly recover global class representatives rather than client-specific samples. The paper proposes mGAN-AI, a malicious-server multi-task GAN that identifies client identity while preserving the standard training process, and experiments show recovery of victim-like samples on two datasets.

  • Problem

    Existing attacks struggle to distinguishably recover private data from a specific federated-learning client beyond global class representatives.

  • Method

    mGAN-AI uses a multi-task discriminator for reality, category, and client identity, with client representatives estimated from updates to guide user-specific generation.

  • Results

    Experiments on MNIST and AT&T show that mGAN-AI reconstructs samples resembling a victim’s training data and outperforms state-of-the-art attacks.

  • Takeaways & Limitations

    A malicious server can target client-level privacy through an attack designed to operate without affecting standard federated-learning training.

Abstract

from arXiv · show

Federated learning, i.e., a mobile edge computing framework for deep learning, is a recent advance in privacy-preserving machine learning, where the model is trained in a decentralized manner by the clients, i.e., data curators, preventing the server from directly accessing those private data from the clients. This learning mechanism significantly challenges the attack from the server side. Although the state-of-the-art attacking techniques that incorporated the advance of Generative adversarial networks (GANs) could construct class representatives of the global data distribution among all clients, it is still challenging to distinguishably attack a specific client (i.e., user-level privacy leakage), which is a stronger privacy threat to precisely recover the private data from a specific client. This paper gives the first attempt to explore user-level privacy leakage against the federated learning by the attack from a malicious server. We propose a framework incorporating GAN with a multi-task discriminator, which simultaneously discriminates category, reality, and client identity of input samples. The novel discrimination on client identity enables the generator to recover user specified private data. Unlike existing works that tend to interfere the training process of the federated learning, the proposed method works "invisibly" on the server side. The experimental results demonstrate the effectiveness of the proposed attacking approach and the superior to the state-of-the-art.

I. INTRODUCTION

Federated learning decentralizes training but remains vulnerable to malicious-server reconstruction attacks. The paper introduces mGAN-AI to recover client-specific data invisibly, beyond generic class representatives.

  • Federated learning trains shared models locally, reducing centralized data storage but preserving privacy concerns under distributed training.
  • Shared model updates encode private-data patterns, enabling malicious servers or clients to mount reconstruction and inference attacks.
  • Existing GAN-based attacks generally recover class-wise representatives rather than exact samples from individual clients.
  • mGAN-AI uses a multi-task GAN that discriminates reality, category, and client identity to recover user-specified private data without modifying federated learning.
  • The method estimates client representatives from accessible updates, supports non-exclusive client class labels, and targets client-level privacy from a malicious server.
  • Experiments on MNIST and AT&T report successful recovery of samples from specific users and superiority over existing attacks.

A. Privacy-preserving Distributed Learning

Prior privacy-preserving learning methods distribute or protect model updates, while inference attacks exploit models to recover private information. The paper positions its malicious-server GAN attack against limitations of earlier approaches.

  • Distributed learning methods differ in how shared-model updates are combined, with averaging offering communication and privacy advantages over asynchronous approaches.
  • Privacy-preserving distributed learning commonly uses differential privacy or secure multiparty computation to protect training information.
  • Membership attacks determine whether a sample belongs to training data, whereas reconstruction attacks construct training samples from model access.
  • The proposed malicious-server attack avoids prior limitations involving single-sample batches, unauthorized model modification, and negative influence on standard training.
  • Federated learning clients train locally and upload updates rather than private data, while the server aggregates those updates into the shared model.
  • GANs generate samples from random inputs and train a discriminator to distinguish generated samples from real training samples.

IV. THREAT MODEL

The threat model treats the server as an adversary targeting one victim client while following federated-learning operation. It supports passive analysis of updates and more powerful active isolation without altering training.

  • The model assumes N clients with non-IID data distributions that train collaboratively on a shared model.
  • The malicious server seeks to reconstruct private data from a designated victim client for a client-level privacy attack.
  • Passive attacks analyze periodic client updates, whereas active attacks intentionally isolate the victim’s shared model for a stronger attack.
  • The proposed framework sends the current shared model to all clients, receives their updates, and targets the vth client as victim.
  • mGAN-AI uses the victim update, shared model, and client representatives, with an auxiliary real dataset supporting discriminator training.
  • The malicious server follows federated-learning rules while performing attacks without changing the shared model, objective, or updates.

C. Communication Protocol

mGAN-AI operates at the server side as a multi-task GAN for reconstructing a specified victim’s private data. Its discriminator handles reality, category, and client identity, while the generator uses category and identity labels to produce user-specific samples.

  • C. Communication Protocol: mGAN-AI is a server-side attack that reconstructs private data from a specified victim in federated learning.The malicious server receives client updates and uses them to train the attack framework.
  • C. Communication Protocol: The victim’s update can be aggregated into the latest shared model for a passive attack or directly into the discriminator for an active attack.The two routes differ in how the victim update is incorporated into the attack model.
  • C. Communication Protocol: The discriminator simultaneously performs real-fake discrimination, categorization, and client identification.The discriminator shares the shared model’s structure except for its output layer, which supports the three task-specific outputs.
  • C. Communication Protocol: The generator accepts category labels, identity labels, and noise to conditionally generate user-specified samples.Identity conditioning distinguishes the target client from other clients during generation.
  • C. Communication Protocol: The server estimates client representatives from updates because victim and other-client samples are inaccessible, using auxiliary data for the real-fake task.The training objectives update D with Lreal + Lid and G with Lcat − Lreal + Lid without affecting federated learning.

C. Passive Attack

The passive attack treats the server as honest-but-curious: it analyzes all client updates without modifying the shared model or introducing adversarial influence. It repeatedly updates the discriminator and generator using auxiliary, generated, and estimated client-representative data.

  • C. Passive Attack: The passive attack analyzes client updates without modifying the shared model or introducing adversarial influence.The server acts as honest-but-curious under this attack setting.
  • C. Passive Attack: At each iteration, the server sends the current shared model to clients, receives their updates, and averages them for the next shared model.The update sequence follows the federated-learning protocol.
  • C. Passive Attack: The discriminator is initialized from the shared model and victim update, biasing it toward the victim while retaining current categorization performance.The victim update is used for the victim identity class, whereas other client updates provide the other-client class.
  • C. Passive Attack: The server calculates client representatives, labels the victim’s representatives separately, and trains the discriminator on auxiliary, generated, and representative samples.The generator is then updated using Lcat − Lreal + Lid.
  • C. Passive Attack: Discriminator training increases with iteration time to compensate for the growing imbalance toward real data after shared-model overwrites.Training stops when federated learning reaches its maximum iteration or the attack achieves the desired result.

D. Active Attack

The active attack isolates the victim by sending an isolated shared model through an affiliated server connection. Training only on victim data is intended to improve sample quality and identity distinguishability, although this violates the original federated-learning rule.

  • D. Active Attack: The active attack isolates the victim by training mGAN-AI through an affiliated server connected only to that client.The isolated shared model M iso replaces the ordinary shared model in this connection.
  • D. Active Attack: Training mGAN-AI solely on victim data is described as producing higher-quality samples with more distinguishable identity.The identification task and client-representative calculation are removed because the victim is the only affiliated-server client.
  • D. Active Attack: The active attack sends M iso to the victim to obtain specific information and therefore violates the original federated-learning rule.The passage states that it does not introduce a negative effect on federated learning.

E. Calculation of Client Representatives

Client representatives are estimated by matching update behavior: synthesized inputs are optimized so their shared-model updates approximate each client’s observed update. Total variation regularization encourages spatially smooth image representatives.

  • E. Calculation of Client Representatives: The representative-recovery method matches each client update with an update produced by candidate representatives on the shared model.Both updates are obtained through backpropagation using the shared model’s classification loss.
  • E. Calculation of Client Representatives: Total variation regularization is added because optimization-based methods can introduce noise or artifacts.The regularizer encourages spatial smoothness in the reconstructed images.
  • E. Calculation of Client Representatives: The total-variation term computes neighborhood distances over image rows and columns for each client’s image set.Here, Xk is a set of images and i, j index image rows and columns.
  • E. Calculation of Client Representatives: Representatives are initialized from random noise and optimized with box-constrained L-BFGS.The method empirically obtains valid representatives after several updates.
  • E. Calculation of Client Representatives: The experiments validate the effectiveness of the estimated representatives for the client-identification task.The representatives are designed to provide the identity supervision unavailable from directly accessed client data.

VI. EXPERIMENTAL EVALUATION

The evaluation tests mGAN-AI’s effectiveness and compares it with model inversion and GAN-based attacks, including reconstruction quality and effects on federated learning.

  • The experiments validate mGAN-AI and compare it with model inversion and GAN-based attacks.The comparison covers reconstruction quality and side effects on federated learning.

A. Datasets and Experiment Setup

Experiments use MNIST handwritten digits and AT&T facial images, providing datasets with multiple classes or client-specific facial properties.

  • MNIST contains 70,000 grayscale 28×28 handwritten-digit images across 10 classes.The dataset is divided into 60,000 training and 10,000 testing samples.
  • AT&T contains grayscale 64×64 facial images derived from 40 persons with variation in expressions and glasses.The original images include 10 images per person and measure 92×112 before resizing.

3) Experiment setup:

The setup combines a classifier, discriminator, and generator for client-level reconstruction experiments on MNIST and AT&T, including victim-specific properties and privacy considerations.

  • mGAN-AI uses a classifier, discriminator, and generator, with the generator receiving random noise, categorical labels, and identity labels.The classifier and discriminator use convolutional networks, while the generator uses deconvolution layers.
  • The passive-attack experiment uses biased client data, allowing each client to own samples from multiple classes.The setup evaluates reconstruction against real victim samples rather than evenly splitting training data.
  • On MNIST, victim digits are rotated to create a client-specific property, and reconstructed samples match the victim’s rotation while differing from other clients.The experiment uses 10 clients, with each drawing 100 samples from three random classes.
  • On AT&T, wearing glasses is assigned as the victim property, and mGAN-AI reconstructs the victim’s first two faces while other outputs are distorted.The experiment uses 10 clients and compares victim faces with faces from other clients.
  • Differential privacy does not prevent generating distinguishable samples, and its perturbation must be carefully controlled to preserve useful federated-learning updates.The paper distinguishes sample recovery from membership inference and notes that convergence requires a relatively loose privacy budget.

C. Quantitative and Qualitative Comparison

mGAN-AI produces higher-quality, more victim-like reconstructions than MI and GAN-based attacks, while passive operation preserves federated-learning accuracy. Active mGAN-AI improves reconstruction quality at a small accuracy cost, whereas GAN-based attacks destabilize training.

  • mGAN-AI generates samples with much higher quality and greater similarity to real victim samples than MI and GAN-based attacks.The comparison uses the same federated-learning setup and AT&T dataset with N = 20 clients.
  • GAN-based attack fails to converge because its adversarial influence becomes trivial after federated averaging.Without that influence, the attack generates unrecognizable images.
  • The proposed mGAN-AI achieves a higher inception score than MI and GAN-based attack across 400 generated samples per method.
  • Passive mGAN-AI achieves the highest shared-model accuracy because it does not affect federated-learning training.
  • Active mGAN-AI provides better reconstruction quality than passive mGAN-AI but slightly degrades the learned model while maintaining stable accuracy.
Loading 1812.00535v3…