Source-linked AI summary
Source Inference Attacks in Federated Learning
Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Xuyun Zhang
TL;DR
Membership inference in federated learning can identify training members, but existing attacks do not identify which client owns a record—a distinction with potential privacy consequences. This paper proposes a Bayesian source inference attack using local-model prediction losses, and experiments on synthetic and real datasets validate its efficacy.
Problem
Existing membership inference attacks on FL distinguish training members from non-members but do not identify which client owns a training member, leaving source privacy unexplored.
Source: §1 · p. 2Method
A Bayesian attack uses local-model prediction losses to let an honest-but-curious server estimate a training member’s source without violating the FL protocol.
Source: §1 · p. 2, §3 · pp. 4–5Results
Experiments on one synthetic and five real-world datasets validate SIA’s efficacy across FL settings, with success linked to local-model generalizability and local-data diversity.
Source: §1 · p. 2Takeaways & Limitations
SIA extends privacy analysis beyond whether a record was used for training to which client contributed it.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Federated learning (FL) has emerged as a promising privacy-aware paradigm that allows multiple clients to jointly train a model without sharing their private data. Recently, many studies have shown that FL is vulnerable to membership inference attacks (MIAs) that can distinguish the training members of the given model from the non-members. However, existing MIAs ignore the source of a training member, i.e., the information of which client owns the training member, while it is essential to explore source privacy in FL beyond membership privacy of examples from all clients. The leakage of source information can lead to severe privacy issues. For example, identification of the hospital contributing to the training of an FL model for COVID-19 pandemic can render the owner of a data record from this hospital more prone to discrimination if the hospital is in a high risk region. In this paper, we propose a new inference attack called source inference attack (SIA), which can derive an optimal estimation of the source of a training member. Specifically, we innovatively adopt the Bayesian perspective to demonstrate that an honest-but-curious server can launch an SIA to steal non-trivial source information of the training members without violating the FL protocol. The server leverages the prediction loss of local models on the training members to achieve the attack effectively and non-intrusively. We conduct extensive experiments on one synthetic and five real datasets to evaluate the key factors in an SIA, and the results show the efficacy of the proposed source inference attack.
1 Introduction
The introduction identifies source privacy as a gap in federated-learning privacy research and proposes an attack that infers which client owns a training record. It presents a server-side, non-intrusive approach and evaluates its effectiveness across varied datasets and FL settings.
1 Introduction
- Federated learning enables collaborative model training without clients directly sharing their private data.The introduction describes countries or regions facing privacy laws and regulations such as GDPR, and cites FL as a way to train jointly [10].Source: §1 · pp. 1–2
- Existing membership inference attacks identify training members but ignore which client owns them, leaving source privacy unexplored.The introduction notes that identifying a source hospital in a high-risk region may increase a record owner’s discrimination risk.Source: §1 · p. 2
- The paper frames SIA as extending membership inference by identifying the client behind a training record.It assumes an honest-but-curious central server that knows client identities and receives their updates; the attack does not interfere with training or affect model prediction performance.Source: §1 · p. 2
- SIA uses a Bayesian perspective and local-model prediction losses to let an honest-but-curious server estimate a training member’s source without violating the FL protocol.The authors report evaluations on one synthetic and five real-world datasets across different FL aspects.Source: §1 · p. 2
- The reported attack success is associated with local-model generalizability and diversity in local data.The paper states that experiments across varied FL settings validate SIA’s efficacy.Source: §1 · p. 2
2 Preliminaries
The preliminaries review federated learning and membership inference attacks, providing the background for the paper’s source-inference problem.
2 Preliminaries
- The preliminaries review federated learning and membership inference attacks as background for the paper.Source: §2 · p. 3
2.1 Federated Learning
- FedAvg trains a central model through communication rounds: clients optimize locally, return models, and the server averages them.Clients share model weights or gradients, not their data.Source: §2.1 · p. 3
2.2 Membership Inference Attacks
- Membership inference determines whether a record appeared in a model’s training dataset, and prior work has demonstrated attacks against FL models.The section contrasts this with the paper’s focus on identifying which client owns a training member.Source: §2.2 · p. 3
3 Source Inference Attacks
The section derives a Bayesian source inference attack for a training record and shows how local-model losses guide an honest-but-curious server’s estimate. It then proposes FEDSIA, which implements the attack within FedAvg without violating its protocol.
3 Source Inference Attacks
- The attack estimates whether a record belongs to client k from its local model and the record, with source status represented by a one-hot K-client vector.The setup assumes supervised classification, independent source variables, and probability λ that a record comes from client k.Source: §3 · pp. 3–4
- A Bayesian formulation gives the optimal posterior estimate of a record’s source from a local optimized model and the record.The formulation conditions on information about the other training records and their source statuses.Source: §3 · pp. 3–4
- The loss-based formulation assumes a posterior distribution over model parameters, with temperature γ controlling parameter stochasticity.It expresses the source estimate through a gap between the local model’s loss on the record and the expected loss under models trained without it.Source: §3 · p. 4
- A client’s lower loss on a record raises the posterior probability that the client owns it, motivating FEDSIA to select the client with the smallest loss.If the local model’s loss matches the average loss of models trained without the record, the posterior equals the prior and inference is a random guess.Source: §3 · pp. 4–5
- FEDSIA lets an honest-but-curious server run source inference on received local models during FedAvg communication rounds without violating the protocol.The proposed procedure selects the client whose local model has the smallest loss on the record.Source: §3 · p. 5
4 Experiments
The experiments evaluate source inference across six datasets and varied FL settings. They report that greater data heterogeneity increases attack success, while the effect of additional local epochs varies across datasets.
4.1 Datasets and Model Architectures
- The study uses convolutional networks for MNIST, CHMNIST, and CIFAR-10, and one-hidden-layer fully connected networks for Synthetic, Location, and Purchase.The authors state that their goal is demonstrating source identification, not achieving the highest classification accuracy.Source: §4.1 · pp. 5–6
- Evaluation uses six datasets, 100 target training records per client, and attack success rate (ASR), reporting the highest ASR across 20 communication rounds.The datasets include Synthetic, Location, Purchase, CHMNIST, MNIST, and CIFAR-10; the setup sets the client fraction C to 1.Source: §4.1 · pp. 5–6
4.2 Factors in Source Inference Attack
- The experiments vary data heterogeneity using a Dirichlet distribution and vary local training epochs to examine factors linked to SIA performance.The reverse of α’s magnitude reflects heterogeneity; the study links heterogeneity and local epochs to overfitting.Source: §4.2 · p. 6, §4.4 · p. 7
4.3 Source Inference on Synthetic Dataset
- On Synthetic data, SIA outperforms random guessing, and ASR increases as client-data heterogeneity rises.The experiments assume 10 clients and E = 5; the authors present the random-guess comparison as evidence that random guessing is a lower bound for optimal source inference.Source: §4.3 · p. 7
4.4 Source Inference on All Dataset
- Increasing non-i.i.d. levels raises SIA ASR across models, while increasing local epochs raises ASR on CHMNIST, CIFAR-10, and Location but changes it little on MNIST, Synthetic, and Purchase.The authors attribute the varied epoch effects to local epochs not increasing overfitting in all models.Source: §4.4 · p. 7
5 Discussion
The discussion evaluates differential privacy (DP) as a defense against source inference. DP reduces attack success in the reported settings, but the model utility costs are substantial.
5 Discussion
- Prior work suggests DP for inference-attack defense based on its theoretical privacy guarantee.Source: §5 · p. 7
- DP lowered ASR from 75.1% to 55.1% on Location and from 55.2% to 24.4% on CIFAR-10.The comparison is reported in Table 1.Source: §5 · p. 8, Table 1 · p. 8
- With DP, model utility fell from 71.2% to 12.3% on Location and from 68.3% to 10.0% on CIFAR-10.The authors conclude that vanilla DP is not an effective SIA solution in these experiments.Source: §5 · p. 8
6 Related work
Prior FL inference-attack work addresses whether records are training members, while source inference addresses which client owns them. The related-work discussion also notes that existing FL defenses have not addressed source inference attacks.
6.1 Inference Attacks in FL
- Existing FL membership inference attacks distinguish training members from non-members but do not identify which client owns a training member.The cited examples include an attack with 0.99 precision and perfect recall on FourSquare location data, and an active attack that extracts more membership information.Source: §6.1 · p. 8
- The paper positions source inference as filling the client-ownership gap left by existing FL inference attacks.Source: §6.1 · p. 8
6.2 Privacy Defenses in FL
- Prior FL privacy defenses include centralized and local DP and secure aggregation, but the discussion says none had been explored for SIAs.It points to DP’s trade-off between model utility and SIA defense performance as an opportunity for future work.Source: §6.2 · p. 8
7 Conclusion
The conclusion presents source inference as an FL attack that identifies the client owning a training example. It reports a formally derived loss-based strategy and experiments demonstrating attack effectiveness.
7 Conclusion
- Source inference lets a malicious server infer which client owns a training example in FL.Source: §7 · p. 8
- The derived attack strategy estimates source from local-model loss and is described as optimal and non-trivial.Source: §7 · p. 8
- Experiments across real datasets and varied settings are reported as demonstrating SIA effectiveness in practice.Source: §7 · p. 8