Unleashing the Tiger: Inference Attacks on Split Learning

Dario Pasquini; Giuseppe Ateniese; Massimo Bernaschi

Source-linked AI summary

Unleashing the Tiger: Inference Attacks on Split Learning

Dario Pasquini, Giuseppe Ateniese, Massimo Bernaschi

arXiv:2012.02670v5Dec 4, 2020cs.CRcs.LG

TL;DR

Split learning reduces the resources required for collaborative deep learning, but the paper investigates whether its privacy assumptions withstand adversarial participants. It develops feature-space hijacking attacks and evaluates server- and client-side inference, finding that private training data can be reconstructed or inferred and that existing defenses can be evaded.

Problem
Organizations need collaborative deep learning without sharing sensitive datasets, but split learning’s security assumptions and privacy against malicious participants had received limited scrutiny.
Method
The paper develops FSHA, in which a malicious server hijacks clients’ feature learning before inferring data from smashed representations, and adapts federated-learning attacks for malicious clients.
Results
The attacks recover precise or approximate private instances, support property inference, apply across datasets and split-learning variants, and circumvent proposed defenses.
Takeaways & Limitations
Split learning does not provide the assumed privacy protection against malicious servers or clients under the examined attack strategies.
Takeaways & Limitations
Property inference extensions are limited by the attacker’s ability to collect suitable labeled data, while detecting feature-space hijacking is difficult because clients lack full-network access.

Abstract

from arXiv · show

We investigate the security of Split Learning -- a novel collaborative machine learning framework that enables peak performance by requiring minimal resources consumption. In the present paper, we expose vulnerabilities of the protocol and demonstrate its inherent insecurity by introducing general attack strategies targeting the reconstruction of clients' private training sets. More prominently, we show that a malicious server can actively hijack the learning process of the distributed model and bring it into an insecure state that enables inference attacks on clients' data. We implement different adaptations of the attack and test them on various datasets as well as within realistic threat scenarios. We demonstrate that our attack is able to overcome recently proposed defensive techniques aimed at enhancing the security of the split learning protocol. Finally, we also illustrate the protocol's insecurity against malicious clients by extending previously devised attacks for Federated Learning. To make our results reproducible, we made our code available at https://github.com/pasquini-dario/SplitNN_FSHA.

1 INTRODUCTION

Split learning offers lightweight collaborative training without directly sharing private datasets, but its security has received far less scrutiny than its practical benefits. This paper argues that malicious servers and clients can exploit the framework to infer or reconstruct private training data.

Deep learning depends on large representative datasets, yet organizations may avoid sharing sensitive data because of competition, privacy, and regulatory concerns.
Distributed learning lets parties train shared models without directly exchanging their training sets, offering an alternative to expensive cryptographic protocols.
Split learning reduces client-side resource requirements relative to federated learning, but its security has been investigated much less extensively than its practical properties.
The paper introduces FSHA, a domain-independent malicious-server strategy that hijacks model learning to recover private training instances without knowing clients’ data or architectures.
The authors also adapt federated-learning inference attacks to show that malicious clients can recover suitable approximations of other clients’ private instances.
The contribution includes precise server-side reconstructions, arbitrary-attribute property inference, defensive-technique circumvention, and prototypical client-side reconstructions.

2 DISTRIBUTED MACHINE LEARNING

Distributed and split learning partition training across clients and servers to reduce data sharing and client resources. The paper’s security analysis challenges the assumption that intermediate representations adequately protect private training data.

Distributed machine learning enables remote clients to train a shared model while keeping their private training instances from direct disclosure.
Federated learning: Federated learning exchanges local parameter updates with a central server, but communication scales linearly with model parameters and participants generally know the full network.
Split learning: Split learning partitions consecutive network layers between clients and a server, which handles most computation while clients send intermediate outputs rather than raw inputs.
Split learning: During split-learning training, clients transmit smashed activations, the server computes forward propagation and loss, and gradients are returned for local back-propagation.
Split-learning variants: A client-side loss variant avoids sending labels to the server, while multi-client training can use sequential round-robin updates or parallel splitfed extensions.
Security analysis: The paper identifies a central vulnerability: a malicious server can control clients’ front network f and guide it toward states that make private inputs recoverable from f(X_priv).
Security analysis: The analysis further states that split learning fails to protect honest clients from malicious clients even when the server is honest.

3 FEATURE-SPACE HIJACKING ATTACK

The feature-space hijacking attack (FSHA) lets a malicious server steer the client network into an attacker-chosen feature space and invert smashed data to reconstruct private instances. Experiments show accurate reconstruction across splits, datasets, mismatched and incomplete public distributions, while property inference and client-side attacks further expose split learning's insecurity.

3.2 Attack foundations: FSHA hijacks the client network's learning process by replacing the original objective with one that shapes its feature space toward an attacker-crafted target.The server forges gradients to force the client network toward the target feature space, even without knowing its architecture or weights.
3.2 Attack foundations: After setup, the attacker applies a known inverse network to smashed data and obtains suitable approximations of private training instances.The attack works because the client network's codomain overlaps the attacker-controlled feature space.
3.3 Attack implementations: Across four splits, reconstruction error differs negligibly; network depth mainly changes setup convergence speed, with limited effect on final performance.Low-entropy datasets reach accurate reconstructions within 10^3 setup iterations, while CelebA and Omniglot require about 3 · 10^3 and 2 · 10^3 iterations, respectively.
3.3 Attack implementations: The deepest split produces very accurate reconstructions across all tested datasets, and Omniglot results suggest generalization over private data.Omniglot contains 1623 classes with few examples per class, making the reported reconstruction performance notable within the experiment.
3.4 Public dataset variations: Distribution mismatch between public and private data affects performance negligibly, while the attack can reconstruct unobserved private classes by interpolating over known representations.A public dataset closer to the attacked distribution yields better reconstruction, but accurate reconstructions remain possible despite discrepancies.
3.5 Property inference attacks: Property inference attacks achieve accuracy higher than 90% within limited iterations and can target multiple attributes, multiclass labels, or regression tasks.The main limitation is the attacker's ability to collect suitable labeled data for setup.

4 ON DEFENSIVE TECHNIQUES

The paper argues that split-learning defenses fail against an adversary who controls the learning process. Distance-correlation regularization can be neutralized by manipulating the target feature space or scaling the server-side task loss, while attack detection remains difficult.

Distance correlation minimization: Distance-correlation regularization reduces dependence between raw inputs and smashed data by jointly weighting privacy and task losses.The distance-correlation term depends only on the client network and private data, so clients can compute it locally.
Distance correlation minimization: α1 = 10000 makes the defense effective on MNIST, but privacy loss then eclipses task loss and prevents practical task utility.At such values, reducing the task loss becomes impossible or extremely slow, regardless of whether the task is client-selected or adversarial.
Distance correlation minimization: A malicious server can scale the adversarial task loss by 25, making even impractically large α1 values ineffective.Because the server controls the training objective, it can indirectly control the ratio between client-side privacy loss and adversarial task loss.
Distance correlation minimization: A malicious server can circumvent distance-correlation minimization by forging a feature space that remains invertible and supports precise input reconstruction.The attack influences the client network’s learning process, negating the intended defensive effect.
Detecting the attack: Clients cannot reliably detect feature-space hijacking because they lack the full distributed model and the ability to query it for anomalies.In the private-label setting, the attacker can additionally send fake inputs to a forged server-side network that minimizes the client’s loss.

5 THE SECURITY OF SPLIT LEARNING AGAINST MALICIOUS CLIENTS

Split learning remains vulnerable to malicious clients: adapted federated-learning attacks can reconstruct private instances by manipulating labels, gradients, and model updates. The attack succeeds in experiments across target classes and can also apply to parallel split-learning variants.

Threat model and attack adaptation: Previously proposed GAN-based attacks remain applicable to split learning with minimal adaptations.The paper challenges claims that splitting prevents client-side attacks developed for federated learning.
Threat model and attack adaptation: A malicious client trains a generator to produce target-class samples and uses the distributed classifier as its discriminator.The attacker labels generated samples with a controlled target label and alternates poisoning with generator-training iterations.
Gradient-scaling workaround: Because the server updates one network split, the attacker scales gradients to make that update negligible while preserving a training signal for the generator.The client scales the forward gradient by a small ε, then rescales the returned gradient by ε^-1 without updating the split networks.
Limitations and detectability: The gradient-scaling mechanism creates a trade-off between attack secrecy and performance because scaled gradients can resemble updates from easy examples.A server may detect the manipulation, but distinguishing it from legitimate high-confidence training batches is difficult.
Experimental validation: The client-side attack successfully reproduced target-class instances after 10000 global training iterations on MNIST and AT&T.The validation used 10 honest clients and one malicious client; the same attack also applies to parallel extensions such as Splitfed learning.

6 FINAL REMARKS

The paper concludes that split learning has structural privacy vulnerabilities exploitable by both servers and clients. It further argues that split learning can leak more information than federated learning while making inference attacks harder to detect.

Final remarks: Attackers can reconstruct training instances or infer properties of them, and proposed split-learning defenses can be evaded.These findings violate the protocol’s intended privacy-preserving property.
Final remarks: Compared with federated learning, split learning appears to leak more information and makes ongoing inference attacks harder to detect.In federated learning, participants hold the full network, enabling simple detection mechanisms that can thwart unsophisticated attacks.

A ADDITIONAL RESULTS

This section provides additional experimental results beyond those presented earlier in the paper.

Additional results: The appendix extends the FSHA evaluation to additional datasets.The detailed dataset-specific results follow in the next subsection.

A.1 On the effect of the public dataset

FSHA can reconstruct private instances even when the attacker’s public dataset differs substantially from the private distribution. The appendix reports this behavior for natural and medical-image datasets, including a difficult cross-domain setup.

Natural images: The appendix evaluates FSHA on TinyImageNet and STL-10, two natural-image datasets with different class and image compositions.STL-10 has 10 classes, while TinyImageNet has 200, with no image intersection between the sets.
Natural images: Despite 95% of TinyImageNet’s private semantic classes being unobserved publicly, FSHA produced suitable private-instance reconstructions using STL-10 as the public set.The reconstructions showed altered colors and missing details but converged toward the private instances.
Medical images: FSHA also reconstructed private HAM10000 instances when ISIC-2016 served as the public dataset.This was tested as a worst-case public-private dataset pairing in the medical-image setting.
Medical images: Recovered images could be used directly to re-identify patients, potentially violating privacy rules.The paper states this consequence for the real-world scenario.
Figures: Figure A.1 compares original TinyImageNet images with reconstructions using TinyImageNet test data or STL-10 as the public set.Figure A.2 analogously compares original HAM10000 images with reconstructions using HAM10000 test data or ISIC-2016.

A.2 Property inference attacks

The attacker infers categorical properties by training a multiclass classifier on the AT&T dataset, identifying which of 40 individuals appears in each training image.

The attacker trains C_att as a multiclass classifier to infer categorical attributes from images used during distributed training.The AT&T dataset contains frontal images of 40 individuals, with 10 images per person.
A single fully connected layer with 40 output units is trained using categorical cross-entropy during the attack setup.
The attacker reaches perfect accuracy within a few initial iterations when classifying images from the 40 individuals.

B ARCHITECTURES AND EXPERIMENTAL SETUPS

The experimental setups use specified CNN architectures, attack hyper-parameters, preprocessing conventions, and deliberately different client and attacker networks for the feature-space hijacking attack.

Architectures: Convolutional layers are specified by filter count, kernel size, stride, and activation function, whereas dense layers use node count and activation function.
Architectures: The discriminator D is built from the residual block described in Algorithm 2, with its architecture listed among the attack setup materials.
Architectures: The client network f is a standard CNN, while attacker network f̃ produces the same-shaped tensor but differs in kernels, activations, and pooling.The attacker network omits pooling layers and reduces kernel width using larger convolutional strides.
Architectures: The architectures of f and f̃ are intentionally different to match the threat model, although similar architectures significantly speed up the attack.
Experimental setup: Additional attack hyper-parameters are reported in Table B.2.
Experimental setup: Images are resized to dataset-specific resolutions, channels are replicated for several datasets, and all color intensities are scaled to [−1, 1].MNIST, Fashion-MNIST, Omniglot, and AT&T use 32 × 32 × 3 tensors; CelebA and UTKFace use 64 × 64 resolution.

B.1 Client-side attack

The client-side attack uses a DCGAN-like architecture for the split networks and generator, with a 100-dimensional Gaussian latent space and the architectures detailed in Table B.1.

The client-side attack relies on a DCGAN-like architecture for the splits f, s, and f′ and for the generator G.
The generator uses a latent space of cardinality 100 with a standard Gaussian prior.
The generator architecture combines a dense layer, batch normalization, LeakyReLU activations, and transposed convolutions, ending with tanh.

C EVADING THE DISTANCE CORRELATION METRIC VIA ADVERSARIAL FEATURE SPACES

The paper constructs adversarial feature spaces that minimize distance correlation while preserving accurate input reconstruction, then validates this behavior experimentally on CelebA.

Limitations: Distance correlation can be evaded by highly complex neural-network mappings that allocate recoverable information outside what the metric captures.
Experimental setup: The appendix reports the architectures and hyper-parameters used for the feature-space hijacking attack.
Adversarial feature spaces: A function f can produce smashed data with minimal distance correlation to the input while a decoder f̃^-1 accurately reconstructs that input.The construction hides information in blind spots of distance correlation metrics.
Adversarial feature spaces: The attack learns f by jointly minimizing distance correlation and the decoder’s reconstruction error.This objective is designed to make the representation privacy-oriented according to distance correlation while remaining reconstructable.
Experimental validation: The CelebA experiment uses MSE as the distance function, α2 = 50, and 10^4 training iterations.Figure C.1 reports average distance correlation and average reconstruction error for the same model under three losses.
Experimental validation: The adversarially trained pair minimizes distance correlation while enabling reconstruction of the input, unlike distance-correlation-only training.The comparison includes a reconstruction-error-only autoencoder objective.