PaperlayerView exact revision on arXiv

Source-linked AI summary

APRIL: Finding the Achilles' Heel on Privacy for Vision Transformers

Jiahao Lu, Xi Sheryl Zhang, Tianli Zhao, Xiangyu He, Jian Cheng

arXiv:2112.14087v1cs.CV

TL;DR

Gradient sharing in federated learning can expose private training data, but gradient-leakage research had largely focused on fully connected and convolutional networks. APRIL analyzes vision-Transformer leakage through self-attention and position embeddings; the paper reports closed-form recovery under certain conditions and stronger optimization-based reconstructions than prior attacks on ImageNet. It also presents fixed position embeddings as an effective defense.

  • Problem

    Gradient-leakage attacks can recover private training data, while the risk for vision Transformers and the role of their position embeddings had not been established.

    Source: §1 · pp. 1–2

  • Method

    APRIL analyzes self-attention gradients and uses position-embedding gradients in closed-form and optimization-based attacks to recover Transformer inputs.

    Source: §3.1 · p. 4, §3.2 · p. 6, §3.3 · p. 7

  • Results

    Under certain conditions, self-attention with learnable position embeddings permits closed-form leakage regardless of network complexity; APRIL-Optimization reveals more ImageNet content than DLG, IG, and TAG.

    Source: §1 · p. 2, §4.1 · p. 9

  • Takeaways & Limitations

    The paper presents fixed position embeddings as an effective defense and points toward privacy-preserving Transformer design.

    Source: §5 · p. 13

    The closed-form attack relies on a strong assumption that limits its use in real-world Transformer designs, while the optimization attack implicitly solves a nonlinear system.

    Source: §5 · p. 14

Abstract

from arXiv · show

Federated learning frameworks typically require collaborators to share their local gradient updates of a common model instead of sharing training data to preserve privacy. However, prior works on Gradient Leakage Attacks showed that private training data can be revealed from gradients. So far almost all relevant works base their attacks on fully-connected or convolutional neural networks. Given the recent overwhelmingly rising trend of adapting Transformers to solve multifarious vision tasks, it is highly valuable to investigate the privacy risk of vision transformers. In this paper, we analyse the gradient leakage risk of self-attention based mechanism in both theoretical and practical manners. Particularly, we propose APRIL - Attention PRIvacy Leakage, which poses a strong threat to self-attention inspired models such as ViT. Showing how vision Transformers are at the risk of privacy leakage via gradients, we urge the significance of designing privacy-safer Transformer models and defending schemes.

1 Introduction

Federated learning shares gradients rather than raw data, but those gradients can reveal private training data, and prior attacks focused mainly on fully connected and convolutional networks. This section introduces APRIL to analyze vision-Transformer leakage, identifying learnable position embeddings as a vulnerability and proposing a fixed embedding defense.

1 Introduction

  • The paper identifies a gap in gradient-leakage research: prior attacks primarily targeted fully connected and convolutional networks, leaving vision Transformers understudied.Earlier work on a Transformer-based language model used optimization, while the inherent reason for Transformer vulnerability remained unclear.Source: §1 · pp. 1–2
  • Federated learning keeps local training data from collaborators by sharing gradients, yet gradient inversion can partially recover that private data.The paper notes this threat in a setting widely used for privacy-sensitive applications such as hospital data.Source: §1 · p. 1
  • Vision Transformers have expanded into varied computer-vision tasks, and their data-hungry nature makes large-scale federated learning attractive.The introduction cites ViT's image-classification success and subsequent Transformer applications across vision tasks.Source: §1 · p. 2
  • APRIL combines an analytic gradient-leakage analysis with optimization-based recovery to investigate vision-Transformer privacy.The authors contrast this vision-Transformer analysis with prior Transformer-language-model attacks that did not explain the inherent vulnerability or address position embeddings.Source: §1 · p. 2
  • Jointly using self-attention and learnable position embeddings creates severe privacy risk, with a closed-form leakage solution available under certain conditions regardless of network complexity.The paper also proves that classic self-attention inputs can be perfectly reconstructed from their gradients without intractable optimization.Source: §1 · p. 2
  • The authors propose fixed rather than learnable position embeddings as a defense and report empirical evidence for its effectiveness.This defense follows their finding that learnable position embeddings are a weak spot for privacy.Source: §1 · p. 2

2 Preliminary

Federated learning trains a global model from client-held data by aggregating locally computed gradients, but a curious server may use those gradients to reconstruct private examples. This section frames optimization-based and closed-form attacks, reviews relevant recovery methods, and motivates extending leakage analysis to vision Transformers.

2 Preliminary

  • Clients keep samples local, compute local losses and gradients, and periodically upload gradients for server aggregation and model updates.The global objective weights clients' local losses, with nonnegative client weights summing to one.Source: §2 · p. 3
  • An honest-but-curious server can attempt to reconstruct private training data from shared gradients without disrupting training.Prior threat models mainly pursue optimization-based or closed-form recovery.Source: §2 · p. 3
  • Optimization-based recovery matches gradients from dummy inputs to those from real inputs, with prior methods adding label inference, image priors, or generative models.The cited examples include DLG, iDLG, GradInversion, and GIML.Source: §2 · p. 3
  • Closed-form gradient leakage can solve some recovery problems analytically, but it depends heavily on network architecture and cannot always guarantee a solution.The section also notes that optimization procedures lack a sufficient condition guaranteeing a unique optimizer.Source: §2 · p. 3
  • Vision Transformers are used across vision tasks, while prior gradient-leakage attacks mainly used CNNs and Transformer-language-model work did not examine analytic solutions or position embeddings.The paper therefore sets out to study both closed-form and optimization-based leakage for vision Transformers.Source: §2 · pp. 3–4

3 APRIL: Attention PRIvacy Leakage

APRIL analyzes gradient leakage in self-attention and uses the exposed gradient of a learnable position embedding to recover inputs. It presents a closed-form attack under specific architectural conditions and an optimization-based attack applicable to different Transformer designs.

3.1 Analytic Gradient Attack on Self-Attention

  • If the loss derivative with respect to a self-attention input is known, that input can be uniquely recovered by solving a linear system.The system is overdetermined when the channel dimension c is much larger than the patch number p, a condition the paper says is common in deep networks.Source: §3.1 · pp. 4–5

3.2 Position Embedding: The Achilles’ Heel

  • A learnable position embedding exposes the loss derivative with respect to the self-attention input, making the analytic recovery condition available to an attacker.In the VGG-style design, closed-form APRIL requires the position-embedding output to be the multi-head-attention input; optimization-based APRIL can be placed in either illustrated design.Source: §3.2 · pp. 5–6, Figure 1 · p. 6
  • The authors suggest replacing learnable position embeddings with fixed ones or keeping their gradient updates on local clients as defenses.They report empirical results certifying the effectiveness of the fixed-embedding defense.Source: §3.2 · p. 6

3.3 APRIL Attacks on Vision Transformer

  • Closed-form APRIL recovers image pixels by applying the patch-embedding projection’s pseudoinverse to the recovered embedding.The patch embedding maps pixels linearly to features; its bias is represented in an augmented matrix.Source: §3.3 · p. 7
  • Optimization-based APRIL matches gradients and regularizes the cosine similarity between real and dummy position-embedding derivatives, extending to typical ViT designs.The loss combines two matching terms, with α balancing their contributions.Source: §3.3 · pp. 7–8

4 Experiments

Experiments assess APRIL attacks and defenses across image-classification Transformer settings. APRIL produces stronger reconstructions than existing attacks, while fixed position embeddings or withholding their gradients can prevent meaningful reconstructions.

4 Experiments

  • Experiments evaluate APRIL’s privacy attack on Transformer image-classification models across MNIST, CIFAR-10, and ImageNet; Table 1 reports mean and standard deviation of MSE over 500 reconstructions.For MNIST and CIFAR-10, 50 images were randomly selected from each class; for ImageNet, one image was selected for each of 500 random classes.Source: §4 · pp. 8–9, Table 1 · p. 9

4.1 APRIL as the Gradient Attack

  • Optimization-based APRIL converges faster than DLG and TAG and generally reaches a better terminal point, producing smoother, cleaner reconstructions.Figure 3 visualizes the optimization process and reports that APRIL avoids bad local minima more readily.Source: Figure 3 · p. 9, §4.1 · p. 10
  • For batched input images, optimization-based APRIL attack results were described as impressive, while the closed-form attack was limited because batch gradients yield only an averaged version of z.Before optimization, they used Yin et al.’s trick to restore batch labels.Source: §4.1 · p. 10
  • APRIL outperforms existing privacy attacks on Transformers, with near-perfect closed-form reconstructions and optimization results that reveal sensitive image details where competing methods struggle on ImageNet.The experiments report best results nearly across task settings; on ImageNet, DLG, IG, and TAG reconstructions are nearly unrecognizable, while optimization-based APRIL reveals color and shape.Source: §4.1 · pp. 9–10

4.2 APRIL-Inspired Defense Strategy

  • With ViT-Base’s original hidden dimension, closed-form APRIL can entirely leak an input image; shrinking dimensions to a small value, such as half the patch count, provides solid protection.The authors question whether such a small channel count would preserve high accuracy.Source: §4.2 · p. 11, Figure 5 · p. 11
  • Gradient-noise defense depends on noise scale relative to gradient norm: Gaussian variance below 0.1 times that norm, or Laplacian variance below 0.01 times it, does not work.The reported defense ability increases as noise variance rises.Source: §4.2 · p. 11, Figure 6 · p. 11
  • Switching learnable position embeddings to fixed ones, or not sharing their gradients, is presented as a practical defense that makes optimization reconstructions semantically meaningless.When position-embedding gradients are not matched, gradient matching can still occur but does not produce meaningful reconstructions.Source: Figure 7 · p. 12, §4.2 · p. 13

5 Discussion and Conclusion

The discussion identifies learnable position embeddings as a Transformer privacy weakness and presents APRIL attacks and defenses, while noting that the closed-form attack relies on a strong assumption.

5 Discussion and Conclusion

  • Twin data emerge when position embeddings are no longer shared, which the paper presents as validating a defense and the embeddings’ critical privacy role.Source: Figure 8 · p. 13
  • APRIL attacks shared Transformer gradients, using closed-form input recovery in certain cases and optimization-based attacks with position embeddings in broader scenarios.Source: §5 · p. 13
  • The closed-form APRIL attack relies on a strong assumption that limits its use in real-world Transformer designs.The optimization-based attack implicitly solves a nonlinear system; the relationship between the two attacks remains open for exploration.Source: §5 · p. 14
  • The authors warn that APRIL could be used to steal user data from existing federated-learning systems and urge privacy-safer Transformer blocks.Source: §5 · p. 14

A Do Twin Data Come From Lacking of Parameters Involved in Gradient Matching?

The experiments test whether twin data result from fewer parameters in gradient matching and report that they do not.

A Do Twin Data Come From Lacking of Parameters Involved in Gradient Matching?

B.1 Optimization-based Attack Towards Single Image

The single-image experiments report mostly successful ImageNet attacks, with some reconstructions failing at local minima and producing block artifacts.

B.1 Optimization-based Attack Towards Single Image

  • Six successful and two failed reconstructions are shown; the failures can occur when optimization reaches a bad local minimum and produces block artifacts.Source: §B.1 · p. 19
  • Attack difficulty has no preference for particular classes, while higher-contrast images can have stronger block artifacts in their reconstructions.Source: §B.1 · p. 19
  • Each Fig. 11 row shows one reconstruction process, with the ground-truth input image on the left.Source: §B.1 · p. 19

B.2 Optimization-based Attack Towards Batched Images

The batched-image experiments report recognizable reconstructions even as batch size increases from 4 to 24 under the same iteration and learning-rate settings.

B.2 Optimization-based Attack Towards Batched Images

  • Optimization-based APRIL reconstructions remain recognizable as batch size increases from 4 to 24 with the same iterations and learning rates.Source: §B.2 · p. 19
  • Inverting several images in a batch is reported as not markedly harder than inverting a single image.Source: §B.2 · p. 19
  • The batched-image results are shown in Fig. 12 and Fig. 14.Source: §B.2 · p. 19

B.3 Closed-Form APRIL Attack Towards Batched Images

For batched inputs, closed-form APRIL averages gradients across samples and produces one image proxy that provides almost no useful information about the originals. Unlike R-GAP’s additive-looking batch results, APRIL’s multiplicative result is not recognizable or easy to interpret.

B.3 Closed-Form APRIL Attack Towards Batched Images

  • R-GAP’s batched outputs look like additive combinations of the original images.Source: §B.3 · p. 19
  • The closed-form attack’s batch result provides almost no useful information about the original images.Figure 10 shows perplexing patterns of tanglement.Source: Figure 10 · p. 19, §B.3 · p. 19
  • APRIL’s multiplicative batch result is not recognizable or easy to interpret.Source: §B.3 · p. 19

Related papers

Loading 2112.14087v1…