PaperlayerView exact revision on arXiv

Source-linked AI summary

CAFE: Catastrophic Data Leakage in Vertical Federated Learning

Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, Tianyi Chen

arXiv:2110.15122v4cs.LGcs.AI

TL;DR

Private data may leak through gradients shared in federated learning, while larger batches are often expected to make recovery harder. This paper proposes CAFE, a theoretically justified large-batch attack for vertical FL and evaluates its recovery performance against existing methods. It concludes that large-scale private data can be recovered from aggregated gradients and proposes fake gradients as a practical countermeasure.

  • Problem

    Existing leakage attacks often do not scale to large-batch recovery and lack theoretical justification, leaving the protective effect of larger batches uncertain.

  • Method

    CAFE uses data indices and internal-representation alignment to recover large-scale data in VFL, with theoretical analysis of recovery performance.

  • Results

    CAFE recovers large-scale private data from shared aggregated gradients in VFL and shows superior data recovery performance over state-of-the-art methods.

  • Takeaways & Limitations

    The results indicate a risk of catastrophic data leakage in VFL, while fake gradients provide a practical countermeasure intended to mitigate CAFE.

  • Takeaways & Limitations

    The method is evaluated using image datasets, although the paper states that CAFE can be applied to any type of data.

Abstract

from arXiv · show

Recent studies show that private training data can be leaked through the gradients sharing mechanism deployed in distributed machine learning systems, such as federated learning (FL). Increasing batch size to complicate data recovery is often viewed as a promising defense strategy against data leakage. In this paper, we revisit this defense premise and propose an advanced data leakage attack with theoretical justification to efficiently recover batch data from the shared aggregated gradients. We name our proposed method as catastrophic data leakage in vertical federated learning (CAFE). Comparing to existing data leakage attacks, our extensive experimental results on vertical FL settings demonstrate the effectiveness of CAFE to perform large-batch data leakage attack with improved data recovery quality. We also propose a practical countermeasure to mitigate CAFE. Our results suggest that private data participated in standard FL, especially the vertical case, have a high risk of being leaked from the training gradients. Our analysis implies unprecedented and practical data leakage risks in those learning settings. The code of our work is available at https://github.com/DeRafael/CAFE.

1 Introduction

Vertical federated learning shares gradients across workers holding different features for the same subjects, creating a privacy risk because shared gradients can reveal private data. CAFE addresses limitations of large-batch leakage attacks with a scalable, theoretically justified attack and a fake-gradient defense.

  • Vertical FL combines different features about the same subjects across workers, supporting applications such as finance and healthcare.
  • Shared gradients are treated as privacy-preserving exchanges, but they can expose information about private training data and batch identities.
  • Existing leakage attacks often struggle with large batches and lack theoretical justification, potentially creating false confidence that larger batches prevent leakage.
  • CAFE uses data indices and internal-representation alignment to recover large-scale data in general VFL protocols.
  • CAFE provides theoretical recovery guarantees and divides leakage into recovering output gradients, first-layer inputs, and original data.
  • The paper also proposes fake gradients to mitigate CAFE while preserving model training performance, and evaluates the method in static and dynamic VFL settings.

2 Related Work

Prior work introduced gradient-based data recovery attacks, but their generality and batch scalability remain limited. The related methods include optimization-based, analytical, and regularization-based approaches, alongside work on multi-round privacy leakage.

  • Deep leakage from gradients infers training data without generative models or prior information but lacks generalizability across model architectures and weight initializations.
  • Cosine similarity derives inputs before a fully connected layer but works only for a single sample and does not extend to data batches.
  • Other approaches recover batch inputs through linear equations or use batch-normalization and group-consistency regularizers, but rely on strong assumptions.
  • Concurrent work studies privacy leakage during multi-round communications through model inversion and its defense, whereas this paper focuses on VFL gradient-based data recovery.

3 CAFE: Catastrophic Data Leakage in Vertical Federated Learning

CAFE targets privacy leakage in vertical federated learning, where a server can exploit shared gradients and known batch indices to recover clients’ private data. It combines theoretically guaranteed recovery of internal representations with data-index alignment and practical gradient-based defenses.

  • 3.1 Preliminaries: Vertical federated learning distributes different feature partitions for the same subjects across clients, with labels retained by the server or a worker.This setting is common in finance and healthcare, where institutions may share patient identities but not raw data.
  • 3.1 Preliminaries: In each iteration, the server selects K samples through binary batch-index vectors and broadcasts the selected indices to participating workers.The selected batch is D(st) = {(xn, yn)|st[n] = 1}.
  • 3.2 Why large-batch data leakage attack is difficult?: Large-batch recovery is difficult because aggregated-gradient dimensions stay fixed while the number of unknown recovered samples increases.CAFE addresses this underdetermination using batch data-index alignment to exclude undesired solutions.
  • 3.3 CAFE implementation: CAFE first recovers gradients with respect to the first fully connected layer’s outputs, then uses the recovered internal representation to regularize reconstruction.The attack exploits the server’s access to model parameters, gradients, and VFL batch indices.
  • 3.3 CAFE implementation: If K < N, F1(V) is strongly convex, and SGD converges almost surely to the ground truth for fixed Θ.If N < d2 and Rank(V*) = N, F2(Ĥ) is strongly convex and SGD converges to H when Θ remains unchanged.
  • 3.3 CAFE implementation: CAFE supports nested-loop and single-loop update protocols, while fake gradients mitigate leakage by making attackers invert incorrect inputs instead of true training data.The cited training analysis states that sufficiently close fake gradients can preserve convergence guarantees.

4 Experiments

CAFE is evaluated against existing leakage attacks across datasets, batch sizes, architectures, regularizers, worker counts, and training conditions. It maintains high recovery quality for large batches, while fake gradients mitigate the attack.

  • Experimental setup: CAFE recovers up to 800 images using four workers and batch size 100 on MNIST, CIFAR-10, and Linnaeus 5.Experiments evaluate CAFE in vertical federated learning using PSNR as the recovery metric.
  • Comparison with state-of-the-art: CAFE outperforms existing methods, with PSNR values always above 30 while other methods remain below 10 across all three datasets.The comparison uses batch size 40 and identical models and iteration budgets.
  • Ablation study: Batch size increases have almost no influence on CAFE: PSNR stays above 30 on CIFAR-10, 32 on MNIST, and 28 on Linnaeus 5.Thus, increasing batch size fails as an effective defense against CAFE under the tested conditions.
  • Theoretical and algorithmic analysis: Theoretical recovery holds with infinite iterations when N < d2 and Rank(V*) = N, because the relevant objectives are strongly convex.The corresponding experimental analysis is reported in the supplementary material.
  • Ablation study: The internal representation norm regularizer contributes most to recovery, while TV regularization removes noise and improves PSNR.CAFE remains effective without the first regularizer and, on MNIST, without the TV-norm regularizer.
  • Additional experiments: CAFE remains comparable across four and sixteen workers, recovers training images at relatively small learning rates, and becomes harder to use as the learning rate increases.Fake gradients are evaluated as a practical defense against CAFE under unchanged model parameters.

5 Conclusions

The paper concludes that CAFE exposes substantial privacy risks in vertical federated learning by recovering large-batch private data from shared aggregated gradients. It also proposes fake gradients as a countermeasure.

  • Conclusion: CAFE performs large-batch data leakage with high recovery quality and theoretical guarantees in vertical federated learning.The attack recovers large-scale private data from shared aggregated gradients and addresses the batch limitation of existing attacks.
  • Conclusion: Fake gradients provide an effective countermeasure for mitigating CAFE’s potential data-leakage risks.The defense is presented as a practical mitigation strategy.

Supplementary Material

The supplementary material defines the paper’s notation and analyzes CAFE’s objective functions. It shows how additional constraints reduce undesired solutions during data recovery.

  • Notation: The notation distinguishes real and fake datasets, training data, labels, model parameters, gradients, internal representations, and batch-index permutations.The symbols include D and D-hat for real and fake datasets, U for pre-activation first-layer outputs, and V for estimated gradients.
  • Attack procedure: CAFE’s attack proceeds by recovering gradients at the first fully connected layer, then its inputs, and finally the original data.The staged procedure uses internal representations as part of the recovery process.
  • Undesired solutions: The basic objective can admit undesired solutions with the same objective value as the ground truth, and their number increases with N.Additional regularization or constraints are needed to eliminate these alternatives.
  • Index-based constraints: CAFE uses multiple objective functions based on batch-index permutations, adding constraints that reduce undesired solutions and help fake data converge to the optimum.For K = 2, the supplementary example lists three objective functions over different two-sample subsets.

C Proof of Theorem 1

The proof establishes strong convexity for the objective F1(vec(V)) through analysis of its Hessian. The argument relies on positive definiteness of the relevant block matrix.

  • Hessian structure: The Hessian of F1(V) is organized into blocks H(q,s), with off-diagonal blocks equal to zero when q ≠ s.The diagonal blocks are identical across q, simplifying the positive-definiteness analysis.
  • Eigenvalue analysis: For 1 < K < N, the proof analyzes the eigenvalues of H(1,1) to establish the corresponding convexity condition.The matrix includes terms involving (N−1)/(K−1) and identity-like structure.
  • Theorem conclusion: The resulting conclusion is that F1(vec(V)) is strongly convex.This property supports the theoretical recovery guarantee used by CAFE.
  • Matrix properties: The all-ones matrix JN is not positive definite, so the proof distinguishes it from the positive-definite Hessian structure required for the result.This matrix property appears as part of the supporting linear-algebra argument.

D Proof of Theorem 2

The proof establishes positive definiteness of the Hessian by decomposing it into block matrices and applying the Schur Product Theorem under rank conditions on V.

  • D Proof of Theorem 2: The Hessian of F2(Ĥ) is represented through block matrices G(q,s) whose off-diagonal blocks vanish when q ≠ s.The diagonal blocks are identical and equal to G(1,1).
  • D Proof of Theorem 2: If G(1,1) is positive definite, then the Hessian ∇2F2(vec(Ĥ)) is positive definite.
  • D Proof of Theorem 2: By the Schur Product Theorem, G(1,1) is positive definite when R is positive definite because H(1,1) is positive definite.
  • D Proof of Theorem 2: When N < d2 and Rank(V) = N, R = V(V)⊤ and G(1,1) are positive definite.

E Theoretical Guarantee on Data Recovery for CAFE

This section gives recovery guarantees for both CAFE stages under explicit stopping criteria, rank assumptions, and bounded quantities, and describes a fake-gradient defense implementation.

  • E.1 Performance Guarantee for CAFE step I: CAFE step I assumes a stopping criterion based on recovery precision ϵ1 for V relative to the ground truth V∗.
  • E.1 Performance Guarantee for CAFE step I: The recovery of V is guaranteed under the stated step-I conditions.
  • E.2 Performance Guarantee for CAFE step II: CAFE step II uses a stopping criterion φ2 and assumes N < d2 with Rank(V) = Rank(V∗) = N.
  • E.2 Performance Guarantee for CAFE step II: Under the stopping criteria and bounded constants λΘ, λV, and λ∗, the recovery precision of Ĥ is bounded.
  • Defense strategy: The defense constructs sorted Gaussian fake gradients, clips real gradients against them, and uploads the resulting gradients to the server.

G Additional Details on Experiments

The appendix records experimental settings for CAFE and states that the listed hyperparameters serve as the basis for adjusting other experiments.

  • G Additional Details on Experiments: The additional-experiments section provides details omitted from the main paper.
  • G Additional Details on Experiments: Table 9 lists CAFE hyperparameters with M = 4, K = 40, batch ratio = 0.05, and Nested-loops.
  • G Additional Details on Experiments: Other experiments, including the ablation study, adjust their hyperparameters based on the Table 9 settings.

G.2 Experiments of CAFE PSNR via epoch

These experiments examine how batch size and iteration count affect CAFE, and compare training loss under no defense, differential privacy, and fake gradients.

  • G.2 Experiments of CAFE PSNR via epoch: In Table 3, larger batch sizes help CAFE approximate H, especially on MNIST, when T is fixed.
  • G.2 Experiments of CAFE PSNR via epoch: When epochs are fixed, increasing batch size K and iteration count T both contribute to attack performance.
  • G.2 Experiments of CAFE PSNR via epoch: With a smaller batch size, the attacker needs more iterations to recover data when the number of epochs is fixed, leading to better performance.
  • G.2 Experiments of CAFE PSNR via epoch: Table 11 compares training loss for true gradients, differential privacy, and fake gradients.
  • G.2 Experiments of CAFE PSNR via epoch: The training loss cannot be effectively reduced using differential privacy with the stated gradient-clipping approach.The gradient norm was clipped to 3.

G.4 Experiments on human face dataset

Figure 8 presents a visualization of CAFE applied to the Yale 32 × 32 human face dataset, including recovered data image 6-25.

  • The visualization includes recovered data image 6-25.
  • Figure 8 visualizes CAFE on the Yale 32 × 32 human face dataset.
  • The figure concerns recovered data from a human face dataset.
Loading 2110.15122v4…