PaperlayerView exact revision on arXiv

Source-linked AI summary

Label Leakage and Protection in Two-party Split Learning

Oscar Li, Jiankai Sun, Xin Yang, Weihao Gao, Hongyi Zhang, Junyuan Xie, Virginia Smith, Chong Wang

arXiv:2102.08504v3cs.LGcs.CR

TL;DR

The paper asks whether split-learning communication can reveal a party’s private labels and how such leakage can be protected. It formalizes a threat model and privacy metric, identifies label-recovery attacks, and develops random-perturbation defenses including Marvell. Experiments show improved privacy-utility tradeoffs for Marvell relative to protection baselines.

  • Problem

    Communicated gradients in two-party split learning may leak sensitive private labels, but the threat and its protection require rigorous characterization.

  • Method

    The paper formalizes a binary-classification threat model and leakage metric, analyzes two attacks, and designs random perturbations including Marvell against worst-case adversaries.

  • Results

    Marvell shows improved privacy-utility tradeoffs compared with other protection baselines.

  • Takeaways & Limitations

    Label leakage can be accurately exploited within the threat model, while perturbation-based protections can counter the identified attacks.

  • Takeaways & Limitations

    The threat model assumes an honest-but-curious non-label party that follows the agreed training procedure and cannot tamper with batches or submitted features.

Abstract

from arXiv · show

Two-party split learning is a popular technique for learning a model across feature-partitioned data. In this work, we explore whether it is possible for one party to steal the private label information from the other party during split training, and whether there are methods that can protect against such attacks. Specifically, we first formulate a realistic threat model and propose a privacy loss metric to quantify label leakage in split learning. We then show that there exist two simple yet effective methods within the threat model that can allow one party to accurately recover private ground-truth labels owned by the other party. To combat these attacks, we propose several random perturbation techniques, including $\texttt{Marvell}$, an approach that strategically finds the structure of the noise perturbation by minimizing the amount of label leakage (measured through our quantification metric) of a worst-case adversary. We empirically demonstrate the effectiveness of our protection techniques against the identified attacks, and show that $\texttt{Marvell}$ in particular has improved privacy-utility tradeoffs relative to baseline approaches.

1 Introduction

The paper examines whether communicated gradients in two-party split learning leak a label party’s private labels and develops protections against such leakage. It formalizes the threat, identifies effective label-recovery attacks, and reports improved privacy-utility tradeoffs for Marvell.

  • 1 Introduction: Two-party split learning aligns records with PSI and communicates intermediate computations between parties without sharing raw data.The non-label party sends cut-layer outputs, while the label party completes forward computation and initiates backpropagation.
  • 1 Introduction: Communicated gradients may leak sensitive labels despite raw features and labels not being directly shared.The paper highlights purchases and disease status as examples of sensitive label information.
  • 1 Introduction: The paper formalizes a binary-classification threat model and proposes privacy metrics to quantify label leakage.These contributions define how leakage threats are measured in two-party split learning.
  • 1 Introduction: Two simple and realistic methods can accurately recover the label party’s private label information within the proposed threat model.The attacks exploit information available during split training rather than requiring raw-data exchange.
  • 1 Introduction: Marvell searches for random-perturbation noise that minimizes label leakage against a worst-case adversarial non-label party.The approach measures leakage using the paper’s quantification metric.
  • 1 Introduction: Experiments show that Marvell improves privacy-utility tradeoffs relative to other protection baselines.The paper evaluates the effectiveness of its protection techniques empirically.

2 Related Work

Prior work shows that communicated gradients and model parameters can reveal sensitive information, while privacy protections include cryptographic, system-based, and perturbation approaches. This paper places its protections in the perturbation category.

  • 2 Related Work: In horizontal federated learning, communicated gradients can enable recovery of raw features and labels when the attacker knows model details.Prior work also showed that ground-truth labels can be extracted from gradient directions.
  • 2 Related Work: Federated-learning privacy protections include cryptographic methods, trusted execution environments, and perturbations that modify communicated messages.The paper’s protection techniques belong to the perturbation category.

3 Label Leakage in Split Learning

The section formalizes label leakage in two-party split learning by modeling an honest-but-curious non-label party that infers hidden labels from communicated gradients. It defines leak AUC and shows that simple norm- and direction-based attacks can recover labels accurately.

  • Threat model: The non-label party receives cut-layer gradients tied to examples in each batch, creating a potential channel for inferring the label party’s hidden labels.The threat model assumes training follows the agreed procedure, while the adversary uses communicated gradients to guess labels.
  • Threat model: The adversary is modeled as an honest-but-curious non-label party using any binary classifier over cut-layer gradients to predict hidden labels.The classifier is represented by a scoring function and threshold, without allowing the adversary to tamper with batch selection or transmitted features.
  • Privacy quantification: Leak AUC measures how accurately a scoring function distinguishes positive and negative examples from their cut-layer gradients.Values near 1 indicate accurate label recovery, whereas values near 0.5 indicate a non-informative scoring function.
  • Practical attack methods: The norm-based attack uses gradient magnitude as a score because positive-instance gradients are generally larger than negative-instance gradients.The norm leak AUC remains above 0.9 throughout training, indicating substantial label leakage.
  • Practical attack methods: A direction-based attack exploits cosine similarity: same-class gradients have positive similarity, whereas opposite-class gradients have negative similarity.With a positive reference gradient, the cosine leak AUC reaches 1 in the reported Figure 2 iterations.

4 Label Leakage Protection Methods

The paper develops perturbation-based protections for label leakage in split learning, progressing from heuristic noise designs to Marvell, which optimizes worst-case leakage under a utility constraint. The methods target both practical attacks and the broader class of measurable label-recovery scoring functions.

  • Random perturbation baseline: Random perturbation aims to preserve unbiased gradients while mixing positive and negative gradient distributions before communication.The isotropic Gaussian baseline adds iid noise, but its covariance structure may overlook gradient direction and class asymmetry.
  • Norm-alignment heuristic: max_norm adds zero-mean, example-dependent rank-1 Gaussian noise to align every perturbed gradient’s expected squared norm with the batch maximum.Its noise follows the gradient line, reflecting the observed near-one-dimensional separation between positive and negative gradients.
  • Norm-alignment heuristic: max_norm requires no tuning parameter but lacks strong theoretical motivation, flexible privacy-utility control, and robustness guarantees against unknown attacks.These limitations motivate the subsequent optimized perturbation method.
  • Marvell: Marvell minimizes the worst-case adversarial scoring function’s leak AUC subject to a constraint on the noise power.Its noise distributions may differ between positive and negative gradients, and the objective protects against all measurable scoring functions in the threat model.
  • Marvell: The optimization replaces worst-case leak AUC with symmetric KL divergence between perturbed class distributions when the divergence sum is below 4.Theorem 1 links a smaller divergence sum to a smaller worst-case leak AUC.
  • Marvell: Marvell constrains class-weighted noise covariance traces because larger noise improves privacy but can slow convergence and reduce model utility.The tunable power bound P controls this privacy-utility tradeoff, with class weights determined by the positive-example proportion.
  • Marvell: The practical optimization assumes Gaussian class gradients and Gaussian perturbations with commuting covariance matrices, reducing the covariance search to a four-variable problem.These assumptions restrict the search space but make optimization tractable and provide insight into the noise structure.

5 Experiments

Experiments on Criteo and ISIC evaluate Marvell’s protection against label-leakage attacks and its privacy-utility tradeoff. Marvell provides strong protection across layers and generally outperforms isotropic noise, while max_norm can perform well but has important flexibility and scope limitations.

  • Label leakage and Marvell’s protection: Marvell reduces norm- and cosine-based leakage from near-complete exposure without protection to leak AUC ≈0.5 at s = 4.0 on ISIC.The protection applies at the cut layer and remains strong when leakage is evaluated using gradients at the first layer.
  • Label leakage and Marvell’s protection: Marvell’s protection remains strong at the first layer despite perturbations being analyzed at the cut layer.This result indicates that protection persists as the non-label party backpropagates toward earlier layers.
  • Experimental setup: ISIC experiments compare protection methods using leak AUC for privacy and test loss or test AUC for utility.Leakage is evaluated at both the cut layer and the first layer; Criteo and ISIC are the main datasets in this section.
  • Privacy-utility tradeoff: Marvell almost always achieves a better privacy-utility tradeoff than isotropic Gaussian noise across both attacks, layers, and the ISIC and Criteo datasets.On ISIC, isotropic noise retains cosine leak AUC above 0.9 at the cut layer even with t > 20, whereas Marvell reaches approximately 0.6.
  • Privacy-utility tradeoff: max_norm can match or sometimes beat Marvell’s leak AUC at the same utility level, but it cannot tune the privacy-utility tradeoff and may fail against attacks beyond those studied.Marvell is designed for the entire class of adversarial scoring functions considered in the threat model, unlike max_norm.
  • Summary: Overall, Marvell significantly outperforms the isotropic Gaussian baseline, while max_norm performs particularly well against the identified norm- and direction-based attacks.The comparison uses the privacy-utility evaluations reported for the proposed protection methods and baselines.

6 Conclusion

The paper formulates and quantifies label leakage in two-party split learning, demonstrates attacks that recover private labels, and proposes perturbation-based protections. It reports strong empirical performance for Marvell and max_norm while identifying broader threat settings as future work.

  • Contributions: The paper introduces a label-leakage threat model and leak AUC metric for binary classification in two-party split learning.The framework addresses label leakage despite communication of intermediate computations rather than raw labels.
  • Contributions: Two simple attack methods within the threat model can accurately recover the label party’s private labels.The attacks motivate protection methods based on random perturbations.
  • Protection methods: Marvell searches for noise distributions that protect against worst-case adversaries, while max_norm provides a heuristic alternative.Experiments demonstrate effectiveness over the isotropic Gaussian perturbation baseline.
  • Open questions: Future work includes attacks using stale gradients, multiclass classification, and split learning with more than two parties or more complex protocols.These settings are outside the paper’s current setup and may require a more complex threat model.

A.1 Expressing AUC(r) as an integral

The appendix expresses the ROC-area quantity for a scoring function as a Riemann-Stieltjes integral. This formulation accommodates monotone bounded-variation rates even when they are not differentiable everywhere.

  • Integral formulation: Because the false-positive and true-positive rates need not be differentiable, the area cannot be written using an ordinary derivative-based integral.The appendix instead uses their bounded-variation structure.
  • Integral formulation: The ROC area is represented by integrating the true-positive rate with respect to the false-positive rate using a Riemann-Stieltjes integral.The integration direction is chosen so the result is positive.

A.2 Toy example of positive example prediction lacking confidence

A balanced toy classification problem still produces weaker positive-class confidence because positive-feature regions overlap with some negative examples. Even the Bayes-optimal classifier therefore has a larger confidence gap for positives.

  • The toy problem places all positives in [0, 1], while negatives mix 10% in [0, 1] with 90% in [1, 2].Both classes have prior probability 0.5.
  • The setup models advertising conversion, where product interest does not guarantee purchase after an ad click.
  • The Bayes-optimal classifier predicts positivity with probability 10/11 in [0, 1] and zero in [1, 2].
  • Every positive example has confidence gap 1/11, whereas 90% of negative examples have gap 0.Thus positive examples remain less confidently predicted even under the optimal classifier.
  • The lack of positive prediction confidence persists in this class-balanced example, so the observation does not require class imbalance.

A.3 Proof of Theorem 1

Theorem 1 bounds an adversary’s label-recovery AUC using the symmetrized KL divergence between perturbed positive and negative gradient distributions. The proof converts the KL bound to total variation and then bounds every thresholded scoring rule.

  • For any mutually absolutely continuous perturbed gradient distributions, the theorem assumes a bound on their symmetrized KL divergence.
  • Pinsker’s and Jensen’s inequalities upper-bound total variation distance by the symmetrized KL divergence.
  • For every measurable set, the difference between positive- and negative-class probabilities is bounded by the resulting total-variation limit.
  • Applying this bound to threshold sets of any scoring function bounds TPR minus FPR at every threshold.
  • Integrating the ROC bound yields the theorem’s AUC upper bound, completing the proof because the argument holds for every scoring function.

A.4 Proof and Interpretation of Theorem 2

Theorem 2 reduces optimal Gaussian perturbation design to a low-dimensional constrained optimization by exploiting shared eigenvectors and rotational structure. The resulting covariances combine isotropic noise with a rank-one component aligned to the class-mean gradient difference.

  • Theorem 2 minimizes the symmetrized KL divergence between two Gaussian gradient distributions over positive-semidefinite covariance matrices.
  • The commuting covariance constraint permits a shared orthogonal eigenbasis with nonnegative, ordered eigenvalues.
  • The optimal orthogonal matrix has first row aligned with the normalized class-mean difference, reducing the objective before solving the remaining variables.
  • Convexity and symmetry force equal values across coordinates beyond the first, with one leading eigenvalue vanishing when u ≥ v.
  • The four-variable optimization can be solved by alternating one-dimensional line searches after fixing variables on its feasible line segment.

A.5 Marvell algorithm description

Marvell estimates batch-specific gradient statistics, optimizes a constrained four-variable noise design, constructs the corresponding covariance matrices, and perturbs gradients. Experiments across Avazu, Criteo, and ISIC evaluate leak AUC and utility, with Marvell showing strong privacy protection and improved tradeoffs.

  • Algorithm: Marvell takes unperturbed gradients, batch labels, and a power-constraint hyperparameter as inputs, returning perturbed gradients.
  • Algorithm: The algorithm estimates positive and negative means and covariances from batch gradients using maximum likelihood estimation.
  • Algorithm: It sets the power constraint to P = s · ||∆g||_2^2 and iteratively optimizes the four-variable problem with constrained one-dimensional line searches.
  • Experiments: The experiments cover Avazu, Criteo, and a modified ISIC dataset, using dataset-specific preprocessing and model architectures.
  • Evaluation: Privacy is evaluated with norm and cosine leak AUC at the cut layer and first non-label-party layer, while utility includes training loss and other utility measures.
  • Results: Marvell provides strong and flexible protection against label attacks across Avazu and Criteo at different model layers and protection strengths.
  • Results: The reported privacy-utility comparisons include Marvell, iso, no_noise, and max_norm on all three datasets.
Loading 2102.08504v3…