Feature Inference Attack on Model Predictions in Vertical Federated Learning

Xinjian Luo; Yuncheng Wu; Xiaokui Xiao; Beng Chin Ooi

Source-linked AI summary

Feature Inference Attack on Model Predictions in Vertical Federated Learning

Xinjian Luo, Yuncheng Wu, Xiaokui Xiao, Beng Chin Ooi

arXiv:2010.10152v3Oct 20, 2020cs.LGcs.DB

TL;DR

Vertical federated learning exposes prediction outputs computed from private passive-party features, raising the question of how much those features can be inferred. The paper develops attacks using individual or multiple predictions under a stringent no-background-information setting, finds considerable leakage, and motivates private prediction mechanisms.

Problem
The paper asks how much information about passive-party feature values can be inferred from vertical FL prediction outputs when only the trained model and predictions are available.
Method
It proposes equality-solving and path-restriction attacks for LR and DT, plus a general multiple-prediction attack for NN and RF models.
Results
The attacks are effective, and prediction outputs can leak considerable passive-party feature information, especially with many classes or highly correlated features.
Takeaways & Limitations
The findings highlight the need for defense mechanisms that protect prediction outputs in vertical federated learning.

Abstract

from arXiv · show

Federated learning (FL) is an emerging paradigm for facilitating multiple organizations' data collaboration without revealing their private data to each other. Recently, vertical FL, where the participating organizations hold the same set of samples but with disjoint features and only one organization owns the labels, has received increased attention. This paper presents several feature inference attack methods to investigate the potential privacy leakages in the model prediction stage of vertical FL. The attack methods consider the most stringent setting that the adversary controls only the trained vertical FL model and the model predictions, relying on no background information. We first propose two specific attacks on the logistic regression (LR) and decision tree (DT) models, according to individual prediction output. We further design a general attack method based on multiple prediction outputs accumulated by the adversary to handle complex models, such as neural networks (NN) and random forest (RF) models. Experimental evaluations demonstrate the effectiveness of the proposed attacks and highlight the need for designing private mechanisms to protect the prediction outputs in vertical FL.

I. INTRODUCTION

The paper investigates whether prediction outputs in vertical federated learning can reveal passive parties’ features, even when the adversary controls only the trained model and predictions. It introduces attacks for individual and accumulated predictions, evaluates them, and motivates defenses.

Motivation: Vertical federated learning lets organizations jointly build models while keeping proprietary data private, but prediction outputs are computed from that private data.The setting uses shared samples with disjoint features, and only the active party owns labels.
Threat setting: The attacks assume the adversary has only the trained vertical FL model and prediction outputs, without intermediate computation information or background data distributions.This is presented as the most stringent attack setting.
Findings and defenses: Experimental evaluations find effective attacks and show that prediction outputs can leak considerable passive-party feature information under conditions such as many classes or highly correlated features.The paper also designs and incorporates defense strategies into Falcon for privacy protection.
Research gap: The paper formulates feature inference against passive-party features from model predictions and identifies this as the first study of this privacy leakage in vertical FL.The active party attempts to infer feature values of new samples belonging to passive parties.
Attacks: The paper proposes individual-output attacks for LR and DT, plus a multiple-output attack for more complex NN and RF models.The individual attacks are equality solving and path restriction; the general attack learns correlations from accumulated predictions.

B. Vertical Federated Learning

This section defines vertical FL, its prediction process, and the threat model used for feature inference. The active party receives model parameters, prediction outputs, and its own features, then seeks to infer the remaining party’s features.

Prediction stage: During prediction, the active party requests a sample by id, passive parties provide corresponding features, and a joint protocol reveals only the prediction output to the active party.The protocol can protect intermediate sensitive information during computation.
System model: Vertical FL parties share sample ids while holding disjoint feature subsets, with aligned common samples established using private set intersection.Each dataset row corresponds to a sample and each column to a feature.
Threat model: The threat model is semi-honest: parties follow the protocol but may infer other parties’ private information from received messages.The active party may collude with passive parties, potentially leaving one passive party as the target.
Threat model: The adversary is assumed to lack background information about passive-party data distributions but may know feature names, types, and value ranges.This knowledge is considered reasonable for assessing the trained model’s effectiveness.
Feature inference attack: The attack input comprises model parameters θ, prediction output v, and adversary-held features x_adv; the goal is to infer target features x_target.Attack performance is measured using MSE, and CBR for tree-based models.

IV. ATTACK BASED ON INDIVIDUAL PREDICTION

Individual-prediction attacks exploit the deterministic outputs of LR models and the structure of DT models to infer passive parties’ unknown features. For multi-class LR, confidence-score differences yield linear equations that can exactly recover features when enough class outputs are available, while overdetermined systems still provide estimates.

Individual-prediction attacks include equality solving for LR and path restriction for DT models.
Binary LR prediction: For binary LR, the adversary combines the prediction score and its own features into an equation whose unknowns are the target party’s features.The prediction is computed from the known model parameters and the concatenated feature vector.
Binary LR prediction: One unknown feature in binary LR has a unique solution, allowing its value to be inferred precisely.
Multi-class LR prediction: Multi-class LR subtracts adjacent log-confidence equations to obtain c − 1 linear equations in the unknown target features.The shared term across classes cancels during subtraction.
Multi-class LR prediction: When dtarget ≤ c − 1, multi-class LR inference has one solution and can recover target features exactly.The equations can be written as Θtargetxtarget = a; a pseudo-inverse provides an estimate when dtarget ≥ c.
Example: In a three-class example, the attack estimates unknown features as (8011.8, 3.046), with loss attributed to precision truncation.

B. Path Restriction Attack

The path restriction attack uses the predicted class and the adversary’s known features to eliminate incompatible decision-tree paths. It can identify the actual path and infer a target feature when the remaining path constraints determine its branch.

The DT attack restricts possible prediction paths using the predicted class and the adversary’s own feature values.DT prediction output contains only the predicted class, with confidence score 1.
In the example, known age and income reduce five candidate paths to two, while predicted class 1 identifies the real path.
The example correctly infers that the target party’s deposit value exceeds 5K.
Generally, the attack compares known features with branching thresholds and filters paths whose leaf labels match the predicted class.If np is the total number of paths, the candidate set is reduced to nr paths.
Algorithm 1 traverses the tree to construct indicator vectors and has complexity O(nf), where nf is the number of nodes in the full binary tree.

V. ATTACK BASED ON MULTIPLE PREDICTIONS

Individual-prediction attacks are difficult to apply to complex models because nonlinear transformations and large tree combinations complicate direct equation solving. The paper therefore introduces GRN, which learns target-feature distributions from accumulated predictions without intermediate or background information.

Neural networks hinder equality solving because hidden layers introduce nonlinear transformations, while random forests may require computationally expensive tree combinations.
The generative regression network attack uses multiple model predictions to learn correlations between known and unknown features.
GRN generates target-feature values conditioned on the adversary’s known features and random inputs, then trains them to match the vertical FL model’s decisions.
Random vectors regularize the generator and provide varied gradient directions during training.
Random inputs reduce reconstruction error by 20% compared with using no random inputs.
The attack assumes only the trained vertical FL model and a set of model predictions, requiring neither intermediate computation information nor target-data distributions.

A. Generative Regression Network

GRN trains a generator on known features, prediction outputs, and random vectors so generated target features reproduce the vertical FL model’s predictions. The method can infer features for differentiable LR and NN models and uses accumulated predictions from the attack samples.

The adversary collects n prediction outputs paired with known-feature samples to train the generator.
For each sample, the generator receives known features and a random vector and outputs an estimate of the target party’s features.
The objective includes a regularization term for generated unknown features, including a penalty when their variance is too large.
Training concatenates generated target features with known features, compares the federated prediction with the observed output, and back-propagates the loss.
After training, the generator directly produces inferred target features for samples in the prediction dataset.
GRN treats the trained vertical FL model as a black box and applies to LR and NN models when the model objective is differentiable.

B. Adopt GRN Attack on the Random Forest Model

The path restriction attack does not apply effectively to random forests, so the paper adapts GRN by approximating the forest with a differentiable model before inference.

Motivation: Path restriction attack is unsuitable for random forests, particularly when the forest contains many trees.Random-forest outputs aggregate predictions across trees, while the model objective is not differentiable for direct back-propagation.
Differentiable approximation: The adversary trains a differentiable neural network to approximate the random forest using dummy samples and their forest prediction outputs.The approximation model is trained on pairs of generated samples and predictions produced by the released random forest.
Attack adaptation: The resulting approximation enables the generative regression network attack to infer unknown feature values from random-forest predictions.The GRN training procedure uses the approximating model parameters to train a generator and reconstruct target features.

VI. EXPERIMENTAL EVALUATION

The evaluation examines the proposed attacks experimentally, separating attacks based on individual predictions from those based on multiple predictions.

Evaluation scope: The paper evaluates the performance of its proposed attack algorithms experimentally.
Evaluation organization: Attacks based on individual model predictions are evaluated separately from attacks based on multiple model predictions.
Section structure: The evaluation is divided into Sections VI-B and VI-C for individual- and multiple-prediction attacks, respectively.

A. Experimental Setup

The experiments use real-world and synthetic datasets, several vertical FL model types, and task-specific metrics to assess feature reconstruction and branch inference.

Datasets: Experiments evaluate attacks on four real-world datasets and two synthetic datasets with varying sample, feature, and class configurations.The synthetic datasets are used to study how the number of prediction samples affects GRNA performance.
Models: The study trains LR, NN, DT, and RF vertical FL models and releases only the final trained models to the adversary.DT and RF use specified tree settings, while centralized training provides the released models for evaluation.
Metrics: ESA and GRNA are measured by MSE per feature, while PRA is measured by correct branching rate.MSE evaluates reconstruction of multiple target features; CBR measures the fraction of inferred values assigned to correct branches.
Experimental variables: The experiments vary the target-feature fraction from 10% to 60% for attacks based on individual predictions.
ESA results: 0 MSE per feature is achieved by ESA whenever dtarget ≤ c − 1 holds across the evaluated datasets.For the 11-class Drive diagnosis dataset, unknown features are precisely inferred when dtarget = 10, corresponding to 20%.
ESA results: ESA accuracy generally worsens as the target-feature fraction increases, with dataset-specific upper bounds of 0.60, 0.14, 0.45, and 0.34 for Bank, Credit, Drive, and News.The paper attributes the faster Bank MSE increase to its larger upper bound and reports better ESA results on Drive than News because Drive provides more equations.

C. Evaluation of Attacks Based on Multiple Predictions

The multiple-prediction evaluation studies GRNA across model types, feature correlations, ablated components, random forests, and prediction-dataset size.

GRNA configuration: GRNA uses a multilayer perceptron generator whose input dimension is d and output dimension is dtarget.The generator has three hidden layers with 600, 200, and 100 neurons, followed by layer normalization.
Ablation study: Removing output constraints increases inference error by 17%, while adding a random vector to xadv reduces reconstruction error by 20%.The ablation also finds that a naive regression model performs worse than random guessing and that excluding xadv degrades performance most.
GRNA results: GRNA outperforms random guessing even when target features comprise 60% of the total, and performs best with NN models among LR, RF, and NN.The paper attributes this pattern to neural networks’ more complicated decision boundaries and greater capacity to capture feature correlations.
Prediction-set size: Increasing the number of prediction samples lowers GRNA’s MSE per feature across the evaluated datasets.The study tests prediction-set sizes of 10%, 30%, and 50% of each dataset’s remaining samples.
Data correlations: Features more strongly correlated with adversary-held features and model outputs are reconstructed more accurately by GRNA.Weaker correlations correspond to lower inference accuracy, while the overall MSE averages errors across target features.

VII. COUNTERMEASURES

The paper evaluates defenses against feature inference attacks in vertical FL, including score rounding, dropout, preprocessing, verification, model hiding, and differential privacy. The reported results show that rounding can disrupt ESA, while GRNA is insensitive and dropout only slightly degrades NN attack performance.

Rounding confidence scores: Rounding confidence scores disrupts ESA because its inference relies on exact linear equations involving ln v.Rounding to 0.1 causes a large change in the equation result, whereas rounding to 0.001 has only a small impact.
Rounding confidence scores: Rounding to 0.1 yields ESA MSE per feature higher than the uniform random-guess method on two datasets.The result is relatively stochastic under one-decimal rounding.
Rounding confidence scores: GRNA for logistic regression is insensitive to rounding confidence scores.This result is reported for the two datasets shown in Fig. 11c–11d.
Dropout for neural networks model: Dropout training slightly increases MSE per feature for GRNA against neural networks, thereby degrading attack performance.Dropout is applied as a regularization technique to avoid overfitting.
Pre-processing before collaboration: Preprocessing can target conditions associated with stronger attacks, including many classes for ESA and highly correlated adversary and target features for GRNA.The paper also reports that PRA can correctly infer most decision-tree branches when the number of classes is relatively large.
Other countermeasures: Verification can withhold prediction outputs when estimated leakage exceeds a predefined threshold, while hiding the model reduces interpretability for the active party.These alternatives use secure hardware, ciphertexts, or secure multiparty computation.
Differential Privacy: Differential privacy is unsuitable here because protecting an unlabelled input record would require nearly unchanged predictions across arbitrary input changes.The paper states that this would render the model useless.

VIII. RELATED WORK

Prior vertical FL work protects privacy during training or prediction, but this paper studies feature inference from only the trained model and prediction outputs. It positions the attacks as addressing a harder prediction-stage setting without intermediate information or prior distributions of target features.

Vertical federated learning: Existing vertical FL solutions use homomorphic encryption or secure multiparty computation to protect information exchanged during training or prediction.These approaches cover logistic regression, gradient-boosted decision trees, neural networks, and tree-based models.
Vertical federated learning: This paper assumes secure training and prediction computation while allowing the adversary to use only the trained model and model predictions.The target is private feature inference in vertical FL.
Inference attacks on federated learning: Earlier FL inference attacks mainly target horizontal FL training and rely on exchanged gradients to build inference meta-classifiers.Protecting the gradients invalidates those attacks, whereas prediction-stage vertical FL lacks the same memorization signal.
Other related work: Unlike several centralized-model or image-based attacks, the proposed methods do not require background information or prior data distributions for target features.The paper focuses mainly on reconstructing prediction samples rather than training samples.
Other related work: The paper presents equality solving for logistic regression, path restriction for decision trees, and a multiple-output attack for neural networks and random forests.The authors characterize this as the first work investigating privacy leakage in vertical FL.