PaperlayerView exact revision on arXiv

Source-linked AI summary

Membership Inference Attack on Graph Neural Networks

Iyiola E. Olatunji, Wolfgang Nejdl, Megha Khosla

arXiv:2101.06570v3cs.LGcs.CR

TL;DR

The paper asks whether GNNs leak which nodes were used for training, a concern because GNNs encode relational structure in their predictions. It develops two black-box membership-inference settings, analyzes model and dataset factors, and evaluates perturbation defenses that reduce attacker inference while preserving target-model performance.

  • Problem

    The paper investigates whether trained GNNs can reveal which input nodes belonged to their training data, beyond the overfitting-based leakage known in traditional models.

  • Method

    The study introduces two realistic settings, uses posterior-based black-box attacks, compares four representative GNN models, and proposes output and query-neighborhood perturbation defenses.

  • Results

    The experiments show that graph structure is a major contributor to GNN membership-inference vulnerability, while the proposed defenses reduce attacker inference by over 60% without substantially compromising target-model performance.

  • Takeaways & Limitations

    Good generalization alone does not guarantee membership-inference robustness for GNNs, whereas output and neighborhood perturbations can reduce attack performance with little utility loss.

Abstract

from arXiv · show

Graph Neural Networks (GNNs), which generalize traditional deep neural networks on graph data, have achieved state-of-the-art performance on several graph analytical tasks. We focus on how trained GNN models could leak information about the \emph{member} nodes that they were trained on. We introduce two realistic settings for performing a membership inference (MI) attack on GNNs. While choosing the simplest possible attack model that utilizes the posteriors of the trained model (black-box access), we thoroughly analyze the properties of GNNs and the datasets which dictate the differences in their robustness towards MI attack. While in traditional machine learning models, overfitting is considered the main cause of such leakage, we show that in GNNs the additional structural information is the major contributing factor. We support our findings by extensive experiments on four representative GNN models. To prevent MI attacks on GNN, we propose two effective defenses that significantly decreases the attacker's inference by up to 60% without degradation to the target model's performance. Our code is available at https://github.com/iyempissy/rebMIGraph.

I. INTRODUCTION

The paper examines membership inference against GNNs and argues that graph structure, beyond overfitting, contributes substantially to privacy risk. It compares representative graph convolution models and proposes defenses intended to reduce attacks without compromising utility.

  • I. INTRODUCTION: The attack uses black-box prediction posteriors to distinguish target-model behavior on training nodes from behavior on unseen nodes.Membership inference is framed as distinguishing model outputs for inputs encountered during training from outputs for inputs not encountered during training.
  • I. INTRODUCTION: GNNs can identify nodes used during training, even when the models generalize well to unseen data.The paper attributes this vulnerability primarily to graph-structure encoding rather than overfitting alone.
  • I. INTRODUCTION: Output and query-neighborhood perturbation defenses reduce attacker inference by over 60% with negligible loss in target-model inference performance.The defenses are presented as mechanisms that degrade attack performance without compromising target-model utility.
  • I. INTRODUCTION: The study introduces two realistic membership-inference settings and analyzes how dataset properties and model architectures affect attack robustness.The contribution includes comparing differences in aggregation and transformation operations across graph convolution-based models.
  • I. INTRODUCTION: GNN representations recursively aggregate and transform neighbor features, embedding graph structure directly into node predictions.The final softmax outputs class posteriors for each node.
  • I. INTRODUCTION: The evaluated models differ in aggregation or transformation, including attention weighting, neighbor sampling, and removal of nonlinear activation.GCN applies nonlinear transformation, SGC omits it, GAT uses attention weights, and SAGE samples neighbors for scalability.

B. Privacy attacks on Machine Learning

The paper situates membership inference among attacks that exploit model behavior or outputs to expose training membership and other private information. It distinguishes this threat from attribute inference and model inversion, while noting that prior GNN privacy methods do not directly address it.

  • B. Privacy attacks on Machine Learning: Membership inference determines whether a data sample belonged to the training set, whereas attribute inference reconstructs missing attributes.Model inversion instead extracts features characterizing one of a model’s classes.
  • B. Privacy attacks on Machine Learning: Existing privacy-preserving learning methods for GNNs are not directly applicable to the membership-inference risk studied here.The paper contrasts its defenses with later work whose defenses limit target-model utility.

III. OUR APPROACH

The approach represents the target training data as an induced target graph containing the sensitive or member nodes used to train the GNN model.

  • III. OUR APPROACH: The target graph is the induced graph on sensitive or member nodes used to train the target model.The graph is denoted G_t = (V_t, E_t), with V_t representing the training nodes.

1) Notations:

Membership inference is formulated as binary classification of whether a node belongs to the target training set, using its neighborhood and model-derived signals. The attack proceeds through shadow-model training, attack-model training, and final inference.

  • 1) Notations:: The task is to determine whether node v belongs to V_t given v and its L-hop neighborhood.The adversary’s known neighborhood may differ from the neighborhood used to train the target model.
  • 1) Notations:: TSTF exposes the whole graph while hiding the training subgraph, allowing links between member and non-member nodes.TSTS instead treats the target graph as an isolated component, with no member–non-member connections.
  • 1) Notations:: The attack is modeled as binary classification using an attack model A to predict membership.This operationalizes membership inference as distinguishing nodes in V_t from other nodes.
  • 1) Notations:: The adversary’s workflow has three phases: shadow-model training, attack-model training, and membership inference.Figure 1 depicts the corresponding pipeline from shadow predictions to target-model posterior queries.
  • 1) Notations:: The shadow model can be trained with target-model output probabilities or original ground-truth labels, with no significant change in attack success rate reported for the latter.A GCN shadow model also produces good attack performance irrespective of the actual target architecture.

2) Attack model training:

The attack model is trained from shadow-model posterior vectors labeled by membership, then queried with target-model posteriors for inference.

  • 2) Attack model training:: Shadow-model posteriors for training and out-of-training nodes are converted into feature vectors labeled 1 and 0 for attack-model training.The resulting vectors and labels form the attack model’s training data.
  • 2) Attack model training:: For a target node, the adversary queries the target model using the node and known neighborhood, then feeds the posteriors to the attack model.The adversary may not know the exact neighborhood structure used during target-model training.
  • 2) Attack model training:: Experiments compare GCN, GAT, SGC, and SAGE across five benchmark datasets using repeated target, shadow, and test graph splits.Results average 10 random splits and report standard deviations.

B. Model Architecture and Training

The study evaluates black-box membership leakage across GNN architectures, datasets, and attack settings, finding architecture- and setting-dependent vulnerability.

  • B. Model Architecture and Training: The evaluation reports attack AUROC, precision, and recall alongside target-model train and test accuracy.The main paper primarily summarizes results using AUROC.
  • B. Model Architecture and Training: The experiments investigate which GNN models leak training membership and which factors explain differences in vulnerability, including overfitting and attack conditions.The study also asks how to defend against black-box attacks without compromising model performance.
  • B. Model Architecture and Training: The attack model obtains similar scores for GCN and SGC, while GAT is the most robust in the TSTF setting.GCN and SGC use the same feature aggregation, whereas SGC omits the nonlinear transformation after aggregation.
  • B. Model Architecture and Training: In TSTS, disconnected train and test sets reduce target-model test performance, while attack success remains dataset- and architecture-dependent.For REDDIT, GCN and SGC attacks achieve mean precision of 0.81 and 0.74, whereas GAT and SAGE have AUROC scores close to 0.5.

B. Effect of Model and Dataset Properties

Attack robustness varies with GNN aggregation and dataset structure, especially average degree, feature dimensionality, class count, and neighborhood sampling.

  • B. Effect of Model and Dataset Properties: GAT is generally most robust because learnable attention weights distort graph encoding according to supervised class signals, unlike equal-weight GCN and SGC aggregation.SAGE’s sampled neighborhoods provide a different source of robustness.
  • B. Effect of Model and Dataset Properties: GAT’s attack-performance drop relative to GCN is 32% on FLICKR versus 9% on CORA, showing dataset-dependent vulnerability.The comparison uses the percentage drop in attack performance.
  • B. Effect of Model and Dataset Properties: Low-degree CITESEER shows similar vulnerability across GNNs, whereas high-degree REDDIT is highly vulnerable for GCN and SGC.REDDIT’s average degree is around 50 times FLICKR’s in TSTF and around 70 times in TSTS.
  • B. Effect of Model and Dataset Properties: Among similarly low-degree citation datasets, attack performance is highest for CITESEER followed by CORA, consistent with their feature counts and model parameterization.More input features increase first-layer parameters, which the authors connect to better memorization.
  • B. Effect of Model and Dataset Properties: Lower SAGE neighborhood sample counts decrease attack AUROC on CORA and FLICKR because noisy, incomplete neighborhoods weaken posterior correlations.The tested layer-wise sample sizes include [25,10] and [5].

4) Effect of Instance Connectivity:

The analysis links membership-inference behavior to correlations among neighboring nodes’ posteriors and tests whether overfitting strengthens the attack.

  • 4) Effect of Instance Connectivity:: Homophily measures the fraction of one-hop neighbors sharing a node’s membership label, with true and predicted versions defined from ground truth or attack outputs.The neighborhood is computed using the graph available to the adversary.
  • 4) Effect of Instance Connectivity:: Attack predictions are highly correlated across neighboring nodes, enabling the attack model to exploit correlated posterior outputs despite lacking explicit graph structure.Dense regions with high predicted homophily appear across models.
  • 4) Effect of Instance Connectivity:: Higher diagonal density for GCN and SGC corresponds to more accurate attacks, while GAT produces confused labels and SAGE’s orange regions are off-diagonal.The plot compares true and predicted homophily for correctly and incorrectly classified nodes.
  • 4) Effect of Instance Connectivity:: Overfitting consistently decreases attack precision and recall across models except GAT, indicating that overfitting alone may not drive membership inference.The authors also evaluate target-model train and test accuracy.
  • 4) Effect of Instance Connectivity:: Overfitted models assign highly confident predictions to both member and non-member nodes, reducing the confidence distinction available to the attack.This observation is based on maximum-posterior distributions.

D. Sensitivity Analysis of Attack

The analysis tests how attack performance changes with query access, hidden-layer size, overfitting, and shadow-model architecture. Results show that attack success depends on model capacity and architecture, while overfitting can reduce attack effectiveness.

  • Hidden-layer size: Larger hidden layers generally improve attack performance, consistent with greater capacity to store training-set details.Some reduction occurs on PUBMED with hidden sizes of 128 or 64.
  • Effect of overfitting: Overfitted models consistently reduce attack precision and recall except for GAT, because they are highly confident on both member and non-member nodes.Most nodes receive maximum posterior probabilities greater than 0.8 in the overfitted setting.
  • Shadow-model architecture: Using GCN as the shadow model is generally sufficient for successful attacks across differing target architectures, whereas SGC can substantially reduce attack precision.GCN performs better than SGC as a shadow model, including for SAGE targets on REDDIT.

4) Attack using different data distribution (Data transferring attack):

The study finds that membership attacks can transfer across datasets with limited loss, then proposes posterior and neighborhood perturbation defenses. These defenses target posterior similarity and reduce attack performance while preserving node-classification utility.

  • 4) Attack using different data distribution (Data transferring attack):: Relaxing the same-distribution assumption preserves attack precision but lowers recall by 13% on GCN, 16% on GAT and SGC, and not on SAGE.Cora is used for the target model and CiteSeer for the shadow model.
  • VI. DEFENSE MECHANISMS: The proposed defenses are query neighborhood sampling and Laplacian binned posterior perturbation.Both mechanisms are derived from observed attack behavior across GNN models and datasets.
  • VI. DEFENSE MECHANISMS: Laplacian binned posterior perturbation adds shared Laplace noise to shuffled posterior groups before releasing them.Binning reduces the noise needed compared with perturbing each posterior element independently.
  • VI. DEFENSE MECHANISMS: The defenses reduce attack performance without substantially compromising node-classification performance.The reported perturbation experiments vary Laplace noise scale β and the number of posterior bins ψ.
  • VI. DEFENSE MECHANISMS: Query neighborhood sampling removes most first-hop neighbors from a query, disrupting the posterior similarity pattern exploited by the attack.The mechanism retains only k randomly chosen neighbors.

A. Evaluating Defenses

The defenses reduce membership-inference performance while trading off label changes and posterior distortion differently across graph datasets. Neighborhood sampling is especially effective on high-degree graphs, whereas posterior perturbation performs better on low-degree graphs.

  • Dataset comparisons: At label loss 0.1 on CORA, LBP reduces attacker inference by 12%, while NSD provides a 15% reduction with negligible label loss and confidence distortion.LBP achieves a 24% maximum reduction at label loss 0.3, compared with 17% for VANPD at the same loss.
  • Dataset comparisons: At label loss 0.1 on CITESEER, LBP reduces attacker inference by 22%, compared with 5% for VANPD and 12% for NSD.At label loss 0.3, LBP achieves a 30% reduction versus 16% for VANPD.
  • Dataset comparisons: 50% attack-inference reduction is achieved by LBP on PUBMED at label loss 0.3, while VANPD achieves 25%.At label loss 0.1, the two defenses perform similarly, possibly because PUBMED has only three classes.
  • Dataset comparisons: 80% attack-inference reduction at zero label loss is achieved by NSD on REDDIT, compared with 17% for LBP and 15% for VANPD.On FLICKR, NSD likewise achieves a 65% reduction at zero label loss.
  • Cross-dataset patterns: NSD achieves the lowest label loss on all datasets and reduces inference to random guessing on high-degree FLICKR and REDDIT graphs with zero label loss.For the evaluated datasets, values of k greater than 3 produced little additional degradation in attack performance.
  • Confidence distortion: NSD produces the lowest confidence distortion, including 0.05 on REDDIT for a 63% reduction in attack precision, while VANPD produces the highest distortion.Lower distortion better preserves posterior utility and makes the defense less detectable.

VII. CONCLUSION

The paper studies membership-inference vulnerability in GNNs and explains differences through model and dataset properties. It proposes output perturbation and query-neighborhood sampling defenses that reduce attacker inference without substantially compromising target-model performance.

  • Conclusion: The study compares GNN vulnerability to membership-inference attacks across five datasets and attributes differences to model and dataset properties.It uses a simple binary-classifier attack and two realistic attack settings.
  • Conclusion: The defense evaluation compares attack performance using label loss and confidence-score distortion against the undefended attack AUROC.The supplied figure caption identifies the red dashed line as the no-defense baseline.
  • Conclusion: Output perturbation and query-neighborhood sampling significantly decrease attacker inference without substantially compromising target-model performance.These are the paper's two proposed defense families.
Loading 2101.06570v3…