A Unified Framework of Graph Information Bottleneck for Robustness and Membership Privacy

1 INTRODUCTION

• Prior robust-GNN defenses may not preserve membership privacy, while i.i.d.-data privacy methods face sparse labels and neighbor information propagation in semi-supervised node classification.The paper identifies simultaneous robustness and membership privacy as an underexplored graph-learning problem.Source: §1 · p. 2
• Information Bottleneck can filter task-irrelevant information and regularize labeled-sample predictions, which may benefit robustness and membership privacy.The paper links noise filtering to robustness and constrained training-set information to a narrower train–test gap.Source: §1 · p. 2
• Applying i.i.d.-data IB directly to GNNs may miss structural noise, while scarce labels can leave labeled and unlabeled predictions far apart.The paper identifies these as challenges to both robust prediction and membership privacy.Source: §1 · p. 2
• RM-GIB combines attribute and neighbor bottlenecks, self-supervision, and pseudo labels to address attribute and structural noise while preserving membership privacy.Pseudo labels from unlabeled nodes are combined with provided labels during optimization.Source: §1 · p. 2

2.1 Graph Neural Networks

• GNNs use message passing to aggregate neighbor information into center-node representations, supporting tasks including node classification.The section also notes their vulnerability to adversarial and privacy attacks.Source: §2.1 · p. 2

2.2 Robust Graph Learning

• Robust graph-learning defenses include adversarial training, robust aggregation, and graph denoising, while the cited methods do not consider membership inference defense.Examples include changing message passing to restrict perturbations and identifying or down-weighting adversarial edges.Source: §2.2 · pp. 2–3

2.3 Membership Privacy Preservation

• Membership-inference defenses largely target i.i.d. data; graph-focused work is limited, and one cited method sacrifices classification performance for privacy.The section describes regularization and differential privacy as i.i.d.-data approaches.Source: §2.3 · p. 3

2.4 Information Bottleneck

• IB learns task-informative representations with minimal redundant information; RM-GIB is presented as the first IB-based GNN framework for membership privacy and as targeting both attack types.Prior IB work cited here includes robust representations for i.i.d. data and graph representations or subgraphs for other tasks.Source: §2.4 · p. 3

3.1 Notations

• The study focuses on semi-supervised node classification, where only a small node subset is labeled and graph topology or attributes may contain perturbations or inherent noise.The graph is represented as G = (V, E, X), with V_L labeled nodes and V_U unlabeled nodes.Source: §3.1 · p. 3

3.2 Membership Inference Attack

• Membership inference seeks to identify training-set membership using prediction patterns; the described black-box setup uses shadow-model outputs to train a binary attack classifier.The attacker queries prediction vectors from the target model and is assumed to have a shadow graph dataset from the same distribution.Source: §3.2 · p. 3

3.3 Problem Definition

• The formal objective is a GNN that maintains high accuracy on unlabeled nodes while resisting membership inference, given a small labeled set and potentially poisoned edges or attributes..Source: §3.3 · p. 4

3.4 Preliminaries of Information Bottleneck

• The IB objective maximizes information between bottleneck representation z and label y while constraining information between z and input x to filter redundancy.The parameter β balances the two objective terms.Source: §3.4 · p. 4

3.5 Impacts of IB to Membership Privacy

• GCN+IB has lower MIA-F ROC than GCN, but attacks remain effective at low label rates; increasing label rate reduces GCN+IB's ROC score.The figure reports classification and MIA results on Cora; lower MIA-F ROC indicates better membership privacy.Source: §3.5 · p. 4, Figure 1 · p. 4

3.6 Impacts of IB to Adversarial Robustness

• GCN+IB performs better than GCN on Metattack- and Nettack-perturbed graphs, but its perturbed-graph accuracy remains relatively poor compared with clean graphs.The analysis concludes that directly applying the IB objective to a GNN does not properly eliminate noisy information from adversarial edges.Source: §3.6 · p. 4

4.1 Graph Information Bottleneck

• RM-GIB extracts an attribute code and a subset of neighbors for each node, then uses variational upper bounds to make its graph information bottleneck objective tractable.The objective penalizes information in the bottleneck code while supporting classification.Source: §4.1 · p. 5

4.2 Neural Network Parameterization

• RM-GIB parameterizes the attribute bottleneck, neighbor bottleneck, and predictor with neural networks.The predictor can use different GNN models, while each bottleneck models its corresponding conditional distribution.Source: §4.2 · p. 5

4.2.1 Attribute Bottleneck.

• A Gaussian attribute bottleneck uses an MLP to produce a mean and standard deviation for sampling the node’s attribute code.The code is sampled with the reparameterization trick.Source: §4.2.1 · p. 5

4.2.2 Neighbor Bottleneck.

• A multivariate Bernoulli neighbor bottleneck selects a subset of neighbors, with an MLP estimating each neighbor’s inclusion probability from the center and neighbor attributes.Gumbel-Softmax is used during sampling to propagate classifier gradients to the bottleneck.Source: §4.2.2 · p. 6

4.2.3 Predictor.

• The predictor is a GNN that classifies using the bottlenecked attributes and selected neighbors; its depth matches the selected neighborhood’s hop range.The described prediction uses bottlenecked attribute codes for neighbors as well as the center node.Source: §4.2.3 · p. 6

4.3 Self-supervision for Neighbor Bottleneck

• A self-supervisor estimates node-pair mutual information and treats neighbors with low scores as negative samples to guide the neighbor bottleneck.The method’s rationale is that low-mutual-information neighbors are likely noisy and unhelpful for predicting the center node.Source: §4.3 · p. 6

4.4 Privacy-Preserving Optimization with Pseudo Labels

• RM-GIB first trains on provided labels to collect pseudo labels for unlabeled nodes, then retrains using both sets of labels to support membership privacy.Using unlabeled nodes also increases prediction confidence on them and can narrow prediction gaps between training and test samples.Source: §4.4 · pp. 6–7

5.1.1 Datasets.

• Experiments use Cora, Citeseer, Pubmed, and Flickr, but adversarial evaluations exclude Flickr because the attacks run out of memory on that dataset.The adversarial results are averaged over five runs.Source: §5.1.1 · p. 7, §5.3.1 · p. 8

5.1.2 Baselines.

• Privacy baselines include GCN, GIB, Adv-Reg, DP-SGD, LBP, and NSD; robustness comparisons include GCN-jaccard, GCN-SVD, Elastic, and RSGNN.The baselines represent privacy regularization, differential privacy, output or neighborhood perturbation, preprocessing, and robust message passing or denoising.Source: §5.1.2 · p. 7

5.1.3 Evaluation Protocol.

• Membership privacy is evaluated with black-box shadow-model attacks using ROC scores, while robustness is tested with Metattack and Nettack under stated attacker-access assumptions.Lower attack ROC indicates better membership-privacy protection; the adversarial attackers can access the whole graph and use labeled nodes that do not overlap the target model’s training set.Source: §5.1.3 · p. 7

5.2 Privacy Preserving on Clean Graphs

• On clean graphs, RM-GIB obtains lower MIA-F ROC and MIA-S ROC scores than baselines while maintaining high prediction accuracy.The reported table compares membership-inference defenses across clean graphs.Source: Table 3 · p. 8, §5.2 · p. 8

5.3.2 Membership Privacy Preserving.

• On perturbed Cora and Pubmed graphs, RM-GIB simultaneously preserves membership privacy and gives robust predictions, unlike robust GNNs and privacy-preserving baselines.The reported comparison uses accuracy and MIA-F ROC; the figure also presents results on perturbed graphs.Source: Figure 3 · p. 8, §5.3.2 · p. 8

5.4 Ablation Study

• On Cora, RM-GIB with SGC or GraphSage predictors gives comparable results, while removing neighbor self-supervision or pseudo labels worsens results, especially on perturbed graphs.The ablation results associate pseudo labels with better accuracy and membership privacy and neighbor self-supervision with filtering noisy neighbors.Source: Figure 4 · p. 8, §5.4 · p. 9

5.3.1 Robust Classification.

• RM-GIB achieves comparable or better classification results than state-of-the-art robust GNNs on perturbed graphs, and outperforms GIB.The authors attribute its performance to the attribute and neighbor bottlenecks, neighbor self-supervision, and pseudo labels.Source: §5.3.1 · p. 8, Table 4 · p. 9

5.5 Hyperparameter Sensitivity Analysis

• On Metattack-perturbed Cora, stronger bottleneck regularization improves classification and membership privacy until excessive regularization harms generalization; γ has a non-monotonic accuracy effect.The authors report generally good classification performance for γ in [0.0001, 0.001], with negligible effects on membership privacy.Source: Figure 5 · p. 9, §5.5 · p. 9

6 CONCLUSION AND FUTURE WORK

• RM-GIB separately bottlenecks attribute and neighbor information, applies self-supervision to the neighbor bottleneck, and incorporates pseudo labels in optimization.The framework is intended to address attribute and structural noises, while pseudo labels of unlabeled nodes are used to enhance membership privacy.Source: §6 · p. 9
• The study focuses on membership inference attacks; investigating whether IB can help defend against attribute inference attacks remains future work.Source: §6 · p. 9
• The authors propose investigating whether IB’s minimal-sufficient information can remove sensitive user information, such as race, for fairness.Source: §6 · p. 9

A DATASET

• Cora, Citeseer, and Pubmed are citation networks whose nodes are papers, edges are citations, and attributes are bag-of-words.Source: A DATASET · p. 11
• Flickr is a large-scale image graph whose nodes represent images, descriptions provide attributes, and edges connect nodes sharing common properties.Source: A DATASET · p. 11
• Training samples comprise 2% of nodes for Cora, Citeseer, and Flickr and 0.5% for Pubmed; training nodes do not overlap validation or test nodes.Validation and test splits follow the cited papers for consistency.Source: A DATASET · p. 11

B BASELINES

• Membership-privacy baselines comprise GCN, GCN+PL, GIB, Adv-Reg, DP-SGD, LBP, and NSD, covering pseudo-label training, regularization, differential privacy, output noise, and neighbor selection.Source: B BASELINES · p. 11
• GCN and GIB are included in both the privacy and robustness comparisons.Source: B BASELINES · p. 11
• Robustness baselines include GCN-jaccard and GCN-SVD preprocessing, Elastic robust message passing, and RSGNN graph denoising and densification.Source: B BASELINES · p. 11

• Both the attribute bottleneck and neighbor bottleneck use a 2-layer MLP.Source: C IMPLEMENTATION DETAILS · p. 11
• The default predictor is a 2-layer GCN.Source: C IMPLEMENTATION DETAILS · p. 11
• All neural-network hidden dimensions are 256, and the self-supervision negative-neighbor threshold T is 0.5.β and γ are selected by validation accuracy using grid search over the stated candidate sets.Source: C IMPLEMENTATION DETAILS · p. 11

• For each training sample (x_n,y_n), the distribution of z is modeled as fθ(z,x_n), a neural-network probability density function.Source: D PROOF DETAILS · p. 11
• The proof substitutes an empirical joint distribution formed from training samples and Dirac functions for P(x,y,z;θ).Source: D PROOF DETAILS · p. 11
• Iθ(z;y|x) = 0 regardless of model parameters, yielding the first line of Eq. (3).Source: D PROOF DETAILS · p. 11

• Self-supervisor pretraining costs approximately O(Khd|V|).Source: E TIME COMPLEXITY ANALYSIS · pp. 11–12
• Overall training costs O(Kh(4d+3)|V|), which is linear in the number of graph nodes.Here h is the embedding dimension, K the training epochs, and d the average node degree.Source: E TIME COMPLEXITY ANALYSIS · pp. 11–12
• Per epoch, attribute bottleneck training costs O(h|V|), while the neighbor bottleneck and predictor each cost O(hd|V|).Source: E TIME COMPLEXITY ANALYSIS · p. 12

• Additional results on perturbed Cora are reported as showing the same observations as Fig. 3.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 12, Figure 6 · p. 12
• RM-GIB achieves MIA-F ROC scores close to 50% across different labeled-node sizes, while GIB has high scores at small label rates.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 12
• Under Metattack at a 20% perturbation rate, RM-GIB consistently outperforms GIB across different label rates.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 12, Table 6 · p. 12
• On Cora, as pseudo-labels increase, performance in defending against membership-inference and adversarial attacks increases; adversarial attacks use Metattack at a 20% perturbation rate, with Table 7 reporting results in percent.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 12, Table 7 · p. 12
• On attribute-perturbed Cora and Citeseer graphs, RM-GIB performs better than GIB.The reported Metattack attribute-poisoning rate is 20%.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 12, Table 8 · p. 12