Source-linked AI summary
Causal Attention for Interpretable and Generalizable Graph Classification
Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, Tat-Seng Chua
TL;DR
Attention and pooling-based GNNs can learn shortcut correlations instead of causal graph features, harming generalization when shortcut distributions shift. The paper proposes Causal Attention Learning (CAL), which estimates causal and shortcut features and applies backdoor adjustment; experiments support its effectiveness, interpretability, and generalization.
Problem
Learning-to-attend objectives rely on statistical graph-label correlations and can capture noncausal shortcut features, limiting generalization under distribution shifts.
Method
CAL uses attention modules to estimate causal and shortcut features, then applies backdoor adjustment by combining causal estimates with varied shortcut estimates.
Results
CAL’s effectiveness, better generalization, and insightful interpretations are supported by extensive experiments and analyses on synthetic and real-world datasets.
Takeaways & Limitations
CAL encourages GNNs to exploit causal features while ignoring shortcut parts in graph classification.
Takeaways & Limitations
Backdoor adjustment requires an unobservable confounder set, and directly manipulating discrete graph data under domain constraints is difficult.
Abstract
from arXiv · showhide
In graph classification, attention and pooling-based graph neural networks (GNNs) prevail to extract the critical features from the input graph and support the prediction. They mostly follow the paradigm of learning to attend, which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm makes GNN classifiers recklessly absorb all the statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Instead of underscoring the causal features, the attended graphs are prone to visit the noncausal features as the shortcut to predictions. Such shortcut features might easily change outside the training distribution, thereby making the GNN classifiers suffer from poor generalization. In this work, we take a causal look at the GNN modeling for graph classification. With our causal assumption, the shortcut feature serves as a confounder between the causal feature and prediction. It tricks the classifier to learn spurious correlations that facilitate the prediction in in-distribution (ID) test evaluation, while causing the performance drop in out-of-distribution (OOD) test data. To endow the classifier with better interpretation and generalization, we propose the Causal Attention Learning (CAL) strategy, which discovers the causal patterns and mitigates the confounding effect of shortcuts. Specifically, we employ attention modules to estimate the causal and shortcut features of the input graph. We then parameterize the backdoor adjustment of causal theory -- combine each causal feature with various shortcut features. It encourages the stable relationships between the causal estimation and prediction, regardless of the changes in shortcut parts and distributions. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness of CAL.
1 INTRODUCTION
Attention and pooling-based GNNs can exploit noncausal shortcut features because learning to attend follows training correlations rather than causal effects. CAL addresses this confounding by estimating causal and shortcut features and enforcing stable predictions across shortcut variations.
- Learning-to-attend methods maximize the mutual information between attended graphs and labels, but do not distinguish causal from noncausal feature effects.
- Shortcut features from selection biases, noisy features, or trivial graph patterns can support training predictions while bypassing causal features.
- Because shortcut distributions may change outside training conditions, reliance on them contributes to poor generalization in real-world deployment.
- CAL models shortcut features as confounders that open a backdoor path between causal features and predictions, producing spurious correlations.
- CAL estimates causal and shortcut graph features with attention modules, then combines causal estimates with varied shortcuts through backdoor adjustment.
- Experiments on synthetic and real-world datasets, together with visualizations and analyses, support CAL’s effectiveness, interpretability, and rationality.
2 PRELIMINARIES
GNN graph-classification pipelines represent graphs through node features, adjacency structure, attention, pooling, readout, and classification. Their learning-to-attend objective optimizes training prediction and can consequently capture noncausal shortcut features.
- 2.1 Notations: A graph G = {A, X} consists of node features X and adjacency matrix A, while node representations H encode learned features for its nodes.
- 2.2 Attention Mechanism in GNNs: Edge-level attention uses weighted message passing and aggregation to update node representations.
- 2.2 Attention Mechanism in GNNs: Node-level attention uses a self-attention mask to select attentive node representations, followed by pooling when producing graph-level features.
- 2.2 Attention Mechanism in GNNs: A readout function summarizes output node representations into graph representation h_G, which a classifier projects into a probability distribution.
- 2.2 Attention Mechanism in GNNs: Learning-to-attend methods minimize empirical cross-entropy risk over training graphs and labels.
- 2.2 Attention Mechanism in GNNs: This objective relies on statistical graph-label correlations and therefore inevitably captures noncausal shortcut features for prediction.
3 METHODOLOGY
The methodology models shortcut features as confounders that create a backdoor path between causal features and predictions, then uses Causal Attention Learning (CAL) to separate these features and reduce confounding through representation-level intervention.
- 3.1 Causal Analysis: The structural causal model relates graph data, causal features, shortcut features, graph representations, and predictions, identifying shortcuts as confounders between causal features and labels.The backdoor path is C←G→S→R→Y, which can induce spurious correlations and wrong predictions based on shortcuts.
- 3.2 Backdoor Adjustment: Backdoor adjustment estimates P(Y|do(C)) by stratifying shortcut features and averaging conditional predictions using their prior distribution.The confounder set is often unobservable, and direct graph-level intervention is difficult because graph data are discrete and domain-constrained.
- 3.3 Causal and Trivial Attended-graph: CAL uses attention modules to estimate causal and trivial attended-graphs from node- and edge-level soft masks over the input graph.The causal and trivial masks are complementary, and their attention scores identify the relative importance of nodes and edges for each attended-graph.
- 3.4.2 Disentanglement: The disentanglement objectives train the causal attended-graph toward the ground-truth label while pushing trivial-attended-graph predictions toward a uniform distribution.This separates features useful for classification from shortcut patterns considered unnecessary for classification.
- 3.4.3 Causal intervention: CAL performs implicit representation-level causal intervention by pairing each causal attended-graph with stratified trivial representations and enforcing invariant predictions.Random addition supplies the trivial-feature combinations used for the intervention loss.
4 EXPERIMENTS
Experiments on synthetic and real-world graph datasets evaluate CAL’s robustness to distribution shifts, component contributions, and interpretability. CAL generally improves OOD performance and identifies causal graph patterns.
- Datasets: Synthetic datasets contain 8,000 balanced graphs across four classes, each combining a causal subgraph with a trivial subgraph; training bias varies while testing remains unbiased.The task is to predict the causal subgraph type, with bias-levels SYN-b spanning b∈{0.1, 0.2, ..., 0.9}.
- Synthetic performance: Attention- and pooling-based baselines lose 7.37%–12.75% and 7.82%–14.24%, respectively, on SYN-0.1, showing that discriminative feature refinement can harm OOD generalization.On SYN-0.9, the corresponding degradation ranges are 3.79%–13.79% and 3.99%–12.10%.
- Synthetic performance: On unbiased data, GIN reaches 96.74%, GAT 92.69%, and GCN 90.94%, but at bias-levels 0.1 and 0.9 their performance drops by 9.55%/7.36%, 8.71%/5.47%, and 6.60%/5.43%.The robustness ranking reverses from GIN > GAT > GCN in ID evaluation to GCN > GAT > GIN under extreme bias.
- Synthetic performance: CAL consistently outperforms the baselines across bias-levels and achieves performance comparable to IRM and DRO without requiring additional trivial-subgraph annotations.This supports the role of confounder mitigation in stabilizing OOD performance.
- Real-world performance: On real-world datasets, CAL consistently improves different GNN backbones; GCN+CAL yields relative gains of 1.92% on IMDB-B and 4.52% on MNIST.The experiments indicate that distribution shifts also occur in real-world graph datasets.
- Ablation and interpretation: Ablations show that node attention, random causal intervention, and disentanglement each contribute to CAL’s performance, while visualizations place high attention on deterministic causal regions.Removing random combination causes severe degradation, and darker nodes or wider edges concentrate on synthetic causal subgraphs and MNIST digit pixels.
5 RELATED WORK
Prior graph attention work locates informative nodes or edges by fitting statistical correlations with labels, but related methods also expose limitations in interpreting misclassification patterns and transferring causal strategies to graphs.
- Attention-based methods: Attention-based GNN methods use node or edge modules as soft masks to locate attended graphs.These methods belong to a broader “learning to attend” paradigm that maximizes mutual information between attended graphs and labels.
- Misclassification analysis: Figure 8 highlights the concentration degree of misclassification through a red circle.
- Causal intervention: Adversarial training and front-door adjustment have been used for causal intervention, but these approaches were tailored to computer vision.The passage contrasts these approaches with graph-specific causal attention learning.
6 CONCLUSION
The paper frames shortcut features as confounders that create spurious correlations in graph classification and proposes CAL to reduce this confounding while emphasizing causal features.
- Conclusion: Shortcut features establish a backdoor path between causal features and predictions, misleading GNNs into learning spurious correlations.
- Conclusion: CAL uses causal-theory backdoor adjustment to encourage GNNs to exploit causal features while ignoring shortcut parts.
- Conclusion: Extensive experimental results and analyses verify CAL’s effectiveness, while future work targets stronger disentanglement and broader graph-learning tasks.
A.1 Algorithm
The algorithm trains attention modules and classifiers with supervised, uniformity, and causal objectives, then uses causal attended-graphs for inference.
- Inference: At inference, CAL uses the causal attended-graph for prediction.
- Inputs and outputs: CAL takes a dataset, model, attention modules, classifiers, and two weighting parameters as inputs, and returns trained parameters.
- Causal graph construction: The algorithm derives causal masks from attention scores and applies them to adjacency and node-feature matrices to form causal graphs.
- Optimization: Training minimizes a total loss combining supervised, uniformity, and causal terms before updating all trainable parameters.
A.2 Datasets Details
The experiments use synthetic graphs with controlled trivial and causal subgraphs alongside biological, social, and superpixel graph-classification datasets.
- Synthetic graphs: Synthetic graph instances contain trivial and critical subgraphs, including BA-SHAPES and balanced binary Tree structures.
- Synthetic graphs: A causal subgraph is randomly attached to a trivial-subgraph node, after which 10% random edges perturb the resulting graph.
- Synthetic graphs: Synthetic graphs use one-hot node-degree features with dimension 20 and a 7:1:2 training, validation, and testing split.
- Dataset statistics: Table 3 summarizes the statistics of the datasets used in the experiments.
- Real-world graphs: Real-world experiments cover three biological and three social TUDataset benchmarks, plus superpixel graphs.The TUDataset evaluation uses 10-fold cross-validation and reports average accuracy with standard deviation.
A.3 Hyper-parameters
The experiments use fixed training settings across models and datasets, with baseline configurations reproduced from their original papers. Training time is also compared in a dedicated table.
- Models are trained for 100 epochs with batch size 128 using the Adam optimizer.
- GCN, GIN, and GAT encoders use 3 layers and 128 hidden units for SYN-b and TUDataset.
- Superpixel MNIST and CIFAR-10 use 4-layer GNN encoders with 146 hidden units, following the referenced configuration.
- Baselines follow the original papers’ default settings, with missing results reproduced.
- Table 4 compares training time in minutes.