Source-linked AI summary
Overcoming Shortcut Learning in Graph Neural Networks through Active Explanation Guidance
Taraneh Younesian, Steve Azzolin, Antonio Longa, Francesco Ferrini, Vincenzo Marco De Luca, Stefano Teso
TL;DR
GNNs may rely on non-causal shortcuts that harm generalization under distribution shifts. XIGL actively selects explanations for expert corrective feedback, and experiments show its explanation-based strategies mostly improve on prediction-based active-learning baselines across several GNN architectures.
Problem
GNN classifiers can exploit correlational shortcuts instead of task-relevant patterns, limiting their generalization when data distributions change.
Method
XIGL actively prioritizes explanations likely to reveal shortcut reliance, allowing experts to provide corrective feedback on a subset of training examples.
Results
Across several GNN architectures, XIGL’s explanation-based query strategies mostly improve on prediction-based active-learning baselines for graph classification.
Takeaways & Limitations
XIGL suggests that expert corrections to model explanations can reduce shortcut dependencies and annotation costs compared with passive supervision.
Takeaways & Limitations
The evaluation is limited to a small set of datasets and GNN architectures, with broader real-world shortcut settings left for future work.
Abstract
from arXiv · showhide
Graph Neural Networks (GNNs) can solve prediction tasks by unintentionally exploiting shortcuts---that is, edges, nodes, and features that correlate with but are not causal for the prediction---which compromise their reliability in out-of-distribution tasks. We introduce XIGL, an architecture-agnostic human-in-the-loop strategy for removing such shortcuts from GNNs. Our key insight is twofold. On the one hand, reliance on shortcuts can be detected by inspecting GNN explanations. On the other hand, once made aware of such shortcuts, sufficiently expert users can provide tailored corrective feedback, which helps deconfound the model. XIGL supports any query strategy; however, since corrective feedback can be expensive to acquire, we develop an active learning strategy for prioritizing explanations that are more likely to display shortcut behavior, lowering annotation and cognitive costs. We showcase the effectiveness of XIGL, including both existing and proposed explanation-based strategies, on several GNN architectures. Our implementation is available online.
1 Introduction
GNNs may achieve high in-distribution accuracy by relying on non-causal shortcuts, undermining deployment under distribution shifts. XIGL addresses this by actively selecting shortcut-revealing explanations for expert correction and using the feedback to remove shortcut dependence.
- Problem: GNN classifiers can exploit correlational shortcuts such as watermarks, metadata, and simple input statistics, harming generalization when data distributions change.Adding more observational training data is described as insufficient to resolve this problem.
- Method: XIGL uses GNN explanations to expose shortcut reliance and iteratively selects informative instances, obtains expert corrections, and fine-tunes the model with an explanation-guided loss.Experts indicate which nodes the model should not rely on and which it should rely on instead.
- Efficiency: XIGL prioritizes a subset of training examples whose explanations better capture shortcut dependence instead of requiring explanatory supervision across the entire training set.This focuses annotation on subgraphs where corrective feedback is needed and useful, reducing annotation and cognitive costs.
- Results: Across several GNN architectures, XIGL’s explanation-based query strategies mostly improve on prediction-based active learning baselines in graph classification tasks.The baselines select instances using prediction-based informativeness alone.
2 Preliminaries
This section formalizes graph classification and distinguishes causal subgraphs, which determine labels, from spurious subgraphs, which merely correlate with them. Because GNNs may favor easier spurious cues, the desired behavior is to base predictions on causal structure.
- Graph classification: A graph G = (V, E, X) consists of nodes, edges, and node features, with a label y assigned to the graph.The nodes are V = {v1, ..., vN}, edges satisfy E ⊆ V × V, and X ∈ R^N×d.
- Graph classification: Graph classification learns a function f mapping annotated graphs to labels.The target is f : G → Y, learned from training examples of annotated graphs.
- Causal and spurious structure: Input graphs contain causal subgraphs C that determine the ground-truth label and spurious subgraphs S that only correlate with it.In CPatchMNIST, nodes represent superpixels, edges encode adjacency, node features encode average superpixel color, and the label is the digit.
- Shortcut behavior: GNNs tend to exploit easier spurious cues, producing strong in-distribution performance but near-random accuracy when those cues disappear at test time.The example removes color from unseen test instances, exposing the model’s reliance on spurious superpixels.
- Relevant and irrelevant nodes: Relevant nodes Vr belong to the causal subgraph, while irrelevant nodes Vir comprise its complement, including the spurious subgraph and the rest of the graph.Ideally, GNN predictions rely on the causal subgraph and therefore use the “right reason.”
3 Deconfounding through Active Explanation
XIGL deconfounds GNNs by penalizing input-gradient importance assigned to irrelevant nodes, using expert explanation corrections to guide learning away from shortcuts. Because exhaustive explanation annotation is costly, it uses budgeted active learning to select informative graphs, including two explanation-uncertainty query strategies.
- Deconfounding Objective: XIGL minimizes the impact of irrelevant node subgraphs by adding an explanation loss based on input gradients to the classification objective.Input gradients are used because they are simple and require no separately trained explainer.
- Deconfounding Objective: The explanation loss compares gradients for irrelevant and relevant nodes, with λ balancing classification and explanation losses.The ground-truth explanation marks relevant nodes with e_j = 1 and irrelevant nodes with e_j = 0.
- Budgeted Active Learning: Since annotating every graph is costly, XIGL restricts explanation supervision to a budget B and selects a small informative subset for annotation.Training begins with q explained graphs, then repeatedly adds queried batches and fine-tunes for T ≤ B iterations without exceeding the annotation budget.
- Query Strategies: XIGL is compatible with any active-learning query strategy and introduces MaEE and MiEE, which prioritize graphs using uncertainty in their node-level explanations.MaEE selects graphs with high explanation entropy, whereas MiEE selects graphs with low explanation entropy.
- Query Strategies: MaEE targets graphs whose softmax-normalized input gradients are most uniform across nodes, while MiEE targets confident explanations that may reflect shortcut-driven confounders.MiEE is motivated by neural networks’ simplicity bias and their tendency to overfit to spurious patterns.
4 Experiments
Experiments on synthetic ER-color and confounded CPatchMNIST evaluate XIGL across four GNN architectures. Explanation supervision improves generalization to unconfounded data, while active query selection achieves comparable performance with less than 20% of the training data.
- Datasets and setup: The experiments use synthetic ER-color and confounded CPatchMNIST datasets to evaluate GCN, GIN, GraphSAGE, and GAT.ER-color contains 1000 Erdős-Rényi graphs, while CPatchMNIST randomizes label-correlated colors in validation and test sets.
- Passive supervision: Passive explanation supervision generally improves unconfounded-test accuracy and generalization over no-explanation supervision across datasets and architectures.Performance improves further when models are first trained for a few epochs only on Le before training on Lce + λLe.
- Passive supervision: Without supervision, models primarily rely on shortcuts and largely ignore causal patterns; explanation supervision redirects them toward causally relevant features.This interpretation is supported by the observed improvements on unconfounded test data under explanation supervision.
- Passive supervision: On CPatchMNIST, explanation supervision improves every architecture except GraphSAGE, with the largest recovery for GIN after shortcut overfitting causes a sharp test-accuracy drop.Most unsupervised models perform slightly above chance, indicating that the causal signal can be learned only to a limited degree.
- Active learning: Active query selection generally outperforms random sampling, and explanation-based strategies consistently outperform prediction-based strategies on CPatchMNIST across all GNNs.Random sampling nevertheless performs best for GIN on both datasets.
- Active learning: Active learning performs comparably to passive learning while using less than 20% of the training data and requiring significantly fewer annotated explanations.Its lower performance is attributed to the substantially smaller annotation budget, but the resulting gap is relatively small.
5 Discussion, Related Work, & Conclusion
The discussion concludes that expert corrections to model explanations can help remove shortcut dependencies from GNNs while reducing annotation costs relative to passive baselines. Future work will broaden evaluation and study how explanation faithfulness affects debiasing success.
- Discussion & Conclusion: XIGL can use expert corrections to model explanations to reduce shortcut dependencies and annotation costs compared with passive baselines.The authors present this as a conclusion suggested by their work.
- Discussion & Conclusion: Future work will evaluate XIGL on more datasets and GNN architectures, including self-explainable GNNs.The planned extensions explicitly include self-explainable GNNs.
- Discussion & Conclusion: The authors plan to study how explanation faithfulness influences debiasing success.They note that unfaithful explanations may fail to capture the model’s actual behavior.
A GNN Details
The passive-supervision experiments use dataset-specific batch sizes across all GNN models and tune learning rates and λ over specified ranges, while fixing epoch settings for fair comparison.
- Hyperparameters: Batch sizes are 16 for ER-color and 256 for CPatchMNIST across all models.These hyperparameters are reported for the passive supervision scenario.
- Hyperparameters: Learning rates are evaluated over the range 1e −7 to 1e −3, with λ varied between 1 and 1000.The passage describes a wide hyperparameter search for both datasets.
- Training protocol: Epoch counts are fixed for passive methods to support fair comparison, with a fixed value also used across active query settings.The passage ends mid-sentence after describing the active query strategy’s fixed setting.
B AL Details
Active-learning experiments use q = 10 across both datasets, with dataset-specific query budgets and round counts applied consistently to all models. Learning rates and λ are varied as in passive experiments and reported for each model, query strategy, and dataset combination.
- Active-learning setup: Active-learning experiments set q to 10 for both ER-color and CPatchMNIST, using the same settings across all models.ER-color uses 5 queried instances per round for 20 rounds, while CPatchMNIST uses 50 per round for 10 rounds; both use 30 epochs per round.
- Active-learning setup: Learning rates and λ vary over the same range as in passive experiments for every GNN, query strategy, and dataset combination.The corresponding settings are reported in Table 4.