Source-linked AI summary
Explainability Techniques for Graph Convolutional Networks
Federico Baldassarre, Hossein Azizpour
TL;DR
Graph Network decisions are difficult to explain, despite the value of human-understandable interpretations. The paper analyzes gradient-based and decomposition-based techniques on a toy infection task and molecular solubility, finding that the methods identify relevant graph components while LRP often produces more natural causal explanations. It frames the study as groundwork for future Graph Network explanation methods and real-world applications.
Problem
Existing explanation methods had been developed for standard deep networks, but their applicability to Graph Networks had received limited study.
Method
The paper analyzes gradient-based and decomposition-based explanation techniques using a toy infection dataset and a molecular solubility task.
Results
All three evaluated techniques identify relevant nodes and edges in the infection task, while LRP explanations are more natural in cause-and-effect terms and highlight chemistry features correlated with solubility.
Takeaways & Limitations
The study provides case-based analysis, high-level design discussions, and five PyTorch implementations to support future development of Graph Network explanation techniques.
Takeaways & Limitations
The study uses two simple tasks, and LRP is applied to regression although it was originally introduced for classification predictions.
Abstract
from arXiv · showhide
Graph Networks are used to make decisions in potentially complex scenarios but it is usually not obvious how or why they made them. In this work, we study the explainability of Graph Network decisions using two main classes of techniques, gradient-based and decomposition-based, on a toy dataset and a chemistry task. Our study sets the ground for future development as well as application to real-world problems.
1. Introduction
Graph Networks extend deep learning to graph-structured data, but their decisions remain difficult to explain. This work addresses that gap by comparing gradient-based and decomposition-based explanations on toy and chemistry tasks.
- Graph Networks apply deep learning to naturally graph-structured domains including chemistry, life science, and physics.
- Interpretability can improve human trust, diagnose biases, support dataset design, and provide insight into governing laws.
- The study compares gradient-based and decomposition-based explanation methods on a toy dataset and a chemistry task.
- The authors present the work as a first study focused on explainability techniques for Graph Networks.
- The paper identifies challenges and future directions for explaining Graph Network predictions and compares explanation methods on graph-based tasks.
2. Related works
Graph Networks support predictions at vertex, edge, and global levels while preserving relational structure. The paper situates its explanation study within broader model-interpretation work and addresses the limited prior exploration of explanations for Graph Networks.
- Graph Networks embed graphs while preserving relational structure and leveraging properties of Euclidean space.
- End-to-end graph models can make predictions at vertex, edge, or global levels.
- Graph Network variants have introduced gating, spectral and spatial convolutions, skip connections, and attention.
- Model-explanation techniques have developed partly in response to expectations of safety, reliability, and fairness around deep-network decisions.
- Prior work had not focused on explaining Graph Network predictions, motivating explanations for individual decisions in end-to-end molecular tasks.
3. Method
The method uses message-passing Graph Networks that update graph features without changing topology, alongside gradient-based and decomposition-based explanation techniques. These methods distinguish sensitivity to input changes from features’ contributions to the final prediction.
- Graph Networks use message passing to aggregate local information similarly to convolutions in CNNs.
- Each processing layer updates edge, node, and graph features while leaving the graph topology unchanged.
- The mapping f : (E, V, u) → y can produce a graph-level quantity or predictions for individual nodes and edges.
- Sensitivity Analysis explains differentiable predictions using the squared norm of the input gradient, measuring output change under input variation.
- Guided Backpropagation constructs a gradient-based saliency map while clipping negative gradients.
- LRP identifies input features contributing most to the final prediction and can analyze both positive and negative relevance.
4. Experiments
The experiments compare explainability techniques on a toy infection-prediction problem and molecular solubility regression. Gradient-based methods identify relevant inputs, while LRP produces more natural causal explanations and highlights chemically meaningful features, with the caveat that it is adapted here from classification to regression.
- Experiments: The study evaluates explainability methods on both a toy graph infection task and an aqueous solubility regression task.The infection task predicts each node’s state after one disease-spread step; the chemistry task predicts molecular solubility.
- 4.1. Infection: All three techniques identify relevant nodes and edges for individual infection predictions, but LRP explanations align more naturally with cause-and-effect descriptions.Variation-based explanations can diverge from intuitive causal accounts, whereas LRP provides more natural results.
- 4.1. Infection: In the infection example, LRP assigns negative relevance to node 2’s initial health and positive relevance to node 1’s sickness and the non-virtual edge 1 →2.Node 4 is ignored because max pooling considers only one input.
- 4.2. Solubility: The multi-layer GN matches prior performance on aqueous solubility prediction while supporting explanations of individual molecular predictions.The model predicts aqueous solubility from organic compounds’ molecular graphs.
- 4.2. Solubility: LRP assigns positive relevance to R-OH groups and negative relevance to central carbons and non-polar aromatic rings, matching known solubility-correlated features.Similar observations were previously obtained by manually inspecting high-scoring predictions.
- 4.2. Solubility: LRP was originally introduced for classification explanations but is adopted here for the regression task.The paper directs readers to an appendix discussion for interpreting these regression explanations.
5. Discussion
Graph explanations require graph-specific treatment because connections, aggregation choices, heterogeneous features, and structural changes affect what counts as meaningful evidence. The paper discusses structure-aware relevance propagation, search-based max-pooling explanations, feature-level visualization, and challenges in evaluating graph explanations.
- 5.1. The role of connections: Graph-domain explanations must account for connectivity because informative connections may lack edge features and still determine predictions.The paper aggregates explanations across multiple graph-convolution steps so the importance of connections can emerge.
- 5.1. The role of connections: Aggregating relevance across message-passing steps identifies a relevant path between nodes even when edge features are absent.Figure 4 illustrates this for a prediction on node B attributed to node A.
- 5.2. Pooling: Pooling changes explanations: sum and mean distribute relevance to all inputs, whereas max pooling can omit relevant alternatives.The proposed search-based backward pass propagates relevance only to inputs that preserve a similar prediction when selected as maxima.
- 5.3. Heterogeneous Graph Features: Graph explanations may need feature-level rather than node- or edge-level visualization because graph inputs mix semantically meaningful continuous, binary, and categorical features.Graph heatmaps can therefore be insufficient for interpreting individual feature contributions.
- 5.2. Pooling: Lp-norm propagation can make max-pooling explanations more complete but may disperse relevance to unimportant inputs when N is large.The search-based alternative is intended to avoid this over-dispersion.
- 5.4. Perturbation-based evaluation: Perturbation-based evaluation is harder for graphs because altering nodes or edges can drastically change molecular meaning or invalidate the graph.The paper discusses domain-specific changes or learned graph manifolds as possible ways to preserve semantic proximity.
6. Conclusion
The paper analyzes existing explanation techniques for Graph Networks across two complementary tasks and provides an implementation for future GN explanation research. It finds that decomposition-based explanations are more natural than variation-based explanations for describing causes and effects.
- 6. Conclusion: The paper analyzes major existing explanation techniques for Graph Networks using two complementary tasks and discusses design choices for GN explanations.It also provides five PyTorch-autograd explanation implementations intended for general GN definitions.
- 6. Conclusion: Variation-based explanations tend to diverge from human descriptions of causes and effects, whereas decomposition-based explanations are more natural.The authors posit that decomposing the output signal makes LRP suitable for categorical node and edge features.
- 6. Conclusion: The chosen LRP rule is the ϵ-stabilized rule with ϵ = 10^-16 because the αβ-rule was unstable with zero-valued inputs or weights.The instability occurs frequently with one-hot categorical features and L1 weight regularization.
- 6. Conclusion: LRP is extended from classification to regression so atom and bond relevance can represent positive or negative contributions to predicted solubility.Because the networks use bias terms, LRP relevance is not fully conserved.
B.1. Infection
The Infection task uses graph features describing health, immunity, edge virtuality, and uninformative controls to predict each node’s state after one propagation step. A single-layer GN is trained with sum or max pooling over incident edges.
- B.1. Infection: The training set contains 100,000 Barabási-Albert graphs with 30 or fewer nodes.
- B.1. Infection: The Infection network uses one graph-processing layer with shallow ReLU multilayer perceptrons and sum or max pooling over incident edges.Training minimizes per-node binary cross-entropy with Adam.
- B.1. Infection: The synthetic Infection dataset encodes node health and immunity, edge virtuality, and uninformative features that explanations should ignore.Binary features use {-1, +1} encoding.
- B.1. Infection: The Solubility experiment uses a separate dataset of around 1,000 organic molecules represented as atom-and-bond graphs.Molecular features include atom degree, hydrogens, valence, type, and bond properties.
C.1. Infection
The Infection case study compares Sensitivity Analysis, Guided Backpropagation, and Layer-wise Relevance Propagation on node-level predictions from a GN using max pooling. The network correctly predicts the graph’s node states, while the methods produce different feature-level explanations.
- C.1. Infection: The max-pooling GN correctly predicts every node’s state after one infection-propagation step in a graph with sick, healthy, and immune nodes.
- C.1. Infection: The case study examines explanations for an infected node, a node receiving no infection, and an immune node.
- C.1. Infection: Explanations are produced with Sensitivity Analysis, Guided Backpropagation, and Layer-wise Relevance Propagation in that order.
- C.1. Infection: Each method visualizes gradient or relevance values over individual node and edge features and through a graphical representation of the input graph.
C.2. Solubility
The paper examines how pooling and explanation methods behave on Infection graphs and how LRP explains molecular solubility. Max pooling can yield incomplete explanations, while sum pooling can accumulate small contributions into incorrect predictions.
- C.2. Solubility: Valid molecule alterations cannot be generated automatically from atom or bond importance, so domain-specific knowledge is required.The structured graph representation makes small Euclidean perturbations unavailable.
- C.2. Solubility: LRP-guided molecular alterations progressively reduce predicted solubility by converting C-O bonds to C=O bonds and removing oxygen atoms.The modifications are selected to remain chemically valid.
- C.2. Solubility: Max pooling identifies only one possible explanatory neighbor when multiple sick neighbors could contribute to an infection.Sum pooling includes all possible infection sources but can produce incorrect predictions.
- C.2. Solubility: Sum pooling can fail because small contributions from a large neighborhood accumulate and produce a wrong answer.This differs from image convolutions, whose neighborhood size is fixed.
- C.2. Solubility: For glucose, LRP assigns the main negative contribution to the oxygen atom that makes the ring heterocyclic.The predicted solubility is 0.74 log mol/L versus 0.67 log mol/L measured.
- C.2. Solubility: LRP assigns molecular relevance to atom and bond features, including negative relevance for the hydrocarbon chain and positive relevance for OH groups in 4-hexylresorcinol.These assignments agree with common organic-chemistry knowledge.