PaperlayerView exact revision on arXiv

Source-linked AI summary

RelEx: A Model-Agnostic Relational Model Explainer

Yue Zhang, David Defazio, Arti Ramesh

arXiv:2006.00305v1cs.LGstat.ML

TL;DR

Most explainers assume iid data, leaving few options for explaining relational models. RelEx learns relational explanations using black-box outputs, and the paper reports performance comparable to or better than existing relational explainers across its datasets.

  • Problem

    Most explanation approaches assume iid data, while relational explanations require identifying the structure around a target node and have limited existing methods.

    Source: §1 · p. 1

  • Method

    RelEx learns a local differentiable approximation of a black-box relational model from perturbed computation graphs, then learns an interpretable mask.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across synthetic and real-world datasets, RelEx performs comparably to or better than GNN-Explainer and relational extensions of iid explainers despite requiring less black-box information.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Experiments demonstrate RelEx's quantitative and qualitative capability on GNNs and HL-MRFs across two synthetic and one real-world graph dataset.

    Source: §6 · p. 8

Abstract

from arXiv · show

In recent years, considerable progress has been made on improving the interpretability of machine learning models. This is essential, as complex deep learning models with millions of parameters produce state of the art results, but it can be nearly impossible to explain their predictions. While various explainability techniques have achieved impressive results, nearly all of them assume each data instance to be independent and identically distributed (iid). This excludes relational models, such as Statistical Relational Learning (SRL), and the recently popular Graph Neural Networks (GNNs), resulting in few options to explain them. While there does exist one work on explaining GNNs, GNN-Explainer, they assume access to the gradients of the model to learn explanations, which is restrictive in terms of its applicability across non-differentiable relational models and practicality. In this work, we develop RelEx, a model-agnostic relational explainer to explain black-box relational models with only access to the outputs of the black-box. RelEx is able to explain any relational model, including SRL models and GNNs. We compare RelEx to the state-of-the-art relational explainer, GNN-Explainer, and relational extensions of iid explanation models and show that RelEx achieves comparable or better performance, while remaining model-agnostic.

1 Introduction

Most explainers assume iid data, leaving relational models with few options despite their use in structured-data tasks. RelEx is introduced as a model-agnostic explainer for relational predictions, with experiments showing performance comparable to or better than existing relational explainers.

1 Introduction

  • Relational models include statistical relational learning (SRL) methods and graph neural networks (GNNs), which can support molecule classification, knowledge graph completion, and recommendation systems.The paper describes RelEx as model-agnostic and, to the authors’ knowledge, the first general-purpose model-agnostic relational explainer.Source: §1 · pp. 1–2
  • Most existing explainers target iid data, whereas relational explanations must identify the relevant relational structure around the node of interest.Source: §1 · p. 1
  • GNN-Explainer learns important neighbor nodes and links using masks optimized with gradients from the underlying GNN.Source: §1 · p. 2
  • RelEx learns a local differentiable approximation of the black-box model over node perturbations, then learns an interpretable mask over that approximation.Source: §1 · p. 2
  • Across synthetic and real-world datasets, RelEx performs comparably to or better than GNN-Explainer and relational extensions of iid explainers while requiring less black-box information.The experiments compare explanations with ground-truth relational structures and also report qualitative results across datasets.Source: §1 · p. 2
  • RelEx learns diverse explanations, allowing users to choose one that better fits their domain perspective while remaining true to the black-box model.Source: §1 · p. 2

2 Related Work

Related work spans SRL and GNN models, iid post-hoc explainers, and approaches to graph interpretability. The paper positions RelEx against the limited post-hoc relational work and highlights gradient access and input-format constraints in existing approaches.

2 Related Work

  • SRL covers uncertain domain models with relational structure and handcrafted first-order logic rules; HL-MRFs are an example using Probabilistic Soft Logic over knowledge graphs.The paper selects HL-MRFs as a representative SRL model for evaluation.Source: §2 · p. 2
  • GNNs learn node representations by aggregating neighboring representations, while this paper evaluates them for node classification.Source: §2 · p. 2
  • IID explainers include LIME and Anchors for local explanations, plus gradient-based SmoothGrad [7] and Integrated Gradients [6] for image saliency.LIME learns an interpretable local approximation; Anchors clarify where its explanations apply.Source: §2 · p. 3
  • GNN-Explainer is described as the only post-hoc approach among the cited graph methods to explain particular node predictions.Other cited directions include edge attention, simplified GCNs, and disentangled node representations.Source: §2 · p. 3
  • RelEx needs black-box predictions rather than gradients, so it can explain non-differentiable HL-MRFs; it also accommodates models that use edge indices instead of adjacency matrices.Source: §2 · p. 3

3 RelEx: Learning-based Relational Explainer

RelEx explains a target node by selecting influential nodes and edges from its computation-graph neighborhood. It learns a sparse mask using a local approximation of the black-box and supports different mask parameterizations and diverse explanations.

3.1 RelEx Problem Formulation

  • RelEx represents an explanation as a sparse adjacency mask selecting nodes and edges important to the black-box prediction.The objective balances prediction loss with a sparsity measure; L2,1 regularization groups adjacency rows to promote sparsity over nodes as well as edges.Source: §3.1 · pp. 3–4, §3.2 · p. 5

3.2 RelEx Architecture

  • The local approximator is a residual GCN that concatenates each layer's output into the network's final output.The residual architecture is used to increase the GCN's ability to represent graph topology without adding layers or parameters.Source: §3.2 · p. 4
  • RelEx trains a local approximator on black-box outputs for perturbed computation graphs sampled around the target node.Its modified BFS sampling favors closer connected nodes and excludes nodes outside the target's computation graph; the approximator must be locally faithful and differentiable with respect to the adjacency matrix.Source: §3.2 · p. 4
  • RelEx offers a soft sigmoid mask and a Gumbel-softmax mask that directly yields discrete nodes and edges.The sigmoid mask assigns continuous edge-importance values and requires thresholding, while the Gumbel-softmax version produces binary mask entries.Source: §3.2 · pp. 4–5
  • RelEx encourages different explanations by maximizing cross-entropy between masks; experiments report that at least one learned mask corresponds to the right reason.Each mask is compared with previously learned masks using a weighted cross-entropy term.Source: §3.2 · p. 5

4 Evaluation Methods

The evaluation methods adapt Anchors and Saliency Map to relational graphs, then assess explanations using agreement with ground-truth structures and infidelity under perturbations.

4.1 Comparison with State-of-the-art Explainers

  • Relational Anchors treat graph edges as anchor features and estimate precision from perturbed samples that retain those edges.Samples are generated with breadth-first search from the target node; the threshold varies with how much black-box predictions change under perturbations.Source: §4.1 · p. 5
  • The relational Saliency Map uses normalized gradients of the black-box loss with respect to the adjacency matrix as edge-importance values.The gradients are normalized to values between 0 and 1.Source: §4.1 · pp. 5–6

4.2 Relational Explanation Evaluation Metrics

  • AUC-ROC measures how closely an explanation matches the ground-truth right-reason structure; the reported tables cover TREE-GRID and TREE-BA.The text describes comparing learned explanations with known structural reasons, while the tables report AUC-ROC and infidelity for the two synthetic datasets.Source: §4.2 · p. 6, Table 1 · p. 6, Table 2 · p. 6
  • Infidelity quantifies how well an explanation captures changes in the predictor's output under significant perturbations.The metric is motivated by the difficulty of separating errors from the black-box model from errors due to the explainer.Source: §4.2 · p. 6

5 Experiments

Across synthetic and molecular datasets, RelEx produces relational explanations that align with known structures; its performance varies by model and explanation setting, and diverse alternatives can add domain-relevant detail.

5.1 Experiments on Synthetic Relational Datasets with GNNs as the Black-box Model

  • On TREE-GRID and TREE-BA, RelExGumbel achieves the best AUC-ROC and infidelity results among the compared explainers.Saliency Map performs less well than the relationally tailored explainers.Source: §5.1 · p. 6

5.2 Experiments on Synthetic Relational Dataset with HL-MRFs as the Black-box Model

  • On TREE-GRID-BA, RelExSigmoid performs better than RelExGumbel on the reported results for the HL-MRF black box.The authors attribute this to continuous link-importance values captured by learned rule weights; gradient-dependent baselines cannot be applied to HL-MRFs.Source: Table 4 · p. 7, §5.2 · p. 7

5.3 Experiments on Molecule Dataset with GNNs as the Black-box Model

  • On MUTAG, RelExGumbel and GNN-Explainer achieve similar best infidelity, while explanations recover characteristic carbon-ring and nitrogen -NO2 structures.For carbon nodes, RelExGumbel finds a hexagonal ring and RelExSigmoid finds two connected hexagonal rings; all explainers except Relational Anchors identify the nitrogen -NO2 structure.Source: Table 5 · p. 7, §5.3 · pp. 7–8, Figure 3 · p. 8

5.4 Diverse Explanations on Molecule Dataset

  • For the same node, diverse explanations can expose multiple valid motifs, with some alternatives more meaningful from the domain perspective or less noisy.Examples include one explanation finding a pentagon and another finding both pentagon and hexagon rings, and an alternative that excludes noise from a hexagonal structure.Source: Figure 5 · p. 8, §5.4 · p. 8

6 Conclusion

RelEx is presented as a model-agnostic explainer for black-box relational models. Experiments on GNNs and HL-MRFs show quantitative and qualitative explanatory capability, while diverse explanations add practical value for domain-specific predictions.

  • RelEx is presented as able to explain any black-box relational model.Source: §6 · p. 8
  • Experiments demonstrated quantitative and qualitative explanatory capability for GNNs and HL-MRFs on two synthetic and one real-world graph dataset.Source: §6 · p. 8
  • Learning diverse explanations further enhances RelEx's practical value and applicability in explaining domain-specific predictions.Source: §6 · p. 8
Loading 2006.00305v1…