PaperlayerView exact revision on arXiv

Source-linked AI summary

Energy-based Out-of-Distribution Detection for Graph Neural Networks

Qitian Wu, Yiting Chen, Chenxiao Yang, Junchi Yan

arXiv:2302.02914v2cs.LGcs.AIcs.SI

TL;DR

OOD detection for graph data remains underexplored, in part because graph nodes are interdependent while many approaches assume i.i.d. inputs. GNNSafe derives an energy-based detector from standard GNN classifiers, and the paper reports improvements over competing methods across its evaluations. It concludes that GNNSafe is a simple, strong baseline for graph OOD detection, while noting that energy regularization requires exposed OOD data.

  • Problem

    OOD detection on graphs is underexplored, while many existing approaches assume independently sampled inputs despite graph-node interdependence.

    Source: §1 · pp. 1–2

  • Method

    GNNSafe extracts an energy-based OOD discriminator directly from logits of a GNN classifier trained with standard classification loss.

    Source: §1 · p. 2

  • Results

    With outlier exposure, GNNSafe improves average AUROC by up to 17.0% over strong competitors across node-classification datasets and OOD types.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that supervised GNNs can intrinsically serve as effective OOD discriminators, and present GNNSafe as a simple, general baseline for graph OOD detection.

    Source: §6 · p. 9

    Energy-regularized learning requires additional OOD data exposed during training, though the authors report it is unnecessary in quite a few practical cases.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Learning on graphs, where instance nodes are inter-connected, has become one of the central problems for deep learning, as relational structures are pervasive and induce data inter-dependence which hinders trivial adaptation of existing approaches that assume inputs to be i.i.d.~sampled. However, current models mostly focus on improving testing performance of in-distribution data and largely ignore the potential risk w.r.t. out-of-distribution (OOD) testing samples that may cause negative outcome if the prediction is overconfident on them. In this paper, we investigate the under-explored problem, OOD detection on graph-structured data, and identify a provably effective OOD discriminator based on an energy function directly extracted from graph neural networks trained with standard classification loss. This paves a way for a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. It also has nice theoretical properties that guarantee an overall distinguishable margin between the detection scores for in-distribution and OOD samples, which, more critically, can be further strengthened by a learning-free energy belief propagation scheme. For comprehensive evaluation, we introduce new benchmark settings that evaluate the model for detecting OOD data from both synthetic and real distribution shifts (cross-domain graph shifts and temporal graph shifts). The results show that GNNSafe achieves up to $17.0\%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.

1 INTRODUCTION

The introduction frames graph OOD detection as an underexplored problem because graph nodes are interdependent while many existing methods assume independent inputs. It presents GNNSafe as a simple energy-based GNN detector with theoretical support and strong reported results.

Related Work

  • Prior graph OOD generalization studies address distribution shifts with graph-specific approaches, including causal invariance and invariant substructure identification.The introduction distinguishes this generalization work from its focus on OOD detection.Source: §1 · p. 1

Motivation

  • Graph OOD detection matters because node interdependence makes methods built for independently sampled inputs difficult to apply directly to graph data.The introduction notes that OOD detection aims to distinguish OOD from in-distribution data, including in safety-critical applications.Source: §1 · pp. 1–2

Simplicity and Generality

  • GNNSafe extracts an OOD discriminator from GNN classifier logits through an energy function, without requiring a graph generative model or an extra OOD discriminator.The energy-based detector is described as architecture-agnostic and applicable to off-the-shelf GNNs and graph Transformers.Source: §1 · p. 2

Theoretical Soundness

  • The authors state that the energy detector can provably distinguish in-distribution from OOD inputs, with detection consensus further strengthened by learning-free energy belief propagation.They also discuss auxiliary regularization when OOD observations are available for outlier exposure.Source: §1 · p. 2

Practical Efficacy

  • With outlier exposure, GNNSafe improves average AUROC by up to 17.0% over strong competitors across node-classification datasets and OOD types.Without outlier exposure, it improves average AUROC by up to 12.9% over SOTA approaches.Source: §1 · p. 2

2 BACKGROUND

The background formulates graph prediction over partially labeled, interdependent nodes and frames OOD detection as deciding whether an input belongs to the training distribution. It also notes that energy scores tend to be lower for in-distribution data, while energy-based OOD detection has largely focused on i.i.d. inputs.

Predictive tasks on graphs

  • Graph prediction uses a node-level classifier to predict labels for unlabeled in-distribution instances in a partially labeled graph.The graph encodes interdependence among instances, with node features and labels defined for the task.Source: §2 · p. 2

Out-of-distribution detection

  • For OOD detection on graphs, a node's decision depends on its ego-graph and neighboring nodes, unlike independent per-input decisions in vision.The ego-graph comprises the centered node's neighborhood within a specified number of hops and its associated adjacency matrix.Source: §2 · p. 3

Energy-based model

  • The background presents energy-based modeling as a way to associate inputs and class labels with scalar energy values that induce a predictive distribution.It relates classifier logits to energy values.Source: §2 · p. 3
  • Energy scores for in-distribution data tend to be lower than those for OOD data, but energy-based OOD detection has mostly studied i.i.d. inputs.The use of energy models for interdependent data remains under-explored.Source: §2 · p. 3

3 PROPOSED METHOD

The method develops energy-based OOD detection for interdependent node-classification data, then extends it with energy propagation and optional OOD-exposure regularization.

3.1 ENERGY-BASED OOD DETECTION WITH DATA DEPENDENCE

  • Supervised training overall lowers energy for in-distribution instances, supporting energy as an inference-time OOD indicator.The energy is derived from the GNN’s predicted logits and incorporates the node and potentially dependent instances represented in its ego-graph.Source: §3.1 · p. 4
  • The energy-based discriminator is architecture-agnostic across GNNs, while extension to graph Transformers is left for future work.The paper also states that the model applies to Transformer-like node-prediction encoders.Source: §3.1 · p. 4

3.2 CONSENSUS BOOSTING WITH ENERGY-BASED BELIEF PROPAGATION

  • Energy-based belief propagation iteratively spreads initial node energies over graph connections, using a parameter to balance self and linked-node energy.The authors state that propagation pushes energy toward neighboring nodes’ majority and can amplify the in-distribution/OOD energy gap without extra training.Source: §3.2 · pp. 4–5

3.3 ENERGY-REGULARIZED LEARNING WITH BOUNDING GUARANTEES

  • With auxiliary OOD training data, energy-bounded regularization targets lower in-distribution and higher OOD energies while preserving supervised fitting.The stated guarantee assumes margins t_in < t_out and energy bounds for in-distribution and OOD samples.Source: §3.3 · p. 5
  • Energy-regularized learning requires additional OOD data during training, though the authors report it is unnecessary in quite a few practical cases.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments evaluate GNNSAFE variants on synthetic and real graph shifts, comparing OOD detection and in-distribution accuracy. GNNSAFE++ leads the reported comparisons, while backbone and hyperparameter analyses show that performance depends on model and propagation settings.

4.1 SETUP

  • Experiments use five node-classification datasets, with OOD examples drawn from disconnected graphs or subgraphs, unseen nodes within the same graph, or synthetic structure, feature, and label shifts.Twitch uses DE as in-distribution data and other subgraphs as OOD; Arxiv uses papers before 2015 for in-distribution data and papers after 2017 as OOD.Source: §4.1 · p. 6
  • The evaluation measures OOD detection with AUROC, AUPR, and FPR95, and separately tracks in-distribution test accuracy; baselines include vision-derived and graph-specific methods.The setup uses a two-layer, 64-hidden-unit GCN backbone for fair comparison.Source: §4.1 · p. 6

4.2 COMPARATIVE RESULTS

  • 12.8% and 17.0% average AUROC gains, respectively, and 44.8% and 21.0% average FPR95 reductions, respectively, are reported for GNNSAFE++ on Twitch and Cora-Structure, with comparable in-distribution accuracy.GNNSAFE without energy regularization increases average AUROC by 12.9% on Cora-Structure.Source: §4.2 · pp. 6–7, Table 1 · p. 7, Table 2 · p. 8
  • Energy propagation improves inference-time detection over Energy, separating in-distribution and OOD energy scores and leveraging graph topology.The comparison treats Energy as GNNSAFE with propagation removed.Source: §4.2 · pp. 7–8, Figure 1 · p. 7
  • GNNSAFE++ improves relative performance over GNNSAFE across the reported experiments, but AUROCs are comparable on Amazon and Coauthor, where energy regularization may be unnecessary under stronger shifts.The authors suggest regularization can matter more for Twitch and Arxiv, where context-related shifts may be harder to discriminate.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 ANALYSIS OF HYPER-PARAMETERS

  • Detection performance varies by encoder backbone, although GNNSAFE variants outperform competitors in nearly all reported cases and remain competitive with GNN-based counterparts using an MLP backbone.GNN backbones yield significantly better AUROC than MLP, while propagation can use graph topology with MLP.Source: §4.3 · p. 8, Figure 2 · p. 8
  • GNNSAFE++ AUROC first rises and then degrades as propagation depth increases; moderate self-loop strengths such as 0.5 or 0.7 perform better than extremes.The authors attribute excessive depth to over-smoothing energy scores and extreme self-loop strengths to underweighting either neighbors or other nodes.Source: §4.3 · pp. 8–9
  • GNNSAFE++ performance depends strongly on regularization weight λ, whose best value varies by dataset, while overly large margins degrade performance.The tested λ search space was {0.01, 0.1, 1.0}.Source: §4.3 · p. 9

5 RELATED WORKS

Prior OOD detection work largely assumes independent samples, while graph OOD detection for inter-dependent nodes remains under-explored. The paper distinguishes detection from OOD generalization and positions its work within a wider set of graph distribution shifts.

Out-of-distribution detection for neural networks

  • OOD detection methods using confidence scores, auxiliary OOD training data, or generative models mostly assume i.i.d. instances and do not address inter-dependent data.The cited approaches include confidence scores (Hendrycks & Gimpel, 2016), auxiliary OOD training (Bevandić et al., 2018; Liu et al., 2020; Mohseni et al., 2020), and generative models (Ren et al., 2019).Source: §5 · p. 9

Out-of-distribution detection on graphs

  • Graph OOD detection for inter-dependent nodes remains under-explored, and prior graph approaches either address graph-level instances or use Bayesian GNNs for node-level detection.The paper contrasts its logit-derived, architecture-agnostic detector with these approaches and generative density estimation.Source: §5 · p. 9

Out-of-distribution generalization

  • OOD generalization concerns predictive performance on shifted test data, whereas this work studies detection; its experiments focus on cross-domain and temporal shifts among several graph-shift types.Other cited shifts include subgroup attributes, graph size, temporal context, and feature space.Source: §5 · p. 9

6 CONCLUSIONS

The conclusion presents supervised-loss-trained GNNs as intrinsic OOD discriminators and reports theoretical justification and experiments on real-world datasets with varied OOD types.

6 CONCLUSIONS

  • Supervised-loss-trained GNNs can intrinsically discriminate OOD test data that the model should avoid predicting on.The authors describe the approach as simple and general, and report theoretical analysis and extensive real-world experiments across OOD data types.Source: §6 · p. 9

A.1 PROOF FOR PROPOSITION 1

The appendix proof excerpts explain how supervised classification training shapes energy scores and relate regularization-optimal energy to supervised-optimal predictions.

A.1 PROOF FOR PROPOSITION 1

  • The proof says minimizing the supervised loss lowers the true-label energy and raises the energy for other classes, tending to lower energy on in-distribution instances.The cited proof follows reasoning similar to Liu et al. (2020).Source: §A.1 · p. 13

A.2 PROOF FOR PROPOSITION 2

  • For Proposition 2, the excerpt says the opposite case follows by a similar proof, without stating further details.The supplied proof text does not specify the case or its result.Source: §A.2 · p. 13

A.3 PROOF FOR PROPOSITION 3

  • The proof relates energy from a regularization-loss-minimizing model to energy induced by the model with the supervised-loss-optimal predictive distribution.Under the assumption that the optimal energy distinguishes in-distribution from OOD scores, it identifies regularization-loss choices whose optimum meets the stated requirements.Source: §A.3 · pp. 13–14

B EXPERIMENTAL DETAILS

The experiments use Ubuntu, CUDA, PyTorch, and PyTorch Geometric, primarily on NVIDIA 2080Ti GPUs.

B EXPERIMENTAL DETAILS

  • Most experiments ran on NVIDIA 2080Ti GPUs with 11GB memory; higher-memory cases used NVIDIA 3090 GPUs with 24GB.The implementation used Ubuntu 16.04.6, CUDA 10.2, PyTorch 1.9.0, and PyTorch Geometric 2.0.3.Source: B EXPERIMENTAL DETAILS · p. 14

B.1 DATASET INFORMATION

The dataset appendix describes public graph benchmarks and their use for in-distribution training and OOD evaluation, including synthetic, cross-domain, and temporal settings.

B.1 DATASET INFORMATION

  • Cora is a citation network with 2,708 nodes, 5,429 edges, 1,433 features, and 7 classes; the study uses three synthetic OOD constructions and standard semi-supervised splits.The nodes are papers, edges denote citations, and labels represent paper topics.Source: §B.1 · p. 14
  • Amazon-Photo is an item co-purchasing graph with 7,650 nodes, 238,162 edges, 745 features, and 8 classes, evaluated with three synthetic OOD constructions.Products are nodes, frequent co-purchases are edges, and in-distribution data use 1:1:8 random splits.Source: §B.1 · p. 14
  • Coauthor-CS is a computer-science coauthor graph with 18,333 nodes, 163,788 edges, 6,805 features, and 15 classes, using three synthetic OOD constructions.Authors are nodes, coauthorships are edges, and in-distribution data use 1:1:8 random splits.Source: §B.1 · p. 14
  • Twitch-Explicit contains regional social-network subgraphs; DE is in-distribution, ENGB supplies OOD exposure, and ES, FR, and RU are OOD test data.Nodes represent players, edges indicate follows, features are game embeddings, and labels indicate mature-content streaming.Source: §B.1 · pp. 14–15
  • ogbn-Arxiv uses publication years from 1960–2020 to partition in-distribution and OOD data, then random-splits the in-distribution portion 1:1:8.The authors note this prevents using the original time-based train/test split; random splitting keeps in-distribution train, validation, and test nodes identically distributed and distinct from OOD data.Source: §B.1 · p. 15

B.2 IMPLEMENTATION DETAILS

The appendix documents encoder architectures, hyperparameter searches, training procedures, and competitor implementations used in the experiments.

Architectures

  • The experiments use GCN as the basic encoder and also evaluate MLP, GAT, JKNet, and MixHop backbones.GCN, MLP, GAT, and JKNet use two layers with hidden size 64; MixHop uses two layers and hop number 2.Source: §B.2 · p. 15

Hyper-parameters

  • The default propagation settings are K = 2 and α = 0.5, with validation searches for GNNSAFE learning rate and GNNSAFE++ margins and regularization weight.GNNSAFE learning rates are {0.1, 0.01, 0.001}; GNNSAFE++ searches tin ∈ {−9, −7, −5}, tout ∈ {−1, −2, −3, −4}, and λ ∈ {0.01, 0.1, 1.0}.Source: §B.2 · p. 15

Training Details

  • GNNSAFE uses supervised cross-entropy, while GNNSAFE++ adds regularization on OOD training data; results come from the epoch with the lowest validation classification loss.Training uses a fixed 200-epoch budget, with validation loss selecting the reported epoch.Source: §B.2 · p. 15

Competitors

  • Baseline implementations use the same two-layer, hidden-size-64 GCN for MSP, ODIN, Mahalanobis, Energy, and Energy FT; GKDE and GPN use public implementations with dataset-specific fine-tuning.GKDE and GPN settings follow their papers before fine-tuning.Source: §B.2 · p. 15

B.3 EVALUATION METRICS

OOD detection is evaluated with threshold-independent AUROC, AUPR, and FPR95 metrics, each capturing a different aspect of detection performance.

Evaluation Metrics

  • AUROC measures the probability that an in-distribution example receives a higher estimated score than an OOD example; AUPR reflects the precision–recall trade-off and adjusts for class base rates.Both metrics are evaluated across thresholds; AUPR is noted as useful when positive and negative classes are imbalanced.Source: §B.3 · pp. 15–16
  • FPR95 is the rate at which OOD examples are misclassified as in-distribution when the true positive rate is 95%.The evaluation uses AUROC, AUPR, and FPR95, which are independent of threshold τ.Source: §B.3 · pp. 15–16

C ADDITIONAL EXPERIMENT RESULTS

The appendix provides additional dataset-level detection results, runtime comparisons, and hyperparameter analyses that supplement the main-text tables.

Additional OOD results

Runtime comparison

Hyper-parameter analysis

  • Figure 3 analyzes propagation steps K, self-loop strength α, margins tin and tout, and GNNSAFE++ energy-regularization weight λ on Cora with structure manipulation.Source: Figure 3 · p. 18

Related papers

Loading 2302.02914v2…