Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection

Yu Zheng; Ming Jin; Yixin Liu; Lianhua Chi; Khoa T. Phan; Yi-Ping Phoebe Chen

Source-linked AI summary

Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection

Yu Zheng, Ming Jin, Yixin Liu, Lianhua Chi, Khoa T. Phan, Yi-Ping Phoebe Chen

arXiv:2108.09896v2Aug 23, 2021cs.LG

TL;DR

Graph anomaly detection requires methods that capture complex structure, attributes, and contextual information when anomaly labels are unavailable. SL-GAD constructs contextual subgraph views and combines generative attribute reconstruction with multi-view contrastive learning. On six benchmark datasets, it outperforms state-of-the-art methods by a large margin.

Problem
Existing methods either inadequately capture complex graph interdependency or fail to fully exploit contextual information as supervision for anomaly detection.
Method
SL-GAD constructs local contextual subgraph views around target nodes and combines generative attribute reconstruction with multi-view contrastive learning.
Results
SL-GAD outperforms state-of-the-art algorithms by a large margin in experiments on six datasets.
Takeaways & Limitations
Jointly using generative and contrastive self-supervision lets SL-GAD capture anomalies across attribute, structure, and mixed structure-attribute spaces.
Takeaways & Limitations
Performance gains are smaller on high-degree social networks, where fixed-size subgraph sampling may lose information; efficient sampling remains future work.

Abstract

from arXiv · show

Anomaly detection from graph data has drawn much attention due to its practical significance in many critical applications including cybersecurity, finance, and social networks. Existing data mining and machine learning methods are either shallow methods that could not effectively capture the complex interdependency of graph data or graph autoencoder methods that could not fully exploit the contextual information as supervision signals for effective anomaly detection. To overcome these challenges, in this paper, we propose a novel method, Self-Supervised Learning for Graph Anomaly Detection (SL-GAD). Our method constructs different contextual subgraphs (views) based on a target node and employs two modules, generative attribute regression and multi-view contrastive learning for anomaly detection. While the generative attribute regression module allows us to capture the anomalies in the attribute space, the multi-view contrastive learning module can exploit richer structure information from multiple subgraphs, thus abling to capture the anomalies in the structure space, mixing of structure, and attribute information. We conduct extensive experiments on six benchmark datasets and the results demonstrate that our method outperforms state-of-the-art methods by a large margin.

1 INTRODUCTION

Graph anomaly detection must handle complex structure, attributes, and their interaction without relying on known anomaly labels. SL-GAD addresses these challenges by combining contextual subgraph views with generative and contrastive self-supervision, outperforming state-of-the-art methods on six datasets.

Graph anomalies may occur in structure, attributes, or their combination, while unknown ground truth limits supervised classification.
Existing shallow methods quantify anomalies using graph structure and attributes, whereas graph autoencoders reconstruct node attributes and structure.
Existing graph autoencoder methods underuse neighboring nodes, subgraphs, and available contextual information as supervision signals.
SL-GAD samples multiple local subgraph views around each target node and applies generative attribute reconstruction plus multi-view contrastive learning.
SL-GAD combines generative and contrastive anomaly scoring to capture attribute-space, structure-space, and mixed anomalies.
Six-dataset experiments show that SL-GAD outperforms state-of-the-art algorithms by a large margin.

2 RELATED WORK

Related work spans graph anomaly detection, self-supervised learning, and graph representation learning. Prior approaches include shallow anomaly measures, deep graph autoencoders, and graph-based self-supervised representation methods.

2.1 Anomaly detection: Graph anomaly detection has progressed from traditional statistical methods to shallow graph methods and deep learning approaches.
2.2 Self-supervised learning: Self-supervised learning trains neural models from unlabeled data through pretext tasks, and graph variants use contrastive objectives across graph views.
2.3 Graph representation learning: Graph representation learning learns node or graph representations for downstream tasks such as node classification and anomaly detection.
2.3 Graph representation learning: The paper’s related-work discussion includes graph stochastic neural networks and graph geometry interaction learning among graph representation methods.

3 PROBLEM DEFINITION

The paper formulates unsupervised anomaly detection on attributed graphs, where node features and adjacency define the input graph. The goal is to produce anomaly scores and rank nodes without labeling information.

An attributed graph is represented by node features X and an adjacency matrix A over N nodes.
A graph neural network maps node features to low-dimensional node embeddings through local aggregation.
The task focuses on unsupervised anomaly detection on attributed graphs.
The model F maps an attributed graph to node anomaly scores without labeling information, then ranks nodes in descending score order.

4 METHODOLOGY

SL-GAD detects node-level graph anomalies by sampling contextual graph views and combining generative attribute reconstruction with mixed-level contrastive learning. Its two objectives produce complementary attribute- and structure-sensitive anomaly scores.

Graph View Establishment for Anomaly Detection: SL-GAD samples two augmented graph views around each target node to provide contextual information for unsupervised anomaly detection.The target node and associated views are encoded through a GNN, while masking prevents the target’s raw attributes from leaking into reconstruction and view embeddings.
Generative Learning with Attribute Reconstruction: The generative objective reconstructs a target node’s features from neighboring contextual information, using reconstruction mismatch to identify attribute-space anomalies.A GNN-based encoder and decoder form the graph autoencoder, and the model minimizes mean squared error between original and reconstructed features across views.
Self-Supervised Objectives: The generative discrimination pair is the original versus reconstructed target node, whereas contrastive learning uses the target node and two sampled graph views.These pairings connect node-level attribute comparison with node-to-subgraph structural comparison.
Model Optimization and Graph Anomaly Scoring: SL-GAD combines the generative and contrastive objectives during optimization and derives anomaly scores from both attributive and structural evidence.The final framework includes graph view sampling, self-supervised learning, and graph anomaly scoring, with shared encoder and decoder components across views.
Multi-View Contrastive Learning: The contrastive objective compares a target node with its local subgraph representations across multiple topological scales in embedding and structural space.This mixed-level discrimination complements node-level attribute reconstruction by directly using surrounding subgraph structure.

5 EXPERIMENTAL STUDY

The experimental study evaluates SL-GAD on six real-world benchmark datasets against state-of-the-art anomaly detection and self-supervised learning methods. It also includes ablation and parameter-sensitivity studies.

SL-GAD is evaluated on six real-world benchmark datasets.
The experiments compare SL-GAD with state-of-the-art anomaly detection and self-supervised learning methods using their configurations for fair comparison.
Ablation and parameter-sensitivity experiments investigate the properties of SL-GAD.
The benchmark statistics distinguish two upper social-network datasets from the remaining citation-network datasets.

5.1 Dataset Description

The study uses six real-world graph benchmarks: two social networks and four citation networks. Because the datasets lack ground-truth anomalies, the experiments inject synthetic attribute and structural anomalies.

The benchmark collection contains six real-world graph datasets: two social networks and four citation networks.
Social Networks: BlogCatalog and Flickr represent users as nodes, following relationships as links, and personalized website content as node features.
Citation Networks: Cora, CiteSeer, Pubmed, and ACM represent papers as nodes, citations as edges, and paper text as node features.
Anomaly Injection: Synthetic attributive anomalies replace selected node features with features from randomly selected distant nodes.
Anomaly Injection: Structural anomalies are injected by selecting nodes and making them fully connected, with equal numbers of attributive and structural anomalies.
Anomaly Injection: The total anomaly counts are summarized with dataset statistics in Table 2.

5.2 Experimental Setup

The experimental setup defines baseline methods, ROC-AUC evaluation, and implementation settings for SL-GAD. Baselines span shallow anomaly measures, graph autoencoding, and self-supervised contrasting approaches.

The setup covers baseline methods, evaluation metrics, and parameter settings.
Baselines: AMEN, Radar, and ANOMALOUS use ego-network analysis, residual analysis, or CUR decomposition to detect graph anomalies.
Baselines: DOMINANT reconstructs adjacency and feature matrices with a graph autoencoder, while DGI and CoLA use contrasting-based representations.
Metrics: ROC-AUC measures performance by the area under the ROC curve, where larger AUC indicates higher detection performance.
Parameter Settings: The experiments train discrimination modules with Adam while varying β over {0.2, 0.4, 0.6, 0.8, 1} and fixing α to 1.

5.3 Comparison with the State-of-the-art Methods

Across the benchmark comparison, SL-GAD outperforms all baseline methods, with authors attributing the result to jointly using generative and contrastive self-supervision. Gains are larger on citation networks, while fixed-size view sampling may lose information in higher-degree social networks.

SL-GAD outperforms all baseline methods on every benchmark dataset.The comparison uses ROC curves in Figure 3 and AUC values in Table 3.
The authors attribute SL-GAD’s performance to jointly using generative and contrastive self-supervised learning.
Shallow methods AMEN, Radar, and ANOMALOUS show noncompetitive detection performance in experiments involving high-dimensional features and complex structures.
Compared with other deep methods, SL-GAD shows stronger detection performance and generalization ability while combining strategies used separately by DOMINANT and CoLA.
Performance gains are larger on citation networks than on social networks.The authors associate this difference with possible information loss when fixed-size subgraph views sample higher-degree social networks.

5.4 Effectiveness of Components

The full SL-GAD model combines generative and contrastive self-supervised components, with ablations showing that their joint use and calibrated anomaly-score combination are effective.

Benchmark comparison: Table 3 reports AUC values for anomaly detection on six benchmark datasets, with the best performance on each dataset shown in bold.Figure 3 presents corresponding ROC curves, where larger area under the curve indicates better performance and “Ours” denotes SL-GAD.
Component ablations: The full SL-GAD achieves the best performance, supporting the joint use of generative and contrastive self-supervised learning.The ablation study compares variants excluding the generative or contrastive module.
Component ablations: SL-GAD-Con consistently outperforms SL-GAD-Gen across all six datasets, indicating a stronger role for contrastive self-supervision in anomaly detection.The authors relate this to agreement between a target node and its neighbouring substructure.
Score combination: Both anomaly-score scaling and weighting improve detection performance across the six datasets.Without scaling, combining the differently formed generative and contrastive scores can distort the final measurement.

5.5 Parameters Sensitivity

SL-GAD is evaluated across several hyper-parameters, including score-balance factors, evaluation rounds, subgraph size, embedding dimension, and negative ratio. Performance generally benefits from sufficient evaluation rounds and moderate model settings, while negative-sample effects vary by dataset.

Balance factors α and β: Increasing α generally raises AUC, except when β is extremely small, indicating that the contrastive term is dominant in performance.The study varies α and β on Cora, BlogCatalog, and Flickr.
Evaluation rounds: AUC rises with evaluation rounds up to R ≤80, while larger values R ≥160 provide no significant further increase; R=160 is selected.Sufficient rounds reduce bias from random sampling while preserving acceptable running speed.
Subgraph size: Performance increases sharply with subgraph size when K is small and peaks at K=2 or K=4 depending on the dataset.The study evaluates K values from 1 to 14 across six datasets.
Embedding dimension: Embedding performance usually improves for D′ ≤32, then peaks and slightly declines at larger dimensions; D′=64 is selected generally.The authors attribute the decline to over-fitting from the explosive parameter count and recommend D′ from 32 to 128.
Negative ratio: Increasing the negative ratio has no uniform effect: it hurts detection on Cora and CiteSeer but improves performance on ACM.The negative ratio is varied from 1 to 128, with one positive and one negative sample used by default.

6 CONCLUSION

The paper concludes that SL-GAD addresses unsupervised graph anomaly detection by combining contextual subgraph views with generative attribute reconstruction and multi-view contrastive learning. Experiments on six datasets demonstrate strong performance, while the method remains manually designed.

Conclusion: SL-GAD generates two subgraph views around each target node and combines generative attribute reconstruction with multi-view contrastive learning.The two components target attribute-space and hidden structural anomalies, respectively.
Conclusion: Experimental results on six datasets demonstrate the proposed algorithm’s strong performance.
Limitations and future work: The method is manually designed for anomaly detection, and future work will consider neural architecture search for automatic model design.