Source-linked AI summary
EAGLE: Contrastive Learning for Efficient Graph Anomaly Detection
Jing Ren, Mingliang Hou, Zhixuan Liu, Xiaomei Bai
TL;DR
Existing heterogeneous graph anomaly detection approaches are described as focused on specific contexts and lacking generalization. EAGLE combines meta-path-level contrastive learning with graph autoencoders, and experiments report an average improvement over other deep learning methods across three heterogeneous networks.
Problem
Existing heterogeneous graph anomaly detection approaches are described as focused on specific contexts and lacking generalization.
Source: Graph Anomaly Detection · p. 3Method
EAGLE contrasts target nodes with meta-path instances and uses a graph autoencoder to learn node representations for anomaly detection.
Source: The Proposed Model: EAGLE · p. 5, Attributed Graph Encoder · p. 5, Structure Reconstruction Decoder · p. 6Results
28.4% average improvement over other deep learning methods across three heterogeneous networks.
Source: Comparison with the State-of-the-art… · p. 8Takeaways & Limitations
Experiments on three benchmark datasets demonstrate the effectiveness and efficiency of EAGLE for heterogeneous-graph anomaly detection.
Source: Conclusion · p. 9The evaluated setting is unsupervised and uses pre-training on one part of a large heterogeneous graph followed by fine-tuning on the remainder.
Source: Instance Pair Sampling · p. 4
Abstract
from arXiv · showhide
Graph anomaly detection is a popular and vital task in various real-world scenarios, which has been studied for several decades. Recently, many studies extending deep learning-based methods have shown preferable performance on graph anomaly detection. However, existing methods are lack of efficiency that is definitely necessary for embedded devices. Towards this end, we propose an Efficient Anomaly detection model on heterogeneous Graphs via contrastive LEarning (EAGLE) by contrasting abnormal nodes with normal ones in terms of their distances to the local context. The proposed method first samples instance pairs on meta path-level for contrastive learning. Then, a graph autoencoder-based model is applied to learn informative node embeddings in an unsupervised way, which will be further combined with the discriminator to predict the anomaly scores of nodes. Experimental results show that EAGLE outperforms the state-of-the-art methods on three heterogeneous network datasets.
Related Work
Prior graph anomaly detection work includes reconstruction-based autoencoders, while heterogeneous-graph approaches are described as application-specific and limited in generalization. EAGLE connects contrastive learning with graph autoencoders and uses meta-path instances to represent local structure.
Graph Anomaly Detection
- Reconstruction-based autoencoders detect anomalies by reconstructing graph data or attributed-network representations.DOMINANT reconstructs original data; AnomalyDAE uses a dual autoencoder for attributed networks.Source: Graph Anomaly Detection · p. 3
- Heterogeneous graph anomaly detection approaches are described as application-specific and lacking generalization.EAGLE integrates contrastive learning into a deep graph autoencoder as a proposed approach to this limitation.Source: Graph Anomaly Detection · p. 3
Contrastive Learning
- Contrastive learning seeks common features in positive instances while distinguishing positive from negative instances.A central design choice is how to generate those positive and negative samples.Source: Contrastive Learning · p. 3
Preliminaries
The preliminaries describe heterogeneous graphs and meta-paths as a way to represent typed nodes, relations, and higher-order proximity. The sampling setup uses target-node/meta-path instances and specifies an unsupervised pre-training and fine-tuning setting.
Preliminaries
- A heterogeneous graph has node and edge sets, type sets, and mappings from nodes and edges to their types.The definition requires more than two node and edge types in total.Source: Preliminaries · p. 3
- A meta-path represents a sequence of node types connected by relations and captures proximity from a specific semantic perspective.APA represents co-authorship, while PVP represents papers published at the same venue.Source: Preliminaries · pp. 3–4
Instance Pair Sampling
- Instance sampling pairs a target node with meta-path instances to represent local structure in a heterogeneous graph.Positive meta-path samples contain the target node; the stated setting uses unlabeled data, pre-training on one graph part, and fine-tuning on the remainder.Source: Instance Pair Sampling · p. 4
Preliminaries
- The anomaly-detection task ranks node anomaly scores in descending order and treats nodes above a threshold as anomalous.The task is formulated as learning an anomaly-score function for nodes in a heterogeneous graph.Source: Instance Pair Sampling · p. 4
The Proposed Model: EAGLE
EAGLE selects target nodes, samples meta-path instances, learns graph representations through contrastive pre-training, and combines discrimination and reconstruction signals to calculate anomaly scores. Negative instances exclude the target node but are constrained to include a direct neighbor.
The Proposed Model: EAGLE
- The model uses pre-defined meta-paths as the basis for instance sampling.P denotes the set of pre-defined meta-paths.Source: The Proposed Model: EAGLE · p. 4
- EAGLE combines the discriminator's discrimination score with graph-decoder reconstruction error to calculate node anomaly scores.Its framework proceeds from target-node selection and instance sampling through contrastive pre-training to anomaly-score calculation.Source: Figure 2 · p. 5
- A negative meta-path instance excludes the target node and includes at least one of its direct neighbors.This constraint limits negative instances so they are not simply paths far from the target node.Source: The Proposed Model: EAGLE · p. 5
Attributed Graph Encoder
EAGLE’s heterogeneous-graph autoencoder uses two GCN encoders to incorporate paper-author and paper-venue relationships into node representations. Its contrastive model combines graph autoencoding with a discriminator for instance-pair comparison.
Attributed Graph Encoder
- The proposed GAE combines two GCN encoders to incorporate paper-author and paper-venue relationships into heterogeneous-graph node representations.The paper frames learning informative node representations on heterogeneous graphs as a challenge for deep GAE.Source: Attributed Graph Encoder · p. 5
Graph AutoEncoder-based Contrastive Learning Model
- The contrastive model pairs a Graph AutoEncoder for network representation learning with a discriminator for instance-pair contrast.The discriminator uses a bilinear transformation; its score compares selected instance and target-node embeddings.Source: Graph AutoEncoder-based Contrastive… · pp. 5–6
- GCN layers iteratively transform node embeddings using the adjacency matrix and trainable weights, enabling the encoder to learn structure and attribute information together.The model input is the attribute matrix; the layer operation uses a normalized form of the adjacency matrix with self-loops.Source: Graph AutoEncoder-based Contrastive… · pp. 5–6
Objective Function
The contrastive objective trains predicted discrimination scores to match ground-truth labels using binary cross-entropy.
Objective Function
- The contrastive objective uses standard binary cross-entropy to bring each predicted discrimination score closer to its ground-truth label.The passage notes that this loss has been validated by other contrastive learning models.Source: Objective Function · p. 6
Attribute Reconstruction Decoder
The attribute decoder reconstructs node attributes from learned representations, and its reconstruction errors contribute to anomaly scores.
Attribute Reconstruction Decoder
- The attribute decoder reconstructs input node attributes according to the learned representation H.Its structure is a graph convolutional layer.Source: Attribute Reconstruction Decoder · p. 6
- The deep graph autoencoder objective is introduced after defining the contrastive loss across all graph nodes.The supplied passage does not include the objective’s formula.Source: Attribute Reconstruction Decoder · p. 6
- The reconstructed attribute matrix bX is used to calculate anomaly scores through its reconstruction error bX − X.Source: Attribute Reconstruction Decoder · p. 6
Structure Reconstruction Decoder
The structure decoder reconstructs graph adjacency from latent embeddings by predicting meta-path-neighbor relationships. Structure and attribute reconstruction errors are balanced in the objective, and reconstruction error contributes to anomaly scores.
Structure Reconstruction Decoder
- The structure decoder uses latent embeddings to reconstruct adjacency by predicting whether node pairs are meta-path neighbors.Source: Structure Reconstruction Decoder · p. 6
- Parameters α and β balance structure-reconstruction and attribute-reconstruction errors in the joint objective.The objective also balances the contrastive-learning model using γ.Source: Structure Reconstruction Decoder · p. 6
Anomaly Score Computation
- The reconstructed adjacency matrix bA contributes its reconstruction error bA − A to node anomaly scores.The paper states that normal input nodes are reconstructed with less error than abnormal nodes.Source: Structure Reconstruction Decoder · p. 6, Anomaly Score Computation · p. 6
Discriminator
The discriminator contrasts node and instance-pair embeddings to produce anomaly scores, while evaluation uses three heterogeneous graphs with injected contextual anomalies. EAGLE is presented as efficient, self-supervised, and scalable through pre-training.
Discriminator
- For a normal node, the discriminator scores positive pairs near 1 and negative pairs near 0; for an anomalous node, both scores are near 0.5, yielding higher anomaly scores for abnormal nodes.The model contrasts the node and instance embeddings; abnormal-node scores are close to 0, while normal-node scores are close to -1.Source: Discriminator · pp. 6–7, Dataset · p. 7
Dataset
- The evaluation uses DBLP, Aminer, and Yelp, with node features initialized from paper abstracts or reviews, and injects contextual anomalies by replacing a target node’s attributes with those of a distant node.The anomaly injection selects the farthest node among k = 50 randomly selected nodes, based on Euclidean distance.Source: Dataset · p. 7, Table 1 · p. 7
Model Analysis
- EAGLE combines node attributes and structure in informative embeddings, with time complexity O(edFP), and uses self-supervision without requiring large volumes or variety of downstream labels.The model analysis also describes generalization to large-scale networks through pre-training.Source: Model Analysis · p. 7
Baselines
The cited baselines include methods for attributed-network anomaly detection and heterogeneous information network embedding.
- AnomalyDAE uses a dual autoencoder for joint representation learning on attributed networks, while HeGAN uses adversarial learning for heterogeneous information network embedding.Source: Baselines · p. 7
Experiments
Experiments compare anomaly-detection performance and running time on three heterogeneous graph datasets, then examine pre-training, readout functions, and embedding dimensions. EAGLE leads the reported benchmark comparison, while parameter studies characterize performance and resource trade-offs.
Experiments
- The experiments evaluate effectiveness and efficiency on three real-world heterogeneous networks, using DGI, DOMINANT, and FRAUDRE among the described baselines.Source: Experiments · pp. 7–8
Pre-training and Fine-tuning Setting
- Pre-training uses 30% of DBLP and Aminer and 70% of Yelp for initialization before task-specific fine-tuning.The authors state that fine-tuning a learned framework costs less running time than using randomly initialized parameters.Source: Pre-training and Fine-tuning Setting · p. 8
Metrics
- AUC, ranging from 0 to 1, measures detection performance; larger values indicate better performance.It is defined as the area under the ROC curve, which plots true-positive rate against false-positive rate.Source: Metrics · p. 8
Comparison with the State-of-the-art Methods
- 28.4% average improvement over other deep learning methods; EAGLE outperforms the baselines on all three heterogeneous networks.The comparison reports about 15% improvement over DOMINANT across the three benchmark datasets; EAGLE without pre-training has lower AUC and higher running time.Source: Comparison with the State-of-the-art… · p. 8, Table 2 · p. 9
Effect of Readout Function
- Average pooling performs best on DBLP and Yelp and competitively on Aminer, while min pooling performs worst on all three datasets.The authors attribute min pooling’s results to possible loss of important information when selecting minimum embedding values.Source: Effect of Readout Function · p. 8, Table 3 · p. 9, Parameter Settings · p. 9
Parameter Settings
- EAGLE uses 64-dimensional embeddings for efficiency, although 64 dimensions perform less well than 128 and 256 dimensions.The reported settings also specify learning rates of 0.001, 0.006, and 0.001 for the three datasets, respectively, and α = 0.8, β = 0.2, γ = 0.3.Source: Parameter Settings · pp. 8–9
Effect of Embedding Dimension
- Detection performance does not increase monotonically as embedding dimensions rise from 8 to 256, so the study compares AUC across dimensions.The paper frames dimension choice as a balance between performance and computation resources and running time.Source: Effect of Embedding Dimension · p. 9, Figure 3 · p. 9
Conclusion
EAGLE combines contrastive learning and an autoencoder to learn informative node embeddings and detect anomalies self-supervisedly on heterogeneous graphs. Experiments on three benchmark datasets support the reported effectiveness and efficiency; future work will investigate streaming-graph anomaly detection.
- Meta-path-level instance pairs are used to capture semantic and structural properties, and experiments on three benchmark datasets support EAGLE’s effectiveness and efficiency.Source: Conclusion · p. 9
- Future work will investigate dynamic graph learning techniques for detecting anomalies in streaming graphs.Source: Conclusion · p. 10