Source-linked AI summary
Simple and Efficient Heterogeneous Graph Neural Network
Xiaocheng Yang, Mingyu Yan, Shirui Pan, Xiaochun Ye, Dongrui Fan
TL;DR
Existing HGNNs use attention and multi-layer mechanisms whose effectiveness on heterogeneous graphs is insufficiently studied, while these mechanisms add substantial aggregation complexity. The paper studies them and proposes SeHGNN, which pre-computes mean aggregation, uses long metapaths in a single layer, and fuses semantics with a transformer. Across five heterogeneous-graph datasets, SeHGNN outperforms state-of-the-art methods in accuracy and training speed.
Problem
Existing HGNNs inherit attention and multi-layer mechanisms from homogeneous GNNs, but few studies examine their effectiveness on heterogeneous graphs despite their computational complexity.
Method
SeHGNN pre-computes light-weight mean neighbor aggregation, uses a single layer with long metapaths, and applies transformer-based semantic fusion.
Results
SeHGNN outperforms state-of-the-art methods on heterogeneous-graph node classification across five datasets, with higher accuracy and faster training speed.
Takeaways & Limitations
The study finds that semantic attention is essential whereas neighbor attention is unnecessary, supporting a simpler and more efficient HGNN design.
Takeaways & Limitations
Label propagation is included as an extra input, with self-label information removed to avoid label leakage; complexity comparisons may cover only target-node linear projection for HAN and HGB.
Abstract
from arXiv · showhide
Heterogeneous graph neural networks (HGNNs) have powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) over homogeneous graphs, especially the attention mechanism and the multi-layer structure. These mechanisms bring excessive complexity, but seldom work studies whether they are really effective on heterogeneous graphs. This paper conducts an in-depth and detailed study of these mechanisms and proposes Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). To easily capture structural information, SeHGNN pre-computes the neighbor aggregation using a light-weight mean aggregator, which reduces complexity by removing overused neighbor attention and avoiding repeated neighbor aggregation in every training epoch. To better utilize semantic information, SeHGNN adopts the single-layer structure with long metapaths to extend the receptive field, as well as a transformer-based semantic fusion module to fuse features from different metapaths. As a result, SeHGNN exhibits the characteristics of simple network structure, high prediction accuracy, and fast training speed. Extensive experiments on five real-world heterogeneous graphs demonstrate the superiority of SeHGNN over the state-of-the-arts on both accuracy and training speed.
Introduction
Heterogeneous graphs combine structural and semantic information, but existing HGNNs inherit attention and multi-layer mechanisms whose effectiveness is insufficiently studied and whose aggregation can be costly. The paper studies these mechanisms, proposes SeHGNN, and reports superior node-classification performance over state-of-the-art methods.
- Motivation: Heterogeneous graphs contain multiple node and edge types whose compositions form semantically meaningful metapaths.Examples include APA for co-authorship and APSPA for shared research subjects.
- Research gap: Existing HGNNs inherit attention and multi-layer mechanisms from homogeneous-graph GNNs without thoroughly evaluating their effectiveness on heterogeneous graphs.These mechanisms also add hierarchy-attention calculations and repeated neighbor aggregation.
- Research gap: More than 85% of total time in HAN and HGB is spent on neighbor aggregation with attention modules, creating a speed bottleneck for larger heterogeneous graphs.The comparison covers both metapath-based HAN and metapath-free HGB.
- Approach: SeHGNN pre-computes mean-based neighbor aggregation, uses single-layer long metapaths, and applies transformer-based semantic fusion.Pre-computation avoids repeated aggregation during every training epoch, while long metapaths extend the receptive field.
- Results: Experiments on five widely used datasets show that SeHGNN achieves superior heterogeneous-graph node-classification performance over state-of-the-art methods.The reported contribution emphasizes both high prediction accuracy and fast training speed.
Preliminaries
The preliminaries formalize heterogeneous graphs through typed nodes and edges, adjacency matrices, and metapaths that compose multiple edge types into relations.
- Heterogeneous graphs: A heterogeneous graph is G = {V, E, T_v, T_e}, with node and edge type mappings ϕ and ψ.Each node has a node type, and each directed edge has a relation type from its source to target.
- Graph representation: The graph structure is represented by a family of relation-specific adjacency matrices {A_r} indexed by edge types.Each matrix records edges between a source node type and a target node type.
- Metapaths: A metapath is a composite relation represented by a sequence of node types connected through several edge types.Its instances specify node sequences following the metapath schema.
- Metapaths: A metapath neighbor graph connects endpoint nodes whenever a corresponding metapath instance exists in the original graph.The endpoint node types are determined by the first and last types in the metapath.
Related Work
Related work divides HGNNs into metapath-based and metapath-free approaches, while SGC-based methods provide another efficient family for heterogeneous graphs.
- Homogeneous GNNs: Homogeneous GNNs use neighborhood aggregation, while GAT adds attention and SGC removes intermediate nonlinearities for acceleration.GCN uses multi-layer propagation, GraphSAGE improves scalability through sampling, and SGC simplifies consecutive graph-convolution layers.
- Metapath-based HGNNs: Metapath-based HGNNs aggregate neighbors within each semantic relation before fusing the resulting semantic representations.HAN adds neighbor attention during aggregation and semantic attention during fusion, while MAGNN uses internal metapath-instance nodes.
- Metapath-free HGNNs: Metapath-free HGNNs aggregate neighbors of all node types simultaneously while using additional modules to encode node and edge semantics.RSHN uses a coarsened line graph to obtain edge-type embeddings for aggregation.
- SGC-based methods: SGC-based heterogeneous methods such as NARS, SAGN, and GAMLP aggregate node types together without explicitly distinguishing different semantics.These methods nevertheless report impressive results on heterogeneous graphs.
Motivation
The motivation study identifies which attention and depth mechanisms matter in HGNNs, then uses those findings to simplify SeHGNN while preserving semantic modeling and reducing training cost.
- Attention study: Experiments indicate that removing semantic attention decreases model effectiveness, whereas removing neighbor attention does not.The study evaluates HAN and HGB variants, averaging results over 20 runs with different data partitions.
- Attention study: Semantic attention is essential, while neighbor attention is not necessary for the studied HGNNs.The finding motivates replacing neighbor attention with simple mean aggregation.
- Network structure study: Under the same receptive-field size, single-layer models with long metapaths outperform multi-layer models with short metapaths.Long metapaths preserve distinctions between higher-level semantics that multi-layer intermediate mixtures obscure.
- Proposal of SeHGNN: SeHGNN uses mean aggregation without neighbor attention and a simplified network structure to reduce unnecessary complexity.The design retains semantic fusion while extending the receptive field through long metapaths.
- Proposal of SeHGNN: Parameter-free linear neighbor aggregation can be executed once during preprocessing instead of repeatedly in every training epoch.This design significantly reduces training time.
Methodology
SeHGNN separates structural aggregation, feature projection, and semantic fusion. It pre-computes mean aggregation over metapaths, projects semantic features, and uses transformer-based attention to combine them.
- Simplified Neighbor Aggregation: SeHGNN pre-computes mean aggregation over metapath-based neighbors once during preprocessing, producing semantic feature matrices for training.The aggregation is parameter-free and avoids repeating neighbor aggregation each epoch.
- Simplified Neighbor Aggregation: Metapath aggregation multiplies row-normalized adjacency matrices with raw feature matrices to propagate features along each metapath.Aggregation results for short metapaths can be reused to compute longer-metapath results.
- Simplified Neighbor Aggregation: The training pipeline optionally propagates one-hot labels across metapaths while removing self-contributions to avoid label leakage.These propagated label matrices are collected as additional semantic inputs.
- Multi-layer Feature Projection: Each metapath-specific semantic feature matrix is projected through its own multilayer perceptron into a shared representation space.The projection block includes normalization, nonlinear, and dropout layers between consecutive linear layers.
- Transformer-based Semantic Fusion: Transformer-based semantic fusion maps each projected semantic vector to query, key, and value vectors and learns mutual attention between semantic pairs.The output for each semantic is a weighted sum of value vectors with a residual connection.
- Transformer-based Semantic Fusion: The final node embedding concatenates the fused semantic outputs and feeds them to an MLP for downstream node classification.The overall training process computes predictions and updates network parameters using the loss function.
Experiment
Experiments across HGB and OGB benchmarks evaluate SeHGNN against heterogeneous graph baselines, including ablations and time-complexity analyses. SeHGNN achieves strong node-classification performance and favorable training efficiency across the reported evaluations.
- HGB benchmark: SeHGNN achieves the best performance among HGB baselines on four datasets, except for micro-F1 accuracy on Freebase, where it ranks second.
- Ablation studies: Ablations show that removing neighbor attention or replacing the single-layer long-metapath design produces a performance gap relative to SeHGNN.Variant #1 uses GAT for each metapath, while Variant #2 uses two layers with shorter metapaths per layer.
- Ablation studies: Label propagation and transformer-based semantic fusion improve performance: the no-label variant still outperforms most baselines except on Freebase micro-F1.
- ogbn-mag: SeHGNN outperforms other methods on ogbn-mag under the same condition, and randomly initialized features outperform competitors using additional well-trained embeddings.The comparison addresses extra embeddings and multi-stage training settings reported for the large-scale dataset.
Conclusion
The paper proposes SeHGNN, a simple heterogeneous graph neural network built around simplified aggregation and a single-layer long-metapath design. Across five datasets, it reports higher accuracy and faster training than state-of-the-art methods.
- SeHGNN pre-computes neighbor aggregation with a lightweight mean aggregator, avoiding overused neighbor attention and repeated aggregation during training.
- SeHGNN uses a single-layer structure with long metapaths and transformer-based semantic fusion to capture structural and semantic information.
- Experiments on five commonly used datasets show that SeHGNN outperforms state-of-the-art methods in both accuracy and training speed.
Observation in HGB models
The HGB attention analysis finds that attention values are largely determined by edge types rather than varying substantially among neighbors within each relation. This observation motivates examining whether neighbor-level attention is necessary.
- HGB computes edge attention from the embeddings of two endpoint nodes together with an edge-type embedding.
- In re-implementation experiments, attention values within each relation are similar, while values across different relations differ substantially.
- Figure 4(a) illustrates within-relation attention similarity, while Figure 4(b) plots standard deviations within relations and across all relations for each target node on ACM.
- This pattern motivates separate investigation of neighbor attention within relations and semantic attention across relations.
Framework of existing metapath-based methods
Existing metapath-based HGNNs commonly decompose each layer into feature projection, neighbor aggregation, and semantic fusion. SeHGNN changes the ordering and execution of these operations to enable pre-computed aggregation while retaining nonlinear feature projection.
- Metapath-based HGNN layers typically project features, aggregate neighbors for each semantic scope, and fuse the resulting semantic representations.
- After removing neighbor attention, linear feature projection and parameter-free neighbor aggregation can be exchanged and aggregation can be performed once during preprocessing.The pre-computed results are shared across all training epochs.
- SeHGNN uses an MLP block for feature projection because experiments found that a multi-layer projection block further improves performance.
Experiment settings
Experiments use four medium-scale HGB datasets and one large-scale ogbn-mag3 dataset, with benchmark-specific data partitions and comparisons. The study also specifies metapath selection, model settings, and revised complexity accounting for heterogeneous node types.
- Datasets: Four medium-scale HGB datasets and one large-scale ogbn-mag3 dataset are used to evaluate SeHGNN.The datasets come from the HGB benchmark and OGB challenge.
- Data partitions: Medium-scale datasets use 24% of target nodes for training, 6% for validation, and 70% for online testing.Results are compared with HGB baseline scores and averaged over five local data partitions.
- Metapath selection: Metapaths are selected by presetting maximum hops and using all available metapaths within those limits for feature and label propagation.Different maximum-hop combinations are tested, with final choices reported in Figure 8.
- Model settings: Each metapath uses a two-layer MLP with hidden dimension 512, while semantic fusion uses one transformer head and value vectors matching the hidden dimension.Query and key vectors have one-quarter the hidden-vector dimension.
- Complexity analysis: Revised complexity analysis includes nodes of all types for HAN and HGB and assumes n and m are comparable in full-batch training.Table 9 reports the revised time complexity.
- Complexity analysis: In mini-batch training, m can greatly exceed n, causing HAN and HGB complexity estimates in Table 5 to be significantly underestimated.The disparity grows exponentially with metapath length or layer count when projected neighbor features are rarely reusable.
- Complexity analysis: SeHGNN avoids involving other node types in each training batch because neighbor aggregation is computed once during preprocessing.This makes its computation complexity simpler to estimate.