Source-linked AI summary
An Attention-based Graph Neural Network for Heterogeneous Structural Learning
Huiting Hong, Hantao Guo, Yucheng Lin, Xiaoqing Yang, Zang Li, Jieping Ye
TL;DR
Heterogeneous information networks challenge embedding methods because they combine multiple node and relation types, while meta-path approaches require handcrafted schemes and may lose information. The paper proposes HetSANN, which uses cross-type projections and type-aware attention to aggregate heterogeneous neighborhoods without meta-paths. Experiments on three datasets report consistent improvements over state-of-the-art methods, with its extensions further improving the vanilla model.
Problem
Heterogeneous information networks require modeling multiple node-type spaces and relation semantics, while meta-path methods depend on expert-designed schemes and may discard heterogeneous information.
Method
HetSANN projects vertices from different entity spaces into a common target space and uses a type-aware attention layer within a task-guided graph neural network.
Results
Experiments on three heterogeneous graph datasets report that HetSANN solutions outperform state-of-the-art methods in HIN embedding and node classification.
Takeaways & Limitations
HetSANN provides a meta-path-free framework for learning heterogeneous graph representations from structural and semantic information, with extensions for relation modeling, transformation consistency, and information sharing.
Abstract
from arXiv · showhide
In this paper, we focus on graph representation learning of heterogeneous information network (HIN), in which various types of vertices are connected by various types of relations. Most of the existing methods conducted on HIN revise homogeneous graph embedding models via meta-paths to learn low-dimensional vector space of HIN. In this paper, we propose a novel Heterogeneous Graph Structural Attention Neural Network (HetSANN) to directly encode structural information of HIN without meta-path and achieve more informative representations. With this method, domain experts will not be needed to design meta-path schemes and the heterogeneous information can be processed automatically by our proposed model. Specifically, we implicitly represent heterogeneous information using the following two methods: 1) we model the transformation between heterogeneous vertices through a projection in low-dimensional entity spaces; 2) afterwards, we apply the graph neural network to aggregate multi-relational information of projected neighborhood by means of attention mechanism. We also present three extensions of HetSANN, i.e., voices-sharing product attention for the pairwise relationships in HIN, cycle-consistency loss to retain the transformation between heterogeneous entity spaces, and multi-task learning with full use of information. The experiments conducted on three public datasets demonstrate that our proposed models achieve significant and consistent improvements compared to state-of-the-art solutions.
1 Introduction
Heterogeneous information networks contain multiple node and relation types, creating challenges for modeling cross-type entity spaces and relation semantics. HetSANN addresses these challenges without handcrafted meta-paths by combining projections, type-aware attention, and three extensions, with improvements reported across three datasets.
- Motivation: Heterogeneous information networks contain multiple node and edge types, unlike homogeneous graphs where all nodes and relationships share one type.The paper illustrates an academic HIN with author, paper, and conference nodes plus directed and reverse relations.
- Challenges: Cross-type connections require interactions between distinct node-type entity spaces, while multiple relation types must retain differing semantic content.These are identified as challenges C1 and C2 for heterogeneous graph embedding.
- Limitations of prior methods: Handcrafted meta-path methods depend on expert-designed schemes and can lose heterogeneous node or edge information during node-pair generation.The paper identifies both difficulty exhaustively selecting useful schemes and potential embedding degradation from information loss.
- Proposed approach: HetSANN directly learns structural and semantic representations by projecting vertices across entity spaces and applying type-aware attention to heterogeneous neighborhoods.Its task-guided objective uses node classification loss, while the type-aware layer replaces a conventional GNN convolution.
- Extensions: HetSANN extends its base model with multi-task learning, voices-sharing relation attention, and cycle-consistency constraints on transformations.These extensions respectively enhance information sharing, model directed and reversed edge relationships, and preserve transformation consistency.
- Evaluation: Experiments on three heterogeneous graph datasets report superiority over state-of-the-art methods, while ablations show improvement from all three extensions over vanilla HetSANN.The evaluation uses node classification, and the ablation study examines the extensions individually through model variants.
2 Heterogeneous Graph Structural Attention Neural Network (HetSANN)
HetSANN learns heterogeneous graph representations by projecting neighboring nodes into target-type spaces and aggregating raw multi-relational links with type-aware attention. Its extensions model directed-edge pairs, enforce cycle consistency, and jointly train node classifiers.
- Framework: HetSANN represents each vertex in a low-dimensional space specific to its node type and supports downstream node classification.The model is task-guided and learns representations through type-aware attention layers.
- Transformation Operation (C1): Each attention head transforms every neighbor into the target vertex’s type-specific space before neighborhood aggregation.The projection uses a relation- and node-type-specific transformation so heterogeneous neighbors share a space for aggregation.
- Aggregation of Neighborhood (C2): Type-aware attention assigns relation-specific scores to raw links, normalizes them over the target neighborhood, and aggregates the projected states.The attention coefficient represents an edge’s importance to the target vertex; the model uses separate scoring functions for relation types.
- Aggregation of Neighborhood (C2): Multiple attention heads concatenate their low-dimensional outputs, while deeper layers capture high-order proximity from raw links rather than meta-path-generated links.Residual connections are used to facilitate training of the deeper model.
- Multi-task Learning: Multi-task learning shares attention-layer parameters across classifiers for different node types, reducing overfitting risk and improving representation robustness.The classifiers are trained jointly using information from multiple node types.
- Voices-sharing Product: Voices-sharing product shares attention parameters between directed and reversed edge types to model their pairwise relationship.The reversed relation is treated as the passive voice of the directed relation.
- Cycle Consistency: Cycle-consistency loss constrains transformations between node-type spaces so a transformation cycle returns a vertex to its starting position.The paper replaces matrix inversion with a trainable matrix and incorporates the constraints into the objective.
3 Experiments
Experiments evaluate HetSANN and its variants on three heterogeneous datasets using node classification, comparisons with baselines, ablations, and parameter sensitivity analyses. The full model consistently outperforms competing methods, while the extensions generally improve the vanilla model.
- Datasets: Three heterogeneous datasets—IMDB, DBLP, and AMiner—are used for node classification experiments.IMDB contains movie, actor, and director information; DBLP and AMiner represent academic networks.
- Evaluation Metrics: Models are evaluated on a randomly split 0.8:0.1:0.1 labeled dataset using Micro F1 and Macro F1 averaged over 10 repeated processes.The best model is selected on validation data before test evaluation.
- Ablation Study: Ablation results show that the extensions improve vanilla HetSANN overall, although multi-task learning can slightly reduce Paper classification performance on DBLP.The multi-task auxiliary task is Paper classification alongside the main Author classification task and is unavailable for IMDB.
- Ablation Study: HetSANN.M.R.V performs best among model variants in Author and Movie classification, but its gain over HetSANN.M.R is constrained by the cycle-consistency implementation.The paper attributes this constraint to replacing an analytical inverse matrix with a trainable matrix and leaves the issue for future work.
- Parameter Sensitivity Study: On IMDB, performance decreases when the number of type-aware attention layers exceeds 5, while cycle-consistency weighting shows task-dependent sensitivity.Increasing β1 beyond 10^-4 can suppress the main node-classification task, whereas performance stabilizes when β2 exceeds 10^-5 under the reported setting.
4 Related Work
Prior HIN embedding methods commonly adapt homogeneous graph techniques through meta-paths, while GNNs propagate structural information directly across graph neighborhoods.
- metapath2vec guides heterogeneous-graph random walks with meta-paths before applying skip-gram to learn vertex representations.
- HAN combines node-level attention over meta-path-generated neighbors with semantic-level attention across multiple meta-path schemes.
- GNNs generate node embeddings by spatially filtering each node over its neighborhood and propagating structural information layer by layer.
5 Conclusion
The paper proposes HetSANN for meta-path-free heterogeneous graph embedding and reports consistent improvements over state-of-the-art methods across three datasets.
- HetSANN performs meta-path-free embedding by directly learning from structural information in heterogeneous graphs.
- A type-aware attention layer jointly embeds vertices with different types of neighboring nodes and their associated linkages.
- Three extensions add voices-sharing product attention, cycle-consistency loss, and multi-task learning to the HetSANN framework.
- Experiments on three popular datasets show the proposed solutions outperform state-of-the-art methods in HIN embedding and node classification.