MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding

Xinyu Fu; Jiani Zhang; Ziqiao Meng; Irwin King

Source-linked AI summary

MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding

Xinyu Fu, Jiani Zhang, Ziqiao Meng, Irwin King

arXiv:2002.01680v2Feb 5, 2020cs.SIcs.LG

TL;DR

Heterogeneous graph embedding must represent diverse structural, semantic, and content information, but existing metapath methods may omit content, intermediate nodes, or multiple metapaths. MAGNN addresses these gaps with three aggregation components and reports state-of-the-art performance across node classification, node clustering, and link prediction on three real-world datasets.

Problem
Existing heterogeneous graph embedding methods may ignore node content features, discard intermediate metapath nodes, or use only a single metapath.
Method
MAGNN transforms type-specific node content, aggregates information within each metapath, and fuses representations from multiple metapaths.
Results
MAGNN achieves state-of-the-art results across node classification, node clustering, and link prediction on three real-world datasets.
Takeaways & Limitations
MAGNN integrates node attributes, intermediate metapath context, and multiple metapaths into heterogeneous graph embeddings for downstream prediction tasks.

Abstract

from arXiv · show

A large number of real-world graphs or networks are inherently heterogeneous, involving a diversity of node types and relation types. Heterogeneous graph embedding is to embed rich structural and semantic information of a heterogeneous graph into low-dimensional node representations. Existing models usually define multiple metapaths in a heterogeneous graph to capture the composite relations and guide neighbor selection. However, these models either omit node content features, discard intermediate nodes along the metapath, or only consider one metapath. To address these three limitations, we propose a new model named Metapath Aggregated Graph Neural Network (MAGNN) to boost the final performance. Specifically, MAGNN employs three major components, i.e., the node content transformation to encapsulate input node attributes, the intra-metapath aggregation to incorporate intermediate semantic nodes, and the inter-metapath aggregation to combine messages from multiple metapaths. Extensive experiments on three real-world heterogeneous graph datasets for node classification, node clustering, and link prediction show that MAGNN achieves more accurate prediction results than state-of-the-art baselines.

1 INTRODUCTION

Heterogeneous graphs combine diverse node and edge types, making it difficult for conventional GNNs and existing metapath methods to encode their structural, semantic, and content information. MAGNN addresses these limitations through content transformation and aggregation across metapath instances and multiple metapaths, achieving consistently stronger results across evaluated tasks.

Motivation: Heterogeneous graphs contain varied node types, edge types, and feature spaces that challenge models designed for homogeneous or fixed-structure inputs.These graphs require low-dimensional representations that preserve rich structural and semantic information.
Metapath-based modeling: Metapaths encode composite relations and high-order proximity, but traditional GNNs cannot directly model their complex structural and semantic information.Examples include APA for co-authorship and APVPA for authors publishing in the same venue.
Limitations: Existing metapath methods may omit node content features or discard intermediate nodes, causing information loss on heterogeneous graphs with rich content.These limitations are associated with methods including metapath2vec, ESim, HIN2vec, HERec, and HAN.
MAGNN: MAGNN combines node content transformation, intra-metapath aggregation, and inter-metapath aggregation to generate heterogeneous graph node embeddings.It projects type-specific attributes into a shared latent space, incorporates metapath context, and fuses information from multiple metapaths.
Evaluation: MAGNN learns node embeddings for node classification and clustering on IMDb and DBLP, and link prediction on Last.fm.The experiments evaluate the model across three datasets and three downstream task types.
Results: MAGNN’s learned node embeddings are consistently better than those generated by other state-of-the-art baselines across the evaluated datasets and tasks.This is the reported overall experimental outcome.

2 PRELIMINARY

The preliminary section defines heterogeneous graphs, metapaths, metapath instances, metapath-based neighbors, metapath-based graphs, and heterogeneous graph embedding. Figure 1 illustrates these concepts using user, artist, and tag node types.

Heterogeneous Graph: A heterogeneous graph G=(V,E) uses node and edge type mappings ϕ:V→A and ψ:E→R, with more than two predefined node or edge types overall.A and R denote the sets of node types and edge types, respectively.
Metapath: A metapath is a schema-level path of alternating node and edge types that describes a composite relation between its endpoint node types.The composite relation is formed by composing the constituent relations.
Metapath Instance: A metapath instance is a node sequence that follows the schema specified by a metapath.It is a concrete path in the heterogeneous graph corresponding to that schema.
Metapath-based Neighbor: Metapath-based neighbors are nodes connected to a target through metapath instances, with different instances counted separately and the target included for symmetric metapaths.This definition preserves multiplicity induced by distinct metapath instances.
Illustration: Figure 1 organizes the definitions through a user-artist-tag example, showing UATA and UATAU metapaths, their instances, and the resulting metapath-based graphs.The example includes users, artists, and tags as the three node types.
Metapath-based Graph: A metapath-based graph G_P is constructed from all metapath-P-based neighbor pairs and is homogeneous when P is symmetric.The graph represents connectivity induced by one selected metapath.
Heterogeneous Graph Embedding: Heterogeneous graph embedding learns low-dimensional vectors h_v∈R^d for all nodes that capture the graph’s structural and semantic information.The target dimension d is much smaller than the number of nodes.

3 RELATED WORK

Related work covers general graph neural networks and heterogeneous graph embedding methods. It emphasizes that many existing GNNs assume homogeneous inputs, while heterogeneous methods often incompletely exploit metapath and node information.

General GNNs: General GNNs learn low-dimensional node representations for tasks such as node classification, node clustering, and link prediction.The reviewed approaches include spectral and spatial graph convolution methods.
General GNNs: Spectral GNNs such as ChebNet and GCN operate in the graph Fourier domain but suffer from scalability and graph-specific generalization limitations.Their filters depend on the graph Laplacian eigenbasis and may require the entire graph as input.
General GNNs: Spatial GNNs aggregate feature information from local neighborhoods, with GraphSAGE providing a general aggregator framework for sampling and transforming neighbors.This supports efficient and parallel node-embedding generation.
Heterogeneous GNNs: Many GNNs cannot naturally handle heterogeneous graphs because node features may lie in different spaces and the models assume shared embedding spaces.This limitation applies to models built for homogeneous graphs or specialized structures.
Heterogeneous Graph Embedding: Heterogeneous embedding methods such as metapath2vec, ESim, and HIN2vec use metapaths to guide walks, learn from metapath instances, or jointly model nodes and metapaths.These methods represent several strategies for exploiting heterogeneous graph structure.
Heterogeneous Graph Embedding: Existing heterogeneous methods may ignore node content, discard intermediate metapath nodes, or use only one metapath, leaving room to exploit graph information more comprehensively.The limitation is reported across the reviewed heterogeneous graph embedding approaches.

4 METHODOLOGY

MAGNN generates heterogeneous node embeddings through node content transformation, intra-metapath aggregation, and inter-metapath aggregation. It encodes intermediate nodes and weights both metapath instances and metapaths before task-specific projection and training.

Overview: MAGNN has three components: node content transformation, intra-metapath aggregation, and inter-metapath aggregation.The architecture generates node embeddings for downstream tasks.
Node Content Transformation: Type-specific linear transformations project heterogeneous node attributes, including unequal-dimensional features, into a shared latent vector space.The projected features share a dimension, facilitating subsequent aggregation.
Intra-metapath Aggregation: Intra-metapath aggregation encodes target nodes, metapath-based neighbors, and intermediate nodes into metapath-instance vectors.Intermediate nodes are explicitly included rather than discarded, and each encoded instance has dimension d′.
Intra-metapath Aggregation: An attention layer assigns normalized importance weights to metapath instances and combines them, with multiple heads available to stabilize learning.The weighted combination is passed through an activation function.
Inter-metapath Aggregation: Inter-metapath aggregation uses attention to assign different weights to metapaths and fuse their metapath-specific node vectors.The model summarizes each metapath and then computes relative metapath importance before weighted fusion.
Training and Output: Final node representations are projected to task-specific output spaces and learned under semi-supervised or unsupervised training paradigms.Semi-supervised learning minimizes cross entropy using labeled nodes and predicted probabilities.
Metapath Instance Encoders: MAGNN examines mean, linear, and relational rotation encoders for representing metapath instances, with relational rotation modeling sequential structure.The mean and linear encoders otherwise treat instances essentially as sets.

5 EXPERIMENTS

The experiments evaluate MAGNN on three heterogeneous datasets across node classification, node clustering, and link prediction, comparing it with several classes of graph embedding baselines. MAGNN consistently outperforms the baselines, while ablations support the contribution of its major components.

5.1 Datasets: MAGNN is evaluated on IMDb and DBLP for node classification and clustering, and on Last.fm for link prediction.The datasets represent different domains, and nodes without attributes receive one-hot identity vectors.
5.2 Baselines: The comparison includes traditional homogeneous and heterogeneous embedding models, homogeneous GNNs, and heterogeneous GNNs.The baselines include LINE, node2vec, ESim, HERec, GCN, and HAN.
5.3 Node Classification (RQ1): MAGNN consistently outperforms other baselines in node classification across datasets and training proportions.Its gain over HAN is around 4–7% on IMDb and 1–2% on DBLP.
5.4 Node Clustering (RQ2): MAGNN is consistently superior to all other baselines in node clustering on both IMDb and DBLP.The authors attribute poorer IMDb clustering performance to noisy single-genre labels, while heterogeneity-aware GNNs rank highest on both datasets.
5.5 Link Prediction (RQ3): MAGNN outperforms the baselines by a large margin in Last.fm link prediction and improves over HAN by around 6% relatively.The result supports the importance of metapath contexts and indicates that relying on a single metapath is suboptimal.
5.6 Ablation Study (RQ4): Ablations support node content transformation, intra-metapath aggregation, and inter-metapath aggregation as contributors to MAGNN’s performance.Aggregating metapath instances, combining multiple metapaths, and using relational rotation each improve performance, while all tested encoders outperform HAN.

6 CONCLUSION

MAGNN is presented as a framework addressing three limitations of heterogeneous graph embedding through coordinated architectural components. Experiments across multiple tasks and datasets support its effectiveness, while ablations show the value of these components.

MAGNN addresses dropping node content features, discarding intermediate metapath nodes, and considering only a single metapath.
Figure 4 visualizes embedding node pairs in Last.fm.
The model combines node content transformation, intra-metapath aggregation, and inter-metapath aggregation to generate node embeddings.These components respectively target the three limitations identified in existing heterogeneous graph embedding methods.
MAGNN achieves state-of-the-art results on three real-world datasets across node classification, node clustering, and link prediction.Ablation studies demonstrate the efficacy of its three major components in improving embedding performance.