PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources

Xiao Wang, Deyu Bo, Chuan Shi, Shaohua Fan, Yanfang Ye, Philip S. Yu

arXiv:2011.14867v1cs.SIcs.LG

TL;DR

Heterogeneous graph embedding must preserve diverse structures and semantics that homogeneous methods cannot directly capture. This survey synthesizes HG embedding methods, techniques, applications, and resources, finding broad real-world deployment while identifying continuing challenges in dynamic and reliable embedding.

  • Problem

    Heterogeneous graphs are widespread, but their diverse structures, semantics, and attributes make embedding difficult and leave existing methods without a systematic comprehensive survey.

  • Method

    The paper categorizes state-of-the-art HG embedding methods by the information used, analyzes their pros and cons, and reviews techniques, deployed systems, datasets, code, and platforms.

  • Results

    HG embedding methods have been successfully deployed in recommendation, malware detection, and healthcare systems, while HAN captures structural and semantic information through hierarchical attention.

  • Takeaways & Limitations

    Application-specific domain knowledge, including suitable meta-paths, is essential for application-oriented HG embedding.

  • Takeaways & Limitations

    Dynamic HG embedding remains challenging because incremental methods capture only short-term information, whereas retrained neural methods have high computational cost.

Abstract

from arXiv · show

Heterogeneous graphs (HGs) also known as heterogeneous information networks have become ubiquitous in real-world scenarios; therefore, HG embedding, which aims to learn representations in a lower-dimension space while preserving the heterogeneous structures and semantics for downstream tasks (e.g., node/graph classification, node clustering, link prediction), has drawn considerable attentions in recent years. In this survey, we perform a comprehensive review of the recent development on HG embedding methods and techniques. We first introduce the basic concepts of HG and discuss the unique challenges brought by the heterogeneity for HG embedding in comparison with homogeneous graph representation learning; and then we systemically survey and categorize the state-of-the-art HG embedding methods based on the information they used in the learning process to address the challenges posed by the HG heterogeneity. In particular, for each representative HG embedding method, we provide detailed introduction and further analyze its pros and cons; meanwhile, we also explore the transformativeness and applicability of different types of HG embedding methods in the real-world industrial environments for the first time. In addition, we further present several widely deployed systems that have demonstrated the success of HG embedding techniques in resolving real-world application problems with broader impacts. To facilitate future research and applications in this area, we also summarize the open-source code, existing graph learning platforms and benchmark datasets. Finally, we explore the additional issues and challenges of HG embedding and forecast the future research directions in this field.

1 INTRODUCTION

Heterogeneous graphs model diverse entities and relations with rich semantics, making their embeddings important for graph analysis but difficult to learn with homogeneous methods. This survey addresses the gap through a comprehensive, structured review of methods, applications, and resources.

  • Motivation: Heterogeneous graphs represent multiple entity and relation types, enabling rich semantic modeling across bibliographic, social, and recommendation networks.The academic-network example contains authors, papers, venues, and terms connected by write, contain, and publish relations.
  • Motivation: HG embedding learns lower-dimensional representations while preserving heterogeneous structure and semantics for tasks such as classification and clustering.The survey also frames embedding as a response to the computational and statistical drawbacks of large-scale matrix factorization.
  • Challenges: Homogeneous graph embedding methods cannot be directly applied because HG structures vary with semantic relations and node attributes have different meanings.For example, paper neighbors may be authors under write relations or terms under contain relations, while author and paper attributes can encode different information.
  • Survey scope: The survey categorizes HG embedding methods by the information used during learning and analyzes their advantages, disadvantages, and applicability.The categorization is designed to address particular challenges caused by HG heterogeneity.
  • Survey scope: The paper additionally studies industrial transformativeness, summarizes deployed systems and open resources, and discusses future challenges and directions.Covered resources include open-source code, graph-learning platforms, and benchmark datasets.

2 PRELIMINARY

The preliminary section defines heterogeneous graphs and their semantic structures, then explains why heterogeneity creates challenges in structure, attribute fusion, and application-specific modeling.

  • Basic Concepts: A heterogeneous graph contains multiple node or link types, represented by node and link mapping functions and summarized by a network schema.The graph is defined as G = {V, E}, with node types A and link types R satisfying A + R > 2.
  • Basic Concepts: Meta-paths encode semantic relationships as sequences of node and link types, while meta-graphs combine multiple meta-paths to represent more complex relationships.The examples APA and APCPA represent co-author and co-conference relations; a meta-graph can be symmetric or asymmetric.
  • Basic Concepts: HG embedding maps nodes into a low-dimensional Euclidean space with dimension d much smaller than the number of nodes.The embedding function is written as Φ : V → R^d.
  • Challenges: HGs create complex-structure challenges because a node’s neighborhood changes according to the selected relation type.In the academic network, paper neighbors may be authors through write links or terms through contain links.
  • Challenges: HG attribute fusion is difficult because attributes of different node and edge types can have different meanings.Author attributes may describe research fields, whereas paper attributes may consist of keywords.
  • Challenges: Application-oriented HG embedding requires domain knowledge for graph construction, meta-path selection, and encoding side information or advanced application knowledge.Recommendation systems use user-item-user and item-user-item meta-paths to represent different collaborative-filtering relations.

3 METHOD TAXONOMY

The survey organizes HG embedding methods according to the information they use, covering heterogeneous structures and semantics, attributes, application-specific knowledge, and temporal dynamics.

  • Taxonomy: Structure-oriented methods focus on capturing and preserving heterogeneous structures and semantics such as meta-paths and meta-graphs.
  • Taxonomy: Attribute-assisted methods incorporate node and edge attributes beyond graph structure to use neighborhood information more effectively.
  • Taxonomy: Application-oriented methods learn node embeddings tailored to specific applications over heterogeneous graphs.
  • Taxonomy: Dynamic HG embedding methods address heterogeneous graphs that evolve over time.

3.1 Structure-preserved HG Embedding

Structure-preserved HG embedding methods retain heterogeneous graph semantics through links, paths, hybrid relations, and subgraphs. They use relation-aware objectives or guided sequences to capture increasingly complex structural information, while shallow approaches remain limited in memory use and transductive deployment.

  • Link-based HG Embedding: Link-based methods preserve heterogeneous first-order information by modeling typed links through relation-specific metric spaces, similarity functions, or link probabilities.PME transforms nodes into relation-specific metric spaces, while EOE, HeGAN, AspEM, and HEER use similarity or link-probability objectives.
  • Link-based HG Embedding: PME uses a margin-based triple loss to make node pairs connected by relation r closer than node pairs without that relation.The objective compares positive and negative samples for each relation.
  • Path-based HG Embedding: Link-based methods capture only local first-order structure, leaving higher-order semantic relations such as author-author and paper-paper similarity insufficiently represented.Higher-order relations combine multiple edge types and are critical for heterogeneous graph embedding.
  • Path-based HG Embedding: Metapath2vec generates meta-path-guided node sequences and applies heterogeneous skip-gram learning to preserve both first-order and high-order proximity.The PAP example projects a center node and maximizes the probability of its meta-path-based context nodes.
  • Path-based HG Embedding: HIN2vec jointly predicts relations to learn node and meta-path embeddings, allowing its relation set to encode both first-order and high-order structures.Its relation set includes examples such as A-P and A-P-A, enabling different semantic information to be captured in node embeddings.
  • Subgraph-based HG Embedding: Subgraph-based methods incorporate metagraphs and hyperedges, while mg2vec preserves node–subgraph proximity to capture structural information and node–metagraph similarity.mg2vec preserves first-order and second-order proximity using a neural network for node-pair embeddings.

3.2 Attribute-assisted HG Embedding

Attribute-assisted HG embedding methods encode heterogeneous structures and multiple node attributes, while HGNNs use aggregation and attention to preserve type-specific information. The section covers unsupervised and semi-supervised approaches, including HAN and related architectures.

  • Attribute-assisted methods encode complex structures and multiple attributes to learn node embeddings despite heterogeneous attribute meanings.
  • Unsupervised HGNNs: Unsupervised HGNNs use interactions among different attribute types to capture potential commonalities and improve embedding generalization.
  • Unsupervised HGNNs: HetGNN combines content, same-type neighbor, and node-type aggregation, then uses heterogeneous skip-gram loss to preserve structural and attribute heterogeneity.
  • Semi-supervised HGNNs: HAN applies hierarchical attention: node-level attention weights neighbors within meta-paths, while semantic-level attention weights different meta-path embeddings.
  • Semi-supervised HGNNs: HAN uses a task-specific layer and a small number of labels to fine-tune embeddings for downstream tasks such as node clustering and link prediction.
  • Summary: HGNNs support inductive learning and require less memory than shallow models, but they still incur high inference and retraining costs.

3.3 Application-oriented HG Embedding

Application-oriented HG embedding incorporates domain knowledge into heterogeneous graphs for recommendation, identification, and proximity search. Methods combine application-specific structures, attributes, relations, and supervision to support these tasks.

  • Application-oriented methods must construct a suitable heterogeneous graph and determine which domain knowledge to incorporate for the target application.
  • Recommendation: Recommendation systems model user-item interactions as heterogeneous graphs and may add social relationships to capture complex interactions.
  • Recommendation: HERec uses meta-path-guided random walks, node2vec, meta-path fusion, and joint graph-embedding and recommendation objectives.
  • Identification: Identification methods find likely nodes under given conditions, including potential authors of anonymous papers and users across platforms.
  • Author Identification: Camel jointly models paper content, author-paper pairwise relations, and heterogeneous graph context through a content encoder, metric loss, and meta-path-guided walks.
  • Proximity Search: Prox learns embeddings from heterogeneous sequences and applies a ranking loss so preferred nodes receive higher similarity than less-preferred nodes.
  • Summary: Meta-paths encode application-specific semantics, such as user-based and item-based collaborative filtering or semantic relationships in proximity search.

3.4 Dynamic HG Embedding

Dynamic HG embedding methods address graph evolution through either incremental updates or retraining at each timestamp. Incremental approaches are efficient but short-term and shallow, whereas retrained approaches capture longer-term temporal information at higher computational cost.

  • Dynamic HG Embedding: Dynamic HG embedding methods divide into incremental-update and retrained-update approaches.Incremental methods update embeddings for new timestamps, while retrained methods relearn embeddings and then model temporal information.
  • Incremental HG Embedding: DyHNE uses matrix perturbation to update embeddings while preserving meta-path-based first- and second-order proximities.It captures both heterogeneous structure and graph evolution without retraining the entire model.
  • Retrained HG Embedding: DyHATR combines node- and edge-level hierarchical attention to fuse neighbor attributes before feeding timestamp-ordered embeddings into an RNN.The RNN captures temporal information from changes in node embeddings across timestamps.
  • Summary: Incremental methods capture only short-term temporal information, while retrained methods capture long-term information but incur high computational cost.The survey identifies combining their advantages and reducing cumulative incremental-update errors as open problems.

3.5 Miscellanea

The survey highlights heterogeneous graph embedding beyond its main method categories, including NLP, multimodal data, hyperbolic representations, and several specialized applications. These settings exploit heterogeneous entities, relations, and data forms to model task-specific structure and semantics.

  • NLP: NLP tasks can represent words, entities, sentences, or paragraphs as heterogeneous graphs for graph-to-sequence generation and text classification.AMR-to-text generation uses a heterogeneous graph encoder to capture semantic relations among concepts, while HGAT models topics, entities, and documents.
  • Multimodal Data: Multimodal data such as text, images, and videos can be modeled as heterogeneous graphs to capture dependencies and connections across data forms.The survey presents Community Question Answering as an example application.
  • Future Directions: Hyperbolic heterogeneous graph embedding is proposed as a meaningful direction because hyperbolic spaces can capture hierarchical and power-law graph structures.The survey notes that Euclidean spaces are not established as optimal isometric spaces for these graphs.
  • Other Applications: Heterogeneous graph embedding is also applied to entity set expansion, basket recommendation, event categorization, and social-network tasks.These examples extend the application scope beyond the survey’s major categories.

4 TECHNIQUE SUMMARY

The survey organizes heterogeneous graph embedding techniques into shallow and deep models, then compares their information use, task fit, training properties, and complexity. Shallow models emphasize structural proximity, while deep models better integrate attributes and application-specific knowledge.

  • Shallow Model: Shallow models initialize embeddings randomly and optimize objective functions, comprising random-walk-based and decomposition-based techniques.Random-walk methods preserve semantic structure through guided walks, while decomposition methods preserve proximity in graph substructures.
  • Deep Model: Deep models learn from node attributes or node interactions through message passing, encoder-decoder, or adversarial architectures.These categories differ in how they aggregate information, preserve graph properties, or improve embedding robustness.
  • Method Properties: Most message passing-based methods are inductive because they update embeddings by aggregating neighborhood information, but they require additional labels for training.The survey summarizes these properties in its comparison of typical methods.
  • Information and Tasks: Most deep methods target attributed or application-specific HGs, whereas shallow methods mainly use structural information.Deep models support complex domain-knowledge modeling, and HGNNs integrate graph structures with attributes.
  • Training and Efficiency: Shallow models are easy to parallelize but use two-stage training and heavy memory, while deep models are end-to-end and require less memory.Message passing integrates structures, attributes, and semantics; encoder-decoder methods are more flexible in decoder objectives but weaker at information fusion.
  • Adversarial Methods: Adversarial methods use negative samples to improve robustness, but negative-sample selection strongly affects performance and increases variance.The survey contrasts this trade-off with the flexibility and fusion behavior of other deep techniques.

5 REAL-WORLD DEPLOYED SYSTEMS

Heterogeneous graph embedding has been deployed in industrial systems where multiple entity types and relations characterize practical data. The survey reviews applications in e-commerce, cybersecurity, healthcare, and real-time event prediction.

  • E-commerce: E-commerce platforms model users, items, shops, and auxiliary information as heterogeneous graphs for recommendation, profiling, alignment, and fraud detection.The survey describes these graphs as suitable for complex interactions in large-scale commercial systems.
  • E-commerce: Intent recommendation models users, items, and queries in a heterogeneous graph and applies a meta-path-guided HGNN to recommend queries from historical behavior.This reframes intent recommendation as query recommendation.
  • Cybersecurity: Security applications use heterogeneous graphs to represent multiple entities and complex relations for malware detection, key-player identification, and drug-trafficker identification.HinDroid models relations between Android APIs and applications, while other systems use attributed or multi-view heterogeneous graphs.
  • Healthcare: Healthcare systems represent diagnostic and treatment events as nodes and relations extracted from raw text as edges in heterogeneous graphs.These methods support mining electronic health records for disease-diagnosis-related objectives.
  • Real-Time Prediction: Ride-hailing systems dynamically construct heterogeneous graphs for ongoing events and use multilayer GNNs to model historical actions and surrounding conditions.The learned representations are used for real-time event prediction.

6 BENCHMARKS AND OPEN-SOURCE TOOLS

This section surveys commonly used heterogeneous graph datasets, source-code collections, and open-source platforms and toolkits that support research and applications.

  • Datasets: The survey organizes real-world HG datasets into academic, business, and film networks, summarizing node types, link types, and meta-paths.
  • Datasets: DBLP and Aminer represent academic networks, while Yelp and Amazon cover social-media and e-commerce interactions.
  • Datasets: IMDB and Douban are film-rating networks that record user preferences alongside film and user-related information.
  • Resources and source code: The survey collects related source code and lists publicly available graph-embedding websites to support method reproduction.
  • Platforms and toolkits: Open-source platforms include AliGraph, DGL, PyTorch Geometric, and OpenHINE, with support ranging from industrial-scale graphs to heterogeneous embedding methods.

7 CHALLENGES AND FUTURE DIRECTIONS

The survey identifies unresolved challenges in representing HG structure and properties, developing theory and techniques, ensuring reliability, expanding applications, and achieving scalable deployment.

  • Capturing HG structure: Selecting an appropriate meta-path remains challenging, and improper choices can hinder heterogeneous graph embedding performance.
  • Capturing HG properties: Dynamic and uncertain HG properties remain under-modeled, motivating deep dynamic methods and distributional representations such as Gaussian embeddings.
  • Deep graph learning: Theoretical analysis of HGNNs is limited, including whether over-smoothing occurs and which factors cause it under multiple aggregation strategies.
  • Deep graph learning: Self-supervised learning and HGNN pre-training are proposed to reduce dependence on labels, training time, and task-specific retraining.
  • Reliable HG embedding: Fairness, robustness, interpretability, and explainability remain open concerns because HG embeddings can relate to sensitive attributes and models are often black boxes.
  • Applications and scalability: Existing HGNNs are difficult to apply at industrial scale because of high complexity and limited parallelization, despite graphs reaching one billion nodes.
  • Additional directions: Future directions include hyperbolic embedding, joint HG structure learning, and combining knowledge graph embedding with HG embedding.

8 CONCLUSION

The conclusion presents heterogeneous graph embedding as a comprehensive research area supporting HG analysis and related applications, while organizing methods, benchmarks, and resources for future work.

  • The survey systematically studies state-of-the-art HG embedding methods and presents widely used benchmarks and resources.
Loading 2011.14867v1…