PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Heterogeneous Graph Representation Learning: A Survey

Huan Liu, Pengfei Jiao, Jie Yin, Hongjiang Chen, Zhidong Zhao

arXiv:2609.04779v1cs.LGcs.AIcs.SI

TL;DR

DHG representation learning addresses the challenge of representing evolving heterogeneous systems whose structural semantics and temporal dynamics are coupled. This survey provides a unified definition, algorithm-centric taxonomy, and review of applications and benchmarks, then identifies persistent challenges in scalable, transferable, and trustworthy DHG learning.

  • Problem

    Existing research is fragmented, commonly models heterogeneity and dynamics separately, and lacks a unified review of DHG representation learning.

  • Method

    The survey unifies discrete-time and continuous-time DHGs, organizes methods by algorithmic family and temporal modeling bias, and reviews applications, datasets, and benchmarks.

  • Results

    The survey identifies inconsistent graph construction and deterministic temporal evaluation protocols, alongside limitations in current DHG modeling mechanisms and benchmarks.

  • Takeaways & Limitations

    Future DHG foundation models require progress in efficiency, generalizability, and trustworthiness.

  • Takeaways & Limitations

    Current methods remain constrained by quadratic complexity, task-specific supervised training without principled pre-training, and limited interpretability or causal grounding.

Abstract

from arXiv · show

Graph representation learning (GRL) serves as a canonical paradigm for modeling complex networks. However, real-world AI systems inherently manifest as evolving heterogeneous entities with complex interactions, posing significant challenges to static or homogeneous modeling. To address these complexities, representation learning for Dynamic Heterogeneous Graphs (DHGs) has emerged as a vital approach for learning low-dimensional representations that simultaneously preserve structural semantics and temporal dynamics. This survey presents the first systematic review of DHG representation learning methods. We first introduce a unified formal definition that encompasses both discrete-time and continuous-time DHGs from the perspective of temporal granularity. Building upon this formulation, we propose a novel algorithm-centric taxonomy that categorizes existing literature, including early embedding-based approaches, graph neural network (GNN)-based models, and relatively recent Transformer-based DHG methods, while explicitly highlighting their intrinsic modeling biases with respect to dynamic granularity. Furthermore, we summarize representative applications of DHG representation learning, along with commonly used datasets and benchmarks. Finally, we discuss promising research directions that guide future advances in this rapidly evolving field.

1 Introduction

DHG representation learning addresses evolving systems where heterogeneous semantics and temporal dynamics are coupled. The survey unifies fragmented work through a formal definition, algorithm-centric taxonomy, application and benchmark review, and future research directions.

  • Real-world systems combine dynamically evolving, multi-typed entities with complex interactions, making joint modeling of heterogeneity and dynamics necessary.
  • The literature remains fragmented because prior research commonly models heterogeneity and dynamics separately.
  • The survey introduces a unified DHG definition spanning discrete-time and continuous-time formulations.
  • Its algorithm-centric taxonomy organizes methods into embedding-based, GNN-based, and Transformer-based paradigms while characterizing temporal-granularity biases.
  • The survey reviews applications, datasets, and benchmarks, identifying inconsistent graph construction and fixed temporal partitions that hinder fair comparison and reliable evaluation.

2 Preliminaries and Notations

The survey defines a DHG as jointly modeling structural heterogeneity and temporal evolution, with a temporal domain that distinguishes static, discrete-time, and continuous-time cases. DHG representation learning maps such graphs to low-dimensional node embeddings preserving both properties.

  • A DHG jointly models structural heterogeneity and temporal evolution over possibly time-varying nodes and edges.
  • Node and edge type-mapping functions assign graph entities to node-type and edge-type sets.
  • A graph is heterogeneous when it has multiple semantic types, while one node type and one edge type yield a homogeneous graph.
  • The temporal domain distinguishes static graphs, discrete-time snapshot sequences, and continuous-time timestamped-edge streams.
  • DHG representation learning learns an encoder that maps nodes to low-dimensional embeddings jointly preserving heterogeneity and dynamics proximity.

3 DHG Representation Learning Taxonomy

The taxonomy groups DHG representation learning methods by how their algorithms capture coupled type-specific semantics and temporal dependencies. It distinguishes embedding-based, GNN-based, Transformer-based, and application-oriented families.

  • DHG methods must jointly encode type-specific semantics and temporal dependencies because relation types exhibit distinct evolutionary patterns.
  • Embedding-based approaches preserve structural and temporal proximity through random walks or matrix factorization.
  • GNN-based approaches use type-aware message-passing and aggregation across graph snapshots.
  • Transformer-based approaches unify type and temporal signals within attention mechanisms.
  • The taxonomy also includes application-oriented methods as a distinct family.

3.1 Embedding-based

Embedding-based methods span discrete-snapshot random walks and incremental updates, plus continuous-time temporal point processes. They offer structural or computational advantages but rely on restrictive proximity assumptions and can lose accuracy or transferability.

  • Embedding-based methods preserve structural proximity and divide into random-walk, incremental-update, and temporal-point-process approaches by temporal granularity.
  • Random Walk-based: Random-walk methods use predefined meta-structures to capture co-evolving topology and heterogeneous semantics across snapshots.
  • Random Walk-based: These methods preserve structural proximity but often omit rich node attributes because they depend on expert-defined meta-paths or meta-graphs.
  • Incremental Update-based: Incremental-update methods efficiently revise representations from prior states and localized DHG perturbations instead of retraining the entire network.
  • Incremental Update-based: Incremental updates face cumulative approximation drift over long horizons and difficulty handling abrupt structural changes that violate temporal smoothness.
  • Temporal Point Process-based: Temporal point processes model asynchronous heterogeneous events through conditional intensities incorporating base rates, historical excitation, and temporal decay.
  • Embedding-based methods remain limited by predefined proximity measures, shallow architectures, and absent task-specific gradients for downstream transfer.

3.2 GNN-based

GNN-based DHG methods extend message passing with relation-aware aggregation and temporal updates, using either predefined semantic structures or recurrent memory. Their expressiveness is limited by scaling costs, temporal discretization, and over-smoothing risks.

  • GNN-based methods incorporate heterogeneous edge types and temporal evolution, predominantly operating at snapshot granularity.
  • Relation-specific GNNs: Relation-specific GNNs decompose heterogeneous interactions into relation-specific subgraphs and use hierarchical aggregation across neighbors, types, and time.DyHAN and HTGNN exemplify this design, while HTHGN extends the receptive field to heterogeneous hyperedges for group dynamics.
  • Meta-structure-guided GNNs: Meta-structure-guided GNNs constrain message passing along predefined meta-paths or meta-graphs to model semantic dependencies explicitly.HDGAN treats meta-paths as independent channels, whereas DHANE merges paths into a unified meta-graph for cross-path propagation.
  • Limitations: Meta-structure methods face combinatorial explosion and static-template limitations, while GNN architectures also incur temporal-depth costs, discretization losses, scaling overhead, and possible over-smoothing.
  • Recurrent-based GNNs: Recurrent-based GNNs combine heterogeneous snapshot encoding with recurrent units that update node memory across snapshots or interactions.DyHATR adds hierarchical attention and temporal self-attention; DURENDAL compares Aggregate-Then-Update with Update-Then-Aggregate schemes; HGN2T tightly couples structure and temporal dynamics.

3.3 Transformer-based

Transformer-based DHG methods unify heterogeneous and temporal modeling through attention, spanning snapshot neighborhoods, continuous interaction histories, and LLM-derived semantic priors. These designs improve temporal granularity or semantic coverage but remain constrained by scalability, memory, and transfer limitations.

  • Transformer-based methods use self-attention to unify DHG modeling and avoid the hierarchical decomposition inherent to GNN-based approaches.The survey distinguishes structure-oriented, interaction-oriented, and LLM-enhanced paradigms by their modeling granularity.
  • Structure-oriented Transformers: Structure-oriented Transformers attend over dynamic heterogeneous neighborhoods using node types, edge types, and relative temporal positions.HGT uses triplet-based parameter decomposition and relative temporal encoding; later methods add full Transformer encoders, architecture search, or spatio-temporal positional encodings.
  • Limitations: Transformer-based methods remain constrained by quadratic complexity, additive rather than intrinsic type-time coupling, absent pre-training objectives, and underexplored tokenization.
  • Interaction-oriented Transformers: Interaction-oriented Transformers apply sequence attention to each node’s chronological edge-event history, preserving continuous timestamps and heterogeneous event types.STHN uses patching to reduce sequence complexity from quadratic to linear, while MIGNN separates short-term and long-term dynamics through dual-window aggregation.
  • Interaction-oriented Transformers: Interaction-oriented methods capture fine-grained event-level dynamics but require per-node event histories and may underuse local structural context.
  • LLM-enhanced Transformers: LLM-enhanced Transformers fuse semantic embeddings with structural representations to model implicit type properties and domain-specific temporal patterns.CasMLN uses prompts for node types and graph context, while HTGformer aligns semantic and structural spaces through instruction tuning.

3.4 Application-oriented

DHG applications model heterogeneous entities and relations whose interactions evolve over time across cybersecurity, traffic forecasting, recommendation, multimodal learning, and information diffusion. These applications use temporal and type-specific mechanisms, while limitations include delayed detection and vulnerability to missing or evolving signals.

  • Cybersecurity: Cybersecurity models heterogeneous behavioral evidence over time to detect coordinated attacks and fraudulent identities.HTGT combines spatial dependencies with historical sequences, while TimeSAGE uses time-decayed edges and temporal random walks.
  • Cybersecurity: Accumulated behavioral evidence supports long-horizon attack detection but limits responsiveness to zero-day exploits and evolving adversarial tactics.
  • Traffic forecasting: Traffic forecasting represents road, intersection, and transit dynamics with distinct periodicity, transitions, and volatility rather than uniform embeddings.HetETA uses parallel temporal convolutions for recent, daily, and weekly patterns before graph propagation.
  • Recommendation: Recommendation models assign different temporal and semantic weights to clicks, purchases, and reviews through type-specific memory and hierarchical attention.DHIMN first weights items within behavior types and then aggregates across types.
  • Multimodal learning: Multimodal DHG methods model changing cross-modal alignments and missing channels, but robustness to missing or noisy modalities remains underexplored.DHGRNN adapts evolving convolutions to missing EEG channels, while HDGR uses dynamic intra-modal and bipartite cross-modal graphs.
  • Information diffusion: Information diffusion applies type-specific dynamics to processes such as durable citation influence, rapidly decaying endorsements, and regime-dependent financial contagion.HINTS addresses cold-start prediction by imputing pseudo-historical trajectories via network alignment.

4 Open-source Datasets and Benchmarks

DHG benchmarks draw on open-source datasets spanning application domains and temporal settings, but inconsistent graph construction, discretization, and evaluation undermine comparison. Standardized resources improve consistency, while broader tasks and incremental evaluation remain needed.

  • Dataset overview: Open-source DHG datasets are organized by application domain, node and edge types, temporal coverage, node-label availability, and data sources.The datasets are ordered by descending usage frequency in Table 2.
  • Benchmark limitations: Studies using the same public sources often apply inconsistent graph construction rules, temporal discretizations, and evaluation protocols.These inconsistencies undermine reproducibility and cross-method comparability.
  • Benchmark limitations: TGB 2.0 standardizes preprocessing and evaluation for DHG datasets, but primarily targets link prediction and includes relatively few baseline methods.
  • Benchmark limitations: Fixed-split evaluation on only final-period edges fails to capture DHG evolution and tends to overestimate performance.The passage identifies broader task coverage and incremental learning protocols as future needs.

5 Conclusion and Future Directions

The survey consolidates DHG definitions, methods, applications, and benchmarks, then identifies efficiency, generalizability, and trustworthiness as persistent challenges. Future progress requires scalable, transferable, and more interpretable DHG learning approaches.

  • Conclusion: The survey identifies prohibitive computational complexity, limited transferability, and weak interpretability as three persistent DHG challenges.These challenges concern Web-scale computation, transferable representations, and distinguishing causal mechanisms from spurious correlations.
  • Efficiency: Quadratic complexity restricts current DHG methods on Web-scale graphs, while partial linearization retains an expressiveness-efficiency trade-off.Examples include event patching and incremental propagation.
  • Generalizability: Task-specific supervised training leaves DHG methods without principled pre-training objectives for transfer across heterogeneous schemas and temporal distributions.Preliminary efforts use LLM-derived semantics or unified attention mechanisms, but do not resolve the broader gap.
  • Trustworthiness: Post-hoc subgraph identification lacks faithfulness guarantees and causal grounding, limiting interpretability for high-stakes DHG decisions.The survey points toward inherently interpretable architectures and structural causal models.
Loading 2609.04779v1…