PaperlayerView exact revision on arXiv

Source-linked AI summary

Heterogeneous Deep Graph Infomax

Yuxiang Ren, Bo Liu, Chao Huang, Peng Dai, Liefeng Bo, Jiawei Zhang

arXiv:1911.08538v5cs.LGcs.SIstat.ML

TL;DR

Heterogeneous graphs make it challenging to learn node representations that preserve both attributes and structure, especially when labels are unavailable. HDGI uses meta-paths, graph convolution, and semantic-level attention, trained by maximizing local-global mutual information. The authors report strong classification and clustering performance, including competitiveness with supervised GNNs in node classification.

  • Problem

    Heterogeneous graphs combine diverse node and edge types with semantic relations, complicating representation learning that must preserve both node attributes and structure.

    Source: §I · pp. 1–2

  • Method

    HDGI uses meta-path-based graph convolution and semantic-level attention, maximizing local-global mutual information to learn unsupervised node representations.

    Source: §I · p. 2

  • Results

    HDGI representations are effective for classification and clustering and can outperform state-of-the-art supervised GNN models that use labels.

    Source: §I · p. 2

  • Takeaways & Limitations

    The learned representations support downstream node classification and clustering, with reported node-classification performance competitive with supervised methods.

    Source: §VI · p. 8

    The best global encoder for heterogeneous graphs of different scales and structures remains an open question.

    Source: 4) Different global representation… · p. 8

Abstract

from arXiv · show

Graph representation learning is to learn universal node representations that preserve both node attributes and structural information. The derived node representations can be used to serve various downstream tasks, such as node classification and node clustering. When a graph is heterogeneous, the problem becomes more challenging than the homogeneous graph node learning problem. Inspired by the emerging information theoretic-based learning algorithm, in this paper we propose an unsupervised graph neural network Heterogeneous Deep Graph Infomax (HDGI) for heterogeneous graph representation learning. We use the meta-path structure to analyze the connections involving semantics in heterogeneous graphs and utilize graph convolution module and semantic-level attention mechanism to capture local representations. By maximizing local-global mutual information, HDGI effectively learns high-level node representations that can be utilized in downstream graph-related tasks. Experiment results show that HDGI remarkably outperforms state-of-the-art unsupervised graph representation learning methods on both classification and clustering tasks. By feeding the learned representations into a parametric model, such as logistic regression, we even achieve comparable performance in node classification tasks when comparing with state-of-the-art supervised end-to-end GNN models.

I. INTRODUCTION

Unsupervised graph representation learning seeks node representations that preserve attributes and structure when labels are unavailable. HDGI models heterogeneous relation semantics with meta-paths and attention, and its learned representations perform well on classification and clustering.

I. INTRODUCTION

  • Unsupervised graph representation learning aims to preserve node attributes and graph topology in low-dimensional representations when labeled data are unavailable.These representations can be used with conventional sample-based machine-learning algorithms.Source: §I · p. 1
  • Heterogeneous graphs combine multiple node and edge types whose diverse connections convey semantics that homogeneous-graph models may struggle to handle.The bibliographic example includes Author, Paper, and Subject nodes, with Write and Belong-to edges.Source: §I · p. 2
  • HDGI uses meta-paths, graph convolution, semantic-level attention, and local-global mutual information maximization to learn unsupervised heterogeneous-graph representations.It fuses node attributes into representations while learning graph-level structural information.Source: §I · p. 2
  • The paper presents HDGI as the first model applying mutual-information maximization to heterogeneous-graph representation learning.It frames the approach as an unsupervised graph neural network using meta-path attention.Source: §I · p. 2
  • HDGI representations are effective for node classification and clustering and can outperform state-of-the-art supervised GNNs that use labels.The introduction reports this comparison without providing metric values.Source: §I · p. 2

A. Graph representation learning

Prior graph representation methods address structural information in different ways, while heterogeneous graphs add relation-type semantics. This paper formulates a target-node representation task and uses meta-path adjacency to encode semantic connections.

A. Graph representation learning

  • Homogeneous-graph representation methods include random-walk embeddings, local-global structure objectives, asymmetric-transitivity preservation, and matrix factorization, but do not address heterogeneity.For heterogeneous graphs, metapath2vec, HIN2Vec, HAN, SHNE, and HGAT use meta-paths, attention, or joint structural and semantic learning.Source: A. Graph representation learning · p. 2

B. Graph neural network

  • GNNs learn node features by aggregating neighborhood information, and DGI extends this approach with mutual-information maximization but applies only to homogeneous graphs.HDGI is presented as extending mutual-information-based GNNs to heterogeneous graphs.Source: B. Graph neural network · pp. 2–3

C. Mutual information

  • Mutual-information methods include DIM for computer vision, DGI for graph node representations, and a method for graph-level representations.Source: C. Mutual information · p. 3

III. PROBLEM FORMULATION

  • The learning task seeks low-dimensional representations that combine graph structure and node attributes for a selected target-node type.The representations support downstream tasks such as node classification and clustering.Source: §III · p. 3
  • A meta-path specifies composite relations between node types; this paper uses symmetric, undirected meta-paths to represent closeness among target-type nodes.PAP denotes papers written by the same author, while PSP denotes papers belonging to the same subject.Source: §III · p. 3
  • For each meta-path, an adjacency matrix records whether target-type nodes are connected by a path instance, providing input for HDGI's attention-based integration.Source: §III · p. 3

IV. HDGI METHODOLOGY

HDGI builds local node representations from meta-path-specific adjacency matrices, then aggregates them with semantic-level attention. A global encoder produces a graph summary, and a discriminator maximizes mutual information between that summary and positive nodes.

A. HDGI Architecture Overview

B. Meta-path based local representation encoder

The local encoder first learns node representations separately under each meta-path adjacency, then aggregates them across meta-paths with semantic-level attention.

1) Meta-path specific node representation learning:

B. Meta-path based local representation encoder

2) Heterogeneous graph node representation learning:

Semantic-level attention learns how much each meta-path contributes, then combines the meta-path-specific representations into the final heterogeneous graph representation.

2) Heterogeneous graph node representation learning:

  • A semantic attention layer learns weights for combining meta-path-specific representations into a more general node representation.Source: 2) Heterogeneous graph node… · p. 5
  • The learned meta-path weights are normalized with softmax, and their weighted representations form the final graph node representation.Source: 2) Heterogeneous graph node… · p. 5
  • Unlike HAN’s label-guided classification loss, HDGI’s binary cross-entropy guides attention weights by whether a node belongs to the original graph, avoiding known-label bias.Source: 2) Heterogeneous graph node… · p. 5

C. Global Representation Encoder

The global representation encoder produces a graph-level summary from node representations using one of three candidate aggregation functions.

C. Global Representation Encoder

D. HDGI Learning

HDGI trains a discriminator to distinguish local-global node-summary pairs from negative pairs, using binary cross-entropy to maximize mutual information.

1) Mutual information based discriminator:

2) Negative samples generator:

HDGI corrupts negative samples by shuffling node features while keeping meta-path adjacency intact. The evaluation covers three heterogeneous datasets: DBLP, ACM, and IMDB.

2) Negative samples generator:

  • The sampler shuffles rows of the initial feature matrix while keeping meta-path adjacency matrices unchanged, corrupting node-level connections.The paper motivates this choice as preserving structural information in generated negative samples.Source: 2) Negative samples generator · p. 6

V. EVALUATION

  • HDGI is evaluated against state-of-the-art competitive methods on three real-world heterogeneous graphs.The evaluation section introduces datasets and settings before reporting model comparisons.Source: §V · p. 6

A. Datasets

  • DBLP uses author nodes labeled across four research areas, profile bag-of-words features, and the APA, APCPA, and APTPA meta-paths.Source: A. Datasets · p. 6
  • ACM uses paper nodes in three subject classes, keyword TF-IDF features, and the PAP and PSP meta-paths.Source: A. Datasets · p. 6
  • IMDB uses movie nodes in three genres, TF-IDF-encoded movie features, and the MAM, MDM, and MKM meta-paths.Source: A. Datasets · p. 6

B. Experiment Setup

The experiments assess learned representations on node classification and clustering, comparing HDGI with feature-only, homogeneous-graph, and heterogeneous-graph baselines.

B. Experiment Setup

1) Compared Baselines:

  • Baselines include raw features, Metapath2vec, DeepWalk, DeepWalk with raw features, and DGI.Metapath2vec is tested across meta-paths with its best result reported; homogeneous methods are evaluated using meta-path-based adjacency matrices.Source: 1) Compared Baselines · pp. 6–7
  • HDGI-C uses graph convolution for local representations, while HDGI-A uses graph attention for local representations.Source: 1) Compared Baselines · p. 7
  • Heterogeneous-graph comparators include RGCN and HAN, while homogeneous-graph comparators include GAT and GCN.RGCN's classification loss is calculated on target-type nodes; supervised methods produce end-to-end classification results, unlike unsupervised methods that use a trained simple classifier.Source: 1) Compared Baselines · p. 7

2) Reproducibility:

HDGI performs strongly in node classification and learns effective representations for clustering; the experiments also specify implementation settings for reproducibility.

2) Reproducibility:

  • HDGI is implemented in PyTorch and optimized with Adam, using 512-dimensional HDGI-C representations and 64-dimensional HDGI-A representations with four attention heads.Both variants use an 8-dimensional q vector; experiments run on four GTX-1080ti GPUs.Source: 2) Reproducibility · p. 7

1) Node classification task:

  • HDGI-C outperforms the other unsupervised methods, HDGI performs better than homogeneous GCN and GAT, and HDGI is competitive with heterogeneous HAN in node classification.Source: 1) Node classification task · p. 7

2) Node clustering task:

  • Node clustering results support HDGI's effectiveness when representations account for graph structure, semantic information, and node-independent content.K-Means uses the number of target-node classes as K, without labels, and clustering is repeated 10 times for average NMI and ARI.Source: 2) Node clustering task · p. 7

3) HDGI-A vs HDGI-C:

HDGI-C outperforms HDGI-A across node-classification experiments, while clustering results also support the explanation that convolution captures broader hierarchical structure. Simple averaging performs best among global encoders, but only subtly, leaving encoder choice for larger, more complex graphs open.

3) HDGI-A vs HDGI-C:

  • HDGI-C outperforms HDGI-A in all node-classification experiments, and clustering results also verify the broader-structure analysis.Source: 3) HDGI-A vs HDGI-C · pp. 7–8
  • Graph convolution may capture local structure better because hierarchical dependencies let it reach beyond attention’s direct-neighbor limit.Source: 3) HDGI-A vs HDGI-C · pp. 7–8

4) Different global representation encoder functions:

VI. CONCLUSION

  • HDGI combines convolution-style GNNs, semantic-level attention, meta-paths, and local-global mutual-information maximization to learn heterogeneous-node representations.Source: §VI · p. 8
  • Across three heterogeneous graphs, learned representations work for node classification and clustering, and are competitive with supervised methods in node classification.Source: §VI · p. 8
  • The authors identify mutual-information maximization as a promising direction for unsupervised representation learning.Source: §VI · p. 9
Loading 1911.08538v5…