Source-linked AI summary
Unsupervised Attributed Multiplex Network Embedding
Chanyoung Park, Donghyun Kim, Jiawei Han, Hwanjo Yu
TL;DR
Existing embedding methods inadequately handle multiplex relations, node attributes, unsupervised training, and global graph properties. DMGI jointly integrates relation-specific embeddings through consensus regularization and a universal discriminator, with attention for relation importance; experiments report superiority over state-of-the-art methods across unsupervised and supervised tasks.
Problem
Existing methods often assume one relation type, overlook node attributes, require labels, or fail to model global graph properties in attributed multiplex networks.
Method
DMGI uses DGI-based local-global mutual-information learning with consensus regularization and a universal discriminator to jointly integrate relation-specific embeddings.
Results
DMGI and DMGIattn outperform state-of-the-art baselines on unsupervised and supervised tasks, including clustering, similarity search, and node classification.
Takeaways & Limitations
Attention can infer relation-type importance and support filtering unnecessary relation types as a preprocessing step.
Abstract
from arXiv · showhide
Nodes in a multiplex network are connected by multiple types of relations. However, most existing network embedding methods assume that only a single type of relation exists between nodes. Even for those that consider the multiplexity of a network, they overlook node attributes, resort to node labels for training, and fail to model the global properties of a graph. We present a simple yet effective unsupervised network embedding method for attributed multiplex network called DMGI, inspired by Deep Graph Infomax (DGI) that maximizes the mutual information between local patches of a graph, and the global representation of the entire graph. We devise a systematic way to jointly integrate the node embeddings from multiple graphs by introducing 1) the consensus regularization framework that minimizes the disagreements among the relation-type specific node embeddings, and 2) the universal discriminator that discriminates true samples regardless of the relation types. We also show that the attention mechanism infers the importance of each relation type, and thus can be useful for filtering unnecessary relation types as a preprocessing step. Extensive experiments on various downstream tasks demonstrate that DMGI outperforms the state-of-the-art methods, even though DMGI is fully unsupervised.
1 Introduction
Existing network embedding methods often assume a single relation type, overlook node attributes, or require labels, despite multiplex networks containing related relation types and useful attributes. DMGI addresses these gaps with an unsupervised framework that integrates relation-specific embeddings, captures global graph properties, and uses attention to assess relation importance.
- Multiplex networks contain multiple relation types, such as authorship, citation, and keywords in publication graphs.
- Related relation-specific graphs and node attributes can mutually help downstream tasks such as topic inference.
- Prior multiplex embedding methods often overlook node attributes, while attribute-aware methods may require expensive node labels for training.
- DMGI builds on DGI, which maximizes mutual information between local graph patches and the global graph representation while incorporating attributes, unsupervised training, and global properties.
- DMGI jointly integrates relation-specific embeddings using consensus regularization and a universal discriminator, while attention infers relation importance for filtering unnecessary relations.
- DMGI outperforms state-of-the-art attributed multiplex embedding methods on node clustering, similarity search, and especially node classification despite being fully unsupervised.
2 Problem Statement
An attributed multiplex network consists of multiple relation-specific graphs over shared nodes and attributes. The embedding task is to learn a d-dimensional vector for every node without using labels.
- An attributed multiplex network contains multiple relation-specific graphs sharing a node set and node-attribute matrix.
- Each relation type has its own adjacency matrix representing edges between the shared nodes.
- Unsupervised attributed multiplex network embedding learns a d-dimensional representation z_i for each node v_i without using labels.
3 Unsupervised Attributed Multiplex Network Embedding
DMGI extends Deep Graph Infomax to attributed multiplex networks by learning relation-specific embeddings and jointly integrating them through consensus regularization and a shared discriminator. The framework also supports attention over relation types and a semi-supervised extension.
- Deep Graph Infomax: DGI learns node representations by maximizing mutual information between local patches and a graph-level summary, using corrupted attributes to form negative samples.The encoder integrates node attributes and the discriminator distinguishes true patch-summary pairs from shuffled-attribute counterparts.
- Relation-type-specific Node Embedding: DMGI applies a relation-type-specific GCN encoder to each graph and produces node embeddings for every relation type.Each encoder maps node attributes and a relation-specific adjacency matrix to an embedding matrix.
- Joint Modeling and Consensus Regularization: Consensus regularization encourages relation-specific embeddings to agree with a shared consensus embedding while preserving joint multiplex modeling.The method introduces a consensus matrix Z and regularizes disagreements between each relation-specific embedding and Z.
- Joint Modeling and Consensus Regularization: A universal discriminator shares one scoring matrix across relation types so true patch-summary pairs receive comparable treatment regardless of the relation.The shared discriminator is optimized jointly with relation-specific losses and consensus regularization.
- Attention and Extensions: Attention weighting distinguishes relation types by their importance when generating final node embeddings, while average pooling is the efficient default aggregation scheme.The discussion motivates attention because different relation types can benefit particular downstream tasks differently.
- Attention and Extensions: DMGI includes a semi-supervised extension that predicts labeled-node classes from the consensus embedding using a single-layer classifier and cross-entropy loss.The fully unsupervised framework can incorporate a small amount of label information through an added supervised objective.
4 Experiments
The experiments evaluate DMGI and its attention variant on multiplex attributed-network datasets using unsupervised and supervised tasks. Results show broad gains over baselines, while attention can identify and remove unhelpful relation types.
- Datasets: The experiments use ACM, DBLP, and IMDB datasets, with research-topic classification for ACM and DBLP and movie classification for IMDB.The datasets were selected to support comparison with HAN.
- Evaluation metrics: Evaluation measures include NMI for node clustering, Sim@5 for similarity search, and Macro-F1 for node classification.Sim@5 is computed from the proportion of same-class nodes among each node’s five most similar neighbors.
- Overall evaluation: DMGI and DMGIattn outperform all state-of-the-art baselines on both unsupervised and supervised tasks, with larger improvements on unsupervised tasks.The evaluation covers node clustering, similarity search, and node classification.
- Overall evaluation: Attribute-aware multiplex methods generally outperform methods that neglect node attributes, even when raw attributes are concatenated to learned embeddings.This supports systematically incorporating attributes into the representation-learning model.
- Effect of the attention mechanism: Attention weights tend toward 0 or 1 and usually concentrate on one important relation type, but strongly favoring one relation can hurt when relation types perform similarly.The reported favored relations include PAP for ACM and DBLP, MDM for IMDB, and Bought-together for Amazon.
- Effect of the attention mechanism: Removing the apparently useless PATAP relation improves DMGIattn on DBLP, indicating that attention can filter unnecessary relation types before processing.For GCN and DGI, retaining all relation types remains best on the same comparison.
- Ablation study: Ablations show that node attributes and the consensus-regularization term are important, while average pooling performs comparably to alternative readout functions.The semi-supervised module mainly improves node classification, and shuffling adjacency rather than attributes deteriorates performance.
5 Related Work
Multiplex network embedding learns consensus node representations from multiple relation-specific graphs, while addressing their interrelationship and graph importance. Existing approaches integrate multiple graphs but may require labels.
- Multiplex Network embedding: A multiplex network contains multiple relation types among a single set of node types, making it a special type of heterogeneous network.Unlike heterogeneous networks, it does not require modeling semantics between different node types through meta-paths.
- Multiplex Network embedding: The central challenge is learning a consensus embedding for each node while accounting for interrelationships among multiple graphs.
- Multiplex Network embedding: Existing multiplex embedding methods primarily focus on integrating information from multiple graphs.
- Multiplex Network embedding: HAN applies graph attention separately to each graph, then merges graph-specific node representations according to each graph’s importance.
- Multiplex Network embedding: Existing methods either require labels for training or leave related limitations unaddressed.
6 Conclusion
The paper presents DMGI, an unsupervised method that jointly integrates embeddings from multiple relation types using consensus regularization and a universal discriminator. DMGIattn also uses attention to infer relation importance, and experiments verify the framework’s superiority on supervised and unsupervised tasks.
- 6 Conclusion: DMGI jointly integrates embeddings from multiple relation types through consensus regularization and a universal discriminator.
- 6 Conclusion: DMGIattn’s attention mechanism infers the importance of each relation type, facilitating multiplex-network preprocessing.
- 6 Conclusion: Experimental results verify the superiority of the proposed framework on both unsupervised and supervised tasks.