Source-linked AI summary
Representation Learning for Attributed Multiplex Heterogeneous Network
Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou, Jie Tang
TL;DR
Existing network embedding methods do not adequately cover large networks with multiple node and edge types plus node attributes. The paper formalizes AMHEN embedding and proposes GATNE with transductive and inductive models, achieving significant link-prediction gains and successful Alibaba deployment.
Problem
Existing methods mainly focus on single-typed networks, whereas real applications contain multiple node and edge types, rich attributes, partial observations, and massive scale.
Method
GATNE formally defines AMHEN embedding and provides transductive and inductive models with efficient, scalable learning algorithms.
Results
5.99-28.23% lift by F1 scores was reported on Alibaba over state-of-the-art methods, with p ≪0.01 in a t−test; offline A/B tests confirmed Alibaba recommendation effectiveness and efficiency.
Takeaways & Limitations
The framework provides a unified representation-learning approach for attributed multiplex heterogeneous networks and has been used in Alibaba’s recommendation system.
Takeaways & Limitations
GATNE-T cannot handle unobserved data, motivating the inductive GATNE-I extension.
Abstract
from arXiv · showhide
Network embedding (or graph embedding) has been widely used in many real-world applications. However, existing methods mainly focus on networks with single-typed nodes/edges and cannot scale well to handle large networks. Many real-world networks consist of billions of nodes and edges of multiple types, and each node is associated with different attributes. In this paper, we formalize the problem of embedding learning for the Attributed Multiplex Heterogeneous Network and propose a unified framework to address this problem. The framework supports both transductive and inductive learning. We also give the theoretical analysis of the proposed framework, showing its connection with previous works and proving its better expressiveness. We conduct systematical evaluations for the proposed framework on four different genres of challenging datasets: Amazon, YouTube, Twitter, and Alibaba. Experimental results demonstrate that with the learned embeddings from the proposed framework, we can achieve statistically significant improvements (e.g., 5.99-28.23% lift by F1 scores; p<<0.01, t-test) over previous state-of-the-art methods for link prediction. The framework has also been successfully deployed on the recommendation system of a worldwide leading e-commerce company, Alibaba Group. Results of the offline A/B tests on product recommendation further confirm the effectiveness and efficiency of the framework in practice.
1 INTRODUCTION
The paper targets attributed multiplex heterogeneous networks, where multiple node and edge types coexist with node attributes, and introduces GATNE to support scalable transductive and inductive embedding learning. Experiments across four datasets report significant gains over state-of-the-art methods, including deployment in Alibaba’s recommendation system.
- Motivation: Existing network embedding methods mainly address homogeneous networks, while real applications combine heterogeneous nodes, multiple edge types, and rich node attributes.These settings are common in applications such as e-commerce.
- Challenges: AMHENs present challenges from multiplex edges, partial observations including cold-start cases, and networks containing billions of nodes and edges.Different relationships must contribute to unified embeddings while incomplete observations require inductive capability.
- Approach: GATNE is a framework for embedding attributed multiplex heterogeneous networks with both transductive and inductive learning settings.The framework formally defines the problem and is designed for rich attributed information and multiplex topology.
- Approach: GATNE develops efficient learning algorithms intended to handle hundreds of million nodes and billions of edges.Scalability is presented as a core design objective of the framework.
- Results: ∼5.99-28.23% lift by F1 scores was achieved on Alibaba over state-of-the-art methods, with p ≪0.01 in a t−test.Evaluations covered Amazon, YouTube, Twitter, and Alibaba, followed by deployment in Alibaba’s recommendation engine and offline A/B testing.
2 RELATED WORK
Related work spans homogeneous network embedding, heterogeneous network embedding, multiplex network embedding, and attributed network embedding. Prior methods address subsets of these settings, motivating a unified treatment of their combination.
- Network Embedding: Homogeneous network embedding methods include DeepWalk, LINE, node2vec, NetMF, and NetSMF, while GCN and GraphSAGE represent graph neural network approaches.GraphSAGE specifically supports inductive learning on unobserved nodes through functional representations.
- Heterogeneous Network Embedding: Heterogeneous network methods such as HNE, PTE, metapath2vec, and HERec incorporate multiple node or edge types through contents, co-occurrence information, or meta-paths.These approaches construct representations for networks with heterogeneous contents and structures.
- Multiplex Heterogeneous Network Embedding: Multiplex heterogeneous network embedding methods model multiple proximity types or network views using projection, collaboration, or attention mechanisms.Examples include PMNE, MVE, and MNE.
- Attributed Network Embedding: Attributed network embedding preserves both network topology and node-attribute proximity in low-dimensional representations.Representative methods include TADW, LANE, and AANE.
3 PROBLEM DEFINITION
The paper defines an attributed multiplex heterogeneous network by combining node types, edge types, multiple edge-specific views, and node attributes. Its embedding objective assigns each node a unified low-dimensional representation for every edge type.
- Basic Network: A network G = (V, E) contains n nodes V and edges E, with each edge associated with a nonnegative weight indicating relationship strength.The network may be directed or undirected.
- Heterogeneous Network: A heterogeneous network uses mappings ϕ : V →O and ψ : E →R to assign node and edge types, respectively.The network is heterogeneous when |O| + |R| > 2.
- Attributed Network: An attributed network G = (V, E, A) associates each node vi with a feature vector xi, collected in A = {xi |vi ∈V}.The attribute representation augments the network structure with node features.
- AMHEN: An AMHEN partitions edges into views Er for edge types r ∈R, with E = Ð r ∈R Er and Gr = (V, Er, A), requiring |R| > 1.The same node and attribute sets are considered separately for each edge type or view.
- Embedding Problem: AMHEN embedding seeks a unified low-dimensional representation of each node v on every edge type r through fr : V →Rd, where d ≪|V|.The objective is defined over the attributed multiplex heterogeneous network G = (V, E, A).
4 METHODOLOGY
The methodology develops GATNE-T for transductive embedding and extends it to GATNE-I for inductive learning in attributed multiplex heterogeneous networks. Both models aggregate edge-type-specific information and learn embeddings through heterogeneous, meta-path-based skip-gram training.
- 4.1 Transductive Model: GATNE-T: GATNE-T splits each node’s embedding on an edge type into a shared base embedding and an edge embedding aggregated from neighbors.The base embedding is shared across edge types, while edge embeddings are formed through one or more aggregation layers.
- 4.1 Transductive Model: GATNE-T: Self-attention computes coefficients for combining a node’s edge embeddings across relationship types, followed by a trainable edge-type transformation.The coefficients are obtained from the concatenated edge-embedding matrix, and the transformed combination contributes to the overall embedding.
- 4.1 Transductive Model: GATNE-T: Theorem 4.1 shows that GATNE-T can approximate MNE representations with suitable attention and transformation parameters.The construction embeds the MNE transformation in the first s dimensions and uses additional dimensions to reproduce the required attention weights.
- 4.1 Transductive Model: GATNE-T: GATNE-T remains a generalization of MNE under aggregation when all neighborhood matrices and the activation function are invertible at every aggregation level.This expressiveness claim is conditional on invertibility of the neighborhood matrices and activation function.
- 4.2 Inductive Model: GATNE-I: GATNE-I replaces transductive node-specific parameters with attribute-based transformations for base and initial edge embeddings, addressing unobserved nodes.The inductive model defines the base embedding from node attributes and uses type-specific transformations, with an additional attribute term in the overall embedding.
- 4.3 Model Optimization: Both models are optimized by generating meta-path-based random walks separately for each network view and applying heterogeneous skip-gram to the resulting samples.The optimization procedure samples training pairs from these walks and uses negative sampling in the objective.
5 EXPERIMENTS
The experiments evaluate GATNE against embedding baselines on four multiplex or heterogeneous datasets, using link prediction and additional convergence, scalability, and recommendation tests. GATNE achieves strong predictive performance, while GATNE-I is especially effective on large-scale attributed data and practical recommendation workloads.
- Evaluation Setup: The evaluation covers Amazon, YouTube, Twitter, and Alibaba datasets, with link prediction as the main task.The study also examines parameter sensitivity, convergence, scalability, and offline recommendation performance.
- Performance Comparison: GATNE outperforms the evaluated baselines across the various datasets, with model behavior depending on attribute availability.GATNE-T performs better on Amazon with limited attributes, whereas GATNE-I performs best on Alibaba with abundant attributes; on YouTube and Twitter, the two models perform similarly.
- Convergence Analysis: GATNE-I converges faster and achieves better performance than GATNE-T on extremely large-scale real-world datasets.The convergence comparison is reported on Alibaba.
- Scalability Analysis: GATNE scales on distributed workers, with GATNE-I converging in less than 2 hours using 150 workers.Training time decreases significantly as workers are added; GATNE-I scales almost linearly below 150 workers, while GATNE-T approaches a limit beyond 100.
- Offline A/B Tests: GATNE-I improves recommendation hit-rate by 3.26% over MNE and 24.26% over DeepWalk in offline A/B tests.The deployment uses about 100 million users, 10 million items, and 10 billion daily interactions on Alibaba’s distributed cloud platform.
6 CONCLUSION
The paper formalizes attributed multiplex heterogeneous network embedding and proposes GATNE for both transductive and inductive settings. GATNE combines shared base and attribute embeddings with self-attentive edge embeddings, improving link prediction and supporting Alibaba deployment.
- GATNE addresses attributed multiplex heterogeneous network embedding in both transductive and inductive settings.
- GATNE-I splits each node embedding into base, edge, and attribute components.Base and attribute embeddings are shared across edge types, while edge embeddings aggregate neighborhood information using self-attention.
- GATNE significantly outperforms previous state-of-the-art methods on link prediction across multiple challenging datasets.
- The approach was deployed and evaluated on Alibaba’s recommendation system with reported scalability and effectiveness.
A APPENDIX
The appendix documents implementation notes, dataset and parameter details, comparison fairness, and future work.
- The appendix provides implementation notes for the proposed models.
- It describes the datasets and parameter configurations used for all methods.
- It discusses fair comparison questions and future work.
A.1 Implementation Notes
The implementation uses a single-server setup for four datasets and Alibaba’s distributed cloud platform for the full Alibaba dataset. The appendix also specifies software components, aggregation choices, model parameters, optimization, and code availability.
- Experiments use a single Linux server for four datasets and Alibaba’s distributed cloud platform for the full Alibaba dataset.The distributed platform contains thousands of workers.
- The single-server pipeline consists of random walk generation, model training, and evaluation using TensorFlow and scikit-learn components.
- Mean aggregation is reported because mean and pooling aggregators achieve similar experimental performance.A linear transformation is used for attribute functions in GATNE-I.
- The proposed models use embedding dimension d = 200, edge embedding dimension s = 10, and early stopping based on validation ROC-AUC.Training uses 20 walks per node, walk length 10, window size 5, five negative samples, and up to 50 epochs.
- Adam is used with learning rate lr = 0.001, while the offline A/B test uses N = 50.
- Code and dataset partitions are made available, including the Alibaba-S dataset.
A.2 Compared Methods
The compared methods are standardized with a common embedding size and shared random-walk and skip-gram settings.
- All compared methods use embedding size 200.
- Random-walk methods use 20 walks per node, length 10, and window size 5.
- Skip-gram training uses five negative samples per training pair and 100 iterations.
A.2.1 Network Embedding Methods. •
The evaluation includes DeepWalk, LINE, and node2vec as network embedding baselines, with implementations drawn from authors’ code repositories or reimplemented where necessary.
- DeepWalk is evaluated using code from the corresponding author’s GitHub for public and Alibaba-S datasets, and a reimplementation for Alibaba.
- LINE uses combined first- and second-order embeddings, each with dimension 100, and 1000 million samples.
- node2vec is included as an evaluated network embedding baseline.
A.2.2 Heterogeneous Network Embedding Methods. •
The heterogeneous-network baseline is metapath2vec, reimplemented to support arbitrary node types and specialized for the Alibaba dataset’s user–item network.
- metapath2vec is reimplemented in Python for networks with arbitrary node types, based on the original C++ codes.
- With one node type, metapath2vec degrades to DeepWalk on the three public datasets.
- For Alibaba, metapath2vec uses node types U and I, representing users and items, with meta-path schemes considered.
A.2.3 Multiplex Heterogeneous Network Embedding Methods. •
The study evaluates multiplex and heterogeneous-network embedding methods across Amazon, YouTube, Twitter, and Alibaba-related datasets while emphasizing standardized evaluation splits and future dynamic-network extensions.
- Multiplex methods: PMNE applies node2vec to multiplex networks through network aggregation, result aggregation, and layer co-analysis variants.
- Datasets and evaluation: The evaluation uses five datasets, with four represented by subgraphs because of single-server memory and computation limits.
- Datasets and evaluation: YouTube contains five interaction types among 15,088 users, while Amazon uses product metadata with co-viewing and co-purchasing links.
- Evaluation standardization: Different train, validation, and test separations prevent direct comparison of results across previous papers on the same datasets.
- Evaluation standardization: The paper advocates standardized train, validation, test, and full-dataset releases to improve cross-paper comparability and reproducibility.
- Future work: Dynamic attributed multiplex heterogeneous network representation learning is left as future work, including temporal node attributes, edge attributes, or network snapshots.