Source-linked AI summary
MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network
Hao Wang, Tong Xu, Qi Liu, Defu Lian, Enhong Chen, Dongfang Du, Han Wu, Wen Su
TL;DR
A single node vector may not represent users’ different similarities across behaviors. MCNE learns multiple conditional representations using masking, attention, and graph-neighbor information. The authors report significant improvements over state-of-the-art baselines and support for visualization and transfer learning.
Problem
A single vector represents only one similarity aspect, while social users can have different preference similarities across behaviors.
Source: §1 · p. 2Method
Results
Takeaways & Limitations
The learned conditional representations support visualization and transfer learning and are reported to have interpretability and robustness.
Source: §1 · p. 2, §6 · p. 9Exploring weight sampling is identified as an important direction for further work.
Source: §4.2.4 · p. 4
Abstract
from arXiv · showhide
Recently, the Network Representation Learning (NRL) techniques, which represent graph structure via low-dimension vectors to support social-oriented application, have attracted wide attention. Though large efforts have been made, they may fail to describe the multiple aspects of similarity between social users, as only a single vector for one unique aspect has been represented for each node. To that end, in this paper, we propose a novel end-to-end framework named MCNE to learn multiple conditional network representations, so that various preferences for multiple behaviors could be fully captured. Specifically, we first design a binary mask layer to divide the single vector as conditional embeddings for multiple behaviors. Then, we introduce the attention network to model interaction relationship among multiple preferences, and further utilize the adapted message sending and receiving operation of graph neural network, so that multi-aspect preference information from high-order neighbors will be captured. Finally, we utilize Bayesian Personalized Ranking loss function to learn the preference similarity on each behavior, and jointly learn multiple conditional node embeddings via multi-task learning framework. Extensive experiments on public datasets validate that our MCNE framework could significantly outperform several state-of-the-art baselines, and further support the visualization and transfer learning tasks with excellent interpretability and robustness.
1 INTRODUCTION
Network representation learning turns graph structure into low-dimensional node vectors, but one vector cannot capture users’ behavior-specific similarities. MCNE addresses this with conditional embeddings and reports improved benchmark performance alongside support for visualization and transfer learning.
1 INTRODUCTION
- NRL embeds network structure into low-dimensional node vectors used as features in applications including classification, link prediction, recommendation, and influence analysis.The introduction also surveys DeepWalk, Node2vec, LINE, SDNE, TriNRL, GCN, and PinSage.Source: §1 · p. 1
- A single vector represents only one similarity aspect, although users’ preferences can differ across behaviors, such as movies and books.For example, the woman is more similar to the child for movie preferences and to men for book preferences.Source: §1 · p. 2
- MCNE uses a binary mask to divide a node vector into conditional embeddings, then attention and neighborhood aggregation model interacting preferences.BPR learns preference similarity for each behavior, while multi-task learning jointly trains the conditional representations.Source: §1 · p. 2
- MCNE’s conditional representations outperform several state-of-the-art baselines and support visualization and transfer learning with interpretability and robustness.The paper describes these results on public datasets.Source: §1 · p. 2
2 RELATED WORK
Related work spans unsupervised structural embedding and task-specific representation learning. MCNE adapts binary masking to select embedding dimensions for different tasks.
2.1 Network Representation Learning
- Unsupervised NRL methods use random walks, node distances, or deep learning to preserve structural information, while supervised approaches incorporate task-specific signals.Examples include DeepWalk, node2vec, LINE, SDNE, TriDNR, GCN [12], and PinSage.Source: §2.1 · p. 2
2.2 Binary Neural Network
- Binary neural network methods simplify computation or storage through binary weights, and MCNE instead uses binarization to select relevant embedding dimensions for different tasks.The cited approaches include Matthieu et al. and Mohammad et al..Source: §2.2 · pp. 2–3
3 PROBLEM DEFINITION
The problem is to learn low-dimensional conditional user representations across behavior categories while preserving network structure and behavior similarity. The paper formalizes social networks and multi-category behavior records as inputs to this task.
3 PROBLEM DEFINITION
- A social network is represented as vertices and edges, with each edge assigned a nonnegative weight indicating relationship strength.In real social networks, vertices often represent users and edges represent friendship or trust; edge weights may be binary.Source: §3 · p. 3
- Multi-category user behavior is represented by one user-item record matrix per category, with entries indicating whether a user consumed an item.The matrices together form the set of behavior records across categories.Source: §3 · p. 3
- Because user similarity can differ across behavior categories, a single vector is unsuitable for representing all such relationships.The problem definition builds on the introduction’s example of differing movie and book preferences.Source: §3 · p. 3
- The formal task learns low-dimensional conditional vectors for users across behavior matrices, preserving network structure and category-specific behavior similarity.These are the stated properties of the conditional representations.Source: §3 · p. 3
4 MCNE: MULTIPLE CONDITIONAL NETWORK EMBEDDINGS
The section outlines a general model description, detailed component explanations, and model optimization.
4.1 Framework
MCNE jointly learns multiple conditional network representations by combining behavior-specific dimension masks, attention-weighted message passing, and preference learning across behaviors.
4.1 Framework
- MCNE uses a binary mask layer to select behavior-relevant dimensions and attention to weight preferences between adjacent users during graph message passing.It aggregates multi-preference information to update node representations, then uses BPR and multi-task learning to jointly learn conditional representations.Source: §4.1 · p. 3
- Bayesian Personalized Ranking learns users’ preference similarity for each behavior, while a multi-task framework jointly learns their conditional network representations.Source: §4.1 · p. 3
- The framework iteratively aggregates information from local neighborhoods to update node representations and capture different preference similarities.Source: §4.1 · p. 3
4.2 Generating Multiple Conditional Network Representations
MCNE builds multiple conditional representations from an initial user vector, then uses behavior-specific masks and attention-weighted message passing to incorporate neighborhood preferences. Stacked layers capture information from higher-order neighbors before final conditional representations are produced.
4.2.1 Embedding Layer.
- The initial node representation is a low-dimensional comprehensive vector for user preferences across behaviors; user profiles such as age and gender could also serve as input features.The paper leaves inductive representation learning with profile features for future work.Source: §4.2.1 · p. 3
4.2.2 Binary Mask Layer.
- A binary mask partitions each layer’s node embedding into behavior-specific subspaces, with an additional category for preferences absent from the training data.The masks select embedding dimensions, and the mask variables are trained end-to-end despite hard thresholding.Source: §4.2.2 · p. 4
4.2.3 Multi-aspect Similarity Message Sending Operation.
- Attention scores conditional representations of adjacent users by behavior, then weights and combines them into edge messages carrying multiple preferences.More similar behaviors receive greater weight in the message.Source: §4.2.3 · p. 4
4.2.4 Multi-aspect Similarity Message Receiving Operation.
- The receiving operation samples fixed-size neighbor sets and uses average pooling; stacked layers aggregate higher-order preferences before a final mask yields behavior-specific representations.The authors identify exploring weight sampling as an important direction for future work.Source: §4.2.4 · p. 4, §4.2.5 · p. 5
4.3 Jointly Learning of Conditional Network Representations
MCNE learns preference similarity separately for each behavior using BPR, then jointly trains the conditional embeddings so related behaviors can share information.
4.3 Jointly Learning of Conditional Network Representations
- BPR trains each behavior’s preference similarity by ranking an interacted item above a randomly selected non-interacted item.The loss uses the behavior-specific record matrix and conditional user, positive-item, and negative-item embeddings.Source: §4.3 · p. 5
- MCNE treats each conditional embedding as a task and jointly learns the multiple behavior embeddings with a multi-task framework.C is the number of user behaviors and tasks, and Θ denotes all model parameters.Source: §4.3 · p. 5
- The binary mask layer and multi-task framework use a learnable subspace to share relevant information across behaviors, which can alleviate training-data sparsity and improve robustness.Source: §4.3 · p. 5
4.4 Model Optimization
MCNE optimization learns binary masks with real-valued updates and uses mini-batch neighbor sampling to make representation generation more efficient.
4.4 Model Optimization
- Mask weights are initialized from U(−0.5, 0.5), because fixed-value initialization did not achieve competitive experimental performance.Source: §4.4.1 · p. 5
4.4.1 Binary Mask Learning.
- The binary-mask procedure thresholds real-valued weights for propagation, updates those real-valued weights using gradients for the binary masks, constrains them to [−1, 1], and trains with Adam.The update approach is attributed to.Source: §4.4.1 · p. 5
4.4.2 Mini-batch Training.
- Mini-batch training forward-samples neighbors up to layer k+1 and computes only needed representations; the usual setting is k = 2 with |N1| · |N2| ≤ 50, which the authors say gives satisfactory performance.The authors state this sampling can improve optimization speed and that the resulting time complexity is acceptable for real-world applications.Source: §4.4.2 · pp. 5–6
5 EXPERIMENTS
Experiments on Ciao and Epinions compare MCNE with recommendation and network-representation baselines and evaluate its components. The reported results show improvements across behavior tasks, with variant comparisons supporting the binary mask and attention components.
5.1 Experimental Settings
- Experiments use Ciao and Epinions, trust-based social networks where users consume products from multiple categories; three representative behaviors are selected from each dataset.Source: §5.1.1 · p. 6
5.1.1 Datasets.
- Both datasets exclude users with fewer than two social links or fewer than two multi-category behavior records before statistics are reported.Source: §5.1.1 · p. 6
5.1.2 Baselines.
- Baselines span recommendation models BPR, LibFM, and NFM; network-representation methods Node2vec, LINE, GraphSage, and PinSage; and MCNE variants.Source: §5.1.2 · p. 6
5.1.3 Evaluation Protocols.
- Evaluation uses 70/10/20 train-validation-test splits, ranks test items among 100 sampled negatives, reports Recall@5 and NDCG@5, and averages 10 independent runs.MCNE uses two graph layers, layer dimensions [256, 128, 100], and neighbor samples; other listed settings include λ = 0.0001 and batch size 128.Source: §5.1.3 · p. 6, §5.1.4 · pp. 6–7
5.2 Experimental Results
- MCNE consistently achieves significant improvements over all baselines across multiple behavior tasks on both datasets.Table 2 reports Recall@5 and NDCG@5 comparisons for multiple behaviors on Ciao and Epinions; the supplied text gives no cell values.Source: §5.2.1 · p. 7, Table 2 · p. 7
5.2.1 Performance Comparison.
- NRL methods outperform BPR, LibFM, and NFM; task-specific Node2vec-B and LINE-B outperform their counterparts, and MCNE exceeds PinSage through cross-preference weighting and multi-task sharing.Source: §5.2.1 · p. 7
5.2.2 Performance with MCNE Variants.
- Compared with MCNE-F, MCNE performs better; the learned masks select relevant dimensions for each behavior while overlapping dimensions share relevant information.MCNE-F uses fixed disjoint masks and omits attention.Source: §5.1.2 · p. 6, §5.2.2 · p. 7
- The MCNE versus MCNE-A comparison supports the attention network’s role in modeling behavior-weight relationships; its learned weights may also help interpret user connections.The authors state that visualizing and integrating these weights into the interpretable part is future work.Source: §5.2.2 · p. 7
5.3 Further Experiments
Further experiments assess MCNE’s conditional representations through visualization and transfer learning, and examine sensitivity to embedding dimension and sampled-neighbor count.
5.3.1 Conditional Network Embedding Visualization.
- Figure 5 compares MCNE variants on Ciao and Epinions, while Figure 6 visualizes 1,000 Ciao users’ conditional embeddings, whose separation and overlap indicate discriminability and correlation.The embeddings are projected with t-SNE and colored by behavior.Source: §5.3.1 · pp. 7–8, Figure 6 · p. 8, Figure 5 · p. 8
5.3.2 Transfer Learning for MCNE.
- MCNE outperforms PinSage on the new task while updating only mask variables, whereas PinSage must retrain all model parameters.The transfer experiment adds two behavior records from Ciao and Epinions; Table 3 reports the comparison.Source: Table 3 · p. 8, §5.3.2 · p. 8
5.4 Parameter Sensitivity
- Embedding performance rises as dimension d increases from 50, then falls when dimensions become too large, which the authors attribute to sparsity and noise.The tested dimensions range from 50 to 250 in increments of 50.Source: §5.4 · p. 8
- On the beauty task, increasing sampled neighbors from 10 to 50 reduces the performance margin while rapidly increasing runtime; 20 neighbors balances the two.The same neighbor count is used in each layer.Source: §5.4 · p. 8
6 CONCLUSION
MCNE learns multiple behavior-conditional representations from a shared node embedding, using attention and adapted message passing to model preferences across high-order neighbors.
6 CONCLUSION
- A binary mask separates each user’s single embedding into conditional embeddings for different behaviors, while attention models interactions among aspects of similarity.Adapted message sending and receiving aggregates multi-aspect preference information from high-order neighbors.Source: §6 · p. 8
- Bayesian Personalized Ranking learns preference similarity for each behavior, and multi-task learning jointly trains the conditional node embeddings.The authors report significant performance improvements over state-of-the-art baselines and interpretability and robustness of the learned representations.Source: §6 · p. 9
- The authors identify combining node attributes as future work to improve MCNE’s performance and interpretability.Source: §6 · p. 9