Source-linked AI summary
An Attention-based Collaboration Framework for Multi-View Network Representation Learning
Meng Qu, Jian Tang, Jingbo Shang, Xiang Ren, Ming Zhang, Jiawei Han
TL;DR
The paper studies how to learn robust node representations when networks contain multiple proximity-defined views, extending methods that typically focus on a single view. It proposes attention-based collaboration, in which views vote with learned weights, and reports better performance than single-view and competing multi-view approaches on two tasks.
Problem
Existing network representation methods usually model a single proximity type, although real networks contain multiple views whose information can be sparse, biased, and unequally informative.
Method
The approach learns view-specific representations, combines them through collaborative voting, and uses attention to learn view weights for each node from a few labeled data.
Results
The approach outperforms state-of-the-art single-view and competitive multi-view methods on multi-label node classification and link prediction.
Takeaways & Limitations
Multi-view network representation learning can use collaboration and attention-based weighting to produce robust representations across views.
Takeaways & Limitations
The framework uses context representations shared across different views, although the paper also evaluates a variant with view-specific context representations.
Abstract
from arXiv · showhide
Learning distributed node representations in networks has been attracting increasing attention recently due to its effectiveness in a variety of applications. Existing approaches usually study networks with a single type of proximity between nodes, which defines a single view of a network. However, in reality there usually exists multiple types of proximities between nodes, yielding networks with multiple views. This paper studies learning node representations for networks with multiple views, which aims to infer robust node representations across different views. We propose a multi-view representation learning approach, which promotes the collaboration of different views and lets them vote for the robust representations. During the voting process, an attention mechanism is introduced, which enables each node to focus on the most informative views. Experimental results on real-world networks show that the proposed approach outperforms existing state-of-the-art approaches for network representation learning with a single view and other competitive approaches with multiple views.
1 INTRODUCTION
The paper addresses network representation learning when multiple complementary proximity types define different views, rather than assuming a single view. It proposes attention-based collaboration so views can contribute unequally to robust node representations.
- Motivation: Network embedding maps high-dimensional, sparse network structure into low-dimensional node vectors that preserve proximities for downstream tasks.The paper cites node classification, link prediction, and node visualization as applications.
- Motivation: Multiple proximity types, such as co-authorship and citation, create network views that are individually sparse and biased.The paper also gives following-followee, reply, retweet, and mention relationships as social-media examples.
- Problem: Existing multi-view methods often treat views equally, although their importance and information quality can differ.The paper motivates automatically inferring view weights during integration.
- Approach: The proposed approach learns view-specific node representations, combines them for voting, and uses attention to infer node-specific view weights from a few labeled data.The model is trained efficiently through backpropagation.
- Results: Experiments on multi-label node classification and link prediction report that the approach outperforms single-view state-of-the-art methods and competitive multi-view methods.The contribution summary describes evaluations on several multi-view networks and two tasks.
2 PROBLEM DEFINITION
The paper formalizes multi-view network embedding as learning low-dimensional node representations that remain robust and consistent across multiple proximity-defined views. It motivates collaboration and unequal view weighting because views can differ in quality.
- Definitions: An information network is represented as G = (V, E), where weighted edges encode relationships between objects.Each edge weight w_uv > 0 indicates relationship strength.
- Definitions: Network embedding learns a low-dimensional vector x_v ∈ R^d for each node while preserving node proximities.The embedding dimension satisfies d ≪ |V|.
- Definitions: A view is derived from one proximity or relationship type, while multiple proximity types yield a multi-view network.Examples include following, retweet, and mention relationships in social media.
- Problem Definition: Multi-view network embedding learns robust node representations consistent across K views in a low-dimensional space.The problem is denoted G = (V, E_1, E_2, . . . , E_K).
- Problem Definition: The desired approach promotes collaboration among views and weights them differently during voting because their information quality differs.The next section introduces the proposed approach for this purpose.
3 MULTI-VIEW NETWORK EMBEDDING
The approach learns view-specific node representations, combines them into robust representations, and uses attention to learn node-specific view weights. Its objective jointly supports collaboration across views, attention-based voting, and efficient predictive use of the resulting representations.
- 3 MULTI-VIEW NETWORK EMBEDDING: Existing multi-view methods may fail to promote collaboration between views or assign proper weights when combining their information.The proposed framework addresses both limitations during training.
- 3 MULTI-VIEW NETWORK EMBEDDING: The overall objective combines the collaboration objective Ocollab with the attention objective Oattn.The parameter η controls the weight of the regularization term in the integrated objective.
- 3.1 Collaboration Framework: The collaboration framework learns view-specific representations that preserve each view’s node proximities while integrating them into a robust representation.A shared context representation places view-specific representations in the same semantic space.
- 3.1 Collaboration Framework: Robust representations are weighted combinations of view-specific representations, with node-specific weights allowing each node to focus on informative views.The weights are learned as regularization coefficients and used as voting weights.
- 3.2 Learning the Weights of Views through Attention: Attention learns view weights with a softmax from concatenated view-specific representations and view feature vectors, then backpropagates predictive error through the robust representations.The method can learn weights using limited labeled data for tasks such as node classification and link prediction.
- 3 MULTI-VIEW NETWORK EMBEDDING: The algorithm’s time complexity includes learning view-specific representations, robust representations, and voting weights; the first process takes O(|E|dN) time.The paper states that the approach is efficient for most real-world networks because their edge counts are usually small.
4 EXPERIMENT
The experiments evaluate multi-view representation learning on five real-world networks using node classification and link prediction, with comparisons against single-view, merged-view, concatenated-view, clustering, and matrix-factorization methods.
- Datasets: The evaluation uses five networks: DBLP, Flickr, PPI, YouTube, and Twitter, with the first three supporting node classification and the last two supporting link prediction.
- Baselines: The compared methods include single-view LINE and node2vec, merged and concatenated node2vec variants, CMSC, MultiNMF, and MultiSPPMI.
- Settings: Node representations use dimension 100 for most approaches, while node2vec-concat uses dimension 100K, where K is the number of views.
- Evaluation: The node classification evaluation reports quantitative results in Table 2, where MVE-NoAttn outperforms baselines, attention further improves results, and removing collaboration decreases them.
- Evaluation: The link prediction evaluation reports quantitative results in Table 3, where MVE achieves the best results through collaboration and attention.
4.2 Qantitative Results
Across node classification and link prediction, MVE outperforms the evaluated baselines, while ablations show benefits from both view collaboration and attention-based weight learning.
- Node Classification: MVE-NoAttn already outperforms all baseline approaches on node classification, despite not using labeling information to learn voting weights.
- Node Classification: Attention-based weight learning further improves node classification results over MVE-NoAttn.
- Node Classification: Removing collaboration produces inferior node classification results, indicating that collaboration improves performance in this evaluation.
- Link Prediction: On link prediction, merging views improves node2vec results on the evaluated datasets because the views are comparable and complementary there.
- Link Prediction: MVE outperforms all link-prediction baselines, while removing collaboration or attention-based weight learning lowers performance.
4.3 Performances w.r.t. Data Sparsity
The sparsity analysis groups nodes by degree and compares MVE with node2vec-merge and MVE-NoCollab, finding the strongest advantage for MVE among low-degree, sparse nodes.
- Grouping by sparsity: Nodes are grouped by degree to represent different levels of data sparsity, with high-degree groups denser and low-degree groups sparser.
- Overall finding: Overall, MVE performs especially well in the rightmost, more-sparse groups.
- Compared methods: The comparison evaluates robust representations from MVE, node2vec-merge, and MVE-NoCollab across node groups.
- DBLP: On DBLP, all three models perform poorly in high-degree groups, while MVE significantly outperforms node2vec-merge and MVE-NoCollab in low-degree groups.
- YouTube: On YouTube, the models perform similarly in high-degree groups, whereas MVE outperforms both alternatives in low-degree groups.
4.4 Analysis of the Learned Attentions (Weights) Over Views
The learned attention weights generally favor views whose individual representations perform better. Attention patterns also vary across semantic author groups, enabling nodes to emphasize the views most informative for them.
- Views with better individual performance generally receive higher average attention weights.The reported relationship is positive across the examined views.
- The DBLP case study compares learned attentions among authors in hardware, programming language, data mining, and machine learning.The four groups are abbreviated HW, PL, DM, and ML.
- Authors in data mining and machine learning attend less to text similarity and author citation, focusing more on co-authorship.The paper attributes this pattern to overlap in terms and cited papers across those areas.
- MVE learns attention weights so different nodes can focus on the most informative views.The attention mechanism assigns view weights during voting rather than treating all views identically.
4.5 Parameter Sensitivity
Performance is stable across a broad intermediate range of η but declines when the regularization weight becomes too large. Learning attention weights consistently outperforms equal weighting and converges with few labeled nodes.
- η sensitivity: η balances preserving single-view proximities against agreement among views.It controls the regularization term in the collaboration framework.
- η sensitivity: Performance improves as η increases from 0, remains stable over (0.025, 0.1), and drops when η becomes too large.A large η forces views to agree fully and ignores their differences.
- Number of labeled nodes: MVE consistently outperforms MVE-NoAtn, which assigns equal weights to views, on the examined datasets.The comparison evaluates the effect of learning voting weights from labeled nodes.
- Number of labeled nodes: MVE requires only a very small number of labeled nodes to converge.This convergence behavior is reported for both examined datasets.
4.6 Efficiency Study
MVE has running times close to LINE and node2vec while adding attention-weight learning with limited overhead. On a large Twiter network, training takes less than 15 minutes.
- MVE has running times close to LINE and node2vec on both examined datasets.The efficiency comparison also includes MVE-NoAten, the variant without learned voting weights.
- Less than 15% of MVE’s total running time is spent learning voting weights on both datasets.This compares MVE with MVE-NoAtn.
- MVE training takes less than 15 minutes on the Twiter dataset with more than 300 thousand nodes and 100 million edges.The paper presents this as evidence of efficiency on a large network.
4.7 Case Study
The case study uses DBLP author representations to illustrate how view-specific and robust representations differ. It examines the most similar authors returned for a query author.
- The case study compares view-specific and robust node representations in the DBLP author network.View-specific representations preserve proximities from individual views, while robust representations combine information across views.
- For a query author, the study lists the most similar authors under the different representations.This provides an illustrative comparison of the representations’ retrieved neighbors.
5 RELATED WORK
Prior network-representation methods focus on single-view networks, while multi-view learning integrates information across views. This paper differs by using attention to learn node-specific voting weights.
- Network representation learning: Existing scalable network-representation methods exploit network structure but focus on a single view.
- Efficiency: The approach has close running time to LINE and node2vec, while learning view weights takes less than 15% of running time on both datasets.
- Multi-view learning: Multi-view learning seeks representations from multiple views, but existing approaches commonly assign equal weights to all views.
- Attention-based models: The proposed approach applies attention to learn different voting weights for different nodes across views.
- Attention-based models: Attention-based models infer the importance of different training-data components and focus learning on the most informative parts.
6 CONCLUSIONS
The paper studies multi-view node representation learning through collaboration and attention-based voting, evaluating the framework on real-world networks and two tasks. It reports effective and efficient performance and identifies heterogeneous information networks as a future application.
- Conclusions: The framework combines multiple views to vote for robust node representations and automatically learns view weights using attention.
- Conclusions: Experiments on five real-world multi-view networks demonstrate effectiveness and efficiency on node classification and link prediction.
- Future work: A future direction is applying the framework to heterogeneous information networks, where meta-paths define multiple proximity views.
- Empirical analysis: Table 5 compares nearest neighbors produced by view-specific and robust node representations on DBLP.