Attributed Social Network Embedding

Lizi Liao; Xiangnan He; Hanwang Zhang; Tat-Seng Chua

Source-linked AI summary

Attributed Social Network Embedding

Lizi Liao, Xiangnan He, Hanwang Zhang, Tat-Seng Chua

arXiv:1705.04969v1May 14, 2017cs.SI

TL;DR

Existing network embedding methods largely focus on structure, despite rich social-network attributes that reflect homophily. SNE preserves structural and attribute proximity in a neural framework, outperforming other representation-learning methods on link prediction and node classification. The authors identify multimodal, supervised, and temporal extensions as future directions.

Problem
Existing network embedding methods primarily leverage structure, while social networks also contain rich attributes that affect social ties through homophily.
Method
SNE learns node representations by aggregating ID embeddings for structural proximity and attribute embeddings for attribute proximity within a multilayer neural network.
Results
SNE achieves superior performance to other representation-learning methods on link prediction and node classification across extensive experiments on four datasets.
Takeaways & Limitations
Integrating structural and attribute information yields more informative representations for social network data.
Takeaways & Limitations
The work focuses on attribute modeling and leaves multimodal, supervised, and temporal extensions for future work.

Abstract

from arXiv · show

Embedding network data into a low-dimensional vector space has shown promising performance for many real-world applications, such as node classification and entity retrieval. However, most existing methods focused only on leveraging network structure. For social networks, besides the network structure, there also exists rich information about social actors, such as user profiles of friendship networks and textual content of citation networks. These rich attribute information of social actors reveal the homophily effect, exerting huge impacts on the formation of social networks. In this paper, we explore the rich evidence source of attributes in social networks to improve network embedding. We propose a generic Social Network Embedding framework (SNE), which learns representations for social actors (i.e., nodes) by preserving both the structural proximity and attribute proximity. While the structural proximity captures the global network structure, the attribute proximity accounts for the homophily effect. To justify our proposal, we conduct extensive experiments on four real-world social networks. Compared to the state-of-the-art network embedding approaches, SNE can learn more informative representations, achieving substantial gains on the tasks of link prediction and node classification. Specifically, SNE significantly outperforms node2vec with an 8.2% relative improvement on the link prediction task, and a 12.7% gain on the node classification task.

1 INTRODUCTION

Social network embedding must incorporate both network structure and rich actor attributes because attribute homophily shapes social ties. SNE preserves structural and attribute proximity through a neural framework, and experiments evaluate it on link prediction and node classification.

Existing network embedding methods primarily leverage structural information, although social actors also have rich attributes such as demographics, text, and labels.
Attribute-based ordering of a Facebook friendship matrix reveals dense blocks among users sharing attributes, supporting attribute homophily.
SNE aggregates ID embeddings preserving structural proximity with attribute embeddings preserving attribute proximity.A multilayer neural network models interactions between features.
The framework aims to learn more informative node representations by integrating network structure and attributes.
Experiments on four datasets evaluate SNE using link prediction and node classification, with empirical results and case studies demonstrating effectiveness.

2 RELATED WORK

Related work covers homophily studies, classical and modern network embedding, attribute-aware methods, and semi-supervised extensions. SNE focuses on modeling attributes within a unified embedding framework while leaving supervised extensions for future work.

2.1 Attribute homophily in Social Networks: Homophily research reports that similar users tend to form friendships, and that social ties involve both structural and attribute-based processes.
2.2 Network Embedding: Classical methods construct affinity graphs from node features and embed them using leading eigenvectors.
2.2 Network Embedding: Modern network embedding methods generate node sequences with random walks and use language-model objectives to learn embeddings.
2.2 Network Embedding: Attribute-aware approaches have incorporated text or labels, but some methods handle only text attributes or combine separately learned embeddings.
2.2 Network Embedding: SNE leaves semi-supervised extensions for future work and focuses on modeling attributes for network embedding.

3 DEFINITIONS

Social network embedding represents actors in a low-dimensional space using two information sources: links provide structural proximity, while attributes provide attribute proximity. SNE is designed to preserve both.

A social network is represented as G = (U, E, A), containing actors, links, and actor attributes; edges may also have connection-strength weights.
The embedding objective projects social actors into a low-dimensional space that captures both network structure and attributes.
Structural proximity is evidenced by links, including direct links and indirect proximity through shared contexts.
Random walks generate contexts, with node2vec balancing breadth-first and depth-first sampling; both direct neighbors and context nodes are called neighbors.
Attribute proximity is evidenced by the intersection of two actors’ attributes.
Enforcing attribute proximity models homophily by placing actors with similar attributes close together in the embedding space.

4 PROPOSED METHOD

SNE jointly models network structure and heterogeneous attributes, combining structural and attribute representations through early fusion in a deep architecture. Its parameter settings recover node2vec and SVD++ as special cases, while added depth captures nonlinear interactions.

Deep proximity modeling: SNE estimates node proximity with a multilayer neural network whose hidden layers apply learned transformations and nonlinear activation functions.Each node has separate representations for its input and neighbor roles, which can be combined for downstream applications.
Fusion strategy: Early fusion lets attribute modeling complement structure modeling while jointly optimizing their parameters.This differs from late fusion, where separate models are trained independently and combined afterward.
Framework design: SNE models structural proximity from node IDs and attribute proximity from generic feature vectors, integrating both through early fusion.The framework uses separate structural and attribute components whose representations are combined at the input layer.
Relations to prior models: Setting λ=0 and disabling nonlinear and hidden transformations exactly recovers node2vec, while W(1)=[I, I] further recovers SVD++.These connections position SNE as a deeper extension of related shallow models.
Relations to prior models: SNE deepens shallow models to capture interactions between network structure and attributes, increasing expressiveness for complex nonlinear data.The paper presents this relationship as motivation for the framework design.

5 EXPERIMENTS

The experiments evaluate SNE on four publicly accessible social-network datasets. They ask whether SNE improves node representations, why it improves them, and whether deeper hidden layers help.

Experimental scope: Experiments use four publicly accessible social-network datasets to evaluate the proposed framework.
Research questions: RQ1 compares SNE with state-of-the-art network embedding methods on the quality of learned node representations.
Research questions: RQ2 investigates the reasons for SNE’s better representations, while RQ3 tests whether deeper hidden layers improve social-network embeddings.

5.1 Experimental Setup

The experiments evaluate SNE on four social-network datasets using link prediction and node classification, comparing it with structure-only and attribute-aware embedding baselines.

Datasets: Experiments use four public datasets spanning Facebook friendship networks and academic citation networks.The friendship datasets provide user attributes, while the citation datasets use paper title contents as attributes.
Evaluation Protocols: Link prediction holds out 10% of links for testing and 10% for validation, trains on the remaining 80%, and evaluates ranking with AUROC.Negative instances are sampled from non-existing links; higher AUROC indicates better performance.
Evaluation Protocols: Node classification trains LIBLINEAR classifiers using representations learned without class labels, varying labeled training data across ρ ∈{10%, 30%, 50%}.Results are averaged over 10 repetitions using Macro-F1 and Micro-F1.
Baselines: Compared methods include node2vec, LINE, TriDNR, and attribute-augmented node2vec+ and LINE+ variants.The augmented variants concatenate learned node representations with attribute feature vectors for fair comparison with SNE.
Parameter Settings: SNE experiments use softsign hidden-layer activations, mini-batch Adam optimization, and a searched concatenation parameter λ.The embedding dimension is set to 128, and λ is searched over the same range as TriDNR’s text weight.

5.2 Quantitative Analysis (RQ1)

Across link prediction and node classification, SNE performs best and benefits from incorporating attributes, especially under sparse network structure. Its early-fusion design consistently outperforms alternative attribute-integration strategies, while excessive attribute weighting can hurt performance.

Link Prediction: SNE achieves the best link-prediction performance among all methods and remains significantly better than node2vec and LINE with only half the links.Its performance is more stable as training links decrease, particularly where DBLP and CITESEER contain less link information.
Link Prediction: Attribute integration improves performance, but simple concatenation in node2vec+ and LINE+ yields only modest gains over their structure-only counterparts.The result motivates a more principled approach to incorporating attributes into network embedding.
Link Prediction: SNE consistently outperforms TriDNR, which separately trains structure- and attribute-based models, whereas SNE uses early fusion to capture structure–attribute interactions.The authors characterize separate training as potentially sub-optimal for leveraging attributes.
Node Classification: SNE ranks first across all node-classification settings, with all improvements statistically significant according to a one-sample paired t-test at p < 0.05.The performance ordering is SNE, TriDNR, node2vec+/LINE+, then structure-only node2vec and LINE.
Node Classification: SNE learns effective representations for node classification without using ground-truth labels during the network-embedding process.The paper attributes this to jointly modeling network structure and attributes.
Impact of λ: The largest attribute-related improvements occur on DBLP and CITESEER, including a dramatic CITESEER gain as λ increases from 0.0 to 0.2.The authors connect this pattern with the lower link information in those datasets and attribute modeling’s ability to alleviate link sparsity.
Impact of λ: The pure structure SNE model with λ = 0.0 outperforms node2vec on both link prediction and node classification.With the same p and q settings, the authors attribute this improvement to non-linearity from SNE’s hidden layers.

5.3 Qualitative Analysis (RQ2)

SNE retrieves papers more relevant to a query than structure-only methods in a DBLP case study, supporting the value of combining network structure with attributes.

Qualitative retrieval: SNE’s three retrieved papers all concern dynamic social network analysis and community structures, making them relevant to the query paper.The query paper is a well-cited KDD 2006 paper on group formation in large social networks.
Qualitative retrieval: Structure-only methods return less similar papers: node2vec favors highly cited papers, while LINE misses group or community topics.The cited explanation attributes these patterns to random-walk popularity bias for node2vec and limited proximity modeling for LINE.
Qualitative retrieval: Using both network structure and attributes benefits retrieval of similar nodes compared with pure structure-based methods.The comparison is qualitative and deliberately uses a popular query node, which favors structure-based methods by reducing sparsity.

5.4 Experiments with Hidden Layers (RQ3)

Increasing SNE’s hidden-layer depth improves performance, but deeper fully connected models require more training time and may become difficult to optimize.

Performance by depth: More hidden layers improve SNE’s link prediction and node classification performance on DBLP, with similar results reported on other datasets.The experiment retunes hyperparameters for each setting and equalizes the final hidden-layer size across models.
Depth–cost trade-off: A three-layer SNE takes 81.9 seconds per epoch, compared with 25.6 seconds for a one-layer SNE on the reported server.The deeper architecture therefore increases training cost substantially.
Depth–cost trade-off: Deeper fully connected SNE models can become difficult to optimize, overfit, or degrade, while diminishing improvements motivate modern architectures as future work.The paper specifically mentions residual units and highway networks as possible remedies.
Nonlinear interactions: Adding one hidden layer significantly improves performance over no hidden layer, demonstrating the usefulness of nonlinear structure–attribute interactions.The comparison replaces the softsign activation with an identity function to examine nonlinear modeling.

6 CONCLUSION

The paper concludes that social network embeddings should model both structural and attribute information. SNE uses a deep neural architecture and achieves superior link prediction and node classification performance in extensive experiments.

Conclusion: SNE embeds social networks by preserving both structural proximity and attribute proximity.Its deep neural architecture models complex interrelations between structural information and attributes.
Conclusion: Extensive experiments show that SNE learns informative representations and outperforms other representation learning methods on link prediction and node classification.The conclusion reports superiority across both evaluation tasks without specifying dataset-level results here.
Future directions: The paper identifies multimodal fusion, supervised variants, temporal modeling, and hashing-based efficiency improvements as future directions.These directions target additional data modalities, task-specific embeddings, evolving networks, and large-scale industrial use.