PaperlayerView exact revision on arXiv

Source-linked AI summary

SHINE: Signed Heterogeneous Information Network Embedding for Sentiment Link Prediction

Hongwei Wang, Fuzheng Zhang, Min Hou, Xing Xie, Minyi Guo, Qi Liu

arXiv:1712.00732v1stat.MLcs.IRcs.LG

TL;DR

The paper addresses predicting sentiment-link signs when related text is unavailable, a gap left by content-focused sentiment classification. It builds a labeled heterogeneous Weibo dataset and introduces SHINE, which embeds sentiment, social, and profile networks; SHINE outperforms strong baselines across link prediction and node recommendation and retains decent cold-start performance.

  • Problem

    The paper studies how to predict positive or negative sentiment links without observing related content, where existing classifiers cannot identify links lacking prior textual evidence.

  • Method

    SHINE uses multiple deep autoencoders to learn low-dimensional user representations from sentiment, social, and profile networks and fuses them for sentiment prediction.

  • Results

    SHINE outperforms strong baselines by 8.8% to 16.8% in link-prediction Accuracy and by 17.2% to 219.4% in positive-node Recall@100 for node recommendation.

  • Takeaways & Limitations

    The experiments demonstrate effective use of social and profile side information and decent performance in the cold-start scenario.

  • Takeaways & Limitations

    SHINE depends on side information because target-network-only embedding models perform poorly for newly arrived nodes with little interaction.

Abstract

from arXiv · show

In online social networks people often express attitudes towards others, which forms massive sentiment links among users. Predicting the sign of sentiment links is a fundamental task in many areas such as personal advertising and public opinion analysis. Previous works mainly focus on textual sentiment classification, however, text information can only disclose the "tip of the iceberg" about users' true opinions, of which the most are unobserved but implied by other sources of information such as social relation and users' profile. To address this problem, in this paper we investigate how to predict possibly existing sentiment links in the presence of heterogeneous information. First, due to the lack of explicit sentiment links in mainstream social networks, we establish a labeled heterogeneous sentiment dataset which consists of users' sentiment relation, social relation and profile knowledge by entity-level sentiment extraction method. Then we propose a novel and flexible end-to-end Signed Heterogeneous Information Network Embedding (SHINE) framework to extract users' latent representations from heterogeneous networks and predict the sign of unobserved sentiment links. SHINE utilizes multiple deep autoencoders to map each user into a low-dimension feature space while preserving the network structure. We demonstrate the superiority of SHINE over state-of-the-art baselines on link prediction and node recommendation in two real-world datasets. The experimental results also prove the efficacy of SHINE in cold start scenario.

1 INTRODUCTION

The paper asks whether sentiment-link signs can be predicted without observing related content, using heterogeneous information to address sparse and unlabeled sentiment links. It introduces a labeled dataset and SHINE, then reports gains over baselines across link prediction and node recommendation.

  • Motivation: Sentiment links encode positive or negative attitudes from a generator to a recipient, unlike explicit friendship or follow links.Positive links express liking, trust, or support; negative links express dislike or disapproval.
  • Problem: Traditional sentiment classification cannot detect links when no relevant prior content is available, motivating prediction of unobserved sentiment-link signs.The paper frames this as important for services including personalized advertising, friend recommendation, public-opinion analysis, and polls.
  • Approach: The paper addresses missing labels by building a labeled Weibo sentiment dataset with entity-level extraction and adding social-relational and profile side information.These three information types form a heterogeneous network for sentiment prediction.
  • Approach: SHINE uses multiple deep autoencoders to learn nonlinear representations from sentiment, social, and profile networks, then fuses them for sentiment prediction.Its modules can flexibly incorporate or remove social and profile side information.
  • Results: 8.8% to 16.8% higher Accuracy occurred in link prediction, while positive-node Recall@100 gains ranged from 17.2% to 219.4% in node recommendation.These comparisons are reported against strong baselines on two real-world datasets.

2 RELATED WORK

Prior work spans signed-network analysis and network embedding, but existing embedding methods generally address only one network property at a time. This leaves signed, heterogeneous sentiment prediction insufficiently covered.

  • Signed networks: Signed-network research includes trust propagation, spectral analysis, social-media mining, signed-triad prediction, transfer learning, and positive-link exploitation.These approaches provide related techniques for signed-link prediction but do not by themselves resolve the full heterogeneous setting.
  • Network embedding: Classical and recent embedding methods learn low-dimensional representations through affinity graphs, random walks, structure-preserving objectives, or autoencoders.Examples include IsoMap, Laplacian Eigenmap, DeepWalk, LINE, node2vec, and SDNE.
  • Research gap: Existing methods are typically specialized for unsigned homogeneous, heterogeneous, attributed, or signed networks rather than their combination.The paper identifies this mismatch as the gap motivating SHINE for real-world signed heterogeneous sentiment networks.

3 DATASET ESTABLISHMENT

The paper constructs Weibo sentiment data by filtering tweets about celebrities, assigning sentiment polarity, and supplementing sentiment links with social and profile information. Sentiment extraction combines an emoticon-labeled lexicon, PMI-based scores, and contextual analysis, with manual validation.

  • Weibo tweets: The study collects 2.99 billion Weibo tweets from August 14, 2009 to May 23, 2014 and filters tweets containing celebrity names.Jieba part-of-speech tagging identifies words tagged as person names matching the celebrity list.
  • Weibo tweets: Each retained tweet receives a sentiment value from -1 to +1, and high-absolute-value tweets become triples (a,b,s), where s is +1 or −1 polarity.Here, a is the poster and b is the mentioned celebrity.
  • Side information: The dataset adds directed social-relation tuples (a,b), with a as follower and b as followee.This relation provides side information alongside sentiment links.
  • Side information: Ordinary-user profiles use one-hot gender and location attributes, while celebrity profiles use nine discretized attributes extracted from Microsoft Satori.Celebrity attributes include fields such as birthplace, nationality, specialization, and gender; ambiguous names and noise are removed.
  • Sentiment extraction: The sentiment lexicon assigns SO(word) = PMI(word,pos) − PMI(word,neд), normalizing scores to [−1, 1] after separating positive and negative tweet classes.The classes come from manually constructed emoticon-to-sentiment mappings, while Jieba supplies word segmentation.
  • Sentiment extraction: SentiCircle combines each tweet with its mentioned celebrity to calculate contextual sentiment in a polar-coordinate representation.The celebrity is placed at the origin and other terms are positioned using syntactic distance and contextual semantics.
  • Validation: Manual validation of 1,000 tweets found 95.2% precision for positive sentiment and 91.0% for negative sentiment.The sample contained 500 tweets in each class as tagged by the extraction method.

4 PROBLEM FORMULATION

The paper models sentiment prediction using three related network types: sentiment links, social links, and user-profile attributes. Given these heterogeneous networks, the task is to predict sentiments on unobserved user-to-user links.

  • Network Components: The heterogeneous information network is split into sentiment, social, and profile networks for problem formulation.The three single-type networks separately represent sentiment relations, social relations, and profile knowledge.
  • Network Components: Each sentiment link records whether user i expresses positive, negative, or unobserved sentiment toward user j.The sentiment network is directed, with link values +1, −1, or 0.
  • Network Components: Each social link records whether one user follows another in the directed social network.Social links take binary values indicating whether user i follows user j.
  • Network Components: The profile network is an undirected bipartite graph connecting users to attribute values they possess.Profile links are binary and connect users with possible values of user attributes.
  • Prediction Task: The prediction task is to infer sentiments on unobserved links between users from the sentiment, social, and profile networks.The formulation uses all three networks as input while targeting unobserved sentiment links in the sentiment network.

5 SIGNED HETEROGENEOUS INFORMATION NETWORK EMBEDDING

SHINE is an end-to-end framework that extracts user representations from sentiment, social, and profile networks and aggregates them for sentiment prediction. It uses deep autoencoders to compress network structure into low-dimensional representations while emphasizing observed nonzero relations during reconstruction.

  • Framework: SHINE combines sentiment extraction and heterogeneous-network construction, user-representation extraction, and representation aggregation for sentiment prediction.The framework has three major components spanning input construction, embedding, and prediction.
  • Sentiment Network Embedding: A sentiment adjacency vector contains each user’s global incoming and outgoing sentiment information but is too long and sparse for direct processing.The vector includes both s_ij and s_ji values across users.
  • Framework: SHINE applies separate deep autoencoders to learn nonlinear user representations from sentiment, social, and profile networks.Each autoencoder preserves the structure of its corresponding network while mapping users into a latent space.
  • Sentiment Network Embedding: The sentiment autoencoder maps each user’s adjacency vector into a low-dimensional latent representation and reconstructs the original information.Multiple fully connected nonlinear layers perform encoding and decoding.
  • Sentiment Network Embedding: The weighted reconstruction loss penalizes errors on nonzero sentiment links more heavily than errors on zero entries.The weight is α > 1 when s_ij = ±1 and 1 when s_ij = 0; the sentiment embedding is taken from layer K_s/2.

5.3 Social Network Embedding

The social-network autoencoder represents each user through incoming and outgoing follow relationships, then learns a compact embedding by reconstructing this structural information.

  • Social Representation: Each user’s social adjacency vector contains all incoming and outgoing follow relationships in the directed social network.The vector includes both r_ij and r_ji values for every user pair.
  • Social Representation: The social autoencoder transforms each social adjacency vector through multiple hidden layers and reconstructs the original vector.Its notation follows the sentiment autoencoder, with the reconstruction denoted y′_i.
  • Social Reconstruction: The social reconstruction loss assigns weight α > 1 to observed social links and weight 1 to absent links.This weighting emphasizes reconstruction of entries where r_ij = 1.
  • Social Representation: The social embedding of user i is taken from the middle layer, K_r/2, of the social autoencoder.The middle-layer representation serves as the compact social embedding.

5.4 Profile Network Embedding

The profile-network autoencoder encodes each user’s attribute-value connections and reconstructs the profile adjacency vector. Its reconstruction loss uses profile-specific weights, and the resulting middle-layer representation serves as the profile embedding.

  • Profile Representation: The profile network is an undirected bipartite graph linking users with attribute values.The two disjoint node sets are users and possible attribute values.
  • Profile Representation: For each user, the profile adjacency vector records which attribute values the user possesses.The vector is defined as z_i = {p_ij | j ∈ U}.
  • Profile Representation: The profile autoencoder learns hidden representations from each user’s profile adjacency vector and reconstructs that vector.The reconstruction is denoted z′_i, using notation analogous to the earlier autoencoders.
  • Profile Reconstruction: The profile autoencoder uses a reconstruction loss with a profile weight vector defined analogously to the social-network weighting scheme.The profile weights are denoted n_i and are defined similarly to m_i.

5.5 Representation Aggregation and Sentiment Prediction

SHINE aggregates separate sentiment, social, and profile embeddings into a heterogeneous user representation, then predicts sentiment using a selectable similarity function. Its objective combines reconstruction, supervised prediction, and regularization losses, optimized with AdaGrad.

  • Representation Aggregation: SHINE aggregates sentiment, social, and profile embeddings into a final heterogeneous embedding for each user.Available aggregation functions include summation, element-wise max pooling, and concatenation.
  • Sentiment Prediction: The predicted sentiment between two users is computed from their heterogeneous embeddings using a selectable similarity measurement function.The framework considers inner product, negative Euclidean distance, and logistic regression.
  • Objective Function: SHINE’s objective combines three autoencoder reconstruction losses, a supervised sentiment loss, and a regularization term.The balancing parameters λ1, λ2, λ3, and λ4 weight the objective components, while regularization prevents over-fitting.
  • Objective Function: The regularization penalty covers autoencoder weight parameters and, when appropriate, the similarity measurement function.The weight parameters belong to the sentiment, social, and profile autoencoders.
  • Optimization: AdaGrad minimizes the objective by updating trainable parameters from gradients computed on randomly selected batches of sentiment links until convergence.Each iteration samples a training batch and updates each trainable parameter according to AdaGrad.

5.7 Discussions

SHINE’s ability to represent directed links depends on the chosen similarity function, while its use of side information addresses newly arrived nodes and supports adding or removing information components.

  • Asymmetry: SHINE characterizes link asymmetry if and only if its similarity function is direction-sensitive, meaning f(i, j) differs from f(j, i).The choice of similarity measurement function determines whether directed links can be represented.
  • Cold Start: SHINE addresses cold start by incorporating side information alongside the target network when learning representations for newly arrived nodes.The paper contrasts this with models that rely only on the target network, where new nodes have little interaction history.
  • Flexibility: SHINE can incorporate new user side information through parallel components or remove social and profile autoencoders when those inputs are unavailable.The framework is designed to remain flexible as the available user information changes.

6 EXPERIMENTS

Experiments on Weibo-STC and Wiki-RfA evaluate SHINE against network-embedding, signed-link-prediction, and classification baselines for link prediction and node recommendation. SHINE shows stronger performance, retains performance in cold start, benefits from side information, and is sensitive to representation and loss-balancing choices.

  • Datasets and baselines: SHINE is evaluated on Weibo-STC and Wiki-RfA using five baselines spanning network embedding, signed link prediction, and generic classification.Weibo-STC contains heterogeneous sentiment, social, and profile networks; Wiki-RfA is a signed network without side information.
  • Link prediction: 13.8%, 16.2%, and 8.78% Accuracy gains over LINE, node2vec, and SDNE are reported for SHINE on Weibo-STC link prediction.The corresponding Micro-F1 gains are 15.5%, 17.6%, and 9.71%.
  • Cold start: SHINE maintains decent cold-start performance, with Accuracy decreasing 2.46% versus 11.28%–17.90% for the other listed baselines.The cold-start test excludes associated ordinary users from training, so social and profile information compensate for missing sentiment links.
  • Node recommendation: SHINE’s recommendation curves are almost consistently above baseline curves for positive and negative node recommendation on Weibo-STC and Wiki-RfA.The evaluation reports positive and negative Precision@K and Recall@K.
  • Node recommendation: Negative precision is generally lower than positive precision, while negative recall is higher because negative links are fewer in both datasets.Weibo-STC generally yields better precision and recall than Wiki-RfA, consistent with its additional side information.
  • Parameter sensitivity: Concatenation with inner-product similarity achieves the best Accuracy among tested aggregation and similarity combinations, while max pooling performs worst.The authors attribute concatenation’s advantage to preserving more information from the three embeddings.
  • Parameter sensitivity: Embedding dimension and α exhibit non-monotonic effects: larger dimensions initially help before noise reduces performance, while α values that are too small or too large hurt.Large α can make SHINE ignore dissimilarity among users.
  • Parameter sensitivity: Adding social or profile information improves Accuracy, combining both improves it further, and overly large λ3 disrupts the trade-off among loss terms.λ1 and λ2 indicate whether social and profile information are incorporated, while λ3 balances prediction and reconstruction losses.

7 CONCLUSIONS

The paper introduces a labeled heterogeneous sentiment dataset and SHINE, an embedding framework for predicting sentiment-link signs without sentiment-related content. Experiments show competitiveness against strong baselines and highlight benefits from social and profile information in cold-start settings.

  • The authors establish a labeled, heterogeneous, entity-level sentiment dataset from Weibo because explicit sentiment links are scarce.The dataset supports sentiment-link prediction when sentiment links are not directly available.
  • SHINE learns nonlinear user representations from heterogeneous networks while preserving the structure of the original networks.It is presented as a deep-learning-based network embedding framework.
  • SHINE is competitive with several strong baselines in extensive experiments.
  • Social relation and profile information are effective, especially in the cold-start scenario.
Loading 1712.00732v1…