Source-linked AI summary
Modelling Context with User Embeddings for Sarcasm Detection in Social Media
Silvio Amir, Byron C. Wallace, Hao Lyu, Paula Carvalho Mário J. Silva
TL;DR
Sarcasm detection requires contextual information because identical utterances can be interpreted differently depending on the speaker. CUE-CNN jointly learns content and user embeddings without manual feature engineering, outperforming a handcrafted-feature state-of-the-art model by over 2% in absolute accuracy.
Problem
Sarcasm detection requires speaker context because the same sentence can be interpreted literally or sarcastically depending on its author.
Method
CUE-CNN is a convolutional neural network that jointly learns and exploits user embeddings with features derived from utterances, without manual feature engineering.
Results
The model outperforms the strong Bamman and Smith (2015) baseline by more than 2% in absolute accuracy.
Takeaways & Limitations
Learning user embeddings from preceding messages provides contextual speaker information for sarcasm detection without requiring handcrafted user-attribute features or extensive metadata.
Takeaways & Limitations
Historical tweets were unavailable for a significant proportion of users, so messages lacking contextual information were discarded; the historical tweets were posted after the experimental corpus tweets.
Abstract
from arXiv · showhide
We introduce a deep neural network for automated sarcasm detection. Recent work has emphasized the need for models to capitalize on contextual features, beyond lexical and syntactic cues present in utterances. For example, different speakers will tend to employ sarcasm regarding different subjects and, thus, sarcasm detection models ought to encode such speaker information. Current methods have achieved this by way of laborious feature engineering. By contrast, we propose to automatically learn and then exploit user embeddings, to be used in concert with lexical signals to recognize sarcasm. Our approach does not require elaborate feature engineering (and concomitant data scraping); fitting user embeddings requires only the text from their previous posts. The experimental results show that our model outperforms a state-of-the-art approach leveraging an extensive set of carefully crafted features.
1 Introduction
Sarcasm detection requires contextual information because identical wording can be literal or sarcastic depending on the speaker. The paper proposes a neural model that learns user and lexical representations jointly without extensive manual feature engineering.
- Figurative language, especially sarcasm, remains difficult for social media analysis systems to detect accurately.
- The same sentence can be interpreted literally or sarcastically depending on its speaker and context.
- Prior contextual approaches use users’ preceding messages but require complex manually engineered features.
- The proposed neural model learns embeddings for both tweet content and its author, then combines them for sarcasm inference.
- The model outperforms the Bamman and Smith baseline by more than 2% in absolute accuracy while avoiding manual feature engineering.
2 Related Work
Earlier sarcasm and irony systems primarily relied on textual, sentiment, or manually designed contextual features. This work motivates automatically learned representations that combine lexical evidence with information about the author.
- Verbal irony involves saying something other than, often opposite to, what the speaker means; sarcasm is treated as an indirect insult with positive literal meaning.
- Earlier systems used emoticons, laughter expressions, punctuation, quotation marks, interjections, n-grams, sentiment lexicons, and textual markers.
- Contextual approaches analyzed prior sentiment toward entities, behavioral traits, user information, previous posts, and author–audience interactions.
- A major limitation of prior contextual systems was the manual effort required to derive their feature sets.
- The proposed model instead automatically induces predictive representations for message content and authors.
3 Learning User Embeddings
The paper learns user embeddings from author-conditioned word-usage patterns, aiming to encode latent user aspects and contextual signals relevant to sarcasm. Training uses contrastive objectives and negative sampling to handle large vocabularies and limited per-user text.
- Learning objective: User embeddings are learned by optimizing the conditional probability of words given their authors, alongside neighboring-word context.The objective models word occurrence as depending on both the sentence author and words in a prespecified window.
- Learning objective: The training objective combines author and local-context representations to predict words in a sentence.Word and user embeddings share dimensionality d, and the objective captures dependencies between authors, words, and surrounding words.
- Probability estimation: A log-linear model estimates conditional word probabilities, treating vocabulary words as the classes to be predicted.The model uses a feature vector x with class-specific weights Wk and biases bk.
- Efficient training: Hierarchical Softmax approximates the vocabulary-wide denominator to avoid the expense of summing over all words.The denominator is computationally costly because the vocabulary is large.
- User-specific training: A hinge-loss objective learns user vectors that discriminate observed words from negative words not appearing in the user’s sentence.Minimizing this loss makes embeddings predictive of individual word-usage patterns and discriminative with respect to word usage.
- Negative sampling: Negative sampling approximates the objective by distinguishing observed positive examples from pseudo-negative instances, while scarce per-user text motivates careful sample selection.Rather than fitting user-specific unigram models, the method samples from a unigram distribution estimated across all users.
4 Proposed Model
CUE-CNN represents a message with convolutional lexical features and a learned embedding for its author. A hidden layer combines these content and user representations to model sarcasm.
- Content representation: The model applies convolutional filters to pre-trained word embeddings to extract high-level features from the message content.The selected word-embedding columns form a sentence matrix that serves as the convolutional input.
- Content representation: Filters of varying heights slide across the sentence matrix to produce h-gram feature maps.Each filter has dimensions d × h, where h is its height.
- Content representation: Max-pooling reduces each feature map to its largest value, and concatenated outputs from three filters form the sentence vector cS.The model uses ReLU activations before combining the filter outputs.
- User representation: The author is mapped to a user vector by selecting the corresponding column from a learned user embedding matrix U.Each of N users is represented by a d-dimensional column vector.
- Content-context integration: CUE-CNN combines the sentence vector and author embedding through a hidden layer that captures relations between content and context representations.The model estimates convolutional, embedding, hidden-layer, and output parameters jointly during training.
5 Experimental Setup
The experiments compare content-only, feature-engineered contextual, and neural models for sarcasm detection using a replicated Twitter setup and 10-fold cross-validation. CUE-CNN learns content and user representations while avoiding manual feature engineering.
- Dataset: Labels were inferred from self-declared sarcasm hashtags, with unavailable tweets and missing historical user data reducing the experimental corpus.The resulting corpus contained 11,541 tweets involving 12,500 unique users.
- Baselines: The replicated baseline uses logistic regression with rich tweet, author, audience, and response feature sets.These features include lexical, sentiment, topic, profile, interaction, and response information.
- Baselines: Content-only baselines range from unigram and tweet-feature logistic regression to neural bag-of-embeddings, NLSE, and CNN classifiers.The CNN baseline uses pre-trained word embeddings and convolutional features from lexical content.
- Baselines: Contextual baselines combine tweet features with manually engineered feature sets, while shallow CUE-CNN variants learn user and word embeddings during training.Shallow variants differ in whether user embeddings are randomly initialized or pre-trained.
- Proposed models: CUE-CNN jointly models content and users, with variants using random or pre-trained user embeddings and unigram or uniform-random negative sampling.User embeddings are induced from historical tweets using skip-gram-style training with negative samples.
- Evaluation: Evaluation uses fixed 10-fold cross-validation, with 80% training, 10% tuning, and 10% held-out testing in each split.Linear-model regularization is selected over C = [1e−4, 1e−3, 1e−2, 1e−1, 1, 10].
6 Results
Contextualizing tweets with learned user representations improves sarcasm classification and changes predictions for identical text across hypothetical authors. The embeddings also capture user attributes and homophily, while unigram negative sampling provides an additional accuracy gain.
- Contextual features significantly improve linear-classifier performance, with the largest gains attributable to author information.
- Jointly learning user representations, lexical features, and classification improves neural-model accuracy over context-free variants.
- The proposed model outperforms other baselines and exhibits less variance across cross-validation experiments.
- 0.8% accuracy improvement results from sampling negative examples from a unigram distribution rather than uniformly at random.The authors attribute this to pushing user vectors away from non-informative words and toward discriminative words.
- T-SNE visualizations suggest user embeddings capture political preferences, sports interests, and a soft notion of homophily.Users with similar attributes tend to occupy neighboring regions of the embedding space.
- For the same textual content, contextual user embeddings drastically change sarcasm probabilities and can produce similar responses for different authors.The examples were misclassified by a simple CNN without user information.
7 Conclusions
CUE-CNN jointly learns content and user embeddings for sarcasm recognition without manual feature engineering. It outperforms a handcrafted state-of-the-art baseline, requires only preceding user messages, and is presented as easier to deploy across social-media environments.
- CUE-CNN jointly learns embeddings for message content and users to recognize sarcastic utterances without manual feature engineering.
- The model exceeds a state-of-the-art handcrafted baseline by over 2% in absolute accuracy.
- Learning user embeddings requires only users’ preceding messages, rather than social-media-specific structure or user metadata.
- The learned vectors capture relevant user attributes and a soft notion of homophily, supporting deployment across different social-media environments.
- Future work would incorporate interactions between message authors and their audiences into the model.