Source-linked AI summary
Automatic Sarcasm Detection: A Survey
Aditya Joshi, Pushpak Bhattacharyya, Mark James Carman
TL;DR
Automatic sarcasm detection matters because sarcasm combines positive surface wording with implied negative sentiment, complicating sentiment analysis. This survey compiles prior work across formulations, datasets, approaches, performance, trends, issues, shared tasks, and future directions, identifying semi-supervised pattern extraction, hashtag-based supervision, and context beyond target text as research milestones. It concludes by organizing the field's reported resources and approaches while noting annotation variability and dataset skew.
Problem
Automatic sarcasm detection is difficult because sarcastic text may express negative sentiment through positive surface language and nuanced expression.
Method
The survey synthesizes prior sarcasm-detection work across datasets, approaches, formulations, performance, trends, issues, shared tasks, and future directions.
Results
The survey identifies three milestones: semi-supervised pattern extraction for implicit sentiment, hashtag-based supervision, and context beyond target text.
Takeaways & Limitations
The survey provides organized resources for understanding state-of-the-art, including tables summarizing past work across features, annotation techniques, and data forms.
Takeaways & Limitations
Reported annotation agreement varies substantially, and sarcasm datasets are often skewed because sarcasm is infrequent.
Abstract
from arXiv · showhide
Automatic sarcasm detection is the task of predicting sarcasm in text. This is a crucial step to sentiment analysis, considering prevalence and challenges of sarcasm in sentiment-bearing text. Beginning with an approach that used speech-based features, sarcasm detection has witnessed great interest from the sentiment analysis community. This paper is the first known compilation of past work in automatic sarcasm detection. We observe three milestones in the research so far: semi-supervised pattern extraction to identify implicit sentiment, use of hashtag-based supervision, and use of context beyond target text. In this paper, we describe datasets, approaches, trends and issues in sarcasm detection. We also discuss representative performance values, shared tasks and pointers to future work, as given in prior works. In terms of resources that could be useful for understanding state-of-the-art, the survey presents several useful illustrations - most prominently, a table that summarizes past papers along different dimensions such as features, annotation techniques, data forms, etc.
1. INTRODUCTION
Automatic sarcasm detection predicts whether text is sarcastic, a difficult sentiment-analysis problem because sarcasm can express negative sentiment through positive surface language. This survey reviews prior computational work, datasets, approaches, performance, trends, issues, shared tasks, and future directions.
- Automatic sarcasm detection predicts whether a given text is sarcastic.
- Sarcasm is difficult for sentiment analysis because it combines implied negative sentiment with positive surface sentiment.
- Prior work spans speech and text data, including tweets, reviews, and TV-series dialogues, alongside rule-based, supervised, and semi-supervised approaches.
- The survey reviews problem formulation, datasets, approaches, reported performance, trends, issues, related shared tasks, and future areas.
2. SARCASM STUDIES IN LINGUISTICS
Linguistic research characterizes sarcasm through verbal irony, incongruity, speaker-listener context, and relationships between literal and intended propositions. These perspectives identify common knowledge, ridicule, and shared context as challenges for automatic detection.
- Proposed taxonomies include failed expectation, pragmatic insincerity, negative tension, a victim, and four forms ranging from propositional to illocutionary sarcasm.
- A six-tuple represents sarcasm through speaker, hearer, context, utterance, literal proposition, and intended proposition.
- The survey focuses on computational sarcasm-detection techniques rather than linguistic theories and their possible applications.
- Linguistic accounts identify common knowledge, ridicule, and speaker-listener context as recurring challenges for automatic detection.
3. PROBLEM DEFINITION
Sarcasm detection is usually formulated as predicting sarcasm labels for text, but prior work also uses multiclass, sequence-labeling, and sense-disambiguation formulations. Across the surveyed work, tweets predominate and extra-textual context is a recent trend.
- The common formulation is a classification task that predicts whether a piece of text is sarcastic.
- Some studies use labels spanning politics, humor, irony, and sarcasm rather than a binary sarcastic/non-sarcastic distinction.
- Dialogue sarcasm can be modeled as sequence labeling, with utterances as observed units and sarcasm labels as hidden variables.
- Another formulation treats sarcasm detection as sense disambiguation by identifying a word's sense.
- Tweets are the predominant text form in prior work, while incorporating extra-textual context is a recent trend.
- Most sarcasm-detection research is in English, with additional studies reported for Chinese, Italian, Czech, Dutch, Greek, Indonesian, and Hindi.
4. DATASETS
Sarcasm datasets span short text, long text, and other forms, with tweets especially prominent. Labels come from manual annotation, hashtags, supplementary context, and specialized annotation procedures, each carrying distinct data and quality considerations.
- Datasets are divided into short text, long text, and other datasets.
- Tweets are popular because Twitter is widely used and provides API access, although platform word limits make some social-media text short.
- Manual annotation labels tweets and other texts as sarcastic or non-sarcastic, while some corpora use multiple labels or annotate sarcasm with sentiment.
- Hashtag-based supervision uses author-provided hashtags to create large-scale labeled tweet datasets.
- Hashtag-derived labels can require quality control, including additional annotation, because distant supervision poses challenges.
- Supplementary datasets provide context without requiring annotation, including complete timelines and historical tweets from authors.
- Other resources include eye-tracking annotations, Reddit posts and comments, reviews, discussion forums, call-center transcripts, and sarcastic similes.
5. APPROACHES
Sarcasm-detection approaches are organized into rule-based, statistical, and deep learning-based methods, using varied evidence, features, and classifiers. The survey also situates these approaches alongside shared-task evaluation.
- Rule-based approaches: Rule-based methods identify sarcasm through specific evidences encoded as rules.Examples use simile validation, hashtag sentiment incongruity, or sentiment-bearing situation phrases.
- Statistical approaches: Statistical methods vary in both feature design and learning algorithm.Reported features include bag-of-words, extracted patterns, sentiment and pragmatic cues, ambiguity, unexpectedness, and lexical properties.
- Statistical approaches: Most statistical approaches use bag-of-words alongside task-specific textual features.Contextual features are treated separately from features derived from the target text.
- Statistical approaches: Support vector machines are the most frequently used classifiers among surveyed statistical approaches.Other reported classifiers include logistic regression, Naive Bayes, and rule-based techniques.
- Deep learning-based approaches: Embedding-similarity features improve performance when augmented with features based on congruent and incongruent word pairs.Using embedding-similarity features alone was insufficient in the reported approach.
- Shared tasks: Shared tasks provide common datasets for comparative evaluation across teams.A SemEval-2015 task addressed sentiment analysis of figurative-language statements labeled positive, negative, or neutral.
6. REPORTED PERFORMANCE
The survey compiles reported performance values while cautioning that results are not directly comparable across different datasets and metrics. Reported examples range from classifier accuracy below human performance to higher accuracy using historical and flip-based features.
- Cross-study comparison: Reported values are not directly comparable because studies use different datasets and metrics.The survey presents them as a ballpark estimate of sarcasm-detection performance.
- Reported results: 57.41% accuracy was achieved by the best classifier, compared with 62.59% human performance for sarcasm identification.The comparison was reported for unigram-based features against a subset of sentiment-lexicon words.
- Reported results: 81.5% accuracy was reported for the logistic classifier in Rakov and Rosenberg [2013].
- Reported results: 83.46% accuracy was obtained using historical features along with flip-based features.These features were also presented in a rule-based setting by Khattri et al. [2015].
7. TRENDS IN SARCASM DETECTION
Sarcasm-detection research progressed from pattern-focused classification to hashtag-based supervision and, more recently, context beyond the target text. The survey details pattern discovery and several forms of contextual information, motivated by annotators’ requests for context.
- Research trends: The survey identifies four milestones, including supervised or semi-supervised classification, hashtag-based supervision, and context beyond the target text.Figure 1 summarizes these trends and indicates representative work in each area.
- Discovery of sarcastic patterns: Early pattern-discovery methods extracted sarcastic or implicit-sentiment patterns for use as classifier features or rules.Tsur et al. extracted patterns from labeled sentences, while Riloff et al. iteratively expanded positive verbs and negative situation phrases.
- Role of context in sarcasm detection: Context is defined as information beyond the target text and beyond common knowledge.The survey distinguishes supplementary data from supplementary information supplied by the source platform.
- Role of context in sarcasm detection: Annotators often requested additional Reddit comments during sarcasm labeling, highlighting the need for contextual information.The study also reports label changes after annotators viewed the context.
- Role of context in sarcasm detection: Author-specific context uses an author’s textual history to characterize sentiment contrasts involving the target text.Examples use historical tweets and author timelines to estimate the author’s sentiment or sarcasm likelihood.
- Role of context in sarcasm detection: Conversation context incorporates surrounding discussion text, discourse structure, or audience information.Reported methods include concatenating prior posts, using thread features, and sequence labeling over conversational tweets.
- Role of context in sarcasm detection: Topical context uses related tweets and hashtag-based sequences to model topics associated with sarcasm.Wang et al. construct timestamped sequences of tweets containing a target hashtag before sequence labeling.
8. ISSUES IN SARCASM DETECTION
The survey identifies recurring issues involving data quality, sentiment-based features, and dataset skew in automatic sarcasm detection.
- Issues with Data: Hashtag-based supervision provides large-scale labels, but removing hashtags can erase the cue that makes a sentence sarcastic.The survey notes that “I totally love bland food. #not” can become nonsarcastic-looking after preprocessing.
- Issues with Data: Inter-annotator agreement varies widely because sarcasm is subjective, with reported values ranging from 0.34 to 0.81.Cross-cultural annotation can also differ: Indian and American annotators judged some examples differently.
- Issues with features: Sentiment as feature: Sentiment can serve as a sarcasm-detection feature through surface polarity, sentiment imbalance, emotion dimensions, or sentiment from prior text.These approaches depend on estimating apparent rather than implied sentiment.
- Dealing with Dataset Skews: Sarcasm is infrequent in sentiment expression, creating skewed datasets that require specialized evaluation or learning strategies.Prior work uses ensemble voting, F-score optimization, regularization, and AUC reporting to address imbalance.
9. CONCLUSION & FUTURE DIRECTIONS
The survey synthesizes the development of automatic sarcasm detection, identifying milestones, recurring issues, and directions for future research. It highlights opportunities involving implicit sentiment, numerical incongruity, undercovered sarcasm forms, culture-specific traits, and deep learning.
- The survey identifies three milestones: semi-supervised pattern extraction for implicit sentiment, hashtag-based supervision, and context beyond the target text.
- A comparative table organizes prior datasets and approaches by dimensions including methods, annotation techniques, and features.
- Future directions: Implicit sentiment detection may support sarcasm detection through semi-supervised pattern discovery.The survey links sarcasm to sentiment incongruity and gives contrasting examples involving heated phones and induction cooktops.
- Future directions: Numerical incongruity is a future direction because numerical values convey sentiment and can distinguish sarcastic from non-sarcastic statements.For example, taking 6 hours to reach work with “#yay” is contrasted with taking 10 minutes with the same hashtag.
- Future directions: Current approaches are limited in handling like-prefixed and illocutionary sarcasm among four described sarcasm forms.The four forms are propositional, lexical, like-prefixed, and illocutionary sarcasm.
- Future directions: Future work should consider culture-specific traits and deep learning architectures, which have received limited exploration so far.The survey notes that American and Indian annotators may substantially disagree in sarcasm annotations, while deep learning approaches remain few.