Source-linked AI summary
Reasoning with Sarcasm by Reading In-between
Yi Tay, Luu Anh Tuan, Siu Cheung Hui, Jian Su
TL;DR
Sarcasm disrupts opinion mining because its figurative language and polarity shifts often depend on contrast and incongruity. The paper proposes MIARN, an attention-based recurrent model that looks between words to model these relationships. Across six Twitter, Reddit, and Internet Argument Corpus benchmarks, MIARN achieves state-of-the-art performance and interpretable attention maps.
Problem
Sarcasm's figurative language, polarity shifts, and contrastive structure make it difficult for opinion mining systems to interpret reliably.
Method
MIARN uses multi-dimensional intra-attention to model relationships between word pairs, capturing contrast, incongruity, and long-range dependencies.
Results
MIARN achieves state-of-the-art performance across six benchmarks and produces highly interpretable attention weights.
Takeaways & Limitations
Modeling intra-sentence relationships supports effective and explainable neural sarcasm detection.
Takeaways & Limitations
Early experiments found that adding nonlinearity to the intra-attention layer may degrade performance.
Abstract
from arXiv · showhide
Sarcasm is a sophisticated speech act which commonly manifests on social communities such as Twitter and Reddit. The prevalence of sarcasm on the social web is highly disruptive to opinion mining systems due to not only its tendency of polarity flipping but also usage of figurative language. Sarcasm commonly manifests with a contrastive theme either between positive-negative sentiments or between literal-figurative scenarios. In this paper, we revisit the notion of modeling contrast in order to reason with sarcasm. More specifically, we propose an attention-based neural model that looks in-between instead of across, enabling it to explicitly model contrast and incongruity. We conduct extensive experiments on six benchmark datasets from Twitter, Reddit and the Internet Argument Corpus. Our proposed model not only achieves state-of-the-art performance on all datasets but also enjoys improved interpretability.
1 Introduction
Sarcasm is a prevalent, figurative speech phenomenon that challenges NLP and opinion mining. It often involves contrast between sentiments or between conveyed emotion and the author's situation.
- Sarcasm is challenging for NLP because of its highly figurative nature and prevalence in reviews, tweets, and online forums.
- Sarcasm commonly involves explicit sentiment contrast or disparity between conveyed emotion and the author's situation.
1. I absolutely love to be ignored!
The paper motivates modeling relationships between words and phrases because sequential sarcasm detectors do not explicitly capture contrast, incongruity, or long-range dependencies. It proposes MIARN, a multi-dimensional intra-attention recurrent network designed to model these relationships and produce interpretable attention maps.
- Sarcasm relies on semantic relationships and contrast between individual words and phrases, including contradictory word pairs that express incongruity.
- Sequential neural models process words one at a time without explicit word-pair interactions, limiting their ability to model contrast, incongruity, juxtaposition, and long-range dependencies.
- MIARN uses multi-dimensional intra-attention to model similarities between every word pair while retaining recurrent-model effectiveness and capturing long-range dependencies.
- The model is intended to detect contrast and incongruity within end-to-end neural networks and produce interpretable attention maps for explaining outputs.
- MIARN is presented as a new state-of-the-art sarcasm-detection method based on compositional learning through intra-sentence relationships.
- The work evaluates MIARN extensively on benchmarks from Twitter, Reddit, and the Internet Argument Corpus.
2 Related Work
Earlier sarcasm-detection research includes contrast-based theories, engineered linguistic features, deep sequential and convolutional models, and systems using user or personality context. The paper positions intra-attention as a self-targeted pairwise mechanism for word-level interactions and long-term dependencies.
- Sarcasm theories commonly ground the phenomenon in contrast involving sentiment, intention, situation, or context.
- Traditional sarcasm detection treated the task as text classification using syntactic, sentiment, n-gram, frequency, word-shape, readability, and flip features.
- Deep-learning approaches included recurrent models with gated pooling and CNN-LSTM-DNN architectures for sarcasm detection.
- Other work exploited personality information, user context, gaze or cognitive features, neural machine translation, and emoji-based distant supervision.
- Intra-attention models sequences against themselves, learning attention while capturing long-term dependencies and word-word interactions.
3 Our Proposed Approach
MIARN combines intra-sentence relationship modeling with sequential composition to form a joint representation for sarcasm prediction. Its multi-dimensional intra-attention uses word-pair interactions and max pooling, while an LSTM supplies compositional information.
- 3.1 Input Encoding Layer: MIARN encodes input words as low-dimensional embeddings before applying its attention and compositional encoders.The input is a sequence of one-hot vectors mapped through an embedding layer to word embeddings.
- 3.2 Multi-dimensional Intra-Attention: The intra-attention mechanism models semantic relationships between every word pair instead of relying solely on sequential composition.Pairwise embeddings are concatenated and projected into affinity scores for the input sequence.
- 3.2 Multi-dimensional Intra-Attention: Row-wise max pooling converts the pairwise affinity matrix into attention weights that produce an intra-attentive representation.Self-relations are masked, and max pooling emphasizes each word’s largest contribution to the sequence.
- 3.2 Multi-dimensional Intra-Attention: Multi-dimensional intra-attention projects each word pair into a low-dimensional vector to capture multiple relationship views before scoring affinity.This extends scalar pairwise scoring to relationships represented through multiple views.
- 3.3 Compositional Encoder: An LSTM separately learns a compositional representation from the original word embeddings, preserving sequential information missed by intra-attention.The compositional representation is the LSTM’s final hidden output, and the LSTM does not consume intra-attentively scaled representations.
- 3.4 Prediction Layer: The prediction layer jointly projects the intra-attentive and compositional representations before binary classification.The network is trained end-to-end with binary cross-entropy and L2 regularization.
4 Empirical Evaluation
Across six benchmark datasets, MIARN and SIARN outperform strong neural baselines, with larger gains on long debate texts and intra-attention improving over vanilla sequential models. Attention analysis further indicates that MIARN captures contrast and incongruity more interpretably than standard attention.
- Experimental Setup: Experiments evaluate sarcasm detection across six benchmark datasets spanning Twitter, Reddit, and the Internet Argument Corpus.The datasets include two Twitter collections, two Reddit collections, and two hand-annotated debate corpora.
- Experimental Setup: The comparison includes NBOW, CNN, LSTM, ATT-LSTM, GRNN, and CNN-LSTM-DNN baselines alongside MIARN and SIARN.CNN-LSTM-DNN and GRNN are identified as strong document-level state-of-the-art neural baselines.
- Experimental Results: MIARN outperforms GRNN and CNN-LSTM-DNN by approximately 8%–10% on both IAC-V1 and IAC-V2.The improvement on long debate text is substantially larger than on Twitter and Reddit.
- Experimental Results: Removing intra-attention reduces the model to a standard LSTM, while adding it yields almost 10% improvement in F1 and accuracy overall.On short texts, the improvement is typically approximately 2%–3%, whereas long texts benefit more from modeled long-range dependencies.
- In-depth Model Analysis: MIARN attention focuses on contrasting words such as “love” and “ignored,” unlike standard attention, which often selects the final token or misses the contrast.The analysis reports that MIARN identifies contrast and incongruity through intra-sentence relationships, making its attention maps more interpretable.
5 Conclusion
The MIARN model uses intra-attention to detect contrastive sentiment, situations, and incongruity, outperforming strong baselines across six benchmarks while producing interpretable attention weights.
- MIARN incorporates multi-dimensional intra-attention to learn sentence representations that detect contrastive sentiment, situations, and incongruity.
- The model outperforms strong state-of-the-art baselines, including GRNN and CNN-LSTM-DNN, across six public benchmarks.
- Intra-attention scores yield highly interpretable attention weights, supporting more explainable neural sarcasm detection methods.