Source-linked AI summary
Sentiment and Sarcasm Classification with Multitask Learning
Navonil Majumder, Soujanya Poria, Haiyun Peng, Niyati Chhaya, Erik Cambria, Alexander Gelbukh
TL;DR
Sentiment classification and sarcasm detection are often treated separately despite their relationship. The paper uses a shared GRU-based multi-task architecture with task-specific processing and fusion, and reports improvements over standalone classifiers and prior state-of-the-art methods on the evaluated dataset. The authors identify the need for a new dataset for more rigorous experimentation.
Problem
Prior work often treated sentiment classification and sarcasm detection separately, despite evidence that the tasks are related.
Method
The paper uses a GRU-based neural network with shared representations, task-specific processing, and multi-task learning for sentiment and sarcasm classification.
Results
Multi-task classifiers significantly outperform standalone sentiment and sarcasm classifiers, while the reported architecture outperforms the state of the art on the evaluated dataset.
Takeaways & Limitations
The results indicate that sentiment classification and sarcasm detection are related tasks and that multi-task learning can improve both within the evaluated setting.
Takeaways & Limitations
The authors plan to build a new dataset for more rigorous experimentation.
Abstract
from arXiv · showhide
Sentiment classification and sarcasm detection are both important natural language processing (NLP) tasks. Sentiment is always coupled with sarcasm where intensive emotion is expressed. Nevertheless, most literature considers them as two separate tasks. We argue that knowledge in sarcasm detection can also be beneficial to sentiment classification and vice versa. We show that these two tasks are correlated, and present a multi-task learning-based framework using a deep neural network that models this correlation to improve the performance of both tasks in a multi-task learning setting. Our method outperforms the state of the art by 3-4% in the benchmark dataset.
RELATED WORK
Earlier work largely treated sentiment classification and sarcasm detection separately, using task-specific features and models. This paper applies multi-task learning to model the relationship between the two tasks in one system.
- Sentiment research has used convolutional, recursive, recurrent, and memory networks, while knowledge-based methods have explored syntactic patterns and sentiment resources.
- Sarcasm detection has focused on syntactic, surface-pattern, personality-based, and contextual-incongruity features.
- Mishra et al. extracted multimodal cognitive features for both tasks without modeling them in a single system.
- The paper uses multi-task learning for sentiment classification and sarcasm detection, following its successful application to other NLP tasks.
METHOD
The method uses one neural network to assign sentiment and sarcasm labels, sharing representations while adding task-specific processing and inter-task fusion. It trains both classification objectives jointly with equal priority.
- Task Definition: A single network assigns each sentence both a sentiment tag and a sarcasm tag.
- Sentence Representation: Word embeddings are padded into a fixed-length input matrix before sentence representations are computed with a GRU and attention mechanism.
- Sentence Representation: The shared GRU representations are transformed through separate fully connected layers to accommodate sentiment and sarcasm classification.
- Attention Network: Task-specific attention prioritizes words relevant to each classification task when aggregating context-rich word representations.
- Inter-Task Communication: A neural tensor network of size Dntn = 100 fuses sentiment- and sarcasm-specific sentence representations into information relevant to both tasks.
- Classification: Two softmax layers produce task-specific classifications, using ssen for sentiment and ssar ⊕ s+ for sarcasm.
- Training: Training minimizes categorical cross-entropy losses for sentiment and sarcasm with equal priority using ADAM.
EXPERIMENTS
The experiments compare standalone, coerced, and multi-task variants for sentiment and sarcasm classification. The main model uses shared attention, while fusion and separate-GRU alternatives are evaluated as variants.
- Dataset: The dataset contains 994 samples labeled for sarcasm, sentiment, and eye movements; the experiments ignore the eye-movement data.Among the samples, 383 are positive and 350 are sarcastic.
- Baselines and Model Variants: Standalone classifiers use a GRU sentence representation followed by a task-specific fully connected layer and softmax classifier.The same standalone design is applied separately to sentiment and sarcasm.
- Baselines and Model Variants: The coerced variant forces sentences classified as sarcastic to be considered negative by the sentiment classifier.
- Baselines and Model Variants: The simple multi-task classifier shares the GRU, then uses task-specific fully connected and softmax layers for the two tasks.
- Baselines and Model Variants: Fusion variants combine task representations with neural tensor network output, while alternative fusion methods and concatenation choices did not improve results.The reported alternatives include fully connected layers, Hadamard products, and different concatenation configurations.
- Best model: shared attention: The main model adds shared attention and uses the architecture that gave the best results; separate GRUs did not improve performance.
RESULTS AND DISCUSSION
Across the experiments, multi-task models outperform standalone classifiers, with shared representations and task interaction improving both tasks. The strongest architecture combines neural tensor network fusion with shared attention, while examples indicate that sentiment shifts particularly aid sarcasm classification.
- 10-fold cross-validation evaluates the models against standalone classifiers and Mishra et al.’s CNN-based state-of-the-art method.The standalone GRU classifiers slightly outperform that state-of-the-art method, which uses gaze data unavailable in many real-life settings.
- All multi-task classifiers outperform both standalone classifiers, with larger improvement for sentiment than for sarcasm.The authors suggest this may reflect sarcasm detection being a subtask of sentiment analysis.
- The multi-task network mainly improves sarcasm classification when sentences contain a strong sentiment shift indicating possible sarcasm.In one example, detecting the sentiment shift enabled correct predictions for both sentiment and sarcasm.
- Sarcasm cues can correct sentiment errors: the phrase “water spilt on my phone” led the multi-task network to revise a positive standalone sentiment prediction.
- Shared representations outperform standalone models because they provide additional regularization from the other task.
- Neural tensor network fusion gives the best sarcasm performance, while shared attention further improves sentiment; their combination achieves the best average results.
CONCLUSION
The paper presents a GRU-based multitask architecture that outperforms standalone classifiers and the state of the art for sentiment classification and sarcasm detection on the dataset used by Mishra et al. It identifies multitask learning as beneficial because the two tasks are related, while noting that stronger validation requires a new dataset.
- The GRU-based architecture outperforms the state-of-the-art CNN method on the dataset used by Mishra et al.
- Multi-task learning methods significantly outperform standalone sentiment and sarcasm classifiers.
- These results indicate that sentiment classification and sarcasm detection are related tasks.
- Among the tested architecture variants, the presented multi-task learning architecture achieves the best results.
- The authors plan to build a new dataset for more rigorous experimentation to strengthen their claim.