Source-linked AI summary
Interactive Attention Networks for Aspect-Level Sentiment Classification
Dehong Ma, Sujian Li, Xiaodong Zhang, Houfeng Wang
TL;DR
Aspect-level sentiment classification must identify a target’s polarity in context, but prior methods often neglect separately modeling the target. IAN uses interacting attention networks with LSTM representations for targets and contexts, and experiments on SemEval 2014 report state-of-the-art performance, including gains over ATAE-LSTM.
Problem
Prior approaches model contexts with target-specific representations but neglect separate target modeling, despite targets’ importance for sentiment classification.
Method
IAN uses two interacting attention networks with LSTM-based word representations to learn target and context representations separately.
Results
IAN achieves the best performance among the evaluated methods, improving over ATAE-LSTM by about 1.4% on Restaurant and 3.2% on Laptop.
Takeaways & Limitations
Interactive modeling helps IAN learn effective target and context features for judging target sentiment polarity.
Abstract
from arXiv · showhide
Aspect-level sentiment classification aims at identifying the sentiment polarity of specific target in its context. Previous approaches have realized the importance of targets in sentiment classification and developed various methods with the goal of precisely modeling their contexts via generating target-specific representations. However, these studies always ignore the separate modeling of targets. In this paper, we argue that both targets and contexts deserve special treatment and need to be learned their own representations via interactive learning. Then, we propose the interactive attention networks (IAN) to interactively learn attentions in the contexts and targets, and generate the representations for targets and contexts separately. With this design, the IAN model can well represent a target and its collocative context, which is helpful to sentiment classification. Experimental results on SemEval 2014 Datasets demonstrate the effectiveness of our model.
1 Introduction
Aspect-level sentiment classification identifies the polarity of a specific target in context. IAN addresses prior methods’ limited separate modeling of targets by interactively learning target and context representations.
- Task: Aspect-level sentiment classification predicts the sentiment polarity of specific targets within their contexts.For example, the targets staff, pizza, and beef cubes receive positive, neutral, and negative polarities in one sentence.
- Prior work: Traditional approaches rely on manually designed features, while neural methods automatically generate low-dimensional representations from targets and contexts.Feature engineering is labor intensive and approaches a performance bottleneck.
- Research gap: Prior target-aware methods focus on modeling contexts through target-specific representations but ignore separate modeling of targets.The paper argues that targets and contexts should be coordinated rather than treating the target only as context information.
- Motivation: Words can contribute unequally to target and context representations, motivating separate attention weights for both.The example gives “picture” greater importance when representing “picture quality” described by “clear-cut.”
- Proposed approach: IAN uses target-associated attention to represent context, context information to supervise target modeling, and concatenation for sentiment prediction.The model is based on LSTMs and attention and is evaluated on SemEval 2014 datasets.
2 Model
IAN models targets and contexts as two interacting streams. LSTMs produce word representations, cross-stream attention selects important information, and the resulting representations are combined for classification.
- Architecture: IAN comprises two interactive parts that separately model the target and context.Word embeddings are processed by LSTM networks to obtain hidden states for both sequences.
- Representation learning: The model obtains context and target word representations from word embeddings and LSTM hidden states.The paper uses pretrained word embeddings and averages hidden states to form initial context and target representations.
- Interactive attention: Target and context average representations supervise attention vectors that select important words in the opposite sequence.The target representation computes context attention, while the context representation computes target attention.
- Prediction: Attention-weighted hidden states produce context and target representations that are concatenated for classification.The concatenated vector is projected into the targeted sentiment-class space and passed through a softmax classifier.
- Training: IAN optimizes LSTM, attention, softmax, and embedding parameters using cross entropy with L2 regularization.Gradients are computed by backpropagation, and dropout randomly omits half of the feature detectors during training.
3 Experiments
Experiments evaluate IAN on SemEval 2014 Restaurant and Laptop reviews using three-class accuracy against several LSTM-based baselines. IAN achieves the best reported performance, with interactive target–context modeling particularly beneficial for longer targets.
- Experiment Preparation: SemEval 2014 experiments use Restaurant and Laptop reviews labeled positive, neutral, or negative, with Accuracy as the evaluation metric.Accuracy is the percentage of correctly predicted samples.
- Model Comparisons: 53.5% and 65.0% are the majority-class proportions for Restaurant and Laptop, making Majority the weakest baseline.The Majority method assigns the largest training-set sentiment polarity to every test sample.
- Model Comparisons: Over 1 percent and 2 percent are TD-LSTM’s improvements over LSTM on Restaurant and Laptop, respectively.TD-LSTM processes left and right contexts together with the target.
- Model Comparisons: IAN improves over ATAE-LSTM by about 1.4% on Restaurant and 3.2% on Laptop, achieving the best performance among the baselines.The authors attribute this to two connected attention networks that influence one another while modeling targets and contexts.
- Analysis of IAN Model: No-Interaction performs worst among the analyzed interaction variants, while Target2Content outperforms it but remains worse than IAN.The comparisons support the importance of interaction and bidirectional target–context attention.
- Analysis of IAN Model: IAN’s gains are larger on Laptop because that category contains more multi-word targets, which better suit its LSTM and interactive-attention modeling.Restaurant has 9% more one-word-target instances than Laptop.
- Case Study: In the case study, IAN assigns little attention to common words and punctuation while emphasizing sentiment-bearing words and the relevant target collocation.Attention shifts between “fish” and “variety of fish” depending on the target being classified.
4 Related Work
Aspect-level sentiment classification research progressed from handcrafted features to neural methods, increasingly incorporating targets but often modeling them only indirectly. IAN instead learns target and context representations interactively, using each to focus attention on the other.
- Research directions: Aspect-level sentiment classification methods include traditional machine learning and neural network approaches.Traditional methods extract features such as sentiment lexicons and bag-of-words, while neural methods use architectures including recurrent and recursive networks.
- Traditional methods: Traditional methods depend on labor-intensive handcrafted features, creating a performance bottleneck.Their results depend heavily on feature effectiveness.
- Neural methods: Neural network approaches increasingly incorporate target information because targets contribute substantially to judging sentiment polarity.Prior work includes target-dependent LSTMs, deep memory networks, and attention-based LSTMs.
- Interactive attention networks: IAN separately models targets and contexts by using each to compute attention for the other.The model learns target and context representations interactively and concatenates them for classification.
5 Conclusion
The paper concludes that IAN interactively models target and context attention to generate their representations for aspect-level sentiment classification. Experiments on SemEval 2014 and a case study support its ability to learn useful features and focus on sentiment-relevant words.
- Conclusion: IAN uses two attention networks to model targets and contexts interactively and generate their representations.The model concatenates the target and context representations before predicting sentiment polarity.
- Conclusion: Experiments on SemEval 2014 verify that IAN learns effective target and context features for judging target sentiment polarity.The conclusion reports that the case study shows reasonable attention to words important for sentiment classification.