Source-linked AI summary
Exploiting Document Knowledge for Aspect-level Sentiment Classification
Ruidan He, Wee Sun Lee, Hwee Tou Ng, Daniel Dahlmeier
TL;DR
Aspect-level sentiment models are constrained by small datasets because annotating every opinion target is costly. The paper transfers document-level sentiment knowledge through pretraining and multi-task learning, and finds that both help, with their combination improving attention-based LSTM performance across datasets. These gains arise through multiple transfer effects, including useful transfers from embedding and LSTM layers.
Problem
Costly annotation of every opinion target leaves aspect-level sentiment datasets relatively small, limiting neural-model effectiveness.
Method
The paper transfers document-level sentiment knowledge to aspect-level classification using pretraining and multi-task learning with attention-based LSTM models.
Results
Both transfer methods help, with PRET improving accuracy by 1–3% across all datasets and PRET+MULT yielding better overall results.
Takeaways & Limitations
Document-level knowledge benefits aspect-level classification in multiple ways, with embedding and LSTM-layer transfers generally more useful than output-layer transfer.
Abstract
from arXiv · showhide
Attention-based long short-term memory (LSTM) networks have proven to be useful in aspect-level sentiment classification. However, due to the difficulties in annotating aspect-level data, existing public datasets for this task are all relatively small, which largely limits the effectiveness of those neural models. In this paper, we explore two approaches that transfer knowledge from document- level data, which is much less expensive to obtain, to improve the performance of aspect-level sentiment classification. We demonstrate the effectiveness of our approaches on 4 public datasets from SemEval 2014, 2015, and 2016, and we show that attention-based LSTM benefits from document-level knowledge in multiple ways.
1 Introduction
Aspect-level sentiment classification determines sentiment toward a specific opinion target, but costly target annotation leaves datasets too small for neural models. The paper transfers knowledge from accessible document-level sentiment data through pretraining and multi-task learning, finding both helpful and their combination superior to aspect-level-only attention-based LSTM.
- Aspect-level sentiment classification determines sentiment polarity toward an opinion target within a sentence.An opinion target is a word or phrase describing an entity aspect.
- Attention-based LSTMs separate contexts for different targets by combining sequential-pattern modeling with target-specific attention.
- Costly annotation of every opinion target makes public aspect-level datasets relatively small, limiting neural-model effectiveness.
- The paper transfers document-level sentiment knowledge through pretraining and multi-task learning, with both methods helpful and their combination improving over aspect-level-only attention-based LSTMs.
2 Related Work
Prior aspect-level neural models require substantial annotated data, which is expensive to obtain. This paper investigates pretraining and multi-task learning as established approaches for transferring knowledge across tasks, with transferability depending on source–target semantic relatedness.
- Existing aspect-level neural architectures learn continuous features and target–context relations but require substantial, expensive annotated data.
- Pretraining transfers learned neural layers to different tasks, while multi-task learning jointly trains on samples from multiple tasks with a combined objective.
- Prior transfer research indicates that transferability depends substantially on the semantic relatedness of source and target tasks.
3 Models
The section presents an attention-based LSTM baseline for aspect-level sentiment classification and extends it with document-level knowledge through pretraining, multi-task learning, and their combination.
- Baseline model: The baseline LSTM+ATT uses sentence words and an opinion target as inputs, with word embeddings representing the sentence sequence.The target is a subsequence of the sentence, and embeddings come from a shared embedding matrix.
- Baseline model: The LSTM captures sequential information, while attention assigns each word a target-dependent weight to form a sentence representation.The attention score uses the target representation and a content-based function measuring semantic association between words and the target.
- Baseline model: The target-specific sentence representation is fed to an output layer that predicts sentiment-label probabilities, and the baseline is trained by cross-entropy minimization.The training corpus supplies true labels and probabilities for those labels in the objective.
- Transfer approaches: The aspect-level model is extended with document-level classification, using a standard LSTM-based document classifier without attention alongside the aspect model.The aspect model parameters include embeddings, LSTM, attention, and output-layer parameters; the document classifier has a corresponding LSTM-based parameterization.
- Transfer approaches: Pretraining initializes the aspect model from document-level training, whereas multi-task learning jointly trains both tasks with shared embedding and LSTM layers.In multi-task learning, document representations are mean vectors over LSTM outputs, while other parameters remain task-specific.
- Transfer approaches: The combined PRET+MULT approach first pretrains on document-level examples and then uses those weights to initialize joint multi-task training.This combines document-level initialization with simultaneous aspect-level and document-level learning.
4 Experiments
Experiments on four SemEval aspect-level datasets show that document-level knowledge improves attention-based LSTM classification through pretraining, multi-task learning, and their combination. Analyses attribute gains to better sentiment representations, domain-specific word handling, neutral-instance recognition, and robustness under limited or imbalanced aspect-level data.
- Experimental Setup: Experiments use four SemEval datasets and document-level reviews from Yelp2014 and Amazon Electronics, with balanced three-class labels for transfer training.Aspect-level datasets are paired with document-level data from similar domains; each document-level dataset contains 30k balanced instances.
- Model Comparison: PRET consistently improves accuracy over LSTM+ATT by 1–3% across all four datasets, while PRET+MULT performs best overall.Macro-F1 gains are especially pronounced on the highly unbalanced D3 and D4 datasets.
- Ablation Tests: Embedding and LSTM transfer are more useful than output-layer transfer, and every tested transfer setting improves over LSTM+ATT.Embedding transfer is particularly helpful on D3 and D4, where scarce negative and neutral examples limit sentiment information learned from aspect-level data.
- Document-Level Data Analysis: Increasing document-level training data produces stable accuracy improvements across all datasets, while macro-F1 rises sharply for D3 and D4 when the proportion increases from 0 to 0.4.The PRET+MULT analysis varies the document-level training proportion from 0.0 to 1.0.
- Error Analysis: Document-level knowledge helps correct errors involving common opinion words, domain-specific terms, negation, and neutral instances.The authors relate these improvements to richer sentiment information, domain-matched document data, larger training corpora, and balanced document-level labels.
- Error Analysis: PRET+MULT reduces neutral-instance errors because balanced document-level examples compensate for the difficulty of learning neutral features from imbalanced aspect-level data.The effect is especially relevant where neutral examples are scarce or unevenly distributed.
5 Conclusion
The paper addresses limited aspect-level training data by transferring knowledge from document-level corpora. Its approaches were effective and can potentially extend other aspect-level neural models.
- The proposed approaches transfer document-level knowledge to address data scarcity in aspect-level sentiment classification.
- The authors demonstrate the approaches’ effectiveness and analyze the major benefits of knowledge transfer.
- The approaches can potentially integrate with other aspect-level neural models to further boost performance.