Source-linked AI summary
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
Sebastian Ruder, Parsa Ghaffari, John G. Breslin
TL;DR
The paper addresses the limited use of review-level structure in sentence-wise ABSA classification. It introduces a hierarchical bidirectional LSTM that combines sentence representations, aspect embeddings, and review context; the model outperforms sentence-level baselines and reaches or exceeds state-of-the-art performance across multilingual, multi-domain datasets without hand-engineered features or external resources.
Problem
ABSA models often classify review sentences independently, missing inter-sentence discourse and sentiment context that can inform each sentence's sentiment.
Method
H-LSTM combines sentence-level bidirectional LSTMs with aspect embeddings and a review-level bidirectional LSTM to model intra- and inter-sentence relations.
Results
The model outperforms sentence-level CNN and Bi-LSTM baselines, compares favorably with the state-of-the-art, and achieves state-of-the-art results on 5 of 11 datasets.
Takeaways & Limitations
Review structure and sentential context can improve ABSA predictions without hand-engineered features or external resources, including for low-resource languages.
Takeaways & Limitations
Performance gaps indicate limits when the approach uses only data available at training time, especially for some high-resource languages and the Turkish dataset.
Abstract
from arXiv · showhide
Opinion mining from customer reviews has become pervasive in recent years. Sentences in reviews, however, are usually classified independently, even though they form part of a review's argumentative structure. Intuitively, sentences in a review build and elaborate upon each other; knowledge of the review structure and sentential context should thus inform the classification of each sentence. We demonstrate this hypothesis for the task of aspect-based sentiment analysis by modeling the interdependencies of sentences in a review with a hierarchical bidirectional LSTM. We show that the hierarchical model outperforms two non-hierarchical baselines, obtains results competitive with the state-of-the-art, and outperforms the state-of-the-art on five multilingual, multi-domain datasets without any hand-engineered features or external resources.
1 Introduction
Reviews contain rhetorical and sentential interdependencies that can inform aspect-based sentiment classification, but common neural models largely miss inter-sentence discourse structure. The paper introduces H-LSTM to model both intra- and inter-sentence relations without hand-engineered features or sentiment lexica.
- ABSA examines product or service aspects in finer detail than general sentiment analysis.
- RST structures reviews through rhetorical relations such as Elaboration and Background connecting sentences and clauses.
- Surrounding sentiment and discourse relations can inform current-sentence sentiment and disambiguate equivocal sentences.
- Existing CNN, LSTM, and recursive neural approaches capture intra-sentence relations but miss inter-sentence discourse relations.
- H-LSTM leverages intra- and inter-sentence relations, using only sentences and their structure, and avoids hand-engineered features and sentiment lexica.
2 Related Work
Prior ABSA systems rely heavily on engineered features, external resources, or limited sentence context. The paper positions its hierarchical model as more expressive while avoiding feature engineering, positional information, and parser outputs.
- Earlier ABSA approaches use hand-crafted n-gram, part-of-speech, negation, and sentiment-lexicon features.
- Zhang and Lan consider neighboring sentences, but without modeling their discourse structure, making the approach less expressive than the proposed model.
- The proposed approach requires no feature engineering, positional information, or parser outputs, which are often unavailable for low-resource languages.
- Related hierarchical models primarily support representation learning, document reconstruction, dialogue generation, or salient-sentence extraction.
3 Model
The model builds sentence representations with bidirectional LSTMs, combines them with aspect embeddings, and processes the resulting sequence with a review-level bidirectional LSTM. A final softmax predicts sentiment for each sentence.
- Sentence and aspect representation: Reviews are represented as padded sequences of sentences, while each sentence is represented by concatenated word embeddings.
- Sentence and aspect representation: Each aspect combines entity and attribute embeddings by averaging them into an aspect vector.
- Bidirectional LSTM: LSTMs use input, output, and forget gates to model long-range dependencies relevant to sentiment.
- Bidirectional LSTM: Bidirectional LSTMs process words or sentences in both chronological and reverse order, allowing the model to use preceding and successive context.
- Hierarchical bidirectional LSTM: H-LSTM feeds sentence-level Bi-LSTM outputs concatenated with the aspect vector into review-level Bi-LSTMs, whose outputs predict sentiment distributions.
4 Experiments
Experiments evaluate H-LSTM across 11 multilingual, multi-domain ABSA datasets and compare it with competition systems and sentence-level neural baselines. The comparisons isolate the contribution of hierarchical review structure while documenting the training setup.
- Datasets: The evaluation uses 11 datasets spanning five domains and eight languages, with 300–400 reviews and 1250–6000 sentences per dataset.
- Datasets: Each sentence has none, one, or multiple domain-specific aspects, with a sentiment value for each aspect.
- Training details: The experiments use 300-dimensional word embeddings, 200-dimensional one-layer LSTMs, 15-dimensional aspect embeddings, dropout 0.5, and gradient clipping norm 5.
- Training details: Aspects are unrolled within reviews, sentences without aspects are removed, padding predictions are ignored, and Chinese data is segmented before tokenization.
- Comparison models: H-LSTM and HP-LSTM are compared with the SemEval best system, IIT-TUDA, XRCE, a sentence-level CNN, and a sentence-level Bi-LSTM.
5 Results and Discussion
The hierarchical model generally improves aspect-based sentiment predictions by using review-level context, while remaining competitive with state-of-the-art systems without costly external resources. Results also show gains from pre-trained embeddings, but performance remains limited for some high-resource and low-resource settings.
- The hierarchical model outperforms sentence-level CNN and Bi-LSTM baselines for almost all domain-language pairs.The reported improvement is attributed to taking review structure into account.
- The model outperforms the state-of-the-art on four datasets with random embeddings and five datasets with pre-trained embeddings.It remains competitive with the competition's best single models while using no expensive hand-crafted features or external resources.
- Table 2 shows that context from non-neighboring sentences can disambiguate equivocal aspect sentiment predictions.Examples include service and food-quality context changing a sentence-level LSTM prediction from negative to the H-LSTM's positive prediction.
- 5.1 Pre-trained embeddings: Pre-trained embeddings produce significant gains across almost all languages, except Russian, Arabic, and Chinese.They also enable state-of-the-art performance in the Dutch phones domain.
- 5.2 Leveraging additional information: The approach has clear limits: it trails state-of-the-art systems in Turkish and leaves performance gaps for high-resource languages.The authors attribute these limits to relying only on data available at training time; unsupervised language information alone is insufficient for high-resource languages.
- 5.2 Leveraging additional information: Domain-specific sentiment lexicons and other domain information remain open avenues because the tested lexicons did not significantly improve results with pre-trained embeddings.The authors leave better incorporation of such resources for future work.
6 Conclusion
The paper presents a hierarchical review model for aspect-based sentiment analysis that uses review structure and sentential context. It outperforms sentence-only models, remains competitive with resource-intensive systems, and achieves state-of-the-art results on five of eleven datasets.
- The hierarchical model uses review structure and sentential context to predict aspect-based sentiment.
- It outperforms models relying only on sentence information while achieving performance competitive with models using external resources and hand-engineered features.
- It achieves state-of-the-art results on 5 out of 11 aspect-based sentiment analysis datasets.