Source-linked AI summary
Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence
Chi Sun, Luyao Huang, Xipeng Qiu
TL;DR
ABSA must identify sentiment toward specific aspects, while direct BERT use has yielded limited improvement on (T)ABSA. The paper constructs an auxiliary sentence, reformulates the task as sentence-pair classification, and fine-tunes BERT, achieving new state-of-the-art results on SentiHood and SemEval-2014 Task 4.
Problem
ABSA seeks fine-grained polarity toward a specific aspect, but direct use of pre-trained BERT has produced little improvement on (T)ABSA.
Method
The paper constructs an auxiliary sentence from target and aspect information and transforms (T)ABSA into a sentence-pair classification task for fine-tuned BERT.
Results
The fine-tuned BERT sentence-pair models achieve new state-of-the-art results on SentiHood and SemEval-2014 Task 4, with BERT-pair outperforming other models on SentiHood.
Takeaways & Limitations
The auxiliary-sentence construction lets pre-trained BERT's strengths in QA and NLI be utilized for (T)ABSA and can also be applied to similar tasks.
Abstract
from arXiv · showhide
Aspect-based sentiment analysis (ABSA), which aims to identify fine-grained opinion polarity towards a specific aspect, is a challenging subtask of sentiment analysis (SA). In this paper, we construct an auxiliary sentence from the aspect and convert ABSA to a sentence-pair classification task, such as question answering (QA) and natural language inference (NLI). We fine-tune the pre-trained model from BERT and achieve new state-of-the-art results on SentiHood and SemEval-2014 Task 4 datasets.
1 Introduction
ABSA addresses fine-grained sentiment toward specific aspects, but standard sentiment analysis and direct BERT use do not adequately handle multiple targets or improve (T)ABSA. The paper converts (T)ABSA into sentence-pair classification with an auxiliary sentence and fine-tunes BERT.
- ABSA identifies fine-grained polarity toward specific aspects, enabling more granular evaluation of product or service quality.
- TABSA handles comments involving multiple objects by identifying aspects associated with each target and resolving their polarity.
- Pre-trained BERT has been effective in QA and NLI, but its direct use has produced little improvement on (T)ABSA.
- The paper constructs an auxiliary sentence and transforms (T)ABSA into a sentence-pair classification task.
- Fine-tuning pre-trained BERT in this formulation achieves new state-of-the-art results on SentiHood and SemEval-2014 Task 4 datasets.
2 Methodology
The method converts (T)ABSA into sentence-pair classification by constructing auxiliary sentences, then fine-tunes BERT to classify the resulting pairs.
- Task formulation: TABSA predicts positive, negative, or none for each target-aspect pair, while ABSA predicts aspect detection and polarity jointly.
- Auxiliary sentence construction: The paper considers four auxiliary-sentence constructions: QA-M, NLI-M, QA-B, and NLI-B.QA-M uses questions; NLI-M uses pseudo-sentences; QA-B and NLI-B append candidate polarity labels.
- Auxiliary sentence construction: QA-M generates a question from each target-aspect pair, such as asking about the safety of LOCATION1.
- Auxiliary sentence construction: QA-B creates three labeled sequences for positive, negative, and none, then selects the category with the highest yes-matching score.
- Auxiliary sentence construction: The constructed auxiliary sentence transforms single-sentence classification into sentence-pair classification, which significantly improves TABSA results.
- BERT fine-tuning: BERT represents the original and auxiliary sentences as a token sequence using token, segment, and position embeddings, with [CLS] as the classification representation.
- BERT fine-tuning: Fine-tuning feeds the final hidden state of [CLS] through a classification layer and softmax to obtain category probabilities.The pooled vector is C ∈ R^H, the classifier matrix is W ∈ R^{K×H}, and P = softmax(CW^T).
- Model variants: BERT-single trains separate classifiers for target-aspect or aspect sentiment problems, whereas BERT-pair uses the four auxiliary-sentence variants.
3 Experiments
The experiments evaluate the method on SentiHood and SemEval-2014 Task 4 using BERT variants and established baselines. BERT-pair improves substantially over competing models on SentiHood and further improves over BERT-single on SemEval-2014.
- SentiHood: SentiHood evaluation covers target-aspect detection and sentiment polarity for four frequent aspects, using strict accuracy, Macro-F1, AUC, accuracy, and macro-average AUC.The dataset contains 5,215 sentences and includes both single-target and multiple-target cases.
- SemEval-2014 Task 4: SemEval-2014 evaluation jointly addresses aspect category detection and polarity using Micro-F1 and accuracy, respectively.
- SentiHood: BERT-single detects aspects better than Dmu-Entnet but trails SenticLSTM and Dmu-Entnet in sentiment-classification accuracy by 3.8 and 5.5, respectively.
- SentiHood: 9.4 macro-average F1 and 2.6 accuracies improvement over Dmu-Entnet are obtained by BERT-pair on SentiHood aspect detection and sentiment analysis.BERT-pair-NLI models perform relatively better on aspect detection, while BERT-pair-QA models perform better on sentiment classification.
- SemEval-2014 Task 4: BERT-single improves both SemEval-2014 subtasks, while BERT-pair improves further; BERT-pair-NLI-B leads detection and BERT-pair-QA-B leads polarity across all settings.
4 Discussion
The discussion attributes BERT-pair’s gains to constructing auxiliary sentences and matching the sentence-pair format of QA and NLI. The approach also extends from TABSA to ABSA and other similar tasks.
- Discussion: Auxiliary sentences expand each original sentence into target-aspect-specific sentence pairs, increasing the effective corpus examples for classification.
- Discussion: BERT’s strength on QA and NLI supports the sentence-pair formulation, drawing on masked language modeling and next sentence prediction.
- Discussion: Direct BERT fine-tuning does not improve TABSA, whereas separating target and aspect into an auxiliary sentence enables the model’s pre-trained advantage to be utilized.
- Discussion: BERT-pair-QA-B and BERT-pair-NLI-B achieve better sentiment-classification AUC values, probably because they model label information.
5 Conclusion
The paper concludes that auxiliary-sentence construction converts (T)ABSA into sentence-pair classification and, with fine-tuned BERT, achieves new state-of-the-art results.
- Conclusion: The method transforms (T)ABSA from single-sentence classification into sentence-pair classification through an auxiliary sentence.
- Conclusion: Fine-tuned BERT produces new state-of-the-art results, while comparisons between single-sentence and sentence-pair classification verify the conversion method’s validity.
- Conclusion: The authors propose applying the conversion method to other similar tasks in future work.