Source-linked AI summary
A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
Heng Yang, Biqing Zeng, JianHao Yang, Youwei Song, Ruyang Xu
TL;DR
Existing ABSA research often emphasizes aspect polarity classification while giving less attention to aspect term extraction, particularly for Chinese-oriented tasks. The paper introduces LCF-ATEPC, a multilingual multi-task model combining local context focus with domain-adapted BERT, and reports state-of-the-art ATE and APC performance across Chinese and English datasets.
Problem
Existing ABSA models often focus on aspect polarity classification while neglecting aspect term extraction and Chinese-oriented ABSA research.
Method
LCF-ATEPC jointly performs aspect term extraction and polarity classification using local context focus, multi-head self-attention, BERT, and domain-adapted pretraining.
Results
LCF-ATEPC achieves state-of-the-art performance on ATE and APC tasks across Chinese review datasets and commonly used English ABSA datasets.
Takeaways & Limitations
The model provides a multilingual approach that synchronously extracts aspects and predicts their sentiment polarities.
Abstract
from arXiv · showhide
Aspect-based sentiment analysis (ABSA) task is a multi-grained task of natural language processing and consists of two subtasks: aspect term extraction (ATE) and aspect polarity classification (APC). Most of the existing work focuses on the subtask of aspect term polarity inferring and ignores the significance of aspect term extraction. Besides, the existing researches do not pay attention to the research of the Chinese-oriented ABSA task. Based on the local context focus (LCF) mechanism, this paper firstly proposes a multi-task learning model for Chinese-oriented aspect-based sentiment analysis, namely LCF-ATEPC. Compared with existing models, this model equips the capability of extracting aspect term and inferring aspect term polarity synchronously, moreover, this model is effective to analyze both Chinese and English comments simultaneously and the experiment on a multilingual mixed dataset proved its availability. By integrating the domain-adapted BERT model, the LCF-ATEPC model achieved the state-of-the-art performance of aspect term extraction and aspect polarity classification in four Chinese review datasets. Besides, the experimental results on the most commonly used SemEval-2014 task4 Restaurant and Laptop datasets outperform the state-of-the-art performance on the ATE and APC subtask.
Highlights
The paper proposes a Chinese-oriented model for jointly extracting aspect terms and classifying their polarities. The model supports Chinese and English reviews, integrates domain-adapted BERT, and achieves state-of-the-art performance on seven ABSA datasets.
- The paper proposes a model for the joint task of aspect term extraction and aspect polarity classification.
- The Chinese-oriented model is also applicable to English reviews and can handle both languages.
- Domain-adapted BERT is integrated into the proposed model to enhance its performance.
- The model achieves state-of-the-art performance on seven ABSA datasets.
1. Introduction
ABSA combines aspect term extraction with aspect polarity classification, but existing work often emphasizes polarity classification while treating extraction separately. The paper addresses this gap with LCF-ATEPC, a multilingual multi-task model designed to extract aspects and infer their polarity simultaneously.
- Research gap: Existing research concentrates more heavily on APC and often studies ATE independently, leaving joint extraction and polarity inference underexplored.The paper identifies this separation as a challenge for supervised and transfer-learning approaches.
- Task background: ABSA comprises aspect term extraction and aspect polarity classification, requiring models to identify aspects and determine their contextual sentiment polarity.APC predicts the polarity of individual aspects rather than overall sentence- or document-level sentiment.
- Data setting: The seven ATEPC datasets shown in Figure 1 are domain-specific, underscoring the paper’s focus on domain-oriented review analysis.The figure presents several samples from these datasets.
- Proposed approach: LCF-ATEPC is proposed as a multilingual multi-task model that automatically extracts aspects and analyzes their sentiment polarity simultaneously.The model applies self-attention and local context focus to aspect word extraction and integrates a pretrained BERT model.
- Scope and contribution: The model studies APC and ATE jointly for multilingual reviews, including Chinese-oriented aspect extraction and commonly used SemEval-2014 task4 datasets.The contribution is framed as a new direction for Chinese aspect extraction research.
- Training design: Domain-adapted BERT significantly improves APC performance on three datasets, especially the Restaurant dataset, while dual labels support joint-task learning.The dual labels represent aspect-term labels and sentiment-polarity labels for the input sequence.
2. Related Works
Prior ABSA research commonly treated aspect term extraction and polarity classification separately, with greater emphasis on polarity classification. The paper motivates applying newer deep-learning techniques to joint, Chinese-oriented ABSA.
- Joint ABSA: Existing ABSA methodologies often treat ATE and APC as independent tasks, while this paper addresses them jointly for multilingual and Chinese-oriented settings.
- Aspect Term Extraction: ATE research spans rule-based, dictionary-based, machine-learning, and deep-learning approaches for extracting aspect terms.Deep models learn aspect features and can reduce manual annotation effort.
- Aspect Term Extraction: Chinese-oriented ATE research includes multi-aspect bootstrapping and machine-learning methods for restaurant and hotel reviews.
- Aspect Term Extraction: Recent ATE work had not applied multi-head self-attention or pre-trained models, motivating this paper’s exploration of those techniques.
- Aspect Polarity Classification: APC research includes recurrent, convolutional, attention-based, transformer, and BERT-derived architectures for predicting sentiment polarity toward targeted aspects.
3. Methodology
LCF-ATEPC is presented as a multi-task ABSA model that jointly performs aspect term extraction and polarity classification. Its methodology uses IOB-based ATE labels, dual task inputs, and separate BERT layers for local and global context.
- Model Overview: LCF-ATEPC integrates domain-adapted BERT with a local context focus mechanism for multi-task aspect extraction and polarity classification.
- Aspect Term Extraction: The model prepares ATE as sequence labeling with B_aspect, I_aspect, and O labels marking aspect beginnings, interiors, and non-aspect tokens.
- Aspect Polarity Classification: APC predicts polarity for a targeted aspect sequence within the tokenized review.
- Multi-task Training: The joint model tokenizes inputs for two tasks and assigns each token two kinds of labels during simultaneous multi-task training.
- Context Modeling: LCF-ATEPC uses independent BERT layers for local and global context, with local processing through a context-focus layer and MHSA and global processing through MHSA.
3.3. Multi-Head Self-Attention
The model’s context representation uses multi-head self-attention together with semantic-relative distance to focus local features around a target aspect. CDM masks non-local features, whereas CDW decays their weights as alternative local-context mechanisms.
- Multi-Head Self-Attention: Multi-head self-attention extracts deep contextual semantic features and reduces negative effects from long-distance dependence.
- Multi-Head Self-Attention: Scaled-dot attention takes LCFG-learned features as input and computes attention from query, key, and value matrices.The model performs multiple scaled-dot attentions in parallel and concatenates their outputs.
- Semantic-Relative Distance: Semantic-relative distance counts tokens between a context token and the targeted aspect to determine local context.Tokens with SRD below threshold α are treated as local context.
- Context-Feature Dynamic Mask: The CDM layer masks non-local context features at corresponding output positions while retaining a relatively small amount of their semantic context.An additional MHSA encoder rebalances the masked features.
- Context-Feature Dynamic Weighting: The CDW layer provides a more moderate alternative by retaining local features and applying SRD-based weighted decay to non-local features.CDM and CDW are independent alternatives, with an optional concatenation-and-transformation approach also tested.
3.5. Feature Interactive Learning
Feature interactive learning combines local and global context representations before further encoding for polarity classification. The model therefore uses both focused local information and broader contextual features.
- Feature Interaction: LCF-ATEPC combines local and global context features rather than relying only on local features for sentiment polarity classification.
- Feature Interaction: The concatenated local-global representation is transformed and processed with an MHSA encoding step to learn feature interactions.
3.6. Aspect Polarity Classifier
The aspect polarity classifier pools the learned context representation at the aspect’s corresponding first-token position and applies Softmax to predict sentiment polarity.
- Head-pooling extracts the hidden state at the corresponding first-token position, then Softmax predicts the aspect’s sentiment polarity.The classifier operates on learned concatenated context features.
3.7. Aspect Term Extractor
The aspect term extractor performs token-level classification, inferring a category for each token to identify aspect terms.
- The extractor classifies each token using features at that token’s corresponding position.The task is framed as token-level classification over the input sequence.
- The inferred token category is represented as Y_term, with N denoting the number of token classes.
3.8. Training Details
Training uses BERT-BASE or BERT-SPC input formats with dual labels for polarity and token category, optimizing APC and ATE losses with cross-entropy and L2 regularization.
- BERT-BASE uses a standard sequence, whereas BERT-SPC appends the aspect after a separator for aspect-specific classification.BERT-SPC formats input as “[CLS]” + sequence + “[SEP]” + aspect + “[SEP].
- LCF-ATEPC redesigns inputs with dual labels for sentiment polarity and token category to support its two subtasks.Figure 5 illustrates the BERT-BASE and BERT-SPC input samples.
- Cross-entropy loss is used for both APC and ATE, while L2 regularization is applied to LCF-ATEPC.The APC and ATE losses use separate category counts, with k representing tokens in each input sequence.
4. Experiments
Experiments evaluate LCF-ATEPC across seven English and Chinese ABSA datasets, comparing joint-task variants, baselines, domain adaptation, and local-context settings. The model reports strong ATE and APC performance, while results also expose task-specific and dataset-dependent trade-offs.
- 4.1. Datasets and Hyperparameters Setting: The evaluation covers three English datasets and four Chinese review datasets, using IOB labels for ATE and polarity labels for APC.The Chinese datasets are Car, Phone, Notebook, and Camera; English datasets include Laptops, Restaurant, and Twitter.
- 4.2. Compared Methods: LCF-ATEPC achieves state-of-the-art performance on both ATE and APC tasks against current ABSA-oriented methods.Comparisons include Chinese-oriented APC baselines and BERT-based models.
- 4.1. Datasets and Hyperparameters Setting: The experiments test baseline performance, multi-task learning, domain-adapted BERT, and sensitivity to the semantic-relative-distance threshold.The SRD threshold is a controlled hyperparameter examined across datasets.
- 4.4. Overall Performance Analysis: 82%, 89%, and 96% are the reported ATE F1 scores on three English datasets for LCF-ATEPC.These scores are reported as improvements associated with the joint model across the three English datasets.
- 4.4.1. Effectiveness of Multi-task Learning: Multi-task learning can underperform single-task optimization on some datasets because APC and ATE converge differently and use multiple loss functions.Despite this trade-off, the paper reports LCF-ATEPC remains superior to other ABSA-oriented multi-task and single-task models overall.
- 4.4.2. Domain-adaption for LCF-ATEPC: Domain-adapted BERT significantly improves APC performance, with Restaurant accuracy exceeding 90%.The domain-adapted model is pretrained using Yelp reviews and Amazon laptop reviews.
5. Conclusion
The paper proposes a multilingual multi-task approach for Chinese-oriented ABSA that jointly addresses aspect term extraction and polarity classification. Across Chinese and English ABSA datasets, LCF-ATEPC achieves state-of-the-art performance on both subtasks.
- The paper proposes a multi-task learning model for Chinese-oriented ABSA that addresses the previously separate ATE and APC subtasks.
- LCF-ATEPC automatically extracts aspects and infers their polarity in multilingual reviews, including classic English sentiment-analysis tasks.
- The model achieves state-of-the-art performance on ATE and APC across three English and four Chinese ABSA datasets.