Source-linked AI summary
GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis
Huaishao Luo, Lei Ji, Tianrui Li, Nan Duan, Daxin Jiang
TL;DR
Joint approaches often ignore interactions between aspect terms, and label imbalance remains an issue in sequence labeling. GRACE conditions polarity labeling on generated aspect-term labels and uses gradient harmonization, alongside virtual adversarial training and domain post-training. Across three benchmark datasets, it significantly outperforms baselines on aspect term-polarity co-extraction.
Problem
Joint approaches often ignore interactions between aspect terms, and the labeling tasks have imbalanced labels.
Source: §1 · p. 2, §2.2 · p. 4Method
GRACE conditions polarity labeling on generated aspect-term labels and uses gradient harmonization, virtual adversarial training, and domain post-training.
Source: §1 · p. 2Results
Across three benchmark datasets, GRACE significantly outperforms baselines on aspect term-polarity co-extraction.
Source: §5 · p. 9Takeaways & Limitations
The experiments verify GRACE’s effectiveness for jointly solving aspect term extraction and aspect sentiment classification.
Source: §5 · p. 9Gradient-density estimation assumes examples within the same unit region share the same gradient density.
Source: §2.2 · p. 4
Abstract
from arXiv · showhide
In this paper, we focus on the imbalance issue, which is rarely studied in aspect term extraction and aspect sentiment classification when regarding them as sequence labeling tasks. Besides, previous works usually ignore the interaction between aspect terms when labeling polarities. We propose a GRadient hArmonized and CascadEd labeling model (GRACE) to solve these problems. Specifically, a cascaded labeling module is developed to enhance the interchange between aspect terms and improve the attention of sentiment tokens when labeling sentiment polarities. The polarities sequence is designed to depend on the generated aspect terms labels. To alleviate the imbalance issue, we extend the gradient harmonized mechanism used in object detection to the aspect-based sentiment analysis by adjusting the weight of each label dynamically. The proposed GRACE adopts a post-pretraining BERT as its backbone. Experimental results demonstrate that the proposed model achieves consistency improvement on multiple benchmark datasets and generates state-of-the-art results.
1 Introduction
Aspect-based sentiment analysis co-extraction must identify aspect terms and their polarities, yet joint approaches often miss interactions between aspect terms and sequence labeling faces label imbalance. GRACE uses cascaded polarity labeling and gradient harmonization to address these issues.
1 Introduction
- ATE extracts opinion-targeted aspect terms, whereas ASC identifies the polarities expressed on those extracted terms.The introduction illustrates both tasks with positive opinions about “operating system” and “keyboard.”Source: §1 · p. 1
- Co-extraction can use joint aspect and polarity tag sets, collapsed tags that encode both, or a pipeline that classifies extracted terms.The paper follows the joint approach, which it describes as having more apparent responsibilities than the collapsed approach through parallel sequence labels.Source: §1 · pp. 1–2
- Joint approaches usually ignore interactions between aspect terms, while the numerous ‘O’ labels tend to dominate sequence-labeling loss.Most labels also have low gradients, yet their large number affects the global gradient.Source: §1 · p. 2
- GRACE conditions polarity labeling on generated aspect-term labels and applies gradient harmonization, virtual adversarial training, and domain post-training.Its cascaded labeling is designed to account for interactions between aspect terms when assigning sentiment tags.Source: §1 · p. 2
2 Model
GRACE treats aspect-term extraction and polarity prediction as two sequence-labeling tasks, then conditions polarity labeling on generated aspect-term labels. Its design combines shared BERT representations, gradient-harmonized losses, and virtual adversarial training.
2.1 Problem Statement
- The task assigns each word an aspect tag from B, I, and O and a polarity tag from POS, NEU, NEG, CON, and O.The aspect tags mark the beginning and inside of an aspect term or other words; polarity tags represent sentiment categories.Source: §2.1 · p. 2
2.2 GRACE: Gradient Harmonized and Cascaded Model
- GRACE shares shallow BERT layers across its ATE and ASC modules, with ASC representations drawn from an intermediate layer.BERT’s final-layer representation supplies ATE features, while higher layers can remain task-specific.Source: §2.2 · p. 3
- GRACE feeds generated aspect-term labels into the polarity module as key and value, conditioning sentiment-label generation on the extracted spans.The paper motivates this cascade by the need to attend to sentiment tokens and capture interactions between aspect terms.Source: §2.2 · p. 3, Figure 3 · p. 3
- Gradient harmonization reweights label losses to address label and gradient imbalance, where abundant O labels and many low-gradient labels affect the global gradient.The paper extends GHM-C, used in object detection, to this sequence-labeling setting.Source: §2.2 · p. 4
- Virtual adversarial training adds small perturbations to token embeddings and contributes an additional loss to the joint objective.The total loss combines ATE loss, polarity loss, and VAT loss.Source: §2.2 · pp. 4–5
- A consistent-polarity strategy pools representations within each predicted aspect span to produce sentiment labels, but preliminary experiments found no performance improvement from it.The spans are obtained from the generated ASC sequence labels.Source: §2.2 · p. 5
3 Experiments
GRACE is evaluated on laptop, restaurant, and Twitter datasets using domain post-training, two-stage fine-tuning, and comparisons with prior systems. It achieves the best reported F1 across the benchmark datasets, with ablations and case studies examining its components.
3.1 Datasets
- DL is a laptop-review dataset, DR combines restaurant reviews from SemEval 2014–2016, and DT contains English tweets; DL and DR use official splits, while DT uses ten-fold cross-validation.DL and DR results average five runs; evaluation uses precision, recall, and exact-match F1 for aspect-term/polarity pairs.Source: §3.1 · p. 5, Table 1 · p. 5
3.2 Post-training
- Uncased BERT-Base undergoes post-training for the paper’s tasks using combined Amazon laptop-review and Yelp restaurant-review corpora, containing 142.8M and 2.2M reviews, respectively; the maximum post-training length is 320.Training runs for 10 epochs on 8 NVIDIA Tesla V100 GPUs and takes more than 5 days; the batch size is 4,096 with gradient accumulation every 32 steps.Source: §3.2 · pp. 5–6
3.3 Settings
- Training proceeds in two stages: first fine-tune ATE, then fine-tune both ATE and ASC, with virtual adversarial training added during the first stage.The selected configuration shares nine layers and uses two ASC transformer layers; hyperparameters are tuned on DL and DR validation sets.Source: §3.3 · p. 6
3.4 Baseline Methods
- Comparisons include end-to-end, joint, and pipeline baselines, plus multi-task ABSA models evaluated on the datasets they report.The baselines include E2E-TBSA, DOER, SPAN, BERT-E2E-ABSA, and models such as IMN, WHW, and RACL-BERT.Source: §3.4 · p. 6
3.5 Results and Analysis
- GRACE records the best F1 across datasets, beating SPANBase by 8.50%, 6.50%, and 2.07% on DL, DR, and DT, respectively.It also gains at least 7.31%, 1.84%, 2.05%, and 0.81% over IMN, WHW, TAS-BERT, IKTN+BERT, and RACL-BERT on DL, DR-14, DR-15, and DR-16, respectively.Source: §3.5 · p. 7, Table 2 · p. 7
- Without GHL, scores drop more than without VAT, while removing post-training causes the largest drops on laptop and restaurant datasets.GRACE also achieves state-of-the-art ATE results, and its lower scores without the ASC branch indicate that ASC could enhance ATE.Source: §3.5 · pp. 7–8, Table 4 · p. 7
- On DL and DR, GRACE scores 70.71% and 78.07%, exceeding BASE’s 68.35% and 76.76%, respectively.BASE omits generated aspect-term labels from the ATE branch when training ASC.Source: §3.5 · p. 8
- Case studies show GRACE makes errors on two examples but correctly predicts cases where the coordinating conjunction is minimal.The authors attribute the latter results to both cascaded-labeling information and gradient-harmonized label balancing.Source: §3.5 · p. 8, Table 5 · p. 8
4 Related Work
Prior work treats aspect term extraction and sentiment classification separately, while co-extraction methods frame them as collapsed, joint, or pipelined tasks. GRACE follows the joint approach and addresses a label-imbalance issue that prior models largely overlooked.
4 Related Work
- Prior ATE and ASC research includes unsupervised and supervised extraction methods, while sentiment classification uses context relations, neural networks, attention, memory, capsules, and graphs.The cited examples span frequent-pattern mining, sequence labeling, and graph neural networks.Source: §4 · p. 8
- Co-extraction approaches include collapsed labeling, which combines aspect boundaries and polarity in tags, and joint labeling, which assigns separate tag sets.Pipelined methods first extract aspect terms and then classify their polarities; GRACE follows the joint approach.Source: §4 · p. 8
- Prior co-extraction models largely overlooked label imbalance in sequence labeling; the authors present GRACE as the first ABSA work to address it.Source: §4 · p. 9
5 Conclusion
GRACE jointly solves aspect term extraction and sentiment classification using cascaded labeling and gradient harmonization. Experiments on three benchmark datasets show that it significantly outperforms baselines on aspect term-polarity co-extraction.
5 Conclusion
- GRACE uses cascaded labeling and multi-head attention to strengthen interaction between aspect terms and focus on sentiment tokens for each term.Source: §5 · p. 9
- GRACE applies gradient harmonization to alleviate label imbalance, and adds virtual adversarial training and domain-dataset post-training to improve extraction performance.Source: §5 · p. 9
- On three benchmark datasets, GRACE significantly outperforms baselines on aspect term-polarity co-extraction.Source: §5 · p. 9