Source-linked AI summary
Aspect Term Extraction with History Attention and Selective Transformation
Xin Li, Lidong Bing, Piji Li, Wai Lam, Zhimou Yang
TL;DR
ATE seeks to extract explicit aspect expressions from reviews, while existing methods can overlook their relationship with opinion words. The paper proposes a framework using conditioned opinion summaries and aspect-detection history, and reports improved performance across four benchmark datasets.
Problem
Existing ATE methods do not exploit that aspect terms should co-occur with opinion-indicating words, leading to false positives in non-opinionated sentences.
Method
The framework combines a selectively transformed, token-conditioned opinion summary with truncated attention over previous aspect predictions.
Results
The framework consistently obtains the best scores on all four datasets and outperforms RNCRF on every dataset.
Takeaways & Limitations
The results support using opinion summary and aspect-detection history as complementary information for ATE.
Takeaways & Limitations
Dependency parsing can make mistakes on informal online reviews, limiting dependency-based aspect–opinion modeling in that setting.
Abstract
from arXiv · showhide
Aspect Term Extraction (ATE), a key sub-task in Aspect-Based Sentiment Analysis, aims to extract explicit aspect expressions from online user reviews. We present a new framework for tackling ATE. It can exploit two useful clues, namely opinion summary and aspect detection history. Opinion summary is distilled from the whole input sentence, conditioned on each current token for aspect prediction, and thus the tailor-made summary can help aspect prediction on this token. Another clue is the information of aspect detection history, and it is distilled from the previous aspect predictions so as to leverage the coordinate structure and tagging schema constraints to upgrade the aspect prediction. Experimental results over four benchmark datasets clearly demonstrate that our framework can outperform all state-of-the-art methods.
1 Introduction
ATE identifies opinion-targeted phrases in review sentences, but existing sequence and classification models often overlook their co-occurrence with opinion indicators. The proposed framework combines opinion summaries with aspect-detection history to improve aspect prediction.
- Task and motivation: Aspect Term Extraction identifies phrases targeted by opinion indicators in product-review sentences.For example, “operating system” and “preloaded software” are extracted from a sentence expressing love for them.
- Task and motivation: Existing sequence-labeling and token-classification methods achieve reasonable ATE performance but do not exploit aspect–opinion co-occurrence.This omission can produce false positives for frequently used aspect terms in non-opinionated sentences.
- Framework: The framework distills an opinion summary from sentence-level opinion representations conditioned on the current aspect token.Selective transformation strengthens favorable aspect–opinion correlations before attention produces the summary.
- Framework: It also distills previous aspect predictions to use tagging constraints and coordinate structures when predicting the current aspect.Earlier common aspects can guide discovery of less frequent coordinated aspects.
- Framework: The framework is presented as an approach intended to improve ATE by jointly exploiting opinion information and prediction history.Its effectiveness is evaluated against existing ATE approaches in the paper’s experiments.
2 The Proposed Model
The proposed framework combines Truncated History-Attention (THA) for aspect-detection history with Selective Transformation Network (STN) for aspect-conditioned opinion information. It predicts aspects and opinions jointly using representations built by separate LSTMs.
- Task formulation: ATE is formulated as token-level sequence labeling with B-I-O aspect labels.
- Building Memory: Two bidirectional LSTMs generate contextualized representations for the primary ATE task and auxiliary opinion-detection task.
- Capturing Aspect History: THA caches recent aspect representations, assigns normalized importance scores, and distills aspect-detection history for the current prediction.
- Capturing Opinion Summary: STN conditions opinion representations on the current aspect feature to filter noise before forming a global opinion summary.
- Aspect Prediction: The model concatenates the opinion summary with the history-aware aspect representation and feeds both into a fully connected layer for aspect prediction.
- Training: The framework jointly trains aspect and opinion extraction using token-level cross-entropy losses combined into one objective.
3 Experiment
Experiments on four SemEval datasets evaluate the framework against conventional, dependency-based, attention-based, and state-of-the-art ATE systems. Results consistently favor the proposed method, while ablations and case analyses attribute gains to selective transformation and truncated history attention.
- Datasets and settings: The evaluation uses four SemEval ABSA datasets: one laptop domain and three restaurant-domain datasets, with organizer-labeled aspect terms.The experiments use the standard SemEval train-test split; opinion supervision comes from strongly subjective MPQA words because gold opinion annotations are unavailable.
- Datasets and settings: The comparison includes CRF, Semi-CRF, LSTM, SemEval winning systems, WDEmb, MIN, RNCRF, and CMLA baselines.The framework is compared with both dependency-based and attention-based approaches, using an existing opinion lexicon rather than manually annotated opinions.
- Main results: The proposed framework obtains the best F1 scores on all four datasets, with absolute gains of 5.0%, 1.6%, 1.4%, and 1.3% over SemEval winning systems on D1–D4.Table 2 reports F1 scores in percent and combines newly implemented, reproduced, and previously published results.
- Main results: The framework outperforms RNCRF on every dataset, while RNCRF is 3.7% and 3.9% below it on D3 and D4.The authors attribute RNCRF’s degradation on informal reviews probably to dependency-parser errors.
- Main results: The framework consistently performs better than CMLA and MIN, which also use attention to distill opinion information without dependency parsing.The authors associate the gains with selective transformation reducing attention noise and history attention guiding uncommon aspects in coordinate structures.
- Ablation study: Ablations show that both THA and STN help, STN contributes slightly more, and adding STN yields about 1% absolute gains on each dataset.The full model with both components surpasses all state-of-the-art methods, whereas basic attention alone remains less competitive than CMLA.
- Attention visualization and case study: Attention visualizations show the full model assigns larger scores to opinion words related to “maitre-D” and “bathroom,” whereas removing STN introduces unrelated and non-opinion words.The authors interpret this as evidence that STN helps select opinion words relevant to a particular candidate aspect.
- Attention visualization and case study: Case analyses show that opinion summaries help identify common nouns and infrequent or misspelled aspects, while THA helps with coordinated and long aspect spans.Without these clues, the baseline models fail on the cited coordinate-structure and long-aspect cases.
4 Related Work
ATE research progressed from bootstrapping and syntactic co-extraction to neural models, including joint aspect-opinion extraction. Parsing-based approaches remain vulnerable to errors in informal reviews.
- Early ATE methods used bootstrapping or syntactic patterns based on the proximity and relationships between aspect and opinion words.These approaches included co-extraction with sophisticated syntactic patterns.
- Dependency-based approaches can be error-prone on informal online reviews because their network structures depend on dependency parsing.
- Neural models later became mainstream, with LSTM- and CNN-based systems followed by joint aspect-opinion extraction models.RNCRF used a dependency-tree recursive network, while CMLA applied multiple attention layers.
5 Concluding Discussions
The framework combines opinion summary and aspect detection history through selective transformation and truncated history attention. It outperformed joint extraction methods on ATE, while the authors question joint extraction's necessity because it does not preserve aspect-opinion correspondence.
- The framework uses opinion summary and aspect detection history through selective transformation and truncated history attention.
- The model outperformed joint extraction methods such as RNCRF and CMLA on ATE performance.
- Joint extraction methods do not account for correspondence between extracted aspect terms and opinion words.