Source-linked AI summary
Relational Graph Attention Network for Aspect-based Sentiment Analysis
Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, Rui Wang
TL;DR
ABSA models can confuse which opinion words belong to which aspect, especially in sentences containing multiple aspects and complex syntax. The paper reshapes dependency parses into aspect-oriented trees and encodes them with R-GAT; experiments report better aspect–opinion connections and improved GAT and BERT performance. However, error analysis identifies complicated sentence phenomena that remain beyond current models.
Problem
Attention-based ABSA models can confuse aspect–opinion connections because language complexity and multiple aspects make implicit associations unreliable.
Method
The paper roots a reshaped and pruned dependency tree at each target aspect and encodes its labeled edges with a relational graph attention network.
Results
Experiments on three public datasets show that R-GAT better establishes aspect–opinion connections and significantly improves GAT and BERT performance.
Takeaways & Limitations
Aspect-oriented syntax provides a unified structure for focusing sentiment prediction on target-specific connections while supporting batch and parallel operations.
Takeaways & Limitations
Error analysis finds that misleading neutral reviews, difficult comprehension, advice, and double negation remain challenging for current models.
Abstract
from arXiv · showhide
Aspect-based sentiment analysis aims to determine the sentiment polarity towards a specific aspect in online reviews. Most recent efforts adopt attention-based neural network models to implicitly connect aspects with opinion words. However, due to the complexity of language and the existence of multiple aspects in a single sentence, these models often confuse the connections. In this paper, we address this problem by means of effective encoding of syntax information. Firstly, we define a unified aspect-oriented dependency tree structure rooted at a target aspect by reshaping and pruning an ordinary dependency parse tree. Then, we propose a relational graph attention network (R-GAT) to encode the new tree structure for sentiment prediction. Extensive experiments are conducted on the SemEval 2014 and Twitter datasets, and the experimental results confirm that the connections between aspects and opinion words can be better established with our approach, and the performance of the graph attention network (GAT) is significantly improved as a consequence.
1 Introduction
ABSA must identify sentiment toward individual aspects, but attention mechanisms can confuse aspects with nearby opinion words. The paper addresses this by constructing aspect-oriented dependency trees and encoding them with R-GAT.
- Task and motivation: ABSA determines sentiment polarity toward one or more aspects in a sentence, where sentence-level polarity can obscure opposing aspect sentiments.For example, “great food but the service was dreadful” requires separate predictions for food and service.
- Task and motivation: Attention mechanisms can assign high weight to misleading opinion words because language complexity, proximity, and multiple aspects create ambiguous connections.The review example “delicious ... noodles ... terrible vegetables” illustrates how proximity can mislead attention toward the wrong aspect.
- Task and motivation: Existing dependency-tree approaches may ignore dependency relations, encode irrelevant portions of the tree, and make batch optimization inconvenient.These limitations motivate a more targeted syntactic representation.
- Proposed approach: The proposed aspect-oriented tree reshapes and prunes an ordinary dependency tree so the target aspect becomes its root and task-related connections are retained.The structure is designed to focus on aspects and potential opinion words while facilitating batch and parallel operations.
- Proposed approach: R-GAT encodes labeled dependency relations, and experiments on SemEval 2014 and Twitter show significantly improved GAT performance and superior performance to baselines.The paper presents the model as a way to establish aspect–opinion connections more effectively.
2 Related Work
Related ABSA work uses attention, syntactic rules, dependency trees, and graph neural networks to connect aspects with opinion words. These approaches provide useful structure but may overlook dependency labels that identify aspect–opinion relations.
- Attention-based approaches: Attention-based models implicitly exploit sentence structure to identify sentiment information associated with a target aspect.Examples include attention-based LSTMs, multi-layer attention, multi-hop memory attention, and multi-grained attention.
- Syntax-based approaches: Syntactic approaches explicitly establish connections between target aspects and other words using handcrafted rules, partial alignment, or neural models.Dependency relations are treated as important evidence for identifying potential opinion words.
- Graph-based approaches: Graph neural networks use dependency trees to learn word representations for sentiment classification, including GCN- and GAT-based approaches.These methods combine dependency-tree representations with other features or explicitly model dependency relationships.
- Illustration: Figure 1 compares aspect, attention, and syntax using restaurant-review examples, with dependency labels, attention scores, highlighted high-attention words, and bracketed aspect labels.The figure provides a visual basis for examining whether attention aligns with syntactic aspect–opinion relationships.
- Open limitation: A central limitation of existing graph approaches is that they generally ignore dependency relations that might identify connections between aspects and opinion words.This gap motivates the paper’s relational graph attention formulation.
3 Aspect-Oriented Dependency Tree
The paper constructs an aspect-oriented dependency tree by rooting each sentence tree at the target aspect and pruning or redirecting relations to emphasize relevant syntax. This structure reduces interference from unrelated nodes and supports aspect-focused modeling.
- Motivation: Aspect-oriented trees are motivated by attention models that can connect aspects to the wrong opinion words in syntactically complex sentences.Dependency relations can provide explicit connections that attention mechanisms may miss.
- Construction: The construction starts with an ordinary dependency parse and places the target aspect at the root, treating multiword aspects as entities.Nodes directly connected to the aspect become its children.
- Construction: Other dependency relations are discarded and replaced by virtual n:con edges from the aspect, with n representing node distance.A separate tree is constructed for each aspect when a sentence contains multiple aspects.
- Benefits: Each aspect therefore receives its own tree, reducing influence from unrelated nodes and relations.This design focuses the representation on connections between the aspect and potential opinion words.
- Benefits: The unified structure aggregates dependency relations at multiword aspects instead of requiring extra pooling or attention operations.It also facilitates batch and parallel operations during training.
4 Relational Graph Attention Network
R-GAT extends graph attention by incorporating dependency-relation information through relational heads, while using an aspect-oriented tree and attention heads to encode target-centered syntax for sentiment prediction.
- R-GAT extends GAT to encode graphs with labeled dependency edges.
- GAT updates each node by aggregating neighborhood representations with multi-head attention and dot-product attention coefficients.
- Relational heads act as relation-wise gates that control information flow from neighboring nodes with different dependency relations.The model maps dependency relations to vector representations before computing relational heads.
- R-GAT combines K attentional heads with M relational heads to compute each node’s final representation.
- The model encodes word and aspect representations with BiLSTMs, applies R-GAT to the aspect-oriented tree, and predicts sentiment from the root representation.The root representation is passed through a fully connected softmax layer, and training uses cross-entropy loss over sentence-aspect pairs.
5 Experiments
The experiments evaluate the proposed methods using public sentiment-analysis datasets and report dataset statistics as part of the evaluation setup.
- The evaluation section introduces datasets and baseline methods before reporting results, error analysis, and discussion.
- Table 1 provides statistics for the three datasets used in the experiments.
5.1 Datasets
The experiments use three public sentiment-analysis datasets: Laptop and Restaurant reviews from SemEval 2014 and a Twitter dataset.
- Three public sentiment-analysis datasets are used for evaluation.
- The datasets comprise Laptop and Restaurant review datasets from SemEval 2014 Task and a Twitter dataset.Dataset statistics are reported in Table 1.
- Dependency parsing uses the Biaffine Parser, while R-GAT uses 300-dimensional dependency-relation embeddings and GloVe word embeddings.
5.2 Baseline Methods
The comparison includes syntax-aware, attention-based, other recent, and proposed R-GAT methods, with an optional BERT-enhanced variant.
- Syntax-aware models: The baselines include syntax-aware models such as LSTM+SynATT, ASGCN, CDT, GAT, and TD-GAT.
- Attention-based models: Attention-based baselines include ATAE-LSTM, IAN, RAM, MGAN, attention-equipped LSTM, and fine-tuned BERT.
- Other recent methods: Other recent comparison methods include GCAE, JCI, and TNET.
- Our methods: R-GAT is the proposed relational graph attention network, while R-GAT+BERT replaces its BiLSTM and attentional heads with BERT.
5.3 Results and Analysis
Experiments show that R-GAT improves syntax-aware sentiment classification, while performance depends on parser quality, aspect separation, tree design, and relation modeling. Error analysis identifies several difficult linguistic cases that remain beyond current models.
- Overall Performance: R-GAT outperforms most baselines, and R-GAT+BERT further improves the strong BERT model to achieve a new state of the art.The results support improved encoding of important syntactic structures for sentiment analysis.
- Effect of Multiple Aspects: Aspects with nearer distances tend to receive lower accuracy scores, but R-GAT improves GAT and BERT across distance ranges.Multiple-aspect reviews were analyzed using Euclidean distances between averaged aspect embeddings.
- Effect of Different Parsers: Higher-quality dependency parsing produces higher sentiment classification accuracy, indicating further gains may come from advances in parsing techniques.The comparison evaluates Stanford and Biaffine parsers using UAS and LAS alongside aspect-based sentiment classification.
- Ablation Study: R-GAT improves on aspect-oriented trees across all three datasets, while removing the virtual n:con relation causes a considerable performance drop.The maximal n:con setting is 4; longer dependency distances were not more effective in empirical tests.
- Error Analysis: Misclassifications primarily involve misleading neutral reviews, followed by difficult comprehension, advice without sentiment clues, and double negation.The analysis examines 100 misclassified Restaurant examples from R-GAT and R-GAT+BERT.
6 Conclusion
The paper introduces an aspect-oriented dependency tree and R-GAT to encode syntax for aspect-based sentiment classification. Experiments show better aspect–opinion connections and improved GAT and BERT performance across three public datasets.
- 6 Conclusion: The method reshapes and prunes ordinary dependency parse trees to root them at target aspects, then encodes them with R-GAT.The approach targets comprehensive syntax information for aspect-based sentiment analysis.
- 6 Conclusion: Experiments on three public datasets show that R-GAT better establishes aspect–opinion connections and significantly improves GAT and BERT performance.An ablation study validates the new tree structure and relational heads, while error analysis examines incorrectly predicted examples.