Source-linked AI summary
Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks
Chen Zhang, Qiuchi Li, Dawei Song
TL;DR
Existing attention and CNN approaches lack mechanisms for syntactical constraints and long-range word dependencies, which can lead them to use irrelevant contextual clues. The paper proposes ASGCN over dependency trees, combining graph convolution with aspect-specific feature extraction, and reports effective performance across three benchmark datasets against state-of-the-art models.
Problem
Attention and CNN approaches lack mechanisms to account for relevant syntactical constraints and long-range word dependencies in aspect-based sentiment classification.
Method
ASGCN applies multi-layer graph convolution over sentence dependency structures alongside bidirectional LSTM representations and aspect-specific masking.
Results
Experiments on three benchmarking datasets show that ASGCN addresses both identified limitations and outperforms a range of state-of-the-art models.
Takeaways & Limitations
The experiments indicate that GCN benefits overall performance by leveraging syntactical information and long-range word dependencies.
Takeaways & Limitations
On TWITTER, less grammatical sentences restrict the efficacy of syntactic variants ASGCN-DG and ASGCN-DT, which obtain sub-optimal results.
Abstract
from arXiv · showhide
Due to their inherent capability in semantic alignment of aspects and their context words, attention mechanism and Convolutional Neural Networks (CNNs) are widely applied for aspect-based sentiment classification. However, these models lack a mechanism to account for relevant syntactical constraints and long-range word dependencies, and hence may mistakenly recognize syntactically irrelevant contextual words as clues for judging aspect sentiment. To tackle this problem, we propose to build a Graph Convolutional Network (GCN) over the dependency tree of a sentence to exploit syntactical information and word dependencies. Based on it, a novel aspect-specific sentiment classification framework is raised. Experiments on three benchmarking collections illustrate that our proposed model has comparable effectiveness to a range of state-of-the-art models, and further demonstrate that both syntactical information and long-range word dependencies are properly captured by the graph convolution structure.
1 Introduction
Existing attention- and CNN-based approaches can miss syntactic constraints and long-range dependencies in aspect-based sentiment classification. The paper proposes an aspect-specific GCN over dependency structures and reports effective benchmark performance.
- Existing limitations: Attention-based models may attend to syntactically unrelated context words when identifying descriptors for an aspect.For example, they may associate “acceptable” with “size” rather than “weight.”
- Existing limitations: CNN-based models capture consecutive multi-word features but may miss sentiment expressed by non-consecutive words.The phrase “should be” can reverse the sentiment of “more friendly” despite being separated in the sentence.
- Proposed approach: The paper proposes using GCNs over syntactical dependency structures to address syntactic irrelevance and long-range multi-word dependencies.GCN layers update node representations using immediate neighbors and can propagate information through dependency trees.
- Proposed approach: ASGCN combines a bidirectional LSTM, multi-layer graph convolution, masking of non-aspect words, and feedback of aspect-specific features for sentiment prediction.The graph convolution operates on LSTM outputs, while masking retains high-level features specific to the target aspect.
- Empirical findings: Experiments on three benchmarking datasets show that ASGCN addresses both identified limitations and outperforms a range of state-of-the-art models.The experiments also demonstrate the effectiveness of leveraging syntactical information and long-range word dependencies.
2 Graph Convolutional Networks
GCNs encode graph nodes by repeatedly aggregating information from immediate neighbors. Applied to dependency trees, this provides syntactical constraints and captures long-range, non-consecutive word relations relevant to aspect sentiment.
- Each GCN layer updates a node representation using features from its immediate neighbors.For a graph with k nodes, an adjacency matrix A ∈ R^{k×k} represents node connectivity.
- An L-layer GCN allows each node to be influenced by neighboring nodes within L graph steps.
- Dependency-tree convolutions impose syntactical constraints by identifying aspect-related descriptive words according to syntactical distances.
- GCNs aggregate non-consecutive words into a smaller scope, enabling their features to be combined when they jointly describe an aspect’s polarity.
3 Aspect-specific Graph Convolutional Network
ASGCN combines bidirectional LSTM contextual representations with multi-layer graph convolution over dependency trees and aspect-specific masking. Position weighting, retrieval-based attention, and classification layers produce polarity probabilities from aspect-oriented features.
- 3.1 Embedding and Bidirectional LSTM: A bidirectional LSTM first captures contextual information regarding word orders before graph convolution operates on its output.
- 3 Aspect-specific Graph Convolutional Network: ASGCN applies multi-layer graph convolution over syntactical dependency trees and an aspect-specific masking layer to extract aspect-oriented features.
- 3.2.1 Graph Convolution over Dependency Trees: The dependency-tree variants construct an adjacency matrix from sentence words and account for the trees’ directed-graph structure.
- 3.2.1 Graph Convolution over Dependency Trees: Position weights increase the importance of context words close to the aspect to reduce noise and bias from dependency parsing.
- 3.2.2 Aspect-specific Masking: A zero-masking layer removes hidden states for non-aspect words while preserving aspect-word states, retaining contexts shaped by syntactical dependencies and long-range relations.
- 3.3 Aspect-aware Attention: Retrieval-based attention uses semantic relatedness to retrieve features relevant to aspect words before the representation is classified.
- 3.4 Sentiment Classification: A fully connected layer followed by softmax converts representation r into a probability distribution over sentiment polarity labels.
- 3.5 Training: Training uses standard gradient descent with cross-entropy loss and L2 regularization.
4 Experiments
Experiments evaluate ASGCN variants against established baselines across five datasets, showing strong overall performance and evidence that syntax, graph convolution, and aspect-specific masking matter.
- Model comparison: ASGCN-DG consistently outperforms compared models on LAP14 and REST15, while achieving comparable results on TWITTER, REST16, and REST14 under the stated comparisons.It is comparable with TNet-LF on TWITTER and REST16, and with ASCNN on REST14.
- Model comparison: ASGCN-DG outperforms ASCNN on all datasets except REST14, indicating stronger capture of long-range word dependencies.ASCNN replaces the two-layer GCN with a two-layer CNN in ASGCN.
- Model comparison: ASGCN-DT performs substantially worse than ASGCN-DG on TWITTER, LAP14, REST15, and REST16, suggesting information loss from treating dependency trees as directed graphs.The paper identifies parent-node information as a possible reason.
- Ablation study: Removing aspect-specific masking reduces competitiveness against TNet-LF, verifying the significance of filtering representations to retain aspect-specific features.The masking mechanism is part of ASGCN’s aspect-oriented feature extraction design.
- Ablation study: Removing GCN layers makes ASGCN-DG much less powerful on four datasets and on the TWITTER F1 exception, supporting GCN’s contribution to syntax and long-range relation modeling.The no-GCN variant retains position weights and aspect-specific masking.
- Case study: ASGCN-DG correctly handles all three case-study samples, including examples with multiple aspects, subjunctive wording, and negation.The examples are used to examine attention patterns and predictions across competing models.
5 Discussion
Discussion examines sensitivity to GCN depth and sentence-level aspect multiplicity. Two layers perform best, while accuracy becomes less robust beyond three aspects.
- Investigation on the Impact of GCN Layers: ASGCN-DG achieves its best Accuracy and Macro-Averaged F1 when the GCN layer number L is 2 on LAP14.The study varies L over {1,2,3,4,6,8,12}.
- Investigation on the Impact of GCN Layers: Both metrics decline as L increases, and training becomes more difficult at L = 12 because of the larger parameter count.This depth analysis motivates the two-layer configuration used in the experiments.
- Investigation on the Effect of Multiple Aspects: Samples with more than 7 aspect terms are removed because their small group sizes do not support meaningful comparison.The analysis groups training samples from LAP14 and REST14 by aspect count.
- Investigation on the Effect of Multiple Aspects: When sentences contain more than 3 aspects, accuracy fluctuates, indicating low robustness in capturing multiple-aspect correlations.The paper identifies modeling multi-aspect dependencies as future work.
6 Related Work
Related work spans sequence-based neural models and dependency-tree methods. Prior studies motivate dependency structures for capturing distant word relations and position GCNs as complementary to LSTMs.
- Sequence-based sentiment models: CNNs, RNNs, and recurrent convolutional models achieve promising sentiment-analysis performance using word sequences.The paper contrasts these sequence-based architectures with dependency-tree approaches.
- Dependency-based models: Prior work recognizes dependency trees as an effective mechanism for capturing distant word relations, with tree-based models outperforming CNNs or achieving competitive results.Examples include tree-structured LSTMs and adaptive recursive neural networks.
- Graph convolutional networks: GCNs have been applied to dependency trees and other NLP tasks, including semantic role labeling, document dating, and relation classification.Related studies treat GCNs as complementary to LSTMs or use graph convolution over dependency structures.
7 Conclusions and Future Work
The paper concludes that GCN benefits aspect-based sentiment classification by leveraging syntactical information and long-range word dependencies. It also identifies several directions for improvement, including edge labels, domain knowledge, and handling multiple aspects jointly.
- GCN improves overall aspect-based sentiment classification performance by leveraging syntactical information and long-range word dependencies.
- The study proposes extending graph neural networks to exploit labels on syntactical dependency-tree edges.The current work does not use edge-label information.
- Future work could incorporate domain knowledge into the aspect-based sentiment classification model.
- ASGCN could be extended to judge sentiments for multiple aspects simultaneously by modeling dependencies between aspect words.