Source-linked AI summary
Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks
Binxuan Huang, Kathleen M. Carley
TL;DR
Aspect-level sentiment classification requires identifying sentiment for a specific aspect, while prior neural methods largely overlook sentence syntax. The paper proposes TD-GAT, which propagates sentiment through the original dependency graph, and reports improvements over baselines with GloVe and further gains from BERT.
Problem
Prior neural methods largely treat sentences as word sequences and ignore syntactic structure that can connect sentiment words to the correct aspect.
Method
TD-GAT represents sentences as dependency graphs and uses graph attention to propagate sentiment features from syntax context to the aspect target.
Results
TD-GAT outperforms multiple baselines with GloVe embeddings, while BERT representations further substantially boost performance on laptop and restaurant datasets.
Takeaways & Limitations
The approach uses syntactic context for aspect-level classification and remains lightweight while requiring fewer computational resources and less running time than fine-tuning original BERT.
Abstract
from arXiv · showhide
Aspect level sentiment classification aims to identify the sentiment expressed towards an aspect given a context sentence. Previous neural network based methods largely ignore the syntax structure in one sentence. In this paper, we propose a novel target-dependent graph attention network (TD-GAT) for aspect level sentiment classification, which explicitly utilizes the dependency relationship among words. Using the dependency graph, it propagates sentiment features directly from the syntactic context of an aspect target. In our experiments, we show our method outperforms multiple baselines with GloVe embeddings. We also demonstrate that using BERT representations further substantially boosts the performance.
1 Introduction
Aspect-level sentiment classification identifies sentiment for individual aspects, but many neural methods model sentences as word sequences and largely ignore syntax. TD-GAT addresses this gap with dependency-graph propagation and performs better than baselines, while remaining lightweight.
- Task and motivation: Aspect-level sentiment classification identifies each aspect’s polarity within its context, including different polarities for multiple aspects in one sentence.This is more fine-grained than sentence-level sentiment classification, which can struggle with mixed sentiment.
- Prior approaches: Earlier systems manually engineered sentiment and linguistic features, whereas later neural approaches mainly used LSTM networks and some CNNs.Neural methods reduced the need for manual feature engineering.
- Problem: Most neural methods treat sentences as word sequences, making syntactically separated sentiment words and ambiguous modifier relationships difficult to associate with the correct aspect.Dependency structure can place relevant words closer to the target and clarify relations such as modifiers of “food” and “service.”
- Proposed method: TD-GAT represents sentences as dependency graphs and propagates sentiment features from important syntax-neighborhood words toward the aspect target.A multi-layer graph attention network expands the neighborhood, while an LSTM captures aspect-related information across layers.
- Results: TD-GAT outperforms multiple baselines with GloVe embeddings and gains further performance from BERT representations on laptop and restaurant datasets.The analysis also reports better performance than fine-tuning original BERT with fewer computational resources and less running time.
2 Related Work
Related work progresses from feature-engineered classifiers to neural sequence models, including LSTMs, attention, memory networks, and CNNs. Syntax-aware alternatives use dependency information, but earlier approaches transform dependency trees in ways that can separate modifying words from the aspect.
- Task: Aspect-level sentiment classification determines the polarity of an aspect target within a context sentence and is a branch of sentiment analysis.
- Feature-engineered methods: Early systems converted sentiment lexicons, distance measures, and parse context into features for classifiers such as SVMs.Adding parse-context features improved prediction accuracy by more than one percent in one cited study.
- Neural methods: Later neural methods primarily modeled sentence word sequences with LSTMs, while attention selected aspect-related words and other work used memory networks or CNNs.
- Syntax-aware methods: Syntax-aware prior methods converted dependency trees into binary trees with the aspect at the root for recursive propagation.
- Syntax-aware methods: Binary-tree conversion can move modifying sentiment words farther from the aspect, whereas the proposed approach keeps the original syntax order unchanged.
3 Method
TD-GAT represents sentences as dependency graphs and propagates syntax-context features toward an aspect target through layered graph attention, while an LSTM tracks target information across layers before sentiment classification.
- 3.1 Text Representation: Each word becomes a node with an embedding, and undirected edges connect syntactically related words in the dependency graph.For multiword aspects, the target sequence is replaced by a special meta-node whose feature is the average of its word embeddings.
- 3.2 Graph Attention Network: A GAT layer aggregates neighboring hidden states, allowing an L-layer network to propagate features from L hops away to the aspect target.Multi-head attention assigns coefficients to neighboring nodes using learned transformations and a LeakyReLU-based attention function.
- 3.2 Graph Attention Network: The graph propagation process is expressed as Hl+1 = GAT(Hl, A; Θl), where node states are updated using the graph adjacency matrix and layer parameters.Hl contains all node states, A is the graph adjacency matrix, and Θl denotes the GAT parameters at layer l.
- 3.3 Target-Dependent Graph Attention Network: An LSTM incorporates aspect-related information from progressively larger syntax neighborhoods into the target hidden state across graph layers.The target state initially depends only on local target features; each layer adds neighborhood information while recurrent gates control information flow.
- 3.4 Final Classification: The model retrieves the final aspect-node state, maps it linearly into classification space, and computes sentiment-class probabilities with a softmax.The highest-probability class is selected, and training minimizes cross-entropy loss with L2 regularization.
4 Experiments
Experiments on SemEval 2014 laptop and restaurant reviews compare TD-GAT with prior methods, test explicit target information and depth, and examine efficiency. TD-GAT outperforms baselines with GloVe, improves further with BERT, and uses fewer resources than fine-tuning BERT.
- Datasets: The experiments use laptop and restaurant aspect–sentence pairs from SemEval 2014 Task 4, with 500 training instances held out for development.
- Baseline Comparisons: TD-GAT-GloVe outperforms all listed previous methods, while TD-LSTM’s inferior performance supports incorporating syntax information.Feature-based SVM remains a strong baseline, highlighting the relevance of feature engineering and syntax knowledge.
- Baseline Comparisons: BERT representations further boost TD-GAT, whose accuracy reaches about 80% on laptop and 83% on restaurant datasets.TD-GAT consistently improves over both BERT-AVG and BERT-CLS.
- Effects of Target Information: Explicitly modeling aspect target information improves TD-GAT-GloVe over GAT-GloVe by an average of 1.2 percentage points.
- Effects of Target Information: Explicit target information also improves TD-GAT-BERT over GAT-BERT, contributing an average of 0.95 percentage points.
- Effects of Model Depth: Increasing TD-GAT-GloVe depth to three layers greatly improves performance, whereas TD-GAT-BERT remains satisfactory even at one layer and is more depth-robust.
- Model Size: TD-GAT-BERT training on the restaurant dataset takes 1.15 seconds per epoch, compared with about 226.50 seconds for fine-tuning BERT.Switching from GloVe to BERT increases three-layer TD-GAT training time only from 1.12 to 1.15 seconds per epoch.
5 Conclusion
The paper presents TD-GAT, a target-dependent graph attention network that uses sentence dependency structure and aspect-target syntax context for aspect-level sentiment classification. Experiments on SemEval 2014 laptop and restaurant datasets show improvements with GloVe and stronger performance with BERT, while requiring fewer resources and less training time than fine-tuning original BERT.
- TD-GAT uses syntactic dependency structure and aspect-target syntax context for aspect-level sentiment classification.
- Compared with word-sequence methods, TD-GAT places modifying sentiment words closer to the aspect target and can resolve potential syntactic ambiguity.
- Experiments on SemEval 2014 laptop and restaurant datasets show TD-GAT-GloVe outperforms various baseline models.
- TD-GAT-BERT achieves much better performance while requiring fewer computational resources and less training time than fine-tuning the original BERT model.
- Future work includes modeling dependency relation types, part-of-speech tags, aspect-word importance, and sequence information to reduce noise from dependency parsing errors.