Source-linked AI summary
Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis
Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Hua Jin, Dacheng Tao
TL;DR
ABSA requires aspect-specific sentiment judgments, while existing approaches to external knowledge can be complex and its complementarity with linguistic information remains underexplored. KGAN combines context, syntax, and knowledge representations through hierarchical fusion. The authors report state-of-the-art performance across the evaluated datasets with RoBERTa, alongside effectiveness, robustness, and a latency–performance trade-off.
Problem
External knowledge methods for ABSA can require complex, computationally intensive graph operations, and the complementarity of external knowledge with linguistic information has not been thoroughly studied.
Source: §1 · pp. 1–2Method
KGAN learns context-, syntax-, and knowledge-based representations and hierarchically fuses them, using attention to obtain aspect-specific knowledge representations.
Source: §1 · p. 2, §3.4 · p. 5Results
With RoBERTa, KGAN achieves state-of-the-art performance across the evaluated ABSA datasets, and the authors report effectiveness and robustness.
Source: §1 · p. 2, §5 · p. 11Takeaways & Limitations
The authors report that KGAN achieves a better trade-off between latency and performance while combining contextual, syntactic, and external knowledge.
Source: §5 · p. 11The authors identify validation of KGAN’s multiview approach on other language-understanding tasks as future work.
Source: §5 · p. 11
Abstract
from arXiv · showhide
Aspect-based sentiment analysis (ABSA) is a fine-grained task of sentiment analysis. To better comprehend long complicated sentences and obtain accurate aspect-specific information, linguistic and commonsense knowledge are generally required in this task. However, most current methods employ complicated and inefficient approaches to incorporate external knowledge, e.g., directly searching the graph nodes. Additionally, the complementarity between external knowledge and linguistic information has not been thoroughly studied. To this end, we propose a knowledge graph augmented network KGAN, which aims to effectively incorporate external knowledge with explicitly syntactic and contextual information. In particular, KGAN captures the sentiment feature representations from multiple different perspectives, i.e., context-, syntax- and knowledge-based. First, KGAN learns the contextual and syntactic representations in parallel to fully extract the semantic features. Then, KGAN integrates the knowledge graphs into the embedding space, based on which the aspect-specific knowledge representations are further obtained via an attention mechanism. Last, we propose a hierarchical fusion module to complement these multi-view representations in a local-to-global manner. Extensive experiments on five popular ABSA benchmarks demonstrate the effectiveness and robustness of our KGAN. Notably, with the help of the pretrained model of RoBERTa, KGAN achieves a new record of state-of-the-art performance among all datasets.
1 INTRODUCTION
ABSA predicts sentiment polarity for each aspect, but incorporating external knowledge can require complex, computationally intensive graph operations. KGAN combines context-, syntax-, and knowledge-based representations, and its RoBERTa-based version achieves state-of-the-art results across the evaluated benchmarks.
1 INTRODUCTION
- ABSA predicts sentiment polarity toward each given aspect, so one sentence can contain aspects with opposite polarities.The example assigns positive sentiment to food and negative sentiment to service.Source: §1 · p. 1
- Prior external-knowledge methods may require complex model changes or aspect-specific subgraph construction, which adds computation.The authors identify this cost as especially relevant when sentences contain many aspect terms.Source: §1 · pp. 1–2
- KGAN learns context and syntax in parallel, embeds external knowledge graphs, uses attention for aspect-specific knowledge, and hierarchically fuses the views.The fusion is intended to use complementarity among context-, syntax-, and knowledge-based representations.Source: §1 · p. 2
- With RoBERTa, KGAN achieves state-of-the-art accuracy and macro-F1 across the evaluated ABSA datasets.The authors also report a good trade-off between efficiency and performance.Source: §1 · p. 2
2 RELATED WORKS
Prior ABSA work uses contextual features and syntactic structure to connect aspects with relevant opinion words. Other approaches add external knowledge, while KGAN’s related-work framing emphasizes combining that knowledge with both context and syntax through a simpler strategy.
2.1 Aspect-based Sentiment Analysis
- Context-based ABSA methods use CNNs or LSTMs with attention, while syntax-based methods use dependency trees and graph neural networks to connect aspects with opinion words.Context models can confuse aspect–opinion connections when a sentence contains multiple aspects with different polarities.Source: §2.1 · p. 2
2.2 Incorporating External Knowledge
- Prior ABSA approaches incorporate external knowledge through methods such as unified sentiment-and-structure modeling, node sampling, or knowledge-enhanced BERT.The cited work is motivated in part by knowledge not being explicitly expressed in ABSA datasets.Source: §2.2 · p. 3
- KGAN’s related-work framing contrasts its multiview approach with AR-BERT’s focus on aspect relations, combining knowledge with both contextual and syntactic information.The authors characterize their strategy for modeling knowledge graphs as simpler and more efficient.Source: §2.2 · p. 3
3 KNOWLEDGE GRAPH AUGMENTED NETWORK
KGAN models aspect sentiment through contextual, syntactic, and knowledge-based representations, then combines those views hierarchically. Its branches pair sentence-level information with the target aspect to form aspect-specific features.
3.1 Problem Formulation
- ABSA predicts the sentiment polarity of a sentence toward a specified aspect span.The task defines a sentence S and an aspect T within it, with positive, neutral, and negative polarities.Source: §3.1 · pp. 3–4
3.2 Overview of the KGAN model
- KGAN processes each sentence–aspect pair through parallel context, syntax, and knowledge branches before hierarchical fusion.The contextual and syntactic branches extract sentence information, while the knowledge branch supplies external graph information.Source: §3.2 · p. 4
3.3.1 Context-based Representations
- Contextual representations use pretrained embeddings, BiLSTMs, and attention to capture long-range dependencies and aggregate words relevant to the aspect.The method also encodes relative positions between context words and aspects in the word embeddings.Source: §3.3.1 · p. 4
3.3.2 Syntax-based Representations
- The syntax branch shares embeddings and a BiLSTM with the context branch, then applies a two-layer dependency-tree GCN to derive syntax-aware aspect features.Aspect masking and attention focus the features on the target aspect and related opinion words.Source: §3.3.2 · pp. 4–5
3.3.3 Knowledge-based Representations.
- KGAN embeds WordNet entity–relation triples, maps sentence and aspect words into that space, and uses soft attention to obtain aspect-specific knowledge features.WordNet supplies word-form and sense pairs, with synonym sets representing concepts; the knowledge embeddings are concatenated with hidden states.Source: §3.3.3 · p. 5
3.4 Hierarchical Fusion Module
- Hierarchical fusion first combines each pair of context, syntax, and knowledge features, then uses a 3×3 convolution to selectively integrate the resulting sentiment features.The authors describe this local-to-global fusion as a way to integrate external knowledge with contextual and syntactic information.Source: §3.4 · p. 5
4.1 Datasets and Experimental Settings
Experiments use five standard ABSA datasets, evaluate accuracy and macro-F1, and compare KGAN with prior context-, syntax-, and knowledge-based methods across pretrained settings.
4.1 Datasets and Experimental Settings
- Five benchmarks cover Laptop14, Restaurant14, Twitter, Restaurant15, and Restaurant16.The first four named SemEval datasets come from the 2014, 2015, and 2016 ABSA challenges; Twitter is a collection of tweets.Source: §4.1 · p. 6
- Accuracy (“Acc”) and macro-F1 (“F1”) are the reported evaluation metrics.The compared methods also include context-based approaches such as ATAE-LSTM, RAM, and TNet-AS, and knowledge-based approaches such as MTKEN and Sentic GCN.Source: §4.1 · pp. 6–7
- KGAN is evaluated with GloVe, BERT, and RoBERTa, while knowledge embeddings remain fixed during training.The BERT and RoBERTa configurations use base models with 12 layers and 768-dimensional hidden vectors.Source: §4.1 · p. 6
- Comparisons span context-based, syntax-based, and external-knowledge ABSA methods, including ASGCN, DualGCN, SK-GCN, and Sentic-LSTM.The comparison table averages KGAN results over five random seeds; best and second-best results are marked for each pretrained model.Source: §4.1 · pp. 6–7
4.2 Main Results and Analysis
KGAN performs strongly against prior methods in GloVe settings, while RoBERTa-based KGAN reaches state-of-the-art performance across the reported benchmarks. The analysis also contrasts context- and syntax-based models across datasets.
4.2 Main Results and Analysis
- Context-based models average below syntax-based counterparts, especially on Restaurant14.The authors suggest Restaurant14’s higher multi-aspect-instance ratio as one possible reason syntax-based nonlocal modeling helps there.Source: §4.2 · pp. 7–8
- On Twitter, CNN-based context models significantly outperform LSTM-based models, which the authors relate to noisy, ungrammatical text.The authors note that such text hinders LSTM effectiveness.Source: §4.2 · p. 8
- RoBERTa-based KGAN achieves new state-of-the-art performance among all benchmarks in accuracy and macro-F1.In GloVe settings, KGAN outperforms cutting-edge methods on most evaluated datasets; on Laptop14 it exceeds DualGCN by 0.43% in accuracy and 0.47% in macro-F1.Source: §4.2 · pp. 7–8
4.3 Ablation Study
The ablations examine representation combinations, fusion strategies, and knowledge-graph embeddings. Results favor the full multiview representation, hierarchical fusion, and larger, higher-quality graphs, while embedding performance varies by domain.
4.3.1 Effects of Different Multiview Representation Combinations.
- Using the full combination [Rc,Rs,Rk] performs best among the tested representation combinations and remains the default setting.Each view benefits KGAN, and adding Rk to [Rc,Rs] yields an averaged +1.08% improvement.Source: §4.3.1 · p. 8
4.3.2 Effects of Different Fusion Strategies.
- Hierarchical fusion outperforms the alternative fusion strategies on Laptop14 and Restaurant14 for accuracy and macro-F1.The macro-F1 improvement on Restaurant14 is 2.16%, and the accuracy increase on Laptop14 is at least 0.63%; on Twitter, it slightly exceeds VOTING (Acc: 78.13%; F1: 77.10%).Source: §4.3.2 · pp. 8–9, Figure 6 · p. 8
- Local-to-global fusion is presented as a way to exploit complementarity that direct concatenation or summation does not capture as well.The comparison includes concatenation, elementwise summation, multiview attention, voting, and the proposed module.Source: §4.3.2 · pp. 8–9
4.3.3 Effects of Different Knowledge Graph Embeddings.
- On WordNet, TransE performs worse than semantic-matching approaches, while ANALOGY reaches Laptop14 results of Acc: 78.91% and F1: 75.21%.The authors attribute TransE’s weaker results to focusing on entity-relation information rather than semantic information; DistMult’s performance is unstable across Laptop14 and Restaurant14.Source: §4.3.3 · p. 9, Table 4 · p. 9
- Larger, higher-quality Wikidata embeddings improve KGAN across datasets, with Twitter gains up to 1.42% in accuracy and 1.92% in macro-F1.The reported Wikidata embeddings use TransE; the authors note that these results on Laptop14 and Restaurant14 are slightly below WordNet embeddings trained with ANALOGY or DistMult.Source: §4.3.3 · p. 9, Table 4 · p. 9
4.4 Case Study
The case studies illustrate how contextual, syntactic, and knowledge views contribute differently to aspect sentiment predictions. They also expose a complicated-sentence error shared by all models.
4.4 Case Study
- R-GAT outperforms context-based RAM and TNet-AS in the Tab. 5 examples, illustrating how syntax connects aspects with related opinion words.The table marks aspect terms and true and predicted polarities.Source: §4.4 · p. 10, Table 5 · p. 10
- All models misclassify the final case, where the opinion word “thick” may mislead their comprehension of the complicated sentence.The authors identify understanding complicated sentences as an area needing improvement.Source: §4.4 · p. 10
- For “staff,” contextual and knowledge views attend to “bit more friendly,” while the syntactic view also attends to “should.”The paper presents these attention patterns as evidence of complementarity among the views.Source: §4.4 · p. 10
- For “orange donut,” the knowledge view captures “had” and “shot,” while contextual and syntactic views miss aspect-specific cues the authors describe as closely related.The authors suggest the aspect’s graph proximity to “had” helps the knowledge branch capture the relatedness.Source: §4.4 · p. 10
4.5 Discussion
The discussion compares knowledge-graph embedding with subgraph approaches, examines efficiency trade-offs, and tests robustness to noisy knowledge. KGE-based KGAN performs best among the compared models, with competitive efficiency and tolerance for slight noise.
4.5.1 Comparison between KGE-based and subgraph-based Models.
- KGE-based KGAN achieves the best performance among the compared datasets while using less data-processing time and fewer GFLOPs than the subgraph-based variants.Both KGE- and subgraph-based methods outperform the no-external-knowledge baseline [Rc,Rs].Source: §4.5.1 · pp. 10–11
4.5.2 Latency and Model Size.
- KGAN averages +10.87% macro-F1 over ATAE-LSTM and outperforms R-GAT and DM-GCN with comparable or better latency.The authors attribute the trade-off partly to sharing BiLSTM parameters across the three branches.Source: Table 7 · p. 11, §4.5.2 · p. 11
4.5.3 Robustness of our KGAN.
- On Laptop14, KGAN tolerates 1%, 2%, and 5% noise to some extent, but performance deteriorates at 20%.The authors conclude noisy knowledge does not act as regularization and that KGAN benefits more from incorporated knowledge than noise.Source: Figure 9 · p. 11, §4.5.3 · p. 11
5 CONCLUSION
The conclusion presents KGAN as a three-view ABSA model with hierarchical fusion. The authors report effectiveness, robustness, and a favorable latency-performance trade-off, while naming validation on other language-understanding tasks as future work.
5 CONCLUSION
- KGAN augments ABSA semantic information with context, syntax, and knowledge views fused through a hierarchical module.The conclusion says ablations and case studies show complementarity among these views.Source: §5 · p. 11
- The authors report that KGAN is effective and robust, with a better latency-performance trade-off and robustness to slight noise attacks.Source: §5 · p. 11
- Future work will validate KGAN’s multiview approach on tasks including reading comprehension, intent identification, slot filling, and language generation.Source: §5 · p. 11