PaperlayerView exact revision on arXiv

Source-linked AI summary

Aspect Based Sentiment Analysis with Gated Convolutional Networks

Wei Xue, Tao Li

arXiv:1805.07043v1cs.CL

TL;DR

ABSA seeks sentiment toward particular aspects or entities, but existing LSTM-and-attention approaches can be complex and slow to train. GCAE addresses this with convolutional gating, and SemEval experiments report improved performance and efficiency.

  • Problem

    Existing ABSA approaches commonly use LSTMs and attention, whose sequential and normalization computations are time-consuming during training.

  • Method

    GCAE uses convolutional layers with Gated Tanh-ReLU Units to select aspect-specific sentiment features for ACSA and ATSA.

  • Results

    GCAE improves performance by 1.1% to 2.5% compared with ATAE-LSTM and reports higher accuracy on hard ACSA and ATSA datasets.

  • Takeaways & Limitations

    Convolutional gating provides an efficient and effective alternative to recurrent attention-based models for aspect-level sentiment prediction.

  • Takeaways & Limitations

    Training-time comparison with SVM is unavailable because its performance was retrieved from the original paper.

Abstract

from arXiv · show

Aspect based sentiment analysis (ABSA) can provide more detailed information than general sentiment analysis, because it aims to predict the sentiment polarities of the given aspects or entities in text. We summarize previous approaches into two subtasks: aspect-category sentiment analysis (ACSA) and aspect-term sentiment analysis (ATSA). Most previous approaches employ long short-term memory and attention mechanisms to predict the sentiment polarity of the concerned targets, which are often complicated and need more training time. We propose a model based on convolutional neural networks and gating mechanisms, which is more accurate and efficient. First, the novel Gated Tanh-ReLU Units can selectively output the sentiment features according to the given aspect or entity. The architecture is much simpler than attention layer used in the existing models. Second, the computations of our model could be easily parallelized during training, because convolutional layers do not have time dependency as in LSTM layers, and gating units also work independently. The experiments on SemEval datasets demonstrate the efficiency and effectiveness of our models.

1 Introduction

ABSA targets sentiment toward specific aspect categories or entities, addressing the limits of overall sentiment classification. The paper proposes a CNN-based gated model to simplify and accelerate approaches built on LSTMs and attention.

  • ABSA predicts sentiment polarity for aspect categories or target entities rather than only overall review sentiment.
  • The literature distinguishes aspect-category sentiment analysis (ACSA) from aspect-term sentiment analysis (ATSA).
  • Existing ABSA models commonly combine LSTMs with attention to focus on text related to the given aspect or entity.
  • LSTM processing is sequential, while attention requires normalization across alignment scores, making both time-consuming during training.
  • GCAE uses convolutional layers and gating mechanisms to selectively extract aspect-specific sentiment features and support parallelized computation.

2 Related Work

Related work covers recurrent, memory-based, syntactic, and attention-based approaches to aspect sentiment analysis. These methods use contextual, positional, or structural information, but some depend on inaccurate parsers or costly recurrent computation.

  • ABSA research is organized into aspect-term sentiment analysis and aspect-category sentiment analysis.
  • Aspect-Term Sentiment Analysis: Aspect-term methods analyze sentiment toward labeled aspect terms and often use dependency structure or target-context distance as relevance information.
  • Aspect-Term Sentiment Analysis: Recursive neural networks rely on external syntactic parsers that can be inaccurate and slow on noisy texts such as tweets and reviews.
  • Aspect-Term Sentiment Analysis: TD-LSTM and gated neural networks use separate LSTM networks to model left and right contexts around a target.
  • Aspect-Term Sentiment Analysis: Memory networks use multiple-hop attention to focus on informative context, while LSTM and feature engineering improve handling of more complex contexts.
  • Aspect-Category Sentiment Analysis: For aspect-category sentiment analysis, ATAE-LSTM uses aspect-word embeddings to selectively attend to regions generated by LSTMs.

3 Gated Convolutional Network with Aspect Embedding

GCAE replaces sequential recurrent processing with convolutional layers and independent gating units that extract and select aspect-related sentiment features. Aspect information modulates the gates before pooling and sentiment classification.

  • GCAE uses convolutional layers and gating units instead of recurrent networks, enabling more parallel computation because positions are processed independently.
  • A vanilla text CNN maps embeddings through one-dimensional convolutions and max pooling to produce fixed-size features for sentiment prediction.
  • Convolutional filters detect n-gram patterns at multiple granularities, and max-over-time pooling retains the largest response from each filter.
  • GTRU connects tanh and ReLU convolutional gates at each position, with the ReLU gate receiving aspect information to control sentiment-feature propagation.
  • Element-wise gate multiplication feeds max pooling, whose vector is passed to a softmax layer for sentiment-polarity prediction.
  • The aspect embedding is category-specific for ACSA or is computed by another CNN over aspect terms for ATSA.

4 Gating Mechanisms

The proposed gating mechanism controls sentiment information according to the supplied aspect. Its fine-grained, dimension-wise filtering is motivated by the limitations of global attention and related gated architectures.

  • GTRU controls the path of sentiment information toward pooling, allowing different aspects in one sentence to receive different sentiment features.
  • The ReLU gate can produce a relevance-dependent similarity score because it is zero for negative inputs and unbounded for positive inputs.
  • GTU and GLU provide related convolutional gating designs, using nonlinear or linear feature transforms multiplied by sigmoid gates.

5 GCAE on ATSA

For ATSA, GCAE adds a small convolutional layer over aspect terms to provide aspect information while retaining parallel computation.

  • 5 GCAE on ATSA: ATSA extends GCAE with a small convolutional layer on the aspect terms.The added layer extracts important features from multiple aspect-term words.
  • 5 GCAE on ATSA: The aspect-term CNN supplies the information that controls sentiment-feature flow in the Gated Tanh-ReLU Units.
  • 5 GCAE on ATSA: Unlike ACSA, which uses one aspect word, ATSA derives aspect information from multiple aspect-term words.
  • 5 GCAE on ATSA: The additional convolutional layer retains the ability to perform parallel computing.

6 Experiments

Experiments on SemEval restaurant and laptop reviews evaluate ACSA and ATSA, including hard sets with multiple sentiments toward different aspects. GCAE outperforms compared neural models on difficult cases and trains faster than LSTM-based alternatives, while SVM performance depends strongly on sentiment lexicons.

  • 6.1 Datasets and Experiment Preparation: Experiments use SemEval restaurant and laptop reviews for ACSA and ATSA, with additional hard datasets targeting multiple sentiments toward different aspects.The study reimplements compared methods and reports repeated evaluations on public workshop datasets.
  • 6.3.1 ACSA: GCAE improves performance by 1.1% to 2.5% over ATAE-LSTM, while CNN and GCN also exceed ATAE-LSTM despite not being designed for aspect-based sentiment analysis.GCAE separates aspect and sentiment features and applies element-wise gating to context-vector dimensions.
  • 6.3.1 ACSA: SVM performance depends on available features: adding multiple sentiment lexicons increases performance by 7.6%, but training time cannot be compared because the SVM results come from the original paper.The compared SVM uses extensive feature engineering and large-scale labeled resources.
  • 6.3.1 ACSA: GCAE achieves 4% higher accuracy than ATAE-LSTM on Restaurant-Large and 5% higher on SemEval-2014 for ACSA hard-test evaluation.The hard datasets contain sentences with multiple aspect labels associated with different sentiments.
  • 6.3.2 ATSA: On ATSA hard test sets, GCAE exceeds RAM by 1% on restaurant data and 1.7% on laptop data.GCAE uses CNN outputs over aspect terms to guide sentiment-feature composition through a ReLU gate.
  • 6.4 Training Time: LSTM-based models take more training time than convolutional models, and GCAE is much faster because its convolutions and GTRU lack LSTM-like time dependency.The paper reports convergence time on ATSA using a desktop machine with a 1080 Ti GPU.

7 Visualization

The visualization illustrates how GTRU gate outputs vary across words and aspect targets, using a simplified one-filter model to make the behavior interpretable.

  • 7 Visualization: GTRU produces many gate outputs across words, filters, and dimensions, making its weights harder to visualize than attention weights.The visualization therefore uses one filter spanning three words and sums the ReLU-gate outputs across dimensions for each word.
  • 7 Visualization: Different aspect targets change the magnitude of the tanh-gate outputs through the ReLU gates.After normalization, the summed gate outputs are plotted over the words in Figure 3.

8 Conclusions and Future Work

The paper concludes that GCAE is an efficient gated convolutional model for ACSA and ATSA, with GTRU controlling sentiment flow using aspect information.

  • 8 Conclusions and Future Work: GCAE combines convolutional layers with gating mechanisms for aspect-category and aspect-term sentiment analysis.The model is presented as an efficient convolutional neural network for both ABSA subtasks.
  • 8 Conclusions and Future Work: GTRU controls sentiment flow according to aspect information while separate convolutional layers model aspect and sentiment information.The paper reports performance improvement over other neural models in extensive SemEval experiments.
Loading 1805.07043v1…