Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks

• ABSA identifies an entity’s aspects and classifies sentiment with respect to each, rather than classifying only the text’s overall sentiment.Source: §1 · p. 1
• Labeled data are limited, and aspect detection is non-trivial; errors at that stage severely limit overall ABSA performance.Source: §1 · p. 1
• Because general and aspect-level sentiment can differ, a model must distinguish aspects and make independent decisions for each.Source: §1 · p. 1
• The proposed system leverages unlabeled data for word representations and jointly detects aspects and classifies polarity in an end-to-end trainable system.Source: §1 · p. 1
• The evaluation uses GermEval 2017 customer reviews about Deutsche Bahn AG on social media, focusing on subtask C.Source: §1 · p. 1

• The task asks which polarity applies to each mentioned aspect.Source: §1 · p. 1
• The authors infer from their new state-of-the-art results that word-representation modeling has not received the attention it deserves.Source: §1 · p. 1
• The authors also infer that joint modeling of aspects and polarity has not received the attention it deserves.Source: §1 · p. 1

• The English laptop reviews in SemEval 2016 contain 395 training instances for 88 aspect categories, and top-ranked systems rely on feature engineering and hand-crafted rules.Source: §2 · p. 2
• GermEval 2017 has about 20K training instances across 20 categories, making it suited to evaluating fully automatic aspect and polarity prediction.Source: §2 · p. 2
• GermEval winners use neural sequence labeling, but some aspects have no token assignment, and their category and polarity predictions remain separate.Source: §2 · p. 2
• Unlike Ruder et al. (2016), whose pipeline assumes detected aspects, this approach predicts all document aspects jointly and improves over pipeline approaches.Source: §2 · p. 2

3.1 Embedding Algorithm

• FastText incorporates subword information and can generate embeddings for unseen test words from learned character n-grams.The embedding algorithms were trained on about 113K Deutsche Bahn-related tweets and the GermEval training set.Source: §3.1 · p. 2

3.2 Pipeline LSTM (baseline)

• The pipeline LSTM duplicates each tweet for each aspect and uses aspects detected by the GermEval SVM baseline as input.Because the documents are short, the sentence-level hierarchy from Ruder et al. (2016) is omitted.Source: §3.2 · pp. 2–3

3.3 End-to-end LSTM

• The end-to-end LSTM predicts N/A, positive, negative, or neutral for each aspect, integrating aspect detection and polarity classification.Its loss is cross-entropy summed over all aspects.Source: §3.3 · p. 3

3.4 End-to-End CNN

• The end-to-end CNN replaces the BiLSTM encoder while retaining the same aspectwise classification definitions.It uses 300-dimensional embeddings, max-over-time pooling, filter sizes 3, 4, and 5, and 300 filters of each size.Source: §3.4 · p. 3

3.5 Pipeline CNN

• The pipeline CNN experiment substitutes the CNN for the BiLSTM in the pipeline setting.Source: §3.5 · p. 3

4 Experiments

• The main dataset contains about 26K documents split randomly into training, development, and test sets at 80%/10%/10%.A diachronic test set contains about 1,800 documents from the final three months; approximately 4% of training instances were filtered, while development and test data remained unchanged.Source: §4 · p. 4
• Random search on development data selected a learning rate of 0.03 and batch size 5 for CNNs, and 0.01 and 10 for LSTMs.The search ranges were {0.001, 0.003, 0.01, 0.03, 0.1} for learning rate and {5, 10, 20} for batch size.Source: §4 · p. 4
• Training took 1–3 minutes per epoch on a GeForce GTX 1080 GPU, with the end-to-end CNN the fastest model to train.Source: §4 · p. 4

Aspect polarity

• CNN+FastText achieved the best aspect-polarity performance, outperforming the highly adapted GermEval shared-task winner.The comparison is reported for the joint aspect and polarity task.Source: §5 · p. 4
• FastText matched or surpassed word2vec and GloVe across architectures, while end-to-end models improved performance over pipeline settings.The authors attribute FastText’s advantage to subword features for morphologically rich German.Source: §5 · p. 4

Aspect category only

• For aspect-category detection, CNN+FastText outperformed all baselines and the GermEval winner despite not being optimized for this task alone.The SVM GermEval baseline was nearly on par with the best synchronic submission and outperformed it on the diachronic test set, making it suitable pipeline input.Source: §5 · p. 4

Impact of domain-specific corpus

• Wikipedia-trained FastText, from a corpus about 100 times larger, scored slightly lower on development but slightly higher or equally on test than domain-trained FastText.The authors conclude that subword modeling is FastText’s main positive impact and that large domain-independent and small domain-specific corpora yield similar gains.Source: §5 · p. 4

6 Conclusion

• Joint aspect-category and polarity prediction yielded significant performance gains on GermEval 2017 and established a new state of the art in both tasks.The authors present the resulting system as a practical baseline for future research.Source: §6 · pp. 4–5
• Word representations that leverage subword information were crucial for ABSA in morphologically rich German.The conclusion also reports consistently better CNN performance in otherwise comparable scenarios.Source: §6 · p. 5
• The authors suggest CNNs may cope better with irregular social-media text, but leave that question for future work.They also report a new state of the art in aspect detection and polarity classification.Source: §6 · p. 5