Source-linked AI summary
An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis
Ruidan He, Wee Sun Lee, Hwee Tou Ng, Daniel Dahlmeier
TL;DR
ABSA is commonly handled as a pipeline, which can underuse task relationships and available document-level supervision. IMN jointly learns token- and document-level tasks through iterative message passing over shared latent variables. It outperforms multiple pipeline and integrated baselines on three benchmark datasets.
Problem
Pipeline ABSA methods may not fully exploit joint information between aspect extraction and sentiment classification or available document-level labeled sentiment corpora.
Method
IMN jointly trains aspect-level and document-level tasks while iteratively passing information among them through shared latent variables.
Results
IMN outperforms multiple pipeline and integrated baselines on three benchmark datasets.
Takeaways & Limitations
Task interactions and document-level supervision are incorporated into one architecture for jointly learning fine-grained aspect-based sentiment analysis tasks.
Takeaways & Limitations
The AE task also performs aspect and opinion term co-extraction, although it is denoted AE for simplicity.
Abstract
from arXiv · showhide
Aspect-based sentiment analysis produces a list of aspect terms and their corresponding sentiments for a natural language sentence. This task is usually done in a pipeline manner, with aspect term extraction performed first, followed by sentiment predictions toward the extracted aspect terms. While easier to develop, such an approach does not fully exploit joint information from the two subtasks and does not use all available sources of training information that might be helpful, such as document-level labeled sentiment corpus. In this paper, we propose an interactive multi-task learning network (IMN) which is able to jointly learn multiple related tasks simultaneously at both the token level as well as the document level. Unlike conventional multi-task learning methods that rely on learning common features for the different tasks, IMN introduces a message passing architecture where information is iteratively passed to different tasks through a shared set of latent variables. Experimental results demonstrate superior performance of the proposed method against multiple baselines on three benchmark datasets.
1 Introduction
ABSA extracts aspect terms and predicts sentiment toward each term, but prior pipeline and integrated approaches do not fully model task interactions or use broader document-level supervision. IMN jointly trains token- and document-level tasks with iterative message passing and outperforms multiple baselines on three benchmark datasets.
- Task definition: ABSA extracts explicit aspect mentions and predicts each aspect term’s sentiment orientation.For example, “food” is positive and “service” is negative in “Great food but the service is dreadful”.
- Motivation: Pipeline methods separate aspect extraction from sentiment classification, limiting exploitation of joint information between the tasks.
- Motivation: Earlier integrated models link the tasks through unified tags but do not explicitly model their correlation or exploit related document-level sentiment corpora.
- IMN: IMN jointly learns aspect extraction and sentiment classification with document-level sentiment and domain classification tasks.This combines fine-grained token-level tasks with document-level supervision from larger labeled corpora.
- IMN: IMN explicitly models task interactions by iteratively passing information through shared latent variables rather than relying only on common features.
- Results: IMN outperforms multiple pipeline and integrated baselines on three benchmark datasets.
2 Related Work
Prior ABSA research extensively studies extraction and sentiment classification, usually in a pipeline, while integrated models connect them through unified sequence tags. Related multi-task and message-passing work motivates a structure that enables richer task interactions.
- Aspect-Based Sentiment Analysis: Existing ABSA approaches extensively study aspect extraction and sentiment classification, typically solving them in a pipeline.
- Aspect-Based Sentiment Analysis: Integrated ABSA models formulate the problem as sequence labeling with unified tags, but their results have ranged from discouraging to promising.
- Aspect-Based Sentiment Analysis: These integrated models link subtasks through unified tagging without explicitly modeling interactions, motivating a network structure that allows further task interactions.
- Multi-Task Learning: Conventional multi-task learning uses shared and task-specific networks to learn shared and task-specific feature spaces.
- Message Passing Architectures: Message-passing architectures model iterative information propagation through learned update operators, and this work adapts that idea for multi-task learning.
3 Proposed Method
IMN jointly models aspect-term extraction, aspect-level sentiment, and document-level tasks through shared latent representations updated by iterative message passing. Its architecture lets task predictions interact while using document-level sentiment and domain information during training.
- Architecture: IMN uses shared latent vectors initialized by a feature extractor and updated through message passing across task-specific components.The shared sequence feeds all tasks, and the updated representations are used for subsequent computation.
- Aspect-Level Tasks: The aspect-level components perform aspect and opinion term co-extraction and token-level sentiment classification.AE uses BIO labels for aspect and opinion terms, while AS predicts positive, negative, or neutral sentiment for tokens.
- Aspect-Level Tasks: AS receives AE outputs through a self-attention mechanism that weights contextual tokens using semantic relevance, distance, and predicted opinion-term probability.The attention diagonal is zeroed so sentiment inference uses context words rather than the target token itself.
- Document-Level Tasks: Document-level sentiment and domain classification are jointly trained with AE and AS to exploit larger labeled document-level corpora.Document sentiment predictions and attention weights are incorporated into shared-representation updates for the aspect-level tasks.
- Message Passing Mechanism: Message passing aggregates previous-iteration predictions from AE, AS, DS, and DD to re-encode and update the shared latent vectors.The mechanism repeats for multiple iterations, while only AE and AS outputs are used during inference.
- Learning: IMN alternates aspect-level and document-level training instances after pretraining on document-level data.The aspect-level objective applies token-level cross-entropy while ignoring AS predictions for tokens without aspect sentiment annotations.
4 Experiments
Experiments evaluate IMN on three benchmark datasets using aspect-level and document-level data, comparisons with pipeline and integrated baselines, ablations, and case analyses. Results attribute gains to interactive message passing and transferred domain-specific information.
- Datasets: Experiments use three SemEval aspect-level datasets and two balanced document-level corpora from Yelp restaurants and Amazon electronics.Each document-level corpus contains 30k instances; Yelp or electronics data are assigned according to the aspect-level dataset.
- Models under comparison: The comparison includes four combinations of two AE models and two AS models, plus an independently trained IMN pipeline baseline.AE baselines are CMLA and DECNN; AS baselines are ALSTM and dTrans.
- Embedding analysis: IMN−d without domain-specific embeddings remains competitive with DECNN-dTrans and outperforms all other baselines except DECNN-dTrans.The result suggests that the proposed network structure can transfer useful information even without additional domain-specific embeddings.
- Ablation study: Message passing contributes the largest ablation gains, whereas adding document-level tasks through parameter sharing alone yields only marginal improvement.Document-level message passing remains helpful, indicating that aspect-level tasks benefit from document-level predictions.
- Impact of T: F1-I convergence is quickly achieved within two or three message-passing iterations, and further iterations provide no considerable improvement.The maximum iteration count T is tuned through cross-validation and is set to 2 in the reported setup.
- Case analysis: Case analyses show that message passing helps recognize uncommon aspect terms, avoid extracting terms without expressed opinions, and improve sentiment predictions for domain-specific or complex expressions.Examples include recovering “build,” rejecting “Pizza” without an expressed opinion, and recognizing sentiment-relevant opinion words such as “scratches.”
5 Conclusion
The paper proposes IMN to jointly learn aspect and opinion term co-extraction with aspect-level sentiment classification, using message passing to model task interactions and document-level training data.
- IMN jointly learns aspect and opinion term co-extraction with aspect-level sentiment classification.
- Its message passing mechanism enables informative interactions between tasks, better exploiting their correlations.
- IMN lets fine-grained token-level tasks benefit from document-level labeled corpora.
- The architecture can potentially extend to related tasks such as relation extraction and semantic role labeling.
CNN-based Encoder
The proposed network uses multi-layer CNN encoders with shared and task-specific components. Convolution filters capture each token together with a fixed-size context window, with specified filter configurations, ReLU activations, and dropout.
- Multi-layer CNNs serve as both shared and task-specific encoders in the proposed network.
- Each filter uses kernel size k = 2c + 1 to represent a token with 2c neighboring words.
- The first shared layer has 128 filters each for kernel sizes k = 3 and k = 5, while later layers use 256 filters with k = 5.
- ReLU activates every CNN layer, and dropout with p = 0.5 follows the embedding layer and each CNN layer.
Opinion Transmission
During training, the model uses scheduled sampling to transmit opinion information from aspect extraction to aspect sentiment prediction. Gold opinion labels are progressively replaced by predictions according to an inverse-sigmoid schedule.
- Scheduled sampling transmits gold opinion labels from AE to AS with probability ϵ_i during training.
- The gold-label probability decays with epoch i according to ϵ_i = 5/(5 + exp(i/5)).
- This schedule addresses unreliable opinion-label predictions early in training.
B Model Comparison Details
The comparison evaluates released, reimplemented, and proposed models under matched evaluation procedures and additional opinion-label settings. IMN and IMN−d retain stronger performance when opinion-term labels are unavailable, while opinion extraction information benefits performance only with suitable network design.
- Implementation and evaluation: Experiments use official code for CMLA, ALSTM, dTrans, and INABSA, while MNN is reimplemented because its source code is unavailable.
- Implementation and evaluation: All baselines are run multiple times with random initializations, and one evaluation script measures every model's outputs.
- Opinion-label comparison: The proposed IMN co-extracts aspect and opinion terms, whereas several baselines originally do not use opinion-term information during training.
- Statistical reporting: Table 7 reports averages over five random-initialization runs, with significance marked by one-tailed unpaired t-tests at p < 0.05.
- Opinion-label comparison: For fair comparison, baseline label sets are expanded with BP and IP opinion-term labels, and gold opinion terms are supplied to ALSTM and dTrans during training.
- Results without opinion labels: Without opinion-term labels, IMN−d and IMN significantly outperform other baselines in most cases.
- Results without opinion labels: IMN−d and IMN consistently achieve better F1-I scores on all Table 3 datasets when opinion-term extraction is included.
- Results without opinion labels: Other baselines do not show consistent improvements after opinion-label training, suggesting that exploiting opinion information requires an appropriate network structure.