Source-linked AI summary
Explaining Recurrent Neural Network Predictions in Sentiment Analysis
Leila Arras, Grégoire Montavon, Klaus-Robert Müller, Wojciech Samek
TL;DR
The paper addresses how to make recurrent neural network sentiment predictions more transparent. It extends LRP with a multiplicative-interaction rule, applies it to a bi-directional LSTM, and reports more trustworthy word relevances than a gradient-based decomposition. The method also provides deterministic, one-pass explanations directly from the original classifier.
Problem
As neural sentiment models become more predictive, their decisions need to be transparent and interpretable.
Method
The paper extends LRP to recurrent architectures by propagating relevance through multiplicative interactions, then applies it to a word-based bi-directional LSTM.
Results
The resulting word relevances more reliably identify words supporting or opposing a target sentiment class and perform better than gradient-based decomposition.
Takeaways & Limitations
The technique supports understanding and verifying recurrent classifiers using deterministic, one-pass explanations obtained directly from the original classifier.
Takeaways & Limitations
The paper’s recurrent LRP treatment is currently applied to LSTMs and GRUs, with extensions to other recurrent architectures left for future work.
Abstract
from arXiv · showhide
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specific propagation rule applicable to multiplicative connections as they arise in recurrent network architectures such as LSTMs and GRUs. We apply our technique to a word-based bi-directional LSTM model on a five-class sentiment prediction task, and evaluate the resulting LRP relevances both qualitatively and quantitatively, obtaining better results than a gradient-based related method which was used in previous work.
1 Introduction
The paper addresses the need to understand whether increasingly predictive recurrent sentiment models make intended, transparent decisions. It extends LRP to recurrent architectures and compares word-level explanations with a gradient-based approach.
- Semantic composition requires modeling negation and modifiers that reverse or rescale sentiment.
- Deep learning models, including convolutional and recurrent networks, have become prominent for sentiment analysis.
- Transparent decisions are increasingly important as neural sentiment models become more predictive.
- Feature-importance methods explain which input features contribute to state-of-the-art neural network predictions.
- The paper extends LRP with a rule for multiplicative interactions in LSTMs and applies it to a bi-directional LSTM sentiment classifier.
- The resulting word explanations are compared with gradient-based explanations and reported as more reliable.
2 Methods
The methods compute input relevances either from squared partial derivatives or by propagating class scores backward through the network. For recurrent models, LRP adds a rule that assigns multiplicative-interaction relevance to source neurons rather than gates.
- Relevance definitions: Input relevance R_d quantifies how strongly input dimension x_d supports or opposes a target class c.
- Relevance definitions: Word-level relevance is obtained by summing the relevance scores of the embedding dimensions constituting each word.
- Gradient-based Sensitivity Analysis (SA): Sensitivity Analysis defines relevance using squared partial derivatives computed through standard gradient backpropagation.
- Gradient-based Sensitivity Analysis (SA): SA decomposes the squared gradient norm across input dimensions.
- Layer-wise Relevance Propagation (LRP): LRP redistributes the target class score layer by layer from the output toward the input using relevance-conserving rules.
- Layer-wise Relevance Propagation (LRP): Recurrent architectures combine weighted linear connections with multiplicative interactions, so the method restricts its propagation rules to these connection types.
- Layer-wise Relevance Propagation (LRP): For weighted connections, relevance messages are distributed from upper neurons to lower neurons and summed to obtain each lower-layer relevance.
- Layer-wise Relevance Propagation (LRP): The weighted-connection rule uses a small positive stabilizer ε, set to 0.001 in the experiments.
3 Recurrent Model and Data
The experiments use a one-hidden-layer word-based bi-directional LSTM trained for five-class sentiment prediction on the Stanford Sentiment Treebank. The study compares sensitivity analysis and LRP on the test sentences.
- Recurrent Model and Data: The model is a one-hidden-layer bi-directional LSTM trained on Stanford Sentiment Treebank movie reviews for five-class sentiment prediction.
- Recurrent Model and Data: Each input sentence is processed in both forward and reversed word order, with 60-dimensional embeddings and a hidden layer of size 60.
- Recurrent Model and Data: The test input contains 2210 lowercased tokenized sentences from the Stanford Sentiment Treebank.
- Recurrent Model and Data: 46.3% accuracy is achieved for five-class full-sentence sentiment prediction, while binary positive-versus-negative accuracy reaches 82.9%.
- Recurrent Model and Data: The experiments compare sensitivity analysis with Layer-wise Relevance Propagation as relevance decomposition methods.
- Recurrent Model and Data: Implementations of both relevance decomposition methods are made available to support reproducibility and further research.
4 Results
The results compare SA and LRP explanations qualitatively and quantitatively, finding that LRP more reliably identifies words supporting or opposing sentiment classes and models context-dependent sentiment.
- 4.1 Decomposing Sentiment onto Words: LRP more reliably distinguishes words supporting or opposing the target sentiment class than SA in sentence heatmaps.SA sometimes assigns high relevance to sentimentally opposing words, whereas LRP uses signed relevance to separate positive and negative evidence.
- 4.1 Decomposing Sentiment onto Words: LRP explains several misclassified sentences by highlighting negatively signed relevance for terms opposing the true “very negative” class.This behavior is absent from the corresponding SA heatmaps.
- 4.1 Decomposing Sentiment onto Words: LRP assigns negators relevance modulated by the sentiment of subsequent words, yielding negative or positive signs depending on the surrounding phrase.For example, n’t or not is negative before waste or horrible but positive before worth, funny, or good.
- 4.1 Decomposing Sentiment onto Words: Heatmaps indicate that the sentiment attributed to individual words depends on sentence context rather than remaining static.The relevance decomposition provides insight into how the bi-LSTM composes word-level sentiment.
- 4.2 Representative Words for a Sentiment: LRP produces more meaningful extreme word relevances for the “very positive” class than SA across the test set.SA’s highest relevances include semantically strong but not necessarily positive words, while LRP’s highest and lowest relevances align with positive and negative sentiment.
- 4.3 Validation of Word Relevance: Deleting words by LRP relevance has the most pertinent effect in both correctly and falsely classified sentences, while LRPcons is slightly inferior to standard LRP.The experiments delete words in decreasing relevance order for initially correct predictions and increasing order for initially false predictions.
- 4.4 Relevance Distribution over Sentence Length: Relevance distributions differ between the bi-LSTM’s encoders: the left encoder’s relevances tend to be smoother than the right encoder’s.This asymmetry is unexpected because the architecture has no stated mechanism distinguishing original from reversed reading order.
5 Conclusion
The paper extends LRP to recurrent architectures by propagating relevance through multiplicative interactions and applies it to bi-directional LSTM sentiment classification. The resulting explanations support classifier decisions, outperform gradient-based decompositions, and offer deterministic, self-contained analysis, while broader architectures remain future work.
- The authors extend LRP to recurrent architectures such as LSTMs by introducing a rule for propagating relevance through multiplicative interactions.
- The extended LRP produces trustworthy word relevances and performs better than gradient-based decompositions for bi-directional LSTM sentence sentiment prediction.
- The technique supports understanding recurrent classifier behavior and detecting important patterns in text datasets.
- Unlike sampling- or occlusion-based explanation methods, the technique is deterministic, computed in one network pass, and self-contained.It obtains explanations directly from the original classifier without training an external classifier.
- Future work includes applying the technique to character-level models, GRUs, extractive summarization, and recurrent architectures beyond NLP.
Long-Short Term Memory Network (LSTM)
The experimental model uses LSTM recurrence equations over word-embedding sequences, with element-wise nonlinearities and multiplication. Its bi-directional architecture combines forward and reversed-word encoders before producing class prediction scores.
- The LSTM recurrence applies element-wise sigmoid and tanh activations and element-wise multiplication.
- The LSTM receives a sequence of word-embedding vectors representing the input sentence.The recurrence uses connection weights, biases, and zero initial hidden and cell states.
- The model’s final hidden state feeds a fully connected linear layer that produces one prediction score per sentiment class.
- The bi-directional LSTM concatenates two separate LSTM representations, computed from the original and reversed word orders.
- The concatenated encoder states are fed to a fully connected linear layer to retrieve one class prediction score per class.