Source-linked AI summary
Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
W. James Murdoch, Peter J. Liu, Bin Yu
TL;DR
LSTMs learn nonlinear interactions that are difficult to interpret, motivating a method that explains individual predictions without changing the model. CD decomposes outputs into word, phrase, and interaction contributions, and sentiment-analysis experiments show it identifies contrasting sentiment and positive or negative negations.
Problem
LSTMs learn complex nonlinear relationships, but the inability to describe those interactions leaves them characterized as black boxes.
Method
Contextual decomposition mathematically decomposes an LSTM’s output into phrase-only and contextual contributions, using gate linearization and averaged orderings to extract interactions.
Results
CD identifies contrasting-sentiment words and phrases, separates dissenting-subphrase sentiment, captures high-level compositionality, and extracts positive and negative negations.
Takeaways & Limitations
CD moves LSTM interpretation beyond word-level importance toward phrase contributions and meaningful word or variable interactions.
Takeaways & Limitations
The decomposition assumes the gates and updates can be linearized into sums of input contributions.
Abstract
from arXiv · showhide
The driving force behind the recent success of LSTMs has been their ability to learn complex and non-linear relationships. Consequently, our inability to describe these relationships has led to LSTMs being characterized as black boxes. To this end, we introduce contextual decomposition (CD), an interpretation algorithm for analysing individual predictions made by standard LSTMs, without any changes to the underlying model. By decomposing the output of a LSTM, CD captures the contributions of combinations of words or variables to the final prediction of an LSTM. On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting sentiment, and how they are combined to yield the LSTM's final prediction. Using the phrase-level labels in SST, we also demonstrate that CD is able to successfully extract positive and negative negations from an LSTM, something which has not previously been done.
1 INTRODUCTION
LSTMs achieve strong performance by learning nonlinear interactions, but those interactions remain difficult to describe. Contextual decomposition (CD) explains individual LSTM predictions by identifying both contributing words and how they combine.
- LSTMs learn nonlinear feature interactions that support strong performance but remain largely inscrutable.
- CD explains individual LSTM predictions without modifying the underlying model.
- CD decomposes an LSTM’s output to identify word contributions and the combinations that produce the final prediction.
- On sentiment analysis, CD identifies contrasting-sentiment words and phrases and successfully extracts positive and negative negations.
2 RELATED WORK
Prior LSTM interpretation methods emphasize word importance, gate activity, or indirect attention signals, while generally leaving interactions and their contribution to predictions unresolved.
- Earlier LSTM interpretation methods primarily compute word-level importance scores using decomposition, leave-one-out changes, or gradient-based measures.
- Gate-activation analyses can identify some semantically meaningful cell-state coordinates, but most coordinates remain uninterpretable and their predictive combinations are unclear.
- CNN decomposition methods produce pixel-level importance scores while ignoring interactions between pixels.
- Attention provides an indirect importance signal without directionality about which class a word supports.
3 CONTEXTUAL DECOMPOSITION OF LSTMS
CD decomposes an LSTM’s hidden and cell states into phrase-only and contextual contributions, using gate linearization and cross-terms to represent interactions. The resulting phrase score quantifies the phrase’s contribution to the prediction.
- 3 CONTEXTUAL DECOMPOSITION OF LSTMS: CD decomposes each hidden and cell state into contributions from a specified phrase and contributions involving factors outside it.
- 3 CONTEXTUAL DECOMPOSITION OF LSTMS: The decomposition assigns a quantitative phrase score through the phrase-only contribution to the final output state.
- 3.2.1 DISAMBIGUATING INTERACTIONS BETWEEN GATES: LSTM gating creates interactions because products of gate contributions combine current inputs with prior contextual contributions.
- 3.2.1 DISAMBIGUATING INTERACTIONS BETWEEN GATES: After gate linearization, expanded cross-terms are grouped as phrase-only, phrase-context, or other-factor contributions.
- 3.2.1 DISAMBIGUATING INTERACTIONS BETWEEN GATES: The decomposition is computed recursively through the sequence, with all contribution components initialized to zero.
- 3.2.1 DISAMBIGUATING INTERACTIONS BETWEEN GATES: CD linearizes the tanh transformation after decomposing the cell state; decomposing the output gate did not empirically improve results.
- 3.2.2 LINEARIZING ACTIVATION FUNCTIONS: Because the decomposition terms have no natural ordering, the activation linearization averages contributions over all permutations.
- 3.2.2 LINEARIZING ACTIVATION FUNCTIONS: The linearization generalizes from scalar to vector inputs and can be viewed as an approximation to Shapley values.
4 EXPERIMENTS
The experiments validate CD for sentiment-analysis interpretation on SST and Yelp, comparing it with established baselines under standard modeling procedures. The evaluation targets word importance, compositional phrase sentiment, and interactions.
- 4 EXPERIMENTS: CD is evaluated on sentiment analysis for word-level importance, compositional word and phrase sentiment, and positive and negative negation.
- 4 EXPERIMENTS: Models use standard practices including Adam optimization with learning rate 0.001 and early stopping on the validation set.
- 4 EXPERIMENTS: SST provides review-level labels and phrase-level labels from binarized constituency parses, supporting evaluation of phrase interpretations.
- 4 EXPERIMENTS: The Yelp polarity dataset contains 560,000 training reviews and 38,000 test reviews, with binary positive-versus-negative labels.
- 4 EXPERIMENTS: The Yelp LSTM model attains 4.6% error under the stated implementation guidelines.
- 4 EXPERIMENTS: CD interpretations are compared with cell decomposition, integrated gradients, leave-one-out, and gradient-times-input baselines.
4.2 UNIGRAM (WORD) SCORES
CD produces word-level importance scores that compare favorably with prior methods, using logistic-regression coefficients as an interpretability reference.
- CD evaluates LSTM word-level coefficients by comparing them with logistic-regression coefficients for the same validation-set words.Each scatter-plot point represents one word, and Pearson correlation measures agreement between the two scores.
- 0.76 and 0.72 correlations on SST make CD and integrated gradients substantially better than methods reaching at most 0.51.
- 0.52 correlation on Yelp makes CD competitive with methods ranging from 0.34 to 0.56.
4.3 IDENTIFYING DISSENTING SUBPHRASES
CD identifies sentiment differences inside short phrases, whereas existing methods often fail to distinguish subphrases with opposing sentiment.
- For phrases of at most five words, existing methods often miss subphrases whose sentiment differs from the surrounding phrase.
- In “used to be my favorite,” CD identifies “my favorite” as strongly positive and “used to be” as strongly negative.“favorite” alone has a logistic-regression coefficient in the 93rd percentile but is misranked by existing methods.
- Across Yelp, CD yields significantly separated score distributions for positive and negative dissenting subphrases, unlike prior methods.Phrases were selected using absolute logistic-regression n-gram scores above 1.5.
4.4 EXAMINING HIGH-LEVEL COMPOSITIONALITY
CD captures high-level compositionality when substantial portions of reviews express sentiment opposing the review-level prediction, outperforming prior attribution methods.
- CD is the only evaluated method that accurately captures a positive opening phrase followed by a phrase that makes the full review negative.
- The high-level search targets phrases with SST sentiment opposing the review label and lengths between one and two thirds of the review.
- 87% of positive opposing phrases are labelled negative by integrated gradients, while cell decomposition reverses the positive and negative distributions.
- CD achieves a 0.74 one-sided Kolmogorov-Smirnov statistic, compared with 0, 0.33, 0.58, and 0.61 for the other methods.The statistic quantifies separation between positive and negative phrase-score distributions.
4.5 CONTEXTUAL DECOMPOSITION (CD) CAPTURES NEGATION
CD extracts negation interactions from LSTM predictions and distinguishes positive from negative negations more clearly than leave-one-out attribution.
- CD computes a negation interaction by subtracting the scores of the negated phrase and negation term from the full phrase score.The resulting score is interpreted as an n-gram feature.
- CD produces a clear distinction between positive and negative negations, with interactions centered near the distribution’s outer edges.
- Leave one out shows overlap between positive and negative negations around zero, indicating a high rate of false negatives.
4.6 IDENTIFYING SIMILAR PHRASES
CD produces embeddings for phrases and interactions, and their nearest neighbours are qualitatively semantically sensible across sentiment-related examples.
- 4.6 IDENTIFYING SIMILAR PHRASES: CD provides dense embeddings for arbitrary phrases and interactions alongside importance scores.These embeddings are represented by βT.
- 4.6 IDENTIFYING SIMILAR PHRASES: Nearest-neighbour analysis uses average CD embeddings across training and validation sets with cosine similarity.The analysis covers words, binary interactions, and selected sentiment-related categories.
- 4.6 IDENTIFYING SIMILAR PHRASES: Nearest neighbours are qualitatively sensible for positive negation, negative negation, modification, and positive and negative words.For sentiment words, the analysis uses the positive or negative parts of negations to emphasize compositional disentanglement.
5 CONCLUSION
The paper proposes contextual decomposition for interpreting individual LSTM predictions without modifying the underlying model. Across two sentiment-analysis datasets, CD identifies sentiment-bearing phrases and meaningful interactions beyond word-level importance.
- 5 CONCLUSION: CD interprets individual LSTM predictions without modifying the underlying model.It produces scores for words, phrases, and word interactions.
- 5 CONCLUSION: Across two sentiment-analysis datasets, CD compares favorably with prior methods on information such as word-level scores.The paper then evaluates phrase-level sentiment composition and word interactions.
- 5 CONCLUSION: CD identifies phrases with varying sentiment and extracts meaningful word or variable interactions.The authors describe moving beyond word-level importance as critical for understanding complex, highly non-linear LSTMs.
6 APPENDIX
The appendix supplies additional plots, a general recursion formula, and implementation details for evaluating phrase attributions, negations, and interpretation relationships.
- 6 APPENDIX: Appendix plots provide additional visualizations referenced in Sections 4.3 and 4.4, including phrase-attribution distributions.The supplied captions cover Yelp subphrases and SST phrases.
- 6.2 GENERAL RECURSION FORMULA: The general recursion formula distinguishes time steps inside the phrase, q ≤ t ≤ r, from those outside it.The appendix provides separate update expressions for the two cases and lists negation words used in the search.
- 6 APPENDIX: For Yelp subphrases of length at most five, only CD separates positive and negative attributions in opposite-sentiment contexts.Prior methods produce nearly identical positive and negative distributions.
- 6 APPENDIX: For SST phrases spanning one to two thirds of a review, CD yields more separate positive and negative distributions than other methods.The caption states that other methods still struggle at this coarse granularity.
- 6 APPENDIX: Figure 4 compares LSTM-extracted coefficients with logistic regression coefficients using a least squares regression line.Stronger linear relationships correspond to better interpretation techniques.