Source-linked AI summary
Rationalizing Neural Predictions
Tao Lei, Regina Barzilay, Tommi Jaakkola
TL;DR
Neural predictions often lack interpretable justifications, limiting their applicability where decisions must be understood. The paper jointly trains a generator and encoder to extract short, sufficient rationales without rationale annotations, achieving quality rationales on sentiment and question-retrieval tasks. Its discussion notes that additional generator constraints and variance reduction may improve exploration and training stability.
Problem
Complex neural models can improve accuracy while offering little transparency into the basis for their predictions.
Method
A generator proposes short extracted text fragments and an encoder predicts from them; both are trained jointly without explicit rationale annotations.
Results
The framework produces quality rationales without explicit annotations and achieves MAP up to 56.5% on similar-question retrieval, outperforming noisy question-body baselines.
Takeaways & Limitations
The encoder-generator framework supports concise, sufficient neural justifications across multi-aspect sentiment analysis and question retrieval.
Takeaways & Limitations
The generator may explore a large rationale space, and stochastic training may lack stability without additional constraints or variance reduction.
Abstract
from arXiv · showhide
Prediction without justification has limited applicability. As a remedy, we learn to extract pieces of input text as justifications -- rationales -- that are tailored to be short and coherent, yet sufficient for making the same prediction. Our approach combines two modular components, generator and encoder, which are trained to operate well together. The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction. Rationales are never given during training. Instead, the model is regularized by desiderata for rationales. We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases. Our approach outperforms attention-based baseline by a significant margin. We also successfully illustrate the method on the question retrieval task.
1 Introduction
The paper motivates rationales as interpretable text fragments that make neural predictions more transparent. It proposes learning short, coherent, prediction-sufficient extractive rationales without rationale annotations.
- Complex neural models improve accuracy but provide little transparency about their inner workings.
- Rationales are intended to provide interpretable justifications for neural predictions.
- The proposed approach selects short, coherent input fragments that alone suffice for prediction.
- The generator distributes over candidate extracted rationales, while the encoder maps rationale text to task-specific target values.
- Rationales are learned without target rationale annotations through jointly trained generator and encoder components.
2 Related Work
Prior work improves interpretability through network analysis, sparse representations, proxy models, attention, or human-provided rationales. This paper instead learns extracted rationales without rationale annotations.
- Interpretability research includes analyzing activations, learning sparse word vectors, and linking vectors to semantic properties.
- Proxy approaches approximate trained networks with rules, decision trees, or locally valid models for individual samples.
- Attention-based models provide another way to visualize neural model behavior while improving prediction accuracy on NLP problems.
- Rationale-based classification methods use richer human-provided rationale annotations to improve prediction.
- In contrast, this work learns to generate rationales without providing them during training.
3 Extractive Rationale Generation
Extractive rationale generation selects a small subset of input words that remains interpretable and nearly preserves the original prediction. A generator and encoder are trained jointly without explicit rationale annotations.
- The task maps an input sequence to a target vector, while a complex encoder offers little justification for its prediction.
- A valid rationale is interpretable and sufficient to reach nearly the original input’s prediction.
- The generator selects a few input words, and the encoder should produce nearly the same target vector from them as from the full input.
- Rationale selection is an unsupervised latent-variable process because explicit word-level annotations are unavailable.
- The generator and encoder are trained jointly end to end so that they function well together.
4 Encoder and Generator
The encoder predicts target vectors from text, while the generator extracts probabilistic binary selections as rationales. They are trained jointly so selected text remains concise, coherent, and sufficient for prediction.
- Architecture: The framework uses multi-aspect sentiment prediction as a guiding example but is designed to generalize to other tasks.The recurrent units use recurrent convolution, which the authors apply throughout the model.
- Encoder: The encoder maps an input sequence to a target vector, such as a multi-aspect sentiment vector, using a prediction loss.The encoder predicts ỹ = enc(x) and uses squared error between the prediction and gold target.
- Generator: The generator extracts an interpretable subset of the input using binary selection variables and a distribution over possible selections.Each z_t indicates whether word x_t is selected, and z ∼ gen(x) ≡ p(z|x).
- Generator: Dependent selection introduces a hidden state that couples word choices, enabling phrase selection and discouraging repeated selections.This extends independent, context-dependent selection with dependencies among selected words.
- Joint objective: The generator and encoder are trained jointly as modular units, with the generator compressing text into coherent summaries that work with its encoder.Because rationale selections are latent, the expected cost is optimized by sampling rationales from the generator.
- Joint objective: The joint objective combines prediction loss with a regularizer that penalizes many selections and discourages disconnected words.The final cost is cost(z, x, y) = L(z, x, y) + Ω(z).
5 Experiments
Experiments evaluate rationale extraction for multi-aspect sentiment and similar-question retrieval. The joint model preserves prediction quality with short rationales, extracts aspect-relevant text more precisely than baselines, and reaches strong retrieval performance.
- Evaluation setup: The evaluation covers multi-aspect sentiment analysis on product reviews and similar-text retrieval on the AskUbuntu question-answering forum.The sentiment data include sentence-level aspect annotations, while retrieval is evaluated using question-matching performance.
- Sentiment prediction: The RCNN and LSTM encoders obtain test mean squared errors of 0.0087 and 0.0094 respectively, with RCNN selected for the joint model.The RCNN performs slightly better and uses fewer parameters, so the experiments use a two-layer RCNN encoder with 200 hidden states.
- Prediction with rationales: The joint dependent model approaches full-text encoder performance even when it extracts only a few words as rationales.Different rationale lengths are produced by varying the regularization parameters λ1 and λ2.
- Rationale selection: Rationale precision reaches 80%–96% across appearance, smell, and palate, compared with around 30% for the SVM baseline and lower performance for attention.The comparison constrains the models to extract similar numbers of words, while precision is measured against sentence-level aspect annotations.
- Learning dynamics: The dependent selection model learns rationale precision faster than the independent model, although both obtain similar results by the end.Early training produces low-precision rationales before the models improve after several exploratory epochs.
- Similar text retrieval: AskUbuntu rationales achieve mean average precision up to 56.5%, close to titles and above noisy question-body input.The model obtains this retrieval performance while extracting short fragments, including error messages that can be absent from question titles.
6 Discussion
The framework supports flexible encoder and generator choices while using stochastic rationale sampling for training. Additional output constraints and variance reduction can help address exploration and stability challenges.
- The encoder and generator can be implemented in numerous ways without changing the broader algorithm.Possible encoders include convolutional, deep averaging, and boosting models; simpler encoders may work better for stereotypical text patterns.
- Training uses a REINFORCE-style algorithm that estimates parameter gradients by sampling possible rationales.
- Additional constraints on generator outputs can alleviate exploration problems across the large space of rationales and their interactions with the encoder.
- Variance reduction techniques could increase the stability of stochastic training.