Source-linked AI summary
SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers
Dheeraj Rajagopal, Vidhisha Balachandran, Eduard Hovy, Yulia Tsvetkov
TL;DR
Neural text classifiers can be opaque, while word-level attributions have limitations in explaining phrase structure and influential training data. SELFEXPLAIN adds local and global phrase-based explanations to neural classifiers. Across five datasets it maintains end-task performance while improving interpretability, and its explanations receive favorable human assessments.
Problem
Neural classifiers provide limited insight into decisions, while word-level attributions leave phrase-level explanations and explanations based on influential training data as challenges.
Source: §1 · p. 1Method
SELFEXPLAIN adds a local layer that scores input concepts by activation difference and a global layer that retrieves influential training concepts using maximum inner product search.
Source: §1 · p. 2, §2.4 · p. 4Results
Across three text-classification tasks and five datasets, SELFEXPLAIN facilitates richer interpretability while maintaining end-task performance, and its explanations are judged more understandable, adequate, and trustworthy than strong baselines.
Source: §1 · p. 2Takeaways & Limitations
SELFEXPLAIN combines local and global phrase-based explanations in one trainable framework for neural text classifiers.
Source: §7 · p. 9The framework is evaluated for single-input-sentence contexts; future work will extend it to other tasks and longer contexts.
Source: §7 · p. 9
Abstract
from arXiv · showhide
We introduce SelfExplain, a novel self-explaining model that explains a text classifier's predictions using phrase-based concepts. SelfExplain augments existing neural classifiers by adding (1) a globally interpretable layer that identifies the most influential concepts in the training set for a given sample and (2) a locally interpretable layer that quantifies the contribution of each local input concept by computing a relevance score relative to the predicted label. Experiments across five text-classification datasets show that SelfExplain facilitates interpretability without sacrificing performance. Most importantly, explanations from SelfExplain show sufficiency for model predictions and are perceived as adequate, trustworthy and understandable by human judges compared to existing widely-used baselines.
1 Introduction
Neural text classifiers can obscure their decisions despite risks of overfitting to spurious artifacts and amplifying data biases. SELFEXPLAIN adds local and global explanations based on phrase-level concepts, and its experiments report richer interpretability while maintaining task performance.
1 Introduction
- Neural models provide limited insight into decisions and can overfit to spurious artifacts or amplify biases in data.These concerns underscore the need to understand model decision making.Source: §1 · p. 1
- Word-level feature attributions leave open the challenge of explaining higher-level phrase structure and often require additional methods for global explanations.Prior attributions can also be unreliable or unfaithful, and are limited to the input feature space.Source: §1 · pp. 1–2
- SELFEXPLAIN combines local relevance scores for input concepts with retrieval of influential training concepts in a single end-to-end differentiable classifier.Its Locally Interpretable Layer uses activation differences, while its Globally Interpretable Layer uses maximum inner product search.Source: §1 · p. 2
- Across three text-classification tasks and five datasets, SELFEXPLAIN adds richer interpretability while maintaining end-task performance.Its explanations also reflect model predictions and are judged more understandable, adequate, and trustworthy than strong baseline methods.Source: §1 · p. 2
2 SELFEXPLAIN
SELFEXPLAIN builds phrase-based explanations into a neural text classifier: LIL scores input phrases locally, while GIL retrieves influential concepts from training data to explain predictions.
2.1 Defining human-interpretable concepts
- Concepts are human-interpretable phrases, specifically the non-terminals in an input’s constituency parse tree.The model decomposes each input into these phrases for explanation.Source: §2.1 · p. 2
2.2 Concept-Aware Encoder E
- The concept-aware encoder represents each phrase by averaging its constituent word representations and represents the root with the pooled [CLS] output.The phrase representations are derived from the final layer of a pretrained transformer.Source: §2.2 · p. 2
2.3 Local Interpretability Layer (LIL)
- LIL scores each input phrase by comparing its predicted-label score with the score for an input representation that omits that phrase.The relevance score is the difference between the full-input and phrase-omitted classifier scores for the predicted label.Source: §2.3 · p. 3
2.4 Global Interpretability layer (GIL)
- GIL retrieves the top-K influential concepts from the training set by searching against the input representation, and learns this retrieval approach end-to-end.Its concept store contains concepts from the training data.Source: §2.4 · p. 4
2.5 Training
- Training jointly optimizes label prediction and LIL and GIL outputs with task-label cross-entropy losses and regularization hyperparameters.The GIL and LIL losses are based on their respective label distributions.Source: §2.5 · p. 4
3 Dataset and Experiments
Experiments evaluate SELFEXPLAIN across five text-classification datasets with RoBERTa and XLNet. Classification performance is competitive with the base models, apart from a marginal drop on TREC-6 for SELFEXPLAIN-XLNet.
Datasets
- The five datasets cover binary and fine-grained sentiment, question classification with 6 or 50 classes, and subjective/objective classification.Table 1 reports dataset statistics, including class count and average sentence length.Source: §3 · pp. 4–5, Table 1 · p. 4
Experimental Settings
- Experiments incorporate SELFEXPLAIN into RoBERTa and XLNet, compare against the same encoders without GIL and LIL, and test K = 5 or 10.The authors tune α and β over {0.01, 0.1, 0.5, 1.0} and report the best configuration.Source: §3 · p. 5
Classification Results
- SELFEXPLAIN-RoBERTa and SELFEXPLAIN-XLNet show competitive performance against their base models, except for a marginal drop on TREC-6 with XLNet.Table 2 compares models with and without GIL and LIL using the same encoder configurations.Source: §3 · p. 5, Table 2 · p. 5
Ablation
- Ablations suggest that GIL and LIL contribute complementary performance gains, while changing K made no noticeable difference.The ablation results are reported for SELFEXPLAIN-XLNet and SELFEXPLAIN-RoBERTa on SST-2.Source: §3 · p. 5, Table 3 · p. 5
4 Explanation Evaluation
The evaluation tests whether explanations reflect model predictions and whether people find them plausible, understandable, and trustworthy. Results favor SELFEXPLAIN on predictive sufficiency and the reported human judgments.
4 Explanation Evaluation
- Evaluation criteria cover sufficiency, human plausibility and understandability, and trustability; evaluated explanations are top-ranked local LIL phrases or influential global GIL concepts.The local concepts are ranked by LIL relevance scores, while GIL ranks influential concepts from training data.Source: §4 · pp. 5–6
4.1 Do SELFEXPLAIN explanations reflect predicted labels?
- FRESH tests sufficiency by training a BERT classifier on explanations alone and assessing whether they predict the original model’s label.The remaining input is withheld; high accuracy indicates that the explanations can recover the original model prediction.Source: §4.1 · p. 6
- In FRESH, LIL and GIL explanations show high predictive performance against baseline methods, and GIL outperforms full-text performance.The authors interpret this as higher sufficiency: the explanations are more predictive of labels than prior explanation methods.Source: §4.1 · p. 6
4.2 Are SELFEXPLAIN explanations plausible and trustable for humans?
- Human evaluation involved 14 judges annotating 50 SST-2 validation samples; each judge compared local and global explanations, and the models were anonymized and shuffled.Source: §4.2 · pp. 6–7
- 32% gain in perceived adequate justification; understandability improved 29% over the best-performing baseline.Figure 3 compares the percentage of human-evaluated samples; the reported improvements are for adequate justification and understandability, respectively.Source: Figure 3 · p. 7, §4.2 · p. 7
- Human judges perceived concept-based explanations as more trustworthy than the baselines.Trust was rated on a 1–5 Likert scale.Source: Figure 4 · p. 7, §4.2 · p. 8
5 Analysis
The analysis examines whether explanations help people predict model behavior, whether the interpretability layers agree with model predictions, and how concept selection affects performance and stability.
Analysis
- 22% average improvement in users’ ability to predict model decisions when shown SELFEXPLAIN explanations.Three judges evaluated 16 balanced dev-set samples, predicting model decisions with and without explanations.Source: §5 · p. 8
- K = 5 is preferable for sentence-level tasks as a balance between performance and interpretability, with only a marginal performance drop from the base model.The K analysis varies the number of retrieved influential GIL concepts.Source: §5 · p. 8, Table 6 · p. 8
- On SST-2, LIL-linear F1 is 96.6%, GIL-linear F1 100%, and GIL-LIL-linear F1 agreement 96.6%.The reported agreement is for SELFEXPLAIN-XLNet; the authors say the layers agree on the same classification prediction.Source: §5 · p. 8
- Removing top-ranked LIL phrases lowered annotator accuracy 7%, versus ∼14% for classifier accuracy on the same inputs.The analysis used 50 SST-2 dev samples with the most salient local phrases removed.Source: §5 · p. 8
- Minor input variations can change the ranked local concepts while global influential concepts remain stable.Table 7 presents an input-perturbation example; the stability analysis is qualitative.Source: §5 · pp. 8–9, Table 7 · p. 9
6 Related Work
Related work distinguishes post-hoc explanations from inherently interpretable models and surveys local, global, and concept-based approaches to neural text classification.
Related Work
- Post-hoc NLP interpretability includes gradient-based local methods and approaches that decompose hidden states, while global methods retrieve influential training samples.Global interpretability has also been used to facilitate explainability and detect or mitigate data artifacts.Source: §6 · p. 9
- Inherently interpretable approaches include attention heat maps, annotated rationales, training-sample-based explanations, and modular architectures.The passage also describes high-level concept classifiers, including prototypes and interpretable classes.Source: §6 · p. 9
- SELFEXPLAIN is similar in spirit to Melis and Jaakkola (2018), but adds training-data concepts for neural text classification.The cited work describes high-level concepts; the passage notes that such classifiers had not yet been adopted in NLP.Source: §6 · p. 9
7 Conclusion
SELFEXPLAIN uses higher-level concepts to explain classifier predictions and combines local and global explanations in an end-to-end framework. Human evaluators perceived its explanations as more trustworthy, understandable, and adequate than previous approaches; future work will extend the framework beyond single-sentence contexts and to other tasks.
7 Conclusion
- SELFEXPLAIN explains predictions through higher-level concepts rather than low-level word attributions.The framework provides local explanations from input concepts and global explanations from influential training concepts through two end-to-end modules.Source: §7 · p. 9
- Human evaluators perceived SELFEXPLAIN explanations as more trustworthy, understandable, and adequate than previous approaches to explainability.Source: §7 · p. 9
- Future work will extend the framework to other tasks and longer contexts beyond a single input sentence.The authors also plan to explore additional linguistic concepts and the appropriate abstraction level for human-friendly explanations.Source: §7 · p. 9
A.1 Additional Analysis
The additional analysis qualitatively examines explanation stability by comparing extracted concepts for similar examples.
A.1 Additional Analysis
- For similar examples, local phrases can change rank after minor input variations, while global influential concepts remain stable.The authors assess stability by measuring overlap of extracted concepts and provide a representative SST-2 example in Table 8.Source: §A.1 · p. 14
A.2 Qualitative Examples
The section presents qualitative examples from the best-performing SST-2 model.
A.2 Qualitative Examples
- Table 9 presents qualitative examples from SELFEXPLAIN’s best-performing SST-2 model.Source: §A.2 · p. 14
A.3 Relevant Concept Removal
The relevant-concept-removal examples show model predictions before and after removing the most relevant local concept.
A.3 Relevant Concept Removal
- Table 10 compares original inputs and predictions with perturbed inputs and predictions after removing the most relevant local concept.The examples are cases where the model’s prediction flipped after removal.Source: §A.3 · p. 14, Table 10 · p. 16