PaperlayerView exact revision on arXiv

Source-linked AI summary

Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations

Bodhisattwa Prasad Majumder, Oana-Maria Camburu, Thomas Lukasiewicz, Julian McAuley

arXiv:2106.13876v4cs.CLcs.AIcs.LG

TL;DR

Self-rationalizing models often trade task performance for explanations, and extractive rationales and NLEs capture complementary aspects of model reasoning. REXC grounds predictions and both explanation types in background knowledge, reaching strong task performance while improving explanation quality and supporting zero-shot NLE generation. Its perturbation analysis also finds the explanation–prediction association needed for faithful explanations.

  • Problem

    Existing models that produce high-quality extractive rationales or NLEs often underperform task-specific state-of-the-art systems, while the two explanation types are complementary.

  • Method

    REXC jointly grounds prediction, extractive rationale selection, knowledge-snippet selection, and NLE generation in background knowledge.

  • Results

    Across five natural-language and vision-language tasks, REXC reaches new state-of-the-art performance for explanations, closes the task-performance gap, and achieves new state-of-the-art performance on e-SNLI-VE.

  • Takeaways & Limitations

    REXC provides two complementary explanation types while maintaining competitive task performance and enables competitive zero-shot NLE generation from selected knowledge snippets.

  • Takeaways & Limitations

    The faithfulness analysis cannot establish sufficient conditions for faithful NLEs because different NLE realizations may contribute differently to predictions.

Abstract

from arXiv · show

Models that generate extractive rationales (i.e., subsets of features) or natural language explanations (NLEs) for their predictions are important for explainable AI. While an extractive rationale provides a quick view of the features most responsible for a prediction, an NLE allows for a comprehensive description of the decision-making process behind a prediction. However, current models that generate the best extractive rationales or NLEs often fall behind the state-of-the-art (SOTA) in terms of task performance. In this work, we bridge this gap by introducing RExC, a self-rationalizing framework that grounds its predictions and two complementary types of explanations (NLEs and extractive rationales) in background knowledge. Our framework improves over previous methods by: (i) reaching SOTA task performance while also providing explanations, (ii) providing two types of explanations, while existing models usually provide only one type, and (iii) beating by a large margin the previous SOTA in terms of quality of both types of explanations. Furthermore, a perturbation analysis in RExC shows a high degree of association between explanations and predictions, a necessary property of faithful explanations.

1. Introduction

REXC unifies background knowledge with extractive rationales and natural language explanations to address the task-performance gap faced by self-explainable models.

  • 1. Introduction: Extractive rationales offer concise feature-based explanations, whereas NLEs provide more detailed, human-accessible justifications but may lack background knowledge.The two explanation types are complementary because extractive rationales can omit relations between features while NLEs can express richer reasoning.
  • 1. Introduction: REXC grounds prediction and rationalization in background knowledge through a unified framework combining rationale extraction, knowledge snippets, and NLE generation.It extracts input features, queries a knowledge resource, selects relevant snippets, generates an NLE, and predicts from the NLE and input.
  • 1. Introduction: REXC significantly improves the quality of both explanation types and exhibits necessary faithfulness conditions under perturbation analysis.The framework’s explanations show a high degree of association with predictions, while its knowledge snippets can also serve as supporting evidence.
  • 1. Introduction: REXC reaches state-of-the-art task performance while providing both extractive rationales and natural language explanations.It matches state-of-the-art performance on four of five tasks and achieves new state-of-the-art performance on e-SNLI-VE.
  • 1. Introduction: REXC-ZS produces NLEs without direct NLE supervision and remains competitive with its supervised version.The zero-shot model uses selected knowledge snippets as NLEs, and can sometimes outperform models trained with full NLE supervision.

2. REXC

REXC jointly extracts input rationales, grounds them with selected background-knowledge snippets, generates natural-language explanations, and predicts task outputs. Its knowledge module is frozen while the remaining components are trained end-to-end, with latent sparse selectors enabling unsupervised rationale and knowledge selection.

  • 2. REXC: REXC extracts rationales, queries background knowledge, selects relevant snippets, generates an NLE, and predicts the final answer in one pipeline.All five stages are jointly learned, and rationale and snippet selections are modeled with latent variables rather than direct selection supervision.
  • 2.1. Extractive Rationales via Binary Latent Variables: An extractive rationale is a minimal sufficient subset of input units most responsible for the prediction, such as tokens or image super-pixels.The rationale selector uses latent variables with HardKuma reparameterization and L1-based sparsity control.
  • 2.2. Knowledge about an Extractive Rationale: The knowledge module queries each contiguous rationale element to generate a pool of modality-appropriate background-knowledge snippets.REXC uses generative knowledge resources such as COMET for text and VisualCOMET for images; the knowledge module can remain fixed during training.
  • 2.3. Knowledge Selection: A sparse knowledge selector retains relevant snippet representations and passes them to the NLE generator, while REXC+ decodes them as additional supporting evidence.Human evaluation reports higher-quality explanations when this additional evidence accompanies the NLE.
  • 2.4. NLE Generation and Task Prediction: The predictor conditions directly on the generated NLE and input, thereby implicitly conditioning on the rationale and selected knowledge snippets.The decoder and predictor are trained with task-output and ground-truth-NLE supervision, while the knowledge module itself is fixed for computational ease.
  • 2.5. Training: Training combines lower-bound objectives for rationale and knowledge selection with L1 sparsity penalties and a fused-Lasso compactness term.The two objectives are combined with weight α, and gradients are estimated through Monte-Carlo sampling from reparameterized HardKuma variables.

3. Experiments

The experiments evaluate REXC on three natural-language and two vision-language tasks against self-explainable and task-performance baselines. They also ablate rationale selection, knowledge selection, the NLE generator, and generative versus retrieval-based knowledge sources.

  • 3. Experiments: REXC is evaluated on three natural-language and two vision-language understanding tasks.Table 1 summarizes the task suite, while implementation uses BART-based components for natural-language tasks and UNITER-based components for vision-language tasks.
  • 3. Experiments: The baselines include explanation-focused models, black-box task-performance SOTA systems, post-hoc explainers, and vision-language NLE models.Natural-language comparisons include WT5, NILE, and CAGE; vision-language comparisons include PJ-X, FME, RVT, and e-UG.
  • 3. Experiments: The ablations remove the rationale selector, knowledge selector, both selectors, or the NLE generator, and also replace generative knowledge with retrieval-based sources.REXC-ZS uses selected knowledge snippets as NLEs without NLE-generator supervision, while REXC-RB uses ConceptNet and Visual Commonsense Graphs with maximum inner-product search.
  • 3. Experiments: Table 2 reports task accuracy and NLE quality using METEOR, BERTScore, BLEURT, and human evaluation metrics for natural-language and vision-language tasks.Bold values indicate statistically significant best results, and underlining identifies the best task performance among models providing explanations.

4. Results

REXC improves explanation quality across natural-language and vision-language tasks while maintaining strong task performance and supporting zero-shot NLEs. Its knowledge grounding and selection steps contribute to better NLEs, ERs, and explanation–prediction alignment.

  • Automatic Evaluation of NLEs: REXC achieves the best values on all three automatic NLE metrics for NL tasks, with METEOR gains of 4.8 to 11 points over non-grounded models.The comparisons include REXC w/o KN & ER and WT5.
  • Automatic Evaluation of ERs: REXC reaches new SOTA ER quality, including a 56. versus 51.9 F1 comparison against models lacking NLE or background-knowledge guidance.Joint NLE optimization may encourage informative ERs, while high-quality ERs help retrieve better knowledge snippets.
  • Human Evaluation of NLEs: Human evaluators rate REXC NLEs far better than previous SOTA NLEs on NL tasks, and REXC gains substantially over prior models on VL tasks.For VL tasks, previous SOTA NLEs were rated far below ground truths, while REXC also surpassed knowledge-using RVT.
  • Knowledge Ablations: Knowledge selection improves NLE quality: removing it causes large drops in human ratings and provides gains over the no-selection variant.The results describe knowledge selection as having positive effects and a regularizing effect.
  • Qualitative Analysis: Qualitative examples show REXC NLEs are more knowledge-grounded and comprehensive than previous SOTA explanations for COSe and VCR.Previous explanations could omit instance-specific reasoning, such as the role of boredom in COSe.
  • Task Performance: REXC matches task-performance SOTA on four of five tasks and achieves new SOTA on e-SNLI-VE while providing higher-quality ERs and NLEs.This bridges the prior gap between explainability and task performance.
  • Zero-shot NLEs: REXC-ZS produces human-rated NLEs better than previous SOTA models on four of five tasks despite weaker automatic scores caused largely by distribution mismatch.Concatenated knowledge snippets can serve as NLEs when ground-truth NLEs are unavailable, while the NLE module improves fluency and comprehensibility.
  • Knowledge Ablations: Replacing generative knowledge with ConceptNet leaves 23% of e-SNLI instances without snippets and worsens REXC-RB performance.The generative module avoids the no-hit issue typical of indexed knowledge bases.

5. Evaluating Faithfulness

REXC evaluates NLE faithfulness through feature-importance agreement and robustness equivalence, which assess whether perturbations affect predictions and explanations similarly. These are necessary rather than sufficient conditions, and REXC shows aligned behavior across datasets while outperforming random baselines on ER faithfulness metrics.

  • 5. Evaluating Faithfulness: Faithfulness evaluation remains limited because existing NLE criteria are necessary conditions, not sufficient guarantees.Different NLE realizations can contribute substantially and differently to a model’s prediction process.
  • 5.1. Faithfulness of the NLEs: Feature-importance agreement compares gradient-based attribution patterns for task predictions and generated NLEs.For REXC, salient tokens or super-pixels are selected using attribution scores at the top 10%, 20%, and 30% levels.
  • 5.1. Faithfulness of the NLEs: Robustness equivalence tests whether labels and NLE simulatability remain stable or become unstable in the same noise region.The analysis adds zero-mean Gaussian noise to feature representations and compares task accuracy with NLE simulatability.
  • 5.1. Faithfulness of the NLEs: Figure 5 evaluates robustness equivalence under noise applied to inputs and selected knowledge snippets by comparing stable labels, REXC accuracy, and NLE simulatability.Faithfulness is indicated when the sharpest declines in accuracy and simulatability align with the sharpest decline in stable labels.
  • 5.1. Faithfulness of the NLEs: Similar robustness trends appear in other datasets, supporting the reported pattern beyond the primary analysis.The additional observations are reported in Appendix E, Figure 8.
  • 5.2. Faithfulness of the ERs: REXC achieves better ER comprehensiveness and sufficiency than random selections and outperforms all models reported by DeYoung et al. (2020) on both metrics.These metrics provide established necessary and sufficient faithfulness checks for extractive rationales.

6. Related Work

Related work distinguishes post-hoc explanations from jointly trained self-explainable models and covers NLEs, ERs, and knowledge grounding. Existing systems often provide only one explanation type or lack faithfulness analysis, motivating REXC’s joint treatment.

  • Post-hoc and self-explainable models: Post-hoc methods explain fixed black-box models, whereas self-explainable models jointly produce predictions and explanations.Post-hoc explanations can be useful when only a high-performance black-box model is available, but prior work identifies important downsides.
  • Faithfulness: High held-out performance does not guarantee that a model uses the right reasons for its predictions.This concern motivates evaluating explanation faithfulness alongside task performance.
  • NLEs: Most NLE systems focus on generating explanations, while few jointly produce NLEs and ERs or analyze NLE faithfulness.The related work identifies Park et al. (2018) and Wu & Mooney (2019) as exceptions producing both explanation types.
  • ERs: ER research spans natural-language and vision-language tasks, and REXC jointly models ERs and NLEs to improve both explanation types.Prior ER work includes Zaidan & Eisner (2008), DeYoung et al. (2020), Lei et al. (2016), Bastings et al. (2019), Sha et al. (2021), and Strout et al. (2019).
  • Knowledge Grounding: Background knowledge has been used to ground generation and justify predictions, but REXC combines knowledge grounding with joint ER and NLE production.Prior applications include dialogue, creative-text, counterfactual generation, and prediction justification.

7. Summary and Outlook

The paper presents REXC as a knowledge-grounded self-rationalizing framework that jointly provides ERs and NLEs. Across five natural-language and vision-language tasks, it reports improved explanation quality, stronger task performance, faithfulness evidence, and a promising zero-shot direction.

  • 7. Summary and Outlook: Across five natural-language and vision-language tasks, REXC obtains new SOTA performance for both NLEs and ERs and a new SOTA on e-SNLI-VE.The paper also reports closing the task-performance and explainability gap across the five experiments.
  • 7. Summary and Outlook: REXC combines background knowledge with two complementary explanation types: extractive rationales and natural-language explanations.The framework uses knowledge resources while jointly supporting both forms of explanation.
  • 7. Summary and Outlook: Future work could extend REXC with specialized knowledge resources and investigate its principles for zero-shot and few-shot setups.The paper specifically mentions legal and medical knowledge as possible additions.

A. Implementation Details

Implementation uses early-stopped training, task-specific batch sizes and runtimes, tuned loss weights, AdamW optimization, and BART, UNITER, and GPT-2 backbones. Baselines rely on official code, fine-tuned T5, or published implementation details.

  • Training: Models train for at most five epochs with early stopping based on validation perplexity, using different batch sizes for NL and VL tasks.NL models use batch size 4, whereas VL models use batch size 32 on two 2080 Ti GPUs.
  • Hyperparameters: The implementation sets rationale and knowledge-selection weights, mixes rationale and NLE losses, and uses AdamW with a 6.25e−5 learning rate.The rationale weights are 1.0, the knowledge-selection weight is 0.9, and the loss-mixing coefficient is 0.4.
  • Baselines: REXC uses BART, UNITER, and GPT-2, while baselines use official NILE code, fine-tuned T5, or published implementation details.The vision-language baselines are PJ-X, FME, RVT, and e-UG.

B. Tasks

The paper evaluates RExC across five natural-language and vision-language tasks spanning commonsense validation, textual and visual entailment, commonsense QA, and visual commonsense reasoning.

  • B. Tasks: RExC is evaluated on commonsense validation using ComVE, which pairs sentences and requires selecting the sentence that violates commonsense.ComVE includes NLEs and uses 10,000/1,000/1,000 train/validation/test samples.
  • B. Tasks: SNLI and e-SNLI support three-way textual entailment classification with accompanying natural-language explanations.e-SNLI contains 550K/10K/10K train/validation/test samples.
  • B. Tasks: Commonsense QA treats answer selection as multiclass classification while generating an explanation for each predicted answer.COSe extends CQA with NLEs and provides 9,741 training and 1,221 validation samples.
  • B. Tasks: SNLI-VE adapts textual entailment to images as premises and text as hypotheses, retaining entailment, neutral, and contradiction labels.e-SNLI-VE contains 401K/14K/14K train/validation/test samples and adds NLEs.
  • B. Tasks: VCR evaluates visual commonsense reasoning, with RExC generating explanations from scratch rather than selecting them from a fixed pool.VCR contains 212K/26K/26K train/validation/test samples.

C. Automatic Metrics

Automatic evaluation follows prior work by measuring how closely generated NLEs match ground-truth explanations, with additional language-generation metrics reported for NL and VL tasks.

  • C. Automatic Metrics: NLE quality is assessed using metrics that capture similarity between generated explanations and ground-truth explanations.The evaluation follows the metric suite used by Kayser et al. (2021).
  • C. Automatic Metrics: The reported automatic metrics include BLEU-4, ROUGE-L, SPICE, and CIDER.These additional metrics are reported in Table 5 for NL and VL tasks.

D. Human Evaluation

Human evaluation uses trained annotators to judge whether explanations justify correct predictions, while error analysis identifies both improved comprehensiveness and a verbosity drawback.

  • D. Human Evaluation: The human study used two Anglophone Amazon Mechanical Turk annotators per question, each with at least a 90% lifetime HIT acceptance rate.The interface setup is illustrated in Fig. 10.
  • D. Human Evaluation: Annotators first solved the predictive task, then rated whether each explanation justified the answer using Yes, Weak-Yes, Weak-No, or No.The e-ViL score combines these options; only correct predictions were evaluated, with 250 random examples per model and dataset.
  • D. Human Evaluation: Inter-annotator agreement reached kappas of 0.72, 0.76, 0.79, 0.81, and 0.74 across the evaluated datasets.Agreement was measured with Cohen’s kappa.
  • D. Human Evaluation: Adding commonsense knowledge and knowledge selection made RExC explanations more comprehensive and relevant, while RExC+ won across datasets but was often judged too verbose.Supporting knowledge snippets sometimes repeated information already present in the generated NLEs.

E. Faithfulness

Faithfulness analysis finds consistent agreement and robustness between labels and RExC explanations, supporting a strong association between predictions and explanations.

  • E. Faithfulness: Feature-importance agreement between labels and NLEs is observed across all datasets.The relationship is summarized in Fig. 7 for task accuracy and NLE simulatability.
  • E. Faithfulness: Labels and NLEs show equivalent robustness across datasets when noise is introduced.Fig. 8 analyzes perturbations to inputs and selected knowledge snippets with varying σ2.
  • E. Faithfulness: Together, the agreement and robustness findings indicate a strong label–NLE association for RExC, satisfying necessary conditions for faithful explanations.Illustrative NLEs, extractive rationales, and supporting commonsense are shown in Fig. 9.
Loading 2106.13876v4…