Source-linked AI summary
Measuring Association Between Labels and Free-Text Rationales
Sarah Wiegreffe, Ana Marasović, Noah A. Smith
TL;DR
The paper addresses the limited evidence that free-text rationales faithfully reflect model decisions, especially for reasoning tasks where extractive rationales are insufficient. It compares pipeline and joint approaches, then tests label-rationale association using robustness equivalence and feature importance agreement. T5-based joint models show both properties on CommonsenseQA and SNLI, while the proposed measurements remain necessary rather than sufficient checks.
Problem
Free-text rationales are widely used for reasoning tasks, but whether jointly generated labels and rationales are faithfully connected within the model is not yet established.
Method
The paper compares pipeline and self-rationalizing models and proposes robustness equivalence and feature importance agreement to measure label-rationale association.
Results
T5-based joint models exhibit high robustness equivalence and feature importance agreement across three free-text rationalization datasets for CommonsenseQA and SNLI.
Takeaways & Limitations
These measurements provide a necessary sanity check indicating the potential of T5-based joint models to generate faithful free-text rationales.
Takeaways & Limitations
The measurements address necessary but not sufficient properties of faithful interpretability.
Abstract
from arXiv · showhide
In interpretable NLP, we require faithful rationales that reflect the model's decision-making process for an explained instance. While prior work focuses on extractive rationales (a subset of the input words), we investigate their less-studied counterpart: free-text natural language rationales. We demonstrate that pipelines, existing models for faithful extractive rationalization on information-extraction style tasks, do not extend as reliably to "reasoning" tasks requiring free-text rationales. We turn to models that jointly predict and rationalize, a class of widely used high-performance models for free-text rationalization whose faithfulness is not yet established. We define label-rationale association as a necessary property for faithfulness: the internal mechanisms of the model producing the label and the rationale must be meaningfully correlated. We propose two measurements to test this property: robustness equivalence and feature importance agreement. We find that state-of-the-art T5-based joint models exhibit both properties for rationalizing commonsense question-answering and natural language inference, indicating their potential for producing faithful free-text rationales.
1 Introduction
The paper examines faithful free-text rationales for reasoning tasks, where extractive rationales may omit necessary reasoning steps. It compares pipelines with joint models and proposes tests for whether labels and rationales are meaningfully associated.
- Faithful rationales should accurately represent a model’s decision process for debugging, safety validation, and detecting unintended behavior.
- Extractive rationales can fall short on commonsense question-answering and natural language inference because they may omit reasoning or knowledge needed to justify the label.
- Pipelines first generate rationales and then predict outputs, whereas self-rationalizing models jointly predict labels and rationales.
- For CommonsenseQA and SNLI, self-rationalizing rationales better indicate correct labels than pipeline rationales, while rationale sufficiency does not generally hold.
- The paper proposes robustness equivalence and feature importance agreement to test association between labels and free-text rationales within joint models.
2 Tasks, Datasets, and Models
The experiments use T5-based models on English reasoning datasets with human-written rationales, comparing pipeline and joint self-rationalizing architectures. The section defines the task settings, model components, and evaluation choices.
- Tasks and Datasets: E-SNLI asks whether a hypothesis entails, contradicts, or is neutral toward a premise, while CoS-E asks models to select the correct answer choice.
- Tasks and Datasets: The study uses E-SNLI and CoS-E, including CoS-E v1.0 and v1.1, as English textual-reasoning datasets with human-written natural-language rationales.
- T5 Models: T5 is a 220M-parameter transformer encoder-decoder trained by maximizing the conditional likelihood of the correct text output given the input.
- T5 Models: The model components include I→R for rationale generation, R→O for output prediction from rationales, I→OR for joint prediction, and IR→O for prediction from inputs plus rationales.
- Models: The pipeline composes separately trained I→R and R→O models for 440M total parameters, while the self-rationalizing I→OR model predicts both label and rationale jointly.
- Evaluation: BLEU is omitted because reference-overlap metrics do not measure rationale plausibility or faithfulness and assume a single ground-truth rationale.
3 Shortcomings of Free-Text Pipelines
Free-text pipelines face two central problems on reasoning tasks: generated rationales may not reflect labels, and rationales may lack information needed for prediction. These shortcomings make joint self-rationalizing models a more suitable focus for free-text rationalization.
- Pipeline setup: I→R;R→O pipelines are evaluated for rationale quality and whether their sufficiency assumption fits free-text reasoning tasks.The pipeline first generates a rationale and then predicts the output from that rationale alone.
- Label dependence: Rationales should depend on both the input and predicted label; otherwise an I→R model may produce the same rationale across tasks or annotation layers.Without task-specific input, an I→R model trained on multilayer annotations could generate an identical rationale regardless of the task being rationalized.
- Rationale quality: I→OR rationales recover 8–9% more ground-truth performance than R→O rationales on CoS-E and 1% more on E-SNLI.The comparison evaluates an R→O model on ground-truth rationales and rationales generated by I→OR or I→R.
- Rationale quality: I→OR rationales are higher quality than I→R rationales across datasets, indicating that training on label signal improves rationale generation.The reported rationale-quality comparison is summarized in Table 4.
- Sufficiency: The sufficiency assumption is unsuitable for many reasoning tasks because a rationale alone may not provide enough information to predict the label.In CoS-E, adding the original input to the rationale improves accuracy by 5–12% over using the rationale alone.
- Overall shortcomings: Pipelines can suffer cascading errors, missing information, parameter overhead, and manual labor while still performing worse than end-to-end models.These shortcomings motivate turning to self-rationalizing models, which jointly predict labels and rationales.
4 Analyzing Necessary Properties of Joint Models
The paper tests whether jointly generated labels and free-text rationales are meaningfully linked, using robustness equivalence and feature importance agreement as necessary faithfulness checks.
- Motivation: Self-rationalizing models cannot be treated as faithful explanations without testing whether their label and rationale mechanisms are associated.The paper frames association as a necessary sanity check rather than a guarantee of faithfulness.
- Robustness Equivalence: Robustness equivalence compares whether labels and rationales remain stable or unstable under the same input noise.Gaussian noise N(0, σ2) is added to input embeddings, while label changes and rationale quality are measured.
- Robustness Equivalence: If only one output remains stable under noise, the model fails the association test because the two generation mechanisms are not strongly linked.Both outputs being stable or both unstable indicates similar sensitivity to noise.
- Robustness Equivalence: At σ2 = 20, both CoS-E versions reached minimum rationale contribution, while CoS-E v1.0 showed its largest label-accuracy drop between σ2 = 15 and σ2 = 20.At lower noise levels, labels and rationales were stable; at σ2 values of 20 and above, both became unstable, supporting robustness equivalence across the investigated datasets.
- Feature Importance Agreement: Feature importance agreement tests whether tokens important for label prediction are also important for rationale generation, and vice versa.The method uses gradient-based attribution and ROAR occlusion to measure cross-output effects.
- Feature Importance Agreement: Removing top-k% tokens by label attribution degraded rationale quality more than random, while rationale attributions also degraded label performance.The pattern held for CoS-E v1.0, E-SNLI, and CoS-E v1.11, indicating feature-importance agreement for the studied datasets.
5 Related Work
Prior faithfulness analyses largely target extractive rationales, while this work addresses label–rationale association for free-text rationalization and gradient-attribution reliability in NLP.
- Structural and behavioral tests study model properties related to these experiments, but not the same properties.
- Gradient-attribution’s interplay with free-text rationalization has received little prior study.
- This work is presented as the first evaluation of gradient-attribution reliability for NLP tasks using the ROAR test.
- Prior faithful-explanation models and evaluations generally focus on extractive rationales and rely on sufficiency.
- Related natural-language-rationale pipelines use differentiable stages, complexity control, or label-specific explanation generators.
6 Conclusion
The paper evaluates label–rationale association in self-rationalizing models and finds that T5-based models pass a necessary sanity check on free-text rationalization datasets. The authors emphasize that these measurements are not sufficient properties of faithfulness.
- T5-based models exhibit high robustness equivalence and feature importance agreement on three CommonsenseQA and SNLI datasets.
- The measurements assess necessary, rather than sufficient, properties of faithful free-text rationales.
- Future work can expand the analysis to additional properties.
A Additional Information
The paper’s additional information includes an overview of datasets and rationale types used in prior pipeline work.
- Table 2 overviews datasets and rationale types used in prior work on pipelines.
A.2 Details of Datasets
This section provides dataset statistics, distinguishes CoS-E versions, and describes T5’s broad text-to-text pretraining background.
- The paper summarizes dataset statistics in Table 7.
- CoS-E v1.0 and v1.11 correspond to the first and second CommonsenseQA versions, respectively.
- CoS-E v1.11 contains annotation noise, motivating reports on v1.0, which does not exhibit these issues.
- T5 is pretrained on unsupervised and supervised tasks including translation, question answering, summarization, and classification.
A.4 Implementation Details
The experiments use Huggingface datasets, pretrained T5 models, and specified optimization settings, with training performed on an NVIDIA RTX 8000 GPU.
- The experiments access datasets through Huggingface Datasets and pretrained T5 weights and tokenizers through Huggingface Transformers.
- Training uses Adam with ϵ = 1e-8, β1 = 0.9, and β2 = 0.99, plus gradient clipping at 1.0 and dropout of 0.1.
- Each model is trained on an NVIDIA RTX 8000 GPU with 48GB memory for up to 200 epochs using batch size 64.
- The learning rate linearly decays from 5e-5, and training stops after 10 validation epochs without decreased loss.
A.5 Note on Robustness Equivalence Convergence
At large noise levels, reported performance can reach zero because delimiter failures prevent the model’s label from being separated from its rationale.
- Worst-case model performance reaches 0 rather than random accuracy under large noise values in Robustness Equivalence experiments.The behavior is attributed to the structure of the models’ output.
- The I→OR model uses a delimiter to distinguish the label from the rationale in a long output string.
- When high noise prevents delimiter generation, multiple answer choices may appear together and the label is marked incorrect.
A.6 Further Discussion of Rationale Quality Metric
The rationale quality metric is difficult to interpret because low scores can reflect either poor rationales or poor label predictions.
- Traditional simulatability is often lower-bounded at 0 when model-predicted explanations are consistent with model-predicted labels.
- The paper’s rationale quality metric does not share this lower-bound property, and its scores are commonly negative in the experiments.
- Low rationale quality scores may reflect poor-quality rationales or poor label prediction by the rationale-generating model.The authors propose robust simulatability as future work to separate these confounders and handle noisy or ill-defined predictions.
B Additional Results
The appendix reports dataset statistics, T5 formatting, baseline comparisons, robustness and attribution analyses, and additional noised-output examples.
- Additional materials: Tables 7 and 8 report dataset statistics and T5 input-output formatting, while Tables 11-13 show additional noised I→OR outputs for CoS-E v1.0 examples.The examples concern air conditioning, computer-mediated communication, and communication with a boss.
- Model comparisons: Table 9 compares baseline I→O T5 models with rationalizing I→OR variants, reporting some accuracy loss after adding rationalization and stronger performance from T5-Base models.
- Model comparisons: Table 10 compares the joint self-rationalizing I→OR model with a pipeline using natural-language rationales and reports stronger task performance for I→OR.
- Robustness equivalence: Figures 8 and 9 provide robustness-equivalence results for labels on E-SNLI and CoS-E v1.11 and for rationales on E-SNLI.Most label changes occur in the 10-20 σ2 range for CoS-E and 15-30 σ2 for E-SNLI.
- Attribution analyses: Figure 11 compares gradient attribution methods on the CoS-E v1.0 ROAR reliability check, while Figure 12 reports ROAR feature-importance agreement for E-SNLI and CoS-E v1.11.The largest performance drop in Figure 11 comes from the L1 norm embedding-combination method.