When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness

• The study defines predictive utility as the change in downstream task performance when few-shot rationales are added relative to few-shot prompts without explanations.It examines explanation sources, faithfulness-based selection of self-explanations, and robustness to rationale misalignment.Source: §1 · p. 2
• A modular evaluation framework combines error-based sample selection, explanation-quality scoring, and prompt construction to compare explanation sources and selection strategies.The authors state that they will release the study’s code, explanation datasets, and prompts upon acceptance.Source: §1 · p. 3
• Externally LLM-generated explanations typically yield the largest gains on classification-style benchmarks, while math-reasoning benefits are smaller and more model- and source-dependent.The study compares human, self-generated, and externally generated explanations across six benchmarks and four models.Source: §1 · p. 3
• Faithfulness-based selection can help, but its utility depends on the metric and dataset; faithfulness metrics can disagree and change which self-explanations are selected and how much they help.Source: §1 · p. 3
• Misalignment stress tests show that semantic alignment between explanations and examples is important to explanation-augmented prompting.The tests use randomly swapped and out-of-distribution rationales.Source: §1 · pp. 2–3

Explainable datasets and Human-annotated Rationales

• Human-annotated explanations can be costly and non-automatable to collect, inconsistent, performance-impairing, or beneficial only for a constrained subset of LLMs (Hartmann & Sonntag, 2022; Yao et al., 2023; Wei et al., 2022).Explainable datasets provide human explanations for labeled samples across diverse NLP tasks (Mathew et al., 2021; Talmor et al., 2019).Source: §2 · p. 3

LLM-generated NLEs

• LLM-generated NLEs offer human-readable justifications, but self-explanations are post-hoc and may not faithfully reflect LM reasoning, motivating faithfulness evaluation (Bhan et al., 2024; Atanasova et al., 2023; Shailya et al., 2025).Self-explanations can be full natural-language text or salient tokens (Bhan et al., 2024).Source: §2 · pp. 3–4

Leveraging Explanation to Improve Performance of LMs

• AMPLIFY uses proxy-PLM feature attributions for few-shot corrective signals, while self-AMPLIFY uses SLM-generated salient tokens or CoT but does not assess explanation quality and focuses on small models (Krishna et al., 2023; Bhan et al., 2024).AMPLIFY's approach requires computationally expensive attribution techniques and auxiliary PLMs; self-AMPLIFY mitigates some computational burden.Source: §2 · p. 4

Contribution

• The study compares three NLE sources as behavior-shaping in-context rationales and examines how faithfulness and selection strategy affect their utility across SLMs and LLMs.The selection strategies are random and faithfulness-based filtering; the study focuses on NLEs as practical steering signals for prompting.Source: §2 · p. 4

3 Our Approach - Comparative Study Setup

• The four-step framework selects few-shot examples, generates and selects their rationales, then builds an ICL prompt from the selected pairs and explanations.Figure 1 summarizes the experimental setup.Source: §3 · p. 4, Figure 1 · p. 5

3.1 Few-shot Samples Selection

• Few-shot examples are drawn from inputs the evaluated model misclassifies in zero-shot prompting, treating those errors as corrective signals.The study follows the n-shot sampling strategy of Bhan et al. (2024).Source: §3.1 · p. 4

3.2 Natural Language Explanation Generation

• The study compares human-annotated rationales when available, self-generated explanations, and explanations generated by an LLM for their predictive utility in ICL.Human rationales come from datasets with annotations for the correct label.Source: §3.2 · p. 5
• Self-NLEs use post-hoc chain-of-thought generation, but error examples are regenerated conditioned on the gold label for comparison with other corrective rationales.An ablation using explanations conditioned on the model’s incorrect prediction found lower average utility than the gold-conditioned variant across evaluated settings (Table 7 in Appendix H).Source: §3.2 · p. 5
• LLM-NLEs are generated in an annotation-style process for input-output pairs, independently of the evaluated model’s prediction process, and can be reused across evaluation models.The explainer can differ from the evaluated model; even when they are the same model, annotation-style generation differs from the self-NLE Ph-CoT process.Source: §3.2 · p. 5

3.3 NLE Selection

• NLE selection compares random examples with the top-n self-NLEs ranked separately by two faithfulness metrics, using the corresponding input-output pairs for human and LLM rationale comparisons.The two metric-based variants are Faithful-fm1 and Faithful-fm2.Source: §3.3 · p. 6
• Contrastive selection uses the bottom-n self-NLEs under each metric, while robustness ablations swap in same-dataset rationales or out-of-dataset rationales, the latter only for LLM-NLEs.The lowest-faithfulness condition tests whether utility changes consistently with measured faithfulness; the rationale swaps test alignment and cross-domain noise.Source: §3.3 · p. 6

3.4 Final Prompt Design for LMs

• The final ICL prompt combines a preprompt for answering and explaining with the selected n-shot input-rationale-output triplets.The preprompt differs between the self-NLE setup and the human- and LLM-NLE setups.Source: §3.4 · p. 6

4.1 Datasets

• Six benchmarks span commonsense (ECQA), semantic inference (e-SNLI), pragmatics (SNARKS), causal reasoning (cj), Boolean logic (Bool), and multi-step math (GSM8K).The cited datasets are ECQA, e-SNLI (Camburu et al., 2018), Big-Bench-Hard tasks (Suzgun et al., 2023), and GSM8K (Cobbe et al., 2021).Source: §4.1 · pp. 6–7

4.2 Models

• Evaluation uses GPT-4o-mini, Llama-3.1-8B, Llama-3.3-70B, and Mistral-7B-Instruct-v0.3 in a consistent 6-shot setup.GPT-4o-mini and o3-mini generate external LLM-NLEs, while each evaluation model also generates its own self-NLEs.Source: §4.2 · p. 7

4.3 Evaluating Self-NLEs Faithfulness

• Self-NLE faithfulness is measured with fm1, a three-component score, and fm2, a binary test of prediction consistency after a counterfactual edit.For fm1, the top-scoring dataset-specific quartile is labeled faithful; fm2 marks an explanation faithful when the edited explanation changes the prediction to the intended alternative label.Source: §4.3 · pp. 7–8

4.4 Baseline Methods

• Baselines are Zero-Shot, Few-Shot without rationales, zero-shot chain-of-thought, and FS-Random, which uses random few-shot examples without rationales.These comparisons measure the added value of NLEs in in-context exemplars beyond standard prompting strategies.Source: §4.4 · p. 8

5.1 Comparing NLE types and Setups (RQ1)

• On classification benchmarks, NLEs often improve accuracy over FS-R, and LLM-NLEs are often strongest; human rationales slightly outperform them on e-snli.On gsm8k, LLM-NLEs are generally stronger than self-NLEs, while baseline performance is strongly driven by CoT.Source: Table 1 · p. 8, §5.1 · p. 8
• Across selection settings, LLM- and human-NLEs usually provide stronger, more stable utility than self-NLEs, which benefit more selectively from faithfulness-based selection.Exceptions include self-NLEs outperforming LLM-NLEs for some faithfulness metrics on gsm8k and cj; human-NLEs are best on e-snli across settings.Source: §5.1 · pp. 8–9, Table 2 · p. 9

5.2 Relationship Between Measured Faithfulness and Self-NLE Predictive Utility (RQ2)

• High-faithfulness self-NLE selection yields small average gains overall, but effects vary by dataset, faithfulness metric, and model; lowest-faithfulness selection is harmful on average.On gsm8k, fm1 improves average performance by 0.073, while fm2 changes it by −0.003; lowest-faithful-fm1 and lowest-faithful-fm2 average −0.035 and −0.041, respectively, versus random selection.Source: §5.2 · p. 9, Table 3 · p. 10

5.3 Robustness to Random and OOD Explanations (RQ3)

• Under OOD rationales, models retain non-trivial accuracy but are consistently harmed by semantically mismatched explanations.The results support partial rather than complete robustness and motivate careful explanation selection.Source: §5.3 · pp. 10–11
• OOD rationale swaps consistently reduce performance, whereas in-domain random swaps have mixed effects, indicating partial robustness but a role for semantic alignment.In-domain rationales are swapped across examples; some settings show modest drops or slight gains, but drops are more pronounced for LLM-NLEs on e-snli and boolean.Source: §5.3 · pp. 9–10, Table 4 · p. 11

Which NLE to Use? Practical Guidance

• NLEs often improve few-shot performance, and externally generated or human-written explanations offer comparatively strong, stable utility, while self-NLE utility is more sensitive to selection.LLM- and human-NLEs perform strongly under random selection on most evaluated classification-style benchmarks, offering a practical starting point without faithfulness scoring; faithfulness-selected self-NLEs may help math reasoning in some settings.Source: §6 · p. 11

LLM-Generated vs. Human Explanations

• External LLM-generated explanations are broadly competitive with human rationales on ecqa and e-snli, though human rationales remain slightly stronger on average on e-snli.LLM explanations often match or outperform human explanations on ecqa and can be produced faster and more cheaply; human rationales may retain an advantage when fine-grained semantic alignment is needed.Source: §6 · p. 11

Disagreement Between NLE Faithfulness Metrics

• Faithfulness metrics disagree substantially: their average disagreement rate is ∼0.5 across datasets and models, making metric choice consequential for self-NLE evaluation and selection.The disagreement direction is dataset-dependent but consistent across models within each dataset; the authors advise treating disagreement as measurement uncertainty rather than treating one metric as ground truth.Source: §6 · pp. 11–12

Faithfulness-Based NLE Selection Can Help, but Not Always

• Faithfulness-based selection of self-NLEs yields small average gains, but outcomes vary by metric, task, and model and can include substantial harm.On gsm8k, high-faithfulness selection with fm1 improves average performance, while fm2 produces almost no average change because it benefits Llama-70B but strongly degrades Mistral-7B; lowest-faithfulness selection is especially harmful there.Source: §6 · p. 12

Insights on Robustness and Misalignment

• Randomly swapped in-domain and out-of-domain rationales generally reduce performance, although models retain reasonable performance under these perturbations.OOD swaps are consistently harmful, while in-domain swap effects vary, possibly because the rationales still provide task-relevant cues; the findings support using semantically aligned explanations and treating swaps as stress tests.Source: §6 · p. 12

NLEs vs. CoT Prompting

• On several classification-style benchmarks, LLM- or human-generated NLE exemplars often outperform CoT in some model–dataset settings, while CoT is particularly effective for multi-step math reasoning.The discussion presents NLE prompting and CoT as complementary, noting that precomputed NLEs can support modular prompt design and latency-sensitive use.Source: §6 · p. 12

Impact of Model Size on Explanation Sensitivity

• Relative accuracy gains from explanation-augmented prompting are larger for smaller models, while Llama-70B varies less across setups.Llama-8B and Mistral-7B, except on gsm8k, show more pronounced gains with high-quality explanations, especially under faithful selection; both smaller models improve with well-selected explanations.Source: §6 · pp. 12–13

7 Conclusion

• On classification-style benchmarks, adding NLEs often improves accuracy over few-shot prompts without explanations, while external-LLM and available human NLEs more consistently outperform self-NLEs; gsm8k results vary more by model, source, and selection.These findings compare self-generated, externally generated, and human-written NLEs where available.Source: §7 · p. 13
• Faithfulness-based selection of self-NLEs yields small average gains, but effects vary across metrics, tasks, and models, and metric choice can produce different selections and performance outcomes.Selection can sometimes improve and sometimes reduce downstream performance.Source: §7 · p. 13
• Randomly swapped and out-of-distribution rationales generally reduce performance without collapsing accuracy, indicating partial robustness and the importance of semantic alignment.Source: §7 · p. 13
• The study concludes that NLE source, selection strategy, and faithfulness metric are important explanation-augmented ICL design choices that should be reported explicitly.Source: §7 · p. 13

Empirical and Computational Scope

• The controlled comparison uses four instruction-tuned models, six benchmarks, fixed 6-shot prompts, and zero-shot CoT, limiting power and generalizability across models, budgets, and prompting strategies.OOD rationale tests cover only LLM-NLEs, and more runs, few-shot settings, prompting variants, and Auto-CoT are left for future work because of computational cost.Source: A Limitations · pp. 17–18

Human-explanations coverage

• Human-versus-LLM explanation comparisons are dataset-specific because only ecqa and e-snli provide human rationales, requiring broader validation across datasets, domains, and annotators.The paper avoids universal claims from these results.Source: A Limitations · p. 18

Faithfulness of NLEs

• Faithfulness conclusions concern metric-based self-NLE grounding and consistency, not hidden causal reasoning; fm1 is heuristic and fm2 is binary and edit-dependent.The analysis excludes proxy or external LLM explanations, and calls for metrics tailored to self-NLE faithfulness and further study of predictive utility.Source: A Limitations · p. 18

Evaluation-Subset Variation Across Selection Settings

• Very small cross-setting performance differences may partly reflect slightly different evaluation subsets, so the paper interprets them cautiously.Selection strategies can choose different few-shot examples, changing which instances remain for evaluation.Source: A Limitations · p. 18

B Implementation Details

• Results are averaged over 5 runs across all setups.Source: B Implementation Details · p. 18
• All setups, models, and datasets use n = 6 few-shot samples.Source: B Implementation Details · p. 18
• A random seed makes few-shot example and NLE selection reproducible.Source: B Implementation Details · p. 18
• Setup 1 uses 3 Ph-CoT steps, following Bhan et al. (2024).Source: B Implementation Details · p. 18
• Ablations in Setups 4–5 use fm1 to evaluate NLE faithfulness because of high computational cost.Source: B Implementation Details · p. 18
• Models use the default API temperature unless otherwise specified.Source: B Implementation Details · p. 18
• Error-sampling was adopted because it performed better than sampling correctly classified examples, the success setting evaluated by Bhan et al. (2024).Source: B Implementation Details · p. 18

C.1 Datasets

• Experiments use six datasets as shared pools for selecting few-shot examples and evaluation, excluding selected examples from evaluation; Table 5 lists dataset sizes.This prevents any evaluated instance from appearing in its prompt while maximizing data use.Source: §C.1 · p. 19, Table 5 · p. 19

C.2 OOD NLEs Selection

• OOD LLM-NLEs for ecqa come from e-snli, while those for e-snli come from ecqa; both use 4o-mini.The OOD selection maps each dataset to rationales generated for the other dataset.Source: §C.2 · p. 19
• For snarks, OOD rationales come from boolean via 4o-mini; boolean and cj use snarks via o3-mini, while gsm8k uses boolean via 4o-mini.These mappings specify the cross-dataset rationale sources for the remaining benchmarks.Source: §C.2 · p. 19

D Random and OOD Settings

• Random-rationale tests keep selected input–label pairs fixed while swapping rationales; OOD tests use LLM-generated rationales from other datasets to probe cross-domain mismatch.The OOD setting is limited to LLM-NLEs and is contrasted with prior work on synthetic, in-domain rationale noise.Source: D Random and OOD Settings · p. 19

CoT Prompt; Final ICL (n-samples) Prompt Using Ph-CoT Rationales

• The CoT prompt requests step-by-step reasoning and an answer, while the Ph-CoT ICL prompt requests a concise three-step explanation and provides rationale–answer demonstrations.The ICL template ends with a question placeholder after the few-shot rationale and answer pairs.Source: CoT Prompt · p. 20, Final ICL (n-samples) Prompt Using… · p. 20

E.2 fm1 - Faithfulness Evaluation Prompts; Question Generation Prompt; Question Evaluation Prompt

• The fm1 faithfulness prompts generate commonsense questions from explanations and assess whether each explanation contains enough information to answer them.The question-generation instruction requests at least five questions, while the evaluation prompt requires a yes-or-no response.Source: §E.2 · p. 21, Question Generation Prompt · p. 21, Question Evaluation Prompt · p. 21

Explanation Rewriting Prompt; Rephrased Explanation Answer Test Prompt; Choice Letter Extraction Prompt; Redacted Explanation Prediction Prompt; Response Classification Prompt

• Other fm1 prompts rewrite explanations to support alternative choices, test predictions from rephrased or redacted explanations, and classify responses as choices or insufficient.A separate prompt extracts the choice letter from the rewritten response.Source: Explanation Rewriting Prompt · p. 21, Rephrased Explanation Answer Test Prompt · p. 21, Choice Letter Extraction Prompt · p. 21

E.3 fm2 - Faithfulness Evaluation Prompts; E.3.1 1. Prediction Prompt; E.3.2 2. Counterfactual Explanation; E.3.3 3. Prediction Prompt

• The fm2 counterfactual-faithfulness prompts first predict from an explanation, then minimally edit it for an alternative answer and predict again from the edited explanation.The prediction prompts allow “unknown” when the explanation does not answer the question.Source: §E.3 · p. 22, §E.3.1 · p. 22, §E.3.2 · p. 22

F.1 fm1 LExT Faithfulness Metric; F.1.1 Question Answer Generation (QAG) Sub-Metric; F.1.2 Counterfactual Stability Sub-Metric; F.1.3 Contextual Faithfulness Sub-Metric

• fm1 averages QAG answerability, counterfactual label-change, and contextual-redaction scores, then labels each dataset’s top quartile of scores faithful.QAG measures whether an explanation answers questions derived from itself; counterfactual stability scores label changes; contextual faithfulness scores insufficient responses after sequentially redacting identified keywords, with complete-redaction failure set to zero.Source: §F.1.3 · pp. 23–24, §F.1.1 · p. 23, §F.1.2 · p. 23

F.2.1 Counterfactual Explanation Metric; F.3 Comparison of Faithfulness Metrics

• fm2 uses a minimal counterfactual edit and a separate self-consistency check, producing a binary faithfulness judgment; unlike fm1’s composite score, it tests behavioral consistency through intervention.The passage describes the two metrics as complementary approaches with different output granularity and model-interaction requirements.Source: §F.2.1 · p. 24, §F.3 · p. 24

G LLM-NLEs Generation

• LLM-NLEs were generated with GPT-4o-mini at temperature 0.7 and o3-mini without temperature control, using task-specific prompts for the datasets.The passage says GPT-4o-mini’s temperature was chosen to allow some variation in phrasing or reasoning paths.Source: G LLM-NLEs Generation · p. 25

H Effect of Gold-Label Conditioning on Ph-CoT Self-Explanations

• Table 7 compares Step B gold-label-conditioned regenerations for Step A errors with Step A post-hoc self-explanations, reporting performance deltas across datasets and models.Positive deltas indicate improvement; the supplied passage does not provide the table’s cell values.Source: Table 7 · p. 27

I NLE Length Descriptive Statistics

• NLE length shows no clear relationship with utility: longer self-NLEs generally do not outperform LLM- or human-generated NLEs, while shorter types are often competitive or strongest.The passage cautions that controlled experiments are needed to draw firmer conclusions.Source: I NLE Length Descriptive Statistics · p. 27, Table 8 · p. 27

J Lower-Cost LLMs Can Provide Competitive NLE Utility

• Each of GPT-4o-mini and o3-mini yields higher mean accuracy on three of six benchmarks, so the higher-cost explainer does not consistently provide greater predictive utility.The passage suggests lower-cost generators may offer a favorable cost–utility trade-off, while noting that broader conclusions require evaluation across more models.Source: J Lower-Cost LLMs Can Provide… · p. 28

K Model Size Impact on Explanation Sensitivity

• Figure 2 compares explanation-selection strategies for Llama 8B and 70B and Mistral 7B, averaging results across datasets to examine model-size effects.The supplied caption does not report the direction or magnitude of those effects.Source: Figure 2 · p. 29

L Disagreement Between NLE Faithfulness Metrics

• Table 9 summarizes the count, rate, and direction of disagreements between faithfulness metrics fm1 and fm2 across datasets and models.Source: Table 9 · p. 30
• Figure 3 visualizes binary outcomes from two faithfulness metrics on ECQA test samples using the 4o-minimodel.Columns represent test samples; rows represent metrics; green and blue indicate True and False, respectively.Source: Figure 3 · p. 30
• Figure 4 visualizes binary outcomes from two faithfulness metrics on ECQA test samples using Llama-70B.Columns represent test samples; rows represent metrics; green and blue indicate True and False, respectively.Source: Figure 4 · p. 31
• Figure 5 visualizes binary outcomes from two faithfulness metrics on ECQA test samples using Llama-8B.Columns represent test samples; rows represent metrics; green and blue indicate True and False, respectively.Source: Figure 5 · p. 31
• Figure 6 visualizes binary outcomes from two faithfulness metrics on ECQA test samples using Mistral-7B.Columns represent test samples; rows represent metrics; green and blue indicate True and False, respectively.Source: Figure 6 · p. 31

M.1 Random Selection Results

• Figure 7 compares random-selection NLE types and baselines by dataset, reporting mean accuracy across five runs averaged over four evaluation models.It includes ZS, FS-R, CoT, Self-NLEs, available Human-NLEs, LLM-4o, and LLM-o3, with standard-deviation error bars across runs.Source: Figure 7 · p. 32
• Table 10 gives model-specific random-selection results for the error-sample selection setup, covering self-explanations and 4o-mini and o3-mini LLM explanations.Dashes mark unavailable human explanations for non-explainable datasets, and bold indicates each row's highest value.Source: Table 10 · p. 33

M.2 Self-NLEs Results Across Setups

• Table 11 reports five-run mean performance for four models on five datasets across prompting and self-NLE selection conditions, using FS-R as the statistical-test reference.FS-fm1 and FS-fm2 are matched no-rationale baselines using the same selected input-output pairs as their corresponding faithfulness-based conditions.Source: Table 11 · p. 34

M.3 Full Results Across Setups

• Tables 12–13 compare self-, human-, and LLM-generated explanations across selection setups, models, and datasets in the error-sample-selection setup.Table 12 reports five-run averages; Table 13 averages across four evaluation models and five runs.Source: Table 12 · p. 35, Table 13 · p. 35
• Table 14 reports dataset-level accuracy changes relative to random selection for most-faithful and lowest-faithful settings, with absolute accuracies in parentheses.Human-NLE rows are included only for ECQA and e-SNLI, where human rationales are available.Source: Table 14 · p. 36
• Table 15 reports standard deviations across five runs for each model, dataset, explanation source, and selection strategy.The figure captions describe aggregate and model-specific accuracy visualizations across datasets and explanation-selection strategies.Source: Table 15 · p. 36
• Figures 8–9 visualize aggregate and model-specific accuracy across baseline, random, faithful, and lowest-faithful selection strategies on the evaluated datasets.The captions distinguish self-generated, human-provided, and LLM-generated explanations, including explanations from 4o-mini and o3-mini.Source: Figure 8 · p. 37, Figure 9 · p. 39

N Examples of Faithful-Random and OOD Model Outputs

• Figures 10–13 show random-rationale prompts for Boolean or ECQA examples using 4o-mini, o3-mini, or self-generated explanations; captions call predictions correct while labeling each an error case.The examples use 4o-mini and o3-mini explanations with 4o-mini evaluation, or self-explanations with Llama-8B and Llama-70B evaluation.Source: Figure 10 · p. 40, Figure 11 · p. 41, Figure 12 · p. 42
• Figures 14–18 illustrate OOD-random prompts using explanations from a different dataset; the captions report both correct and wrong predictions.Examples use Snarks explanations for Boolean, e-SNLI explanations for ECQA, and Boolean explanations for Snarks.Source: Figure 14 · p. 44, Figure 15 · p. 45, Figure 16 · p. 46

O Examples of Different NLEs

• Figures 19–23 compare explanations for individual ECQA, e-SNLI, Snarks, Boolean, and Causal Judgment inputs across self-generated, human-written where shown, and LLM-generated setups.The captions identify differences in style, detail, and reasoning approach by model size and generation source.Source: Figure 19 · p. 49, Figure 20 · p. 50, Figure 21 · p. 51