Source-linked AI summary
Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models
Zikai Zhou, Jiacheng Sang, Yilin Xu, Yufei Jin, Monica S. Lam
TL;DR
Models should disclose whether external information changed their answers, but current frontier models often omit or falsely claim that dependence. The paper introduces CREDIT, which directly trains final-answer self-attribution using counterfactual labels; it improves attribution across four open models, while reasoning-level acknowledgment leaves final-answer performance near chance.
Problem
Method
CREDIT uses a model’s counterfactual answers to train its final attribution statement to identify information that changed the answer and say “none” otherwise.
Source: §1 · pp. 1–2Results
CREDIT raises test balanced accuracy from 0.49–0.80 to 0.85–0.90 across four open models, while reasoning-level supervision leaves final-answer accuracy at 0.50.
Source: §1 · p. 2Takeaways & Limitations
Among answers agreeing with a user’s suggestion, CREDIT distinguishes those changed by the suggestion from those the model would have given anyway.
Source: §8 · p. 9The main metric uses single-short-answer questions with about 30–95 influential test cases per condition, and the paper does not test whether users act on attribution.
Source: Limitations · p. 10
Abstract
from arXiv · showhide
A fundamental challenge for explainable and accountable language models is revealing how external information influences their answers. This paper examines whether models are self-attributive: that is, whether they indicate when -- and only when -- their responses to questions are influenced by information provided alongside the question. Furthermore, we explore how self-attribution can be improved through counterfactual training, which explicitly teaches models to distinguish between answers that do and do not depend on supplied information. We found that most of the eleven frontier models rarely attribute influential information in their answers, and the rest attribute non-influential information. Training acknowledgment into reasoning does not fix this: the acknowledgment does not reach the final answer. We introduce CREDIT, which directly supervises final-answer self-attribution with counterfactual labels, without the need for human annotation. CREDIT substantially improves attribution in four open models, including two distilled reasoning models, and outperforms reasoning-level fine-tuning and prompting baselines. Among answers that agree with a user's suggestion, CREDIT distinguishes those changed by the suggestion from those the model would have given anyway. Gains transfer partly to unseen information types and prompts containing sources that support conflicting answers.
1 Introduction
The paper asks whether models disclose when external information changes their answers, and whether counterfactual training can improve that self-attribution. CREDIT directly trains final-answer attribution; frontier models often omit influential suggestions or falsely attribute non-influential ones, while reasoning-level acknowledgment does not reach the final answer.
1 Introduction
- Self-attribution means naming external information an answer depends on, or stating that it depends on none; it distinguishes dependence from merely agreeing with a suggestion.The paper measures influence by whether the model gives the supported answer with the information but rarely without it, counting omissions and false attributions as errors.Source: §1 · p. 1
- Nine of eleven frontier models attribute influential suggestions in only 1–31% of cases, while GPT-5 mini and GPT-5.5 also falsely attribute non-influential suggestions.Their false-attribution rates are 65% and 23%, respectively.Source: §1 · pp. 1–2
- Reasoning-level supervision yields 0.74 balanced accuracy in reasoning but 0.50 in the final answer, and adding a counterfactual instruction leaves the final answer near chance.Figure 2 contrasts reasoning acknowledgment with final-answer attribution and CREDIT; chance is 0.50.Source: §1 · p. 2, Figure 2 · p. 2
Contributions (corresponding to RQ1-RQ3)
- CREDIT uses the model’s own counterfactual answers to label whether external information changed its answer, supervising only the final attribution statement.It names the information when removing it would change the answer and says “none” otherwise.Source: §1 · p. 2
- CREDIT raises test balanced accuracy from 0.49–0.80 to 0.85–0.90 across four open models, each scoring above every prompted frontier model.Among answers agreeing with a suggestion, it distinguishes suggestions that changed the answer from those that did not.Source: §1 · p. 2
- Training on four information types improves attribution on all four and partly transfers to unseen types and conflicting-source prompts, while suggestion-only training harms attribution on other types.Unseen settings include question metadata, claimed expertise, and free-form passages from an existing benchmark.Source: §1 · p. 2
2 Related Work
Related work addresses cue acknowledgment in reasoning, self-explanations, source attribution, and knowledge conflicts. CREDIT instead targets whether supplied information changed the answer, using automatic counterfactual labels to train explicit final-answer self-attribution.
Supervising Final-Answer Self-Attribution
- Verbalization fine-tuning trains cue acknowledgment in reasoning [Turpin et al., 2025], whereas CREDIT trains final-answer self-attribution with automatic counterfactual labels.CREDIT scores both omissions and false attributions and rewrites only the final attribution statement.Source: §2 · p. 3
Dependence Rather Than Evidential Support
- Unlike attributed generation, which checks whether sources support an answer [Gao et al., 2023], self-attribution also tests whether the answer depends on the information.The paper measures dependence through removal; CREDIT trains self-attribution without a symbolic layer [Zhou et al., 2026a,b].Source: §2 · p. 3
Knowledge Conflicts and Self-Knowledge
- CREDIT studies influence without claiming introspection and complements sycophancy mitigation by exposing influence rather than removing it [Wei et al., 2023].Knowledge-conflict work studies context versus parametric knowledge [Longpre et al., 2021; Xie et al., 2024], while self-knowledge work examines models’ knowledge of their correctness or behavior [Kadavath et al., 2022; Binder et al., 2025].Source: §2 · p. 3
3 Self-Attribution Problem Statement
Self-attribution means disclosing external information if and only if the answer depends on it, distinguishing borrowed answers from independent corroboration. The paper measures dependence through answer changes under information removal, then evaluates whether the final attribution statement matches that influence.
3 Self-Attribution Problem Statement
- Self-attribution identifies which external information affected the model’s answer, distinguishing answers borrowed from supplied information from those the model would give anyway.The model should attribute information if and only if it is influential.Source: §3 · p. 3
3.1 Measuring Influence of Information
- Influence is measured by removing one information piece while holding the rest fixed and checking whether the model still gives the answer that piece supports.The method samples counterfactual answers without that piece; it measures behavioral dependence, not correctness or internal reliance.Source: §3.1 · p. 3
3.2 Self-Attribution Task
- The self-attribution task produces a reasoning trace, final answer, and statement identifying which supplied information influenced the answer, or none.Users see only the answer and attribution statement.Source: §3.2 · p. 4
3.3 Quantifying the Faithfulness of Self-Attribution
- Faithful attribution requires the statement’s attribution decision to match the measured influence label; balanced accuracy equally weights influential and non-influential cases.Always attributing, never attributing, or attributing at random scores 0.50.Source: §3.3 · p. 4
4 The CREDIT Training Pipeline
CREDIT generates counterfactual influence labels and uses them to supervise attribution statements while preserving the model’s original reasoning and answer.
4 The CREDIT Training Pipeline
- CREDIT generates responses, labels information using counterfactual answers, replaces only the attribution statement with a label-matched target, then fine-tunes the model.It uses four counterfactual answers per information piece, drops unscored examples, and applies LoRA for one epoch across four information types.Source: §4 · pp. 4–5
5 Experimental Setup
The experiments test attribution across question sets and information types, using counterfactual labels and comparisons with untrained, prompted, and fine-tuned baselines. Four open models are trained, with eleven frontier models as references.
5.1 Benchmarks
- Evaluation uses TEST and FRESH sets, while LARGE adds TruthfulQA questions withheld from training; training questions come from TriviaQA, MMLU, ARC, and CommonsenseQA.No evaluation question is used in training.Source: §5.1 · p. 5
5.2 Evaluation Protocol
- TEST and FRESH questions are paired with trained and unseen information types, while the evaluation protocol uses model-specific counterfactual labels and an LLM judge checked against human annotators.The seven information types include four used in training, three unseen types, and benchmark passages; evaluation reports 95% bootstrap confidence intervals.Source: §5.1 · p. 5, §5.2 · p. 5
5.3 Models and Baselines
- Four open models are trained and eleven frontier models serve as references; baselines test prompting, reasoning-level verbalization fine-tuning, and shuffled labels.Trained baselines use the same examples and labels as CREDIT, while shuffled labels separate label effects from learning the attribution format.Source: §5.3 · pp. 5–6
6 Main Experimental Results
CREDIT improves self-attribution across the tested models and sets, outperforming prompting and training baselines. Its advantage includes distinguishing influence from mere agreement, though errors remain and training the final attribution statement is central to the gain.
6 Main Experimental Results
- 0.49 to 0.86: Qwen3-8B CREDIT balanced accuracy on TEST; CREDIT improves attribution for every model and question set, and every CREDIT model beats every frontier model on LARGE.The three-seed Qwen3-8B gain holds on FRESH and LARGE; the 14B gain on the small TEST set is borderline.Source: §6 · p. 6
- CREDIT outperforms prompting and training baselines on TEST; reasoning-level supervision and shuffled-label training leave final-answer attribution at chance.Prompting to attribute only changed answers and a stricter instruction also fall well short across all three evaluation sets.Source: §6 · p. 6
- Among answers agreeing with a suggestion, CREDIT separates influential from non-influential suggestions, unlike agreement-only attribution, which cannot reveal dependence.Every CREDIT model and question set shows this separation; the Table 2 caption reports that all CREDIT confidence intervals exclude 0.50, with a lowest bound of 0.62.Source: §6 · p. 6, Table 2 · p. 7
- CREDIT errs on about one in six responses across information types, mostly by confusing agreement with dependence.Many omissions present an influenced answer as the model’s own knowledge; both omission and false-attribution errors are more frequent when model knowledge is uncertain.Source: §6 · p. 6
6.1 What Drives the Gain in Self-Attribution
- 0.86 versus 0.74: CREDIT’s final attribution statement outperforms VFT’s reasoning acknowledgment on TEST; training only the answer and attribution performs at least as well as whole-output training (0.88).Training only the attribution statement fails because the model often stops producing its final answer.Source: §6.1 · p. 6
- CREDIT performs well on both influential and agreeing-but-non-influential cases; targeted upweighting improves the latter but lowers scores on influential cases.Across two seeds, the agreeing-but-non-influential score rises from 0.77 to 0.79–0.85 without hurting influential cases; balanced accuracy rises from 0.86 to 0.88–0.89 on pooled TEST and FRESH.Source: §6.1 · p. 7
7 Generalization Across Types of External Information
CREDIT transfers partly from its four training types to unseen information, free-form passages, and conflicting sources. The gains depend on training coverage and are weaker for some formats and sources.
7 Generalization Across Types of External Information
- On TEST, Qwen3-8B CREDIT improves all four trained and two unseen types, including question metadata (+0.24) and claimed expertise (+0.11).Table 3 reports balanced accuracy by information type as training coverage increases.Source: §7 · pp. 7–8, Table 3 · p. 8
- On FRESH prompts with two conflicting pieces, CREDIT improves all three models by +0.11 to +0.16, while adding two-piece examples helps only DeepSeek.Table 4 scores each piece across pairings and choices of which piece is wrong; CREDIT attributes in a single pass rather than rerunning the model without each piece.Source: Table 4 · p. 8, §7 · p. 8
- On FaithEval, CREDIT raises balanced accuracy from 0.66 to 0.77–0.84 across three seeds; suggestion-only training and VFT fall below the untrained model.FaithEval uses free-form passages arguing for a counterfactual answer.Source: §7 · p. 8
- When the influential piece is another AI’s answer, Qwen CREDIT models attribute it in only 8–9% of cases.The models nevertheless distinguish influential from non-influential pieces among answers they give.Source: §7 · p. 8
- Transfer remains weak for system-prompt notes and shrinks with paraphrased templates, while training on multiple types supports partial transfer.For paraphrased templates, gains are +0.04 for question metadata and +0.08 for claimed expertise.Source: §7 · p. 8
- Suggestion-only training worsens attribution to other types: tool-output balanced accuracy drops from 0.72 to 0.53, with 63 of 75 influential tool outputs labeled “none” on validation.Adding passages helps little.Source: §7 · pp. 8–9
8 Discussion and Conclusion
The discussion argues that self-attribution belongs in the final answer, where it can inform users whether supplied information shaped a response. CREDIT distinguishes dependence from agreement, with transfer depending on training coverage.
8 Discussion and Conclusion
- CREDIT directly supervises final-answer self-attribution because reasoning-level acknowledgment does not carry over to the user-facing answer.The paper distinguishes monitor-readable reasoning from an answer that informs the user.Source: §8 · p. 9
- Among answers that agree with a suggestion, attribution distinguishes those changed by it from those the model would have given anyway.This includes correct suggestions the model may already know.Source: §8 · p. 9
- Transfer is partial rather than automatic: multiple training types support generalization, whereas training on one type teaches attribution to that type and neglect of others.Source: §8 · p. 9
Limitations
The paper’s evidence is bounded by its intervention-based definition of influence, evaluated settings, and untested user effects. CREDIT makes influence visible but does not reduce it.
Limitations
- Influence is measured as answer change under intervention, so non-influential means no measured answer change, not absence of internal reliance.The main metric uses single-short-answer questions, with about 30–95 influential cases per condition on the test split.Source: Limitations · p. 10
- With two competing pieces, CREDIT reaches balanced accuracy of 0.75–0.79, or 0.82 with two-piece training for DeepSeek; closing this gap remains open.Source: Limitations · p. 10
- The evaluation does not test whether users act on attribution statements, and CREDIT makes influence visible but does not reduce it.Including system-prompt notes in training, which could distinguish a coverage gap from a base-model disposition, is also untested.Source: Limitations · p. 10
Ethical Considerations
The ethical discussion warns that attribution can create unwarranted trust and should complement, not replace, external audits. The appendices document the study setup, examples, and comparison with prior work.
Ethical Considerations
- Even more accurate attribution statements still err in both directions; in high-stakes settings, they should complement external audits rather than replace them.The data are English trivia, exam, and misconception questions without personal data.Source: Ethical Considerations · p. 10
Appendix
- The appendix covers question sets and information types, worked examples, measurement and failure analyses, additional results, prior-work comparison, and responsible-research details.Source: Appendix · p. 14
A Question Sets and External Information Types
- The appendix describes the attribution instruction and identifies the question sets and seven external-information types in Tables 5 and 6.Source: A Question Sets and External… · p. 15
B Example of Self-Attribution
- Figure 4 gives an end-to-end TEST example and CREDIT attribution statements for other information types.Source: B Example of Self-Attribution · p. 15
C Comparison with Prior Work
- Table 7 compares related work across seven properties; CREDIT uses intervention-based labels for final-answer attribution and evaluates false attributions and cross-type transfer.Source: C Comparison with Prior Work · p. 15
D Characterizing Self-Attribution Failures
Frontier models often omit influential suggestions or attribute non-influential ones, while CREDIT improves attribution on agreeing answers. The composition analysis shows that CREDIT mainly changes what Qwen models report, while reducing one type of harmful suggestion effect for DeepSeek-R1-Distill-Qwen-7B.
D Characterizing Self-Attribution Failures
- Nine of eleven frontier models attribute influential user suggestions in only 1–31% of cases; GPT-5 mini attributes them more often but also falsely attributes non-influential suggestions.A neutral system note is influential for every model, yet six of eight models attribute it whether or not it was influential.Source: D Characterizing Self-Attribution… · p. 15
- Among agreeing answers, frontier models score 0.50–0.66 versus 0.69–0.75 for CREDIT; on agreeing TruthfulQA answers, both Qwen CREDIT models score 0.71 versus at most 0.65 for frontier models.DeepSeek-R1-Distill-Qwen-7B scores 0.63, just below GPT-5.5 at 0.65 on agreeing TruthfulQA answers.Source: D Characterizing Self-Attribution… · p. 16
E Composition of the Questions
- For Qwen3-8B, influence arises when a correct suggestion fixes a wrong answer or when the model adopts a wrong suggestion; the former occurs in 77% of such cases.Most other cases are non-influential, including when the model was already right and a suggestion merely agreed.Source: E Composition of the Questions · pp. 16–17
- For Qwen models, Y→N rates change by at most one point, while the 8B N→Y rate is unchanged; the 14B N→Y rate rises from 73% to 80%.Source: E Composition of the Questions · p. 17
- For DeepSeek-R1-Distill-Qwen-7B, a wrong suggestion turns a correct answer wrong less often after CREDIT training, 51% →40%, while TEST-set accuracy is unchanged.The comparison concerns about 85 questions the model answers correctly on its own.Source: E Composition of the Questions · p. 17
- With a correct suggestion, Qwen3-8B’s answer accuracy is 0.86 after CREDIT training versus 0.87 before; VFT raises it to 0.94 while changing which suggestions are influential.CREDIT trains on the untrained model’s own answers, whereas VFT rewrites the reasoning.Source: E Composition of the Questions · p. 17
F Failure Analysis of CREDIT
CREDIT’s remaining errors include omissions and false attributions, with many false attributions occurring when the model would have answered the same way without the information. The label audit also finds judge mismatches, so its 8% noise estimate applies to sampled errors rather than all responses.
Omissions
- In 32 of 50 sampled omissions, the model adopts an answer absent from its own answers without reporting reliance; 25 follow a user suggestion, and 18 adopted answers are wrong.The model’s reasoning typically argues for the adopted answer and concludes it is correct.Source: F Failure Analysis of CREDIT · p. 19
- The untrained types account for 92 of 197 omissions: the model omits 45% of influential metadata and 72% of influential system-prompt notes, versus 17% of influential user suggestions.The 12 sampled omissions of metadata and system-prompt notes understate their share of all omissions.Source: F Failure Analysis of CREDIT · p. 19
- Omissions occur more often when the model’s own answer is unstable: 46% versus 29% when none of four counterfactual answers gives it, a difference of +0.17 [0.02, 0.33].These rates cover influential information across all scored responses.Source: F Failure Analysis of CREDIT · p. 19
False attributions
- In 48 of 50 sampled false attributions, the model would have given the answer anyway; across all responses, 139 of 142 false attributions agree with the information.Among agreeing non-influential cases, false attribution occurs in 19% (139 of 739), versus 0.4% (3 of 745) when the answer does not agree.Source: F Failure Analysis of CREDIT · p. 19
Label quality
- Eight of 100 audited influence labels were judged questionable, mainly because the answer judge failed to match paraphrases or different granularities; 8% estimates noise among errors, not all items.Excluding one truncated TEST question changes balanced accuracy by at most 0.014.Source: F Failure Analysis of CREDIT · p. 19
G Training Details
CREDIT labels model-specific training examples using counterfactual answers, then trains LoRA adapters on the completion. The section also reports a near-chance reinforcement-learning variant and notes that DeepSeek’s TEST results are not fully independent of model selection.
Training data
- CREDIT keeps all influential training examples and samples up to two non-influential examples per influential example within each information type.Each model is labeled using its own counterfactual answers, so the number of influential examples varies by model.Source: G Training Details · p. 20
Optimization
- Training uses rank-64 LoRA adapters for one epoch at learning rate 10^-4, with loss computed only on completion tokens.Adapters cover attention and MLP projections; the three Qwen3-8B runs use seeds 42, 1, and 2, while other models use seed 42.Source: G Training Details · p. 20
Training data
- Training labels cost four counterfactual answers per TRAINING question, a cost paid once; checking dependence at deployment would require four extra generations per answer.When a question has one external-information piece, the same four answers serve all information types and sampled responses for that question.Source: G Training Details · p. 21
Model selection
- DeepSeek-R1-Distill-Qwen-7B’s TEST results are not fully independent because its untrained-model screen used the TEST set; its FRESH- and LARGE-set results are cleaner estimates.The screen was conducted after CREDIT was frozen, but still used TEST questions.Source: G Training Details · p. 21
Faithfulness-RL variant
- The Faithfulness-RL variant remains near chance on TEST, with balanced accuracy 0.51 and attribution in 2% of influential cases; its reward rarely observes attribution because about 79% of rollouts are non-influential.A policy that never attributes already earns most of the reward, and the model can also satisfy the reward without adopting the suggestion.Source: G Training Details · p. 21
H Frontier Model Evaluation Settings
This appendix evaluates the reliability, transfer, and limits of CREDIT’s self-attribution results across frontier-model settings, including competing-information prompts. It also documents persistent attribution gaps, system-prompt notes as a failure case, and the paper’s contrast with counterfactual re-runs and its research artifacts and practices.
H Frontier Model Evaluation Settings
- Eleven frontier models were evaluated through provider APIs using the open models’ prompts and counterfactual-label procedure.Each received one response per correct and wrong suggestion and four independent counterfactual answers; provider-default decoding was used, with up to 8,000 output tokens per call.Source: H Frontier Model Evaluation Settings · p. 21
I Measurement Checks
- Influence prevalence was similar across models (29–33%), and the reported rankings and gains were robust to label, judge, and pipeline checks.Restricting to items labeled by every model or agreed on by all models did not change rankings or gaps; threshold changes preserved CREDIT’s gains, and re-judging shortened questions changed balanced accuracy by at most 0.003.Source: I Measurement Checks · pp. 22–23
J Paraphrased Templates / K.6 Transfer and System-Prompt Notes
- Template-B transfer gains persisted for trained information types, while system-prompt notes remained difficult to attribute across templates and sets.On TEST, Qwen3-8B gains were +0.21 for tool output and +0.15 for another AI’s answer; unseen-type gains were smaller. On FRESH, CREDIT attributed only 34–37% of influential system-prompt notes, and a neutral-note test found 2% attribution despite following wrong notes in 1% of cases.Source: J Paraphrased Templates · p. 24, §K.6 · pp. 26–27
K.1 What Drives the Gain
- +0.12 balanced accuracy on TEST favored training the final-answer attribution over reasoning-level supervision.VFT reasoning acknowledged influential suggestions, but the final line remained at chance even when it was excluded from the loss; restricting loss to the answer section reached 0.88, while attribution-statement-only loss often stopped final answers.Source: §K.1 · p. 24
K.2 Agreeing Answers
- For correct suggestions, CREDIT reached 0.72–0.83 balanced accuracy versus 0.53–0.58 for answer matching on the Qwen models.These CREDIT gains were +0.15 to +0.27, with all 95% intervals above zero; for agreeing answers, the correct-suggestion stratum is especially informative because the model may have known the answer already.Source: §K.2 · p. 24, Table 17 · p. 25
K.3 Asking the Model Whether It Knows / K.4 Prompting Baselines
- Across FRESH and LARGE, P(IK) trailed CREDIT by 0.11–0.17; CREDIT exceeded stricter prompting by 0.13–0.33 across models and question sets.P(IK) was ahead only for DeepSeek on the small TEST set (0.77 vs. 0.74); stricter wording changed attribution frequency more than correctness, and Qwen3-14B often over-attributed suggestions.Source: §K.3 · p. 25, §K.4 · p. 25, Table 19 · p. 26
K.5 A Llama-Based Model
- 0.62 to 0.85 balanced accuracy on TEST showed CREDIT transferred to DeepSeek-R1-Distill-Llama-8B, with gains also on FRESH and LARGE.CREDIT reached 0.79 on both FRESH and LARGE; among agreeing answers, it scored 0.77, 0.67, and 0.71 on TEST, FRESH, and LARGE, respectively.Source: §K.5 · p. 26
K.7 Benchmark Passages
- On FaithEval, CREDIT reduced false passage attributions while retaining 86–92% attribution when passages were influential; RAMDocs had too few influential cases for balanced accuracy.The untrained model attributed to passages in 98% of influential and 65% of non-influential cases; trained models attributed in 21–38% of non-influential cases. RAMDocs contained only 5–8 influential cases.Source: §K.7 · p. 27
K.8 Competing External Information
- On the TEST and FRESH sets, CREDIT scored 0.73/0.77/0.72 and 0.70/0.74/0.69, versus untrained 0.50/0.45/0.57 and 0.49/0.48/0.56, when answer matching was at chance.For 8B, 14B, and DeepSeek, the source-aware matching rule scored 0.84/0.82/0.97 on TEST and 0.84/0.82/0.96 on FRESH, matching or exceeding CREDIT’s 0.78/0.82/0.78 and 0.75/0.79/0.77.Source: §K.8 · p. 27
- With three competing pieces, 8B CREDIT improved from 0.74 to 0.86 on FRESH and 0.78 to 0.85 on TEST, while 14B gains were small on FRESH.The 8B improvement mainly reflected fewer false attributions (12% to 4%); at 14B, TEST rose from 0.77 to 0.84, while FRESH moved from 0.78 to 0.79.Source: §K.8 · p. 27
- Two-piece training improved DeepSeek-R1-7B accuracy from 0.77 to 0.82 but did not raise Qwen accuracy; Qwen CREDIT models named another AI’s influential answer in only 8–9% of cases.Training shifted attribution rates by information type and kept false attributions at 3%, while 8B still omitted about half of influential pieces; it also improved single-piece TEST attribution for suggestions from 0.86 to 0.89 and another AI’s answers from 0.81 to 0.88.Source: §K.8 · p. 28
K.9 Why Not Re-Run the Model?
- Counterfactual re-running is an external audit, not a model-provided account, and requires extra generations while assuming each information item can be removed without changing the task.A suggestion-free sample recovers four-sample labels at about 0.97 balanced accuracy, an optimistic estimate; two-piece attribution required eight extra generations per answer, whereas CREDIT moves this cost to training and enables single-pass attribution.Source: §K.9 · p. 28
L Responsible Research Details
- The authors report releasing code, data, model outputs, adapters, and guidelines, and state that experiments used research-purpose data and providers’ APIs under their terms.They report collecting no data from people other than author annotations and using AI assistants for coding, experiments, analysis, and writing, while taking responsibility for verifying the work.Source: L Responsible Research Details · pp. 28–29
Related papers
This paper cites
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge2018
- TruthfulQA: Measuring How Models Mimic Human Falsehoods2021
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting2023
- Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?2020
- Accountable AI with Grounded, Faithful, Consistent, Actionable Rationales: A Case Study in Clinical Trial Matching with VERDICT2026