A Survey of Large Language Models Attribution

• Attribution lets developers and users inspect possible answer sources and assess factuality and reliability, avoiding direct truthfulness judgments that are difficult beyond basic queries.Source: Figure 1 · p. 1
• Pre-training on unfiltered texts and next-word prediction can leave models reproducing falsehoods, and reinforcement learning from human feedback does not eliminate external hallucinations.Source: §1 · p. 1
• Comprehensive attribution is framed as high recall: every generated claim should be backed by appropriate sources except debatable or subjective text.Source: §1 · p. 1

• Attribution means providing evidence from identifiable sources to substantiate generated claims, enabling users to validate them and assess support.Source: §2 · p. 3
• The formal task takes a query and corpus and produces distinct statements, each associated with citations drawn from that corpus.Source: §2 · p. 3
• For practical use, model outputs can be segmented at sentence boundaries, while citations may attach to phrases rather than only whole sentences.Source: §2 · p. 3
• The task differs from scholarly or Wikipedia citation-text generation, where citing and cited documents are usually inputs.Source: §2 · p. 3

3.1 Pre-training Data

• Pre-training corpora are extensive and difficult to inspect manually, leaving potential inaccuracies and biases in data and uncertainty about memorization versus reasoning.Source: §3.1 · p. 3
• Pre-training-data attribution traces a model's behavior on a task to portions of its training data, helping researchers examine whether it captures relevant patterns or memorizes data.The section also connects this tracing to understanding model behavior and trustworthiness.Source: §3.1 · pp. 3–4

• The paper distinguishes direct model-driven attribution, citation generation based on retrieved documents, and attribution applied after an answer is generated.Source: Figure 3 · p. 4

3.2 Out-of-model Knowledge

• Models can use out-of-model knowledge such as web sources or knowledge graphs; retrieval-augmented generation adds retrieved documents or passages to question-answer generation.Other described methods fetch similar training contexts or search for relevant external documents.Source: §3.2 · p. 4

• Recent datasets measure or improve citation accuracy using supplied quotes, varying by question domain and citation granularity.Some earlier long-form QA datasets were designed for factual accuracy rather than direct citation verification.Source: §4 · p. 4
• Most datasets are open-domain, while ExpertQA covers 32 domain-specific scenarios and includes domain experts.Source: §4 · p. 4
• Attribution granularity is commonly entity-level or sentence-level; ExpertQA and BioKaLMA use entity-level attribution, while other methods use sentence-level attribution.Source: §4 · p. 5

5.1 Direct Generated Attribution

• Self-detection and self-attribution can reduce hallucination, improve truthfulness, and improve downstream-task performance, while models may still give weak or mistaken evidence for domain-specific answers.Weller et al. (2023) reports that according-to prompting can affect groundedness and information-seeking performance; Anonymous (2023) describes blueprint questions as a basis for attribution at each answering step.Source: §5.1 · p. 5

5.2 Post-retrieval Answering

• Post-retrieval methods align questions with knowledge, check document support, add reflection tokens, and verify or repair gaps in reasoning chains.Examples include MixAlign, LLatrieval (Li et al., 2023a), Self-RAG (Asai et al., 2023), and SearChain.Source: §5.2 · p. 5

5.3 Post-Generation Attribution

• Post-generation attribution retrieves evidence after an answer, then uses it for citation, verification, and editing; RARR (Gao et al., 2023a) revises unsupported content while preserving the original where feasible.Huo et al. (2023) use retrieved material to check consistency and make edits; Chen et al. (2023b) retrieve web evidence, create claim-focused summaries, and verify complex political claims.Source: §5.3 · pp. 5–6, Figure 4 · p. 6

5.4 Attribution Systems

• Attribution systems use external evidence in different ways, including browsing and displaying references, while citation alone cannot guarantee safety because evidence-backed claims can still be false.Table 2 presents features of different systems; examples include WebGPT (Nakano et al., 2021) gathering references while browsing and GopherCite (Menick et al., 2022) abstaining when uncertain.Source: §5.4 · p. 6, Table 2 · p. 7

• Human evaluation is costly and time-intensive, with annotation of a single query-answer-reference example costing around $1.In practice, users also bear responsibility for checking attributions manually.Source: §6 · p. 7
• Evaluation categories distinguish full, partial, or absent support, with a refined scheme separating attributable, extrapolatory, and contradictory statements.Earlier work also repurposed binary categorization, including natural language inference, to judge whether references support answers.Source: §6 · p. 7
• Attribution quality is assessed through correctness, precision, and recall, with F1 derived from precision and recall; future work is expected to refine automated methods for practical reliability.The QUIP-Score evaluates how much of a section consists of exact spans in a corpus.Source: §6 · p. 7

7.1 Attribution Error Analysis

• Three attribution errors are identified: insufficient citation from granularity mismatch, mistaken entity-event relationships, and hallucinated answers from irrelevant evidence or conflicts with model knowledge.The authors acknowledge that other error types may also exist.Source: §7.1 · p. 7

7.2 Limitations of Attributions

• Attribution remains difficult when distinguishing general from specialized knowledge, and LLMs cannot attribute their own parameter knowledge.The general-versus-specialized boundary can lead to inconsistent attribution.Source: §7.2 · p. 7
• Attributions may cite irrelevant or incorrect sources and become outdated, particularly in rapidly changing fields such as computer science and clinical medicine.Misattribution can confuse users and reduce information reliability.Source: §7.2 · p. 8

7.3 Challenges for Attributions

• Over-attribution may overwhelm users and expose private information, while source or information biases may favor some views over others.The authors recommend varied training data and improved attribution methods to address bias.Source: §7.3 · p. 8
• Without solid validation, LLMs could spread misleading details; future models should recognize ambiguous references and withhold claims when evidence is unclear.The discussion describes reliable credit as important to LLMs’ growth, approval, and effectiveness.Source: §7.3 · p. 8

7.4 Future Directions for Attributions

• Future directions include refreshing model information, evaluating source credibility and accuracy, and balancing creative generation with proper credit.The proposed refresh would keep information current, while source evaluation targets output reliability.Source: §7.4 · p. 8

• Citing sources can enhance language models’ utility, but their limitations require careful consideration.Source: Limitation · p. 9
• The paper does not provide a solution for navigating the complex limitations of attribution.Source: Limitation · p. 9
• The authors call for future work on these limitations and continued user education about relying solely on machine-generated text.Source: Limitation · p. 9

• Open-domain QA targets answer accuracy and relevance, whereas attribution checks whether generated text can be traced to retrieved documents (Chen et al., 2017; Bohnet et al., 2022).Source: §A.1 · p. 14
• Fact-checking emphasizes claim truth, while attribution checks whether generated text reflects its sources (Thorne et al., 2018; Guo et al., 2022).Both tasks rely on retrieved evidence for verification (Wang and Shu, 2023).Source: §A.1 · p. 14
• Natural language inference evaluates entailment, contradiction, or neutrality; summarization attribution instead checks fidelity to source text and helps avert unsupported summaries (Bowman et al., 2015; Ji et al., 2023).Source: §A.1 · p. 14
• Across these tasks, retrieval and comparison with reference material support evaluation of accuracy, relevance, and attribution.Source: §A.1 · p. 14

A.2 Interpretability of NLP Models

• Explicit attribution traces generated information or decisions to source material, supporting source checks and accountability in critical domains.The paper contrasts this with feature attribution, which identifies input words or phrases important to a model’s decision or output.Source: §A.2 · p. 14
• Attribution is a component of interpretability, which more broadly examines model behavior, decision-making, and input-feature significance.Interpretability aims to make the model’s inner workings transparent and comprehensible to users.Source: §A.2 · pp. 14–15
• Attribution evaluation measures whether an answer is verifiable from its reference, not whether it correctly responds to the query.The paper relates this distinction to faithfulness versus plausibility evaluation.Source: §A.2 · p. 15