PaperlayerView exact revision on arXiv

Source-linked AI summary

Measuring Attribution in Natural Language Generation Models

Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, David Reitter

arXiv:2112.12870v2cs.CL

TL;DR

NLG models often produce statements that are not faithful to underlying sources, while existing evaluations lack a unified framework for measuring attribution. This paper formalizes Attributable to Identified Sources (AIS) and validates it through human evaluations, finding moderate-to-high inter-annotator agreement and expected differences in model AIS scores across tasks.

  • Problem

    Existing NLG evaluations lack a unified, formally expressed framework for measuring whether generated statements are supported by underlying sources.

  • Method

    The paper formalizes Attributable to Identified Sources (AIS) and operationalizes it through a two-stage, human-annotation evaluation pipeline across multiple generation tasks.

  • Results

    Human evaluations achieved moderate-to-high inter-annotator agreement and showed generally expected differences in models’ AIS scores across tasks.

  • Takeaways & Limitations

    AIS provides a common framework for evaluating attribution to underlying sources as a necessary but not sufficient step in assessing generated-text quality.

  • Takeaways & Limitations

    AIS applies only when a system sentence has an explicature that is a standalone proposition, leaving questions and similar cases for future work.

Abstract

from arXiv · show

With recent improvements in natural language generation (NLG) models for various applications, it has become imperative to have the means to identify and evaluate whether NLG output is only sharing verifiable information about the external world. In this work, we present a new evaluation framework entitled Attributable to Identified Sources (AIS) for assessing the output of natural language generation models, when such output pertains to the external world. We first define AIS and introduce a two-stage annotation pipeline for allowing annotators to appropriately evaluate model output according to AIS guidelines. We empirically validate this approach on generation datasets spanning three tasks (two conversational QA datasets, a summarization dataset, and a table-to-text dataset) via human evaluation studies that suggest that AIS could serve as a common framework for measuring whether model-generated statements are supported by underlying sources. We release guidelines for the human evaluation studies.

1. Introduction

The paper introduces Attributable to Identified Sources (AIS), a unified framework for evaluating whether generated statements are supported by underlying sources across NLG tasks. Human studies indicate that AIS evaluations can achieve moderate-to-high inter-annotator agreement and distinguish model outputs’ attribution scores.

  • Motivation and contribution: AIS provides a uniform framework for measuring whether natural-language statements are derivable from identified underlying sources, addressing hallucination and faithfulness concerns.Prior NLG work examined these issues across summarization, dialogue, and data-to-text but lacked a formally expressed common measurement framework.
  • AIS framework: The framework formalizes sentence meaning through explicatures and tests attribution by asking whether “According to P, s”.AIS also accommodates uninterpretable outputs and can complement metrics for source quality, fluency, and other aspects outside this work’s scope.
  • Empirical validation: Human evaluation experiments trained annotators to apply task-specific AIS guidelines to outputs from multiple models and found moderate-to-high inter-annotator agreement.The work also releases detailed guidelines for human evaluation.
  • Empirical validation: The human studies observed differences in models’ AIS scores that followed generally expected trends, supporting AIS as a framework for evaluating system-generated utterances across NLG tasks.Accurate evaluation is important because large pretrained NLG models often hallucinate and may lack faithfulness to underlying sources.

2. Background

Prior work studies hallucination and source-groundedness across data-to-text, summarization, and dialogue, but evaluation practices remain fragmented. AIS addresses this gap by formalizing replicable attribution judgments using explicatures while distinguishing attribution from factuality and related verification tasks.

  • Hallucination and groundedness: Hallucination research spans data-to-text, summarization, and dialogue, using automatic metrics, human studies, and task-specific methods to assess faithfulness or groundedness.Wiseman, Shieber, and Rush (2017) and Parikh et al. (2020) studied data-to-text; summarization work includes Maynez et al. (2020), Wang, Cho, and Lewis (2020), Nan et al. (2021), and Durmus, He, and Diab (2020); dialogue work examines conversational consistency and external grounding (Welleck et al. 2019; Mehri and Eskenazi 2020; Gupta et al. 2021; Santhanam et al. 2021).
  • Research gap: Existing studies lack unified evaluation criteria, consistent terminology, and detailed reproducible annotation instructions for determining whether generated statements are supported by sources.General-purpose benchmarking has advanced, but attribution has not received standardized treatment across NLG tasks.
  • AIS framework: AIS formalizes attribution as a replicable, extendable framework across NLG tasks and uses explicatures to define the information conveyed by generated text.The framework is designed to accommodate different forms of context, source documents, and generated text.
  • Fact verification: Fact verification retrieves evidence and may classify claim–evidence relations, whereas AIS evaluates generated utterances against given sources without making absolute factuality judgments.The paper treats source quality as complementary to AIS and assumes the source is a reference whose trustworthiness may be selected by a system.
  • Decontextualization: Decontextualization is related because it rewrites contextual sentences for interpretation outside context, connecting directly to AIS’s use of explicatures.Choi et al. (2021) introduce decontextualization as preserving sentence meaning while enabling out-of-context interpretation.

3. A Formal Definition of Attributable to Identified Sources

This section formalizes AIS from standalone propositions to context-dependent utterances, using explicatures to preserve meaning while requiring attribution to identified source parts. It also specifies practical annotation scope, including interpretability limits, granularity choices, and human-judgment constraints.

  • 3.1 An Initial Definition of AIS: Attribution of Standalone Propositions: AIS for a standalone proposition requires source parts P, interpretability at time t, and a generic hearer’s confident affirmation of “According to P, s.”Standalone propositions are declarative sentences interpretable once time is specified; P may reference corpus passages or knowledge-graph parts.
  • 3.2 Extending AIS: Attribution of Sentences in Context: The framework extends AIS from standalone propositions to utterances comprising multiple sentences, questions, commands, exclamations, or fragments in linguistic context.Interactions are ordered sequences of user and system utterances, covering conversational QA, summarization, and table-to-text settings.
  • 3.3 Explicatures: Explicatures convert context-dependent or fragmentary sentences into context-independent paraphrases that preserve truth-conditional meaning for AIS evaluation.For example, “he was 25 years old” can be explicated as a proposition about George Harrison and his album’s release.
  • 3.3.1 The Full Definition of AIS: The full AIS definition requires an interpretable sentence, a standalone-proposition explicature, and attribution of that explicature to identified source parts at time t.AIS therefore applies attribution to the sentence’s explicature rather than directly to its context-dependent surface form.
  • 3.3.1 The Full Definition of AIS: AIS excludes sentences whose explicatures are uninterpretable or not standalone propositions, leaving cases such as question explicatures for future work.Disfluencies, ambiguous references, and unclear meanings can produce NULL explicatures or otherwise prevent the full AIS conditions from holding.
  • 3.3.2 Attribution of Entire Utterances: AIS may be assessed at sentence or entire-utterance level: sentence-level judgments are finer-grained, whereas single-shot utterance evaluation simplifies annotation but is coarser.Utterance-level evaluation requires extending explicatures, standalone propositions, and attribution to multi-sentence utterances.
  • 3.4 Towards Operationalization of AIS: Operationalization translates the formal framework into intuitive annotation procedures centered on the “according to” test, interpretability, and context-sensitive paraphrase.Annotators are instructed to apply the attribution test strictly without assumptions beyond the text.
  • 3.4 Towards Operationalization of AIS: Practical AIS judgments relax the idealized generic hearer because annotators’ cultural backgrounds and domain expertise can affect interpretation.Such variation is inherent when AIS is implemented through human judgments.

4. Human Evaluation Study

The study evaluates AIS judgments across conversational QA, summarization, and table-to-text generation using a two-stage process that separates interpretability from source attribution. Human judgments use majority consensus, with flags excluding malformed or otherwise unsuitable cases and limitations separating attribution from broader output quality.

  • 4. Human Evaluation Study: The evaluation tests human AIS assessment across conversational QA, summarization, and table-to-text outputs from different models.The study presents evaluators with model outputs for each task to quantify human judgments.
  • 4.1 Task Design: The two-stage task first assesses whether output is interpretable without the source, then evaluates whether all shared information is attributable to that source.Scores report the percentage of all examples judged interpretable and the percentage of interpretable examples judged AIS; flagged cases are reported separately and excluded.
  • 4.1.1 Interpretability Rating.: Interpretability judgments target ambiguity from unclear anaphora, syntactic ambiguity, or disfluency, so the source is withheld to enforce strict assessment.Annotators receive conversational context for conversational QA, while context is empty for summarization and table-to-text; authors note that source visibility can make judgments more forgiving.
  • 4.1.2 AIS Rating.: For AIS ratings, annotators inspect the source and mark output attributable only when every stated or context-derived part is directly supported, including paraphrases and details whose wording misleads.Sources comprise retrieved documents, original news articles, or tables with highlighted cells and metadata, depending on the task.
  • 4.1.3 Flag Rating.: Flag ratings remove cases too malformed or underspecified for judgment, including interface failures, corrupted text, ambiguous sources, or sources requiring expert knowledge.A flagged item leaves the flagging annotator’s queue, while other annotators’ non-flagged ratings remain available for aggregation and agreement calculations.
  • 4.1.4 Limitations.: The framework uses binary whole-output judgments, treating any non-AIS span as making the entire output non-AIS while excluding relevance and other quality dimensions.The authors note that finer-grained AIS measures may be useful for some applications.
  • 4.2 Human Evaluation Procedure: Nine trained annotators evaluated task-specific interfaces, with pilot studies, written justifications, author feedback, edge-case adjudication, expanded guidelines, and internal audits supporting procedure refinement.The interfaces hide sources during interpretability judgments and include task-specific modifications such as conversational history for QA.

5. Experiments

Across conversational QA, summarization, and table-to-text generation, human studies show that AIS distinguishes how model outputs use evidence and can be annotated with generally consistent quality. Results also reveal task-dependent annotation difficulty and limitations in existing gold references and outputs.

  • 5. Experiments: AIS was instantiated for four datasets spanning conversational QA, summarization, and table-to-text generation, using varied models and human evaluations of outputs and gold references.The experiments also analyze annotation reliability, interpretability, and recurring AIS patterns.
  • 5.1 QReCC Answer Generation: Fine-tuning increased QReCC interpretability by about 50 points, while evidence-free generation was interpretable and AIS only 15% of the time.The fine-tuned model using document evidence achieved the highest AIS subscore; model-size differences were generally nonsignificant.
  • 5.2 WoW Answer Generation: The Wizard of Wikipedia model trained with controllable faithfulness inputs achieved the highest AIS scores overall, while gold references scored lower than model outputs.More Wizard of Wikipedia examples were flagged than QReCC examples.
  • 5.3 CNN/DM Summarization: More extractive CNN/DM summarization systems generally achieved higher AIS subscores, consistent with copying source text and reducing hallucination risk.AIS evaluates the full summary because individually attributable spans can become unsupported when composed incorrectly.
  • 5.4 Table-to-Text ToTTo data: T5-base produced the most interpretable ToTTo responses, whereas ByT5 responses were significantly less interpretable; AIS differences across architectures were generally nonsignificant.Larger architectures tended toward slightly lower AIS scores, and T5 outputs were more often flagged for encoding artefacts.
  • 5.5 Annotation Quality: Crowd annotations showed moderate-to-high interannotator agreement, with generally high AIS F1 scores, while summarization had the lowest alpha scores because longer outputs increase rating complexity.Annotation quality audits were in the high nineties across tasks; summarization quality increased with experience, while table-to-text quality was generally lowest.
  • 5.5.3 Crowd Annotator Performance.: Completion times decreased as annotators gained experience and stopped writing required justifications, but summarization remained longest and conversational QA shortest, paralleling agreement differences.Experts and crowds generally agreed on AIS, though experts were more critical under ambiguity; disagreements often involved vague summaries or inconsistent entity naming.
  • 5.5.5 Limitations: Gold-reference AIS quality was limited: Wizard of Wikipedia references were AIS 16% of the time and CNN/DM summaries 54% of interpretable examples.Examples show that ambiguity, missing entity-identifying information, prior-turn grounding, and unsupported external details can separate AIS from general relevance or coherence.

6. Discussion

The discussion frames AIS as a necessary but insufficient source-verification step whose judgments can be crowdsourced with training and interface support, while highlighting noisy ground truth, scope limits, and unresolved contextual challenges.

  • Implications: AIS supports crowd-worker judgments of attribution to underlying sources with appropriate training, instructions, and interfaces, but judgments remain noisy and policy-dependent.Raters need standards for identifying entities and inevitably use world knowledge; AIS does not determine how much generated text should be synthesized versus extractive.
  • Limitations: AIS applies only to propositions assessable through an “according to” framework, excluding unconstrained questions and imperatives.Its treatment of hybrid systems combining informative and entertaining goals, or outputs with unclear attribution boundaries, remains unresolved.
  • Future work: AIS deliberately limits context, although richer channels and multi-session histories may affect attribution judgments and require annotators to resolve explicit references and implicit topics.Close reading is difficult on crowdsourcing platforms, motivating future use of models to extract elementary propositions and improve rater performance.

7. Conclusion

The paper introduces Attributable to Identified Sources (AIS), an evaluation framework for assessing whether generated information is supported by source documents, and validates it through human studies across three NLG tasks.

  • Conclusion: AIS provides formal definitions and task-specific application guidance for conversational QA, summarization, and table-to-text generation.The framework evaluates whether information in generated text can be supported by source documents.
  • Conclusion: Human evaluations using a two-stage annotation pipeline show that AIS ratings can be obtained with high empirical quality.Annotators rated model outputs according to AIS in quantitative studies spanning the three NLG tasks.
  • Conclusion: The results position solid AIS as a basis for addressing ongoing challenges in training NLG models.The studies connect source-supported generation evaluation with broader training challenges.

Appendix A: Examples of Annotations … 1. Interpretability

This appendix provides examples and operational guidance for annotation, covering how to judge interpretability and attribution, flag tasks, and evaluate summaries as faithful representations of source articles. It describes a sequential process in which interpretability is assessed from the summary alone before attribution.

  • Appendix A: Examples of Annotations: The annotation examples distinguish responses that are uninterpretable, interpretable but not AIS, and interpretable and AIS.These categories are illustrated in Tables 1–3 and a table-to-text annotation figure.
  • 1. Interpretability: Interpretability is assessed first by asking whether all response information is understandable, with vague meanings, unclear pronouns, and malformed sentences constituting failures.Responses judged interpretable proceed to attribution assessment.
  • 2. Attribution: Attribution asks whether all information in a system response is fully supported by the cited source document and accurately represents it.Any unsupported, partially supported, or opinion-based content should receive a negative attribution rating.
  • A. Definition of “the Information Provided by the System Response”: Annotators determine the response’s information using the query and conversation context while ignoring the source document, then assess whether that contextualized meaning is accurately represented in the source.Context can resolve abbreviated answers and pronouns, but genuinely unclear responses should be marked uninterpretable.
  • Some Final Important Notes.: Attribution judgments should ignore source correctness and response relevance, focusing only on whether the response accurately represents information in the source document.The guidance frames accurate representation similarly to journalistic evaluation of whether a source would accept the interpretation.
  • Flag: Tasks should be flagged for missing components, severely malformed text, underspecified sources, or documents requiring expertise beyond the annotator’s ability to evaluate.Examples include unclear source–query correspondence and scientific material requiring specialized knowledge.
  • Overview: Annotators evaluate summary quality through two successive categories, Interpretability and Attribution, with the interface controlling the task flow and skipped categories.
  • 1. Interpretability: A summary is interpretable only when all information is understandable on its own, without vague references or malformed, difficult-to-understand sentences.Annotators view the system summary while the source news article is hidden, answer whether all information is interpretable, and justify a “No” rating when prompted.

Interpreting the information provided in the system summary. … Interpretability

The evaluation distinguishes interpretability—whether a summary is understandable and stands on its own—from attribution—whether all summarized information is accurately supported by the source article. Annotators are instructed to judge these properties using explicit checks for missing references, malformed language, unsupported information, and source quality.

  • Interpreting the information provided in the system summary.: A summary is interpretable when its meaning is clear without the original article; missing context, unclear references, or malformed language warrant an “unclear” rating.Annotators should err toward “No, the summary is unclear” when the intended meaning is uncertain.
  • Interpreting the information provided in the system summary.: Interpretability problems include noun phrases needing context, pronouns without clear antecedents, and malformed or poorly connected sentences.Examples include unclear references to “the project,” “the museum,” “the city,” or “it,” as well as missing objects and ambiguous uses of “both.”
  • Some Final Important Notes.: Attribution examples require separating source support from interpretability: a summary may contain understandable content yet still need evaluation for whether every statement is supported by the article.The provided guidance treats attribution as a separate category and supplies examples of ratings and justifications.
  • 2. Attribution: A summary is fully attributable only when all its information is an accurate representation that can be verified in the source news article.The attribution judgment concerns support by the source, not whether the source article itself is correct or trustworthy.
  • 2. Attribution: Annotators should check every claim against the article and mark “No, not fully attributable” if any information is missing from or inaccurately represented relative to the source.The recommended test is whether it is accurate to say “the provided news article says...” or “according to the news article...” before the summary.
  • 2. Attribution: The attribution standard is close to a journalist’s notion of accurate representation, including whether the source author would agree with the summary’s interpretation.Annotators are shown the generated summary together with the source article and may provide a succinct justification for their rating.
  • Interpretability: The rating “No, the summary is unclear” denotes a summary that is unclear and/or difficult to understand.Examples identify missing noun-phrase references, vague pronouns, omitted objects, and poorly connected clauses as reasons for this rating.

Attribution … 1. Interpretability

The evaluation framework assesses table-to-text captions first for standalone interpretability and then for attribution to the source table, while providing procedures for instructions, flagging, and attribution judgments. Captions should be carefully inspected because fluent wording may conceal inaccuracies.

  • Attribution: Attribution is rated as fully attributable when all information in the system summary is supported by the document.The justification states that the summary accurately represents information in the source news article.
  • Attribution: A summary is not fully attributable when any information lacks support in the source article, including partially supported summaries.The example identifies “at stade geoffroy-guichard” and calling the opposing team “the sarries” as unattributed information.
  • Flag: Tasks should be flagged when required components are missing, text is severely malformed, or evaluating the source requires expertise the annotator does not have.Annotators may rate understandable expert documents but should flag cases where expertise is necessary for proper evaluation.
  • Appendix D: Evaluation Instructions for Table-to-Text: The table-to-text instructions evaluate system-generated captions for highlighted table regions according to how well they represent source-table information.The instructions were developed after conversational QA and summarization instructions, with annotator training preceding this task.
  • Overview: The interface evaluates Interpretability and Attribution successively, while warning annotators that fluent captions may contain subtle inaccuracies requiring careful proofreading.Some ratings can cause later categories to be skipped, and the interface guides annotators through the task flow.
  • 1. Interpretability: Interpretability is assessed with the table hidden, asking whether all information in the system-generated caption can be understood on its own.Annotators mark “No” when any part is unclear or difficult to interpret and may provide a succinct justification.
  • 1. Interpretability: An uninterpretable caption has vague or ambiguous meaning, insufficient context, or malformed phrases and sentences that are difficult to understand.If the caption is interpretable, annotators proceed to the attribution category.

Interpreting the information provided in the system caption. … Attribution

The framework separates caption interpretability from attribution: annotators first judge whether captions are understandable, then verify whether all stated information accurately represents the source table and its descriptions. Ratings require checking every claim, while source quality and highlighted-cell coverage do not determine attribution.

  • Interpreting the information provided in the system caption.: Interpretability requires captions to state their subject clearly and stand alone; missing context, unclear references, malformed syntax, or missing nouns warrant “No, the caption is unclear.”Annotators should use their best judgment and err toward the unclear rating when intended meaning is uncertain.
  • 2. Attribution: Attribution is “fully supported” only when every piece of caption information is an accurate, verifiable representation of the source table and its descriptions.Any missing, unattested, or inaccurate information requires a “No, not fully attributable” rating.
  • 2. Attribution: Annotators assess attribution using the table, title, section title, table-section text, and highlighted cells, answering whether the complete caption is supported by these sources.Highlighted cells guide attention but are not the only permissible evidence.
  • Assessing the accuracy of the information in the caption against the source table.: Accuracy judgments ask whether it is appropriate to say “according to the table” before the caption and whether the source author would view it as an accurate representation.The guidance treats accuracy similarly to journalistic paraphrase, requiring review of all caption information.
  • Some Final Important Notes.: Source-table trustworthiness is excluded from attribution judgments, which evaluate only whether the caption accurately represents the underlying table.Source quality is assessed separately.
  • Interpretability: Interpretability examples show that absent context, malformed sentences, contradictory descriptions, and unspecified temporal references make captions difficult to understand.Examples include “Bradman scored 299,” a verb-less hardware phrase, and two albums labeled number one for an unspecified year.
  • Attribution: Examples distinguish fully attributable captions, which accurately reflect table information or justified inferences, from errors involving swapped values, incorrect entities, or unsupported labels.A caption can fail attribution when it assigns numbers to an unstated concept such as “seats.”

Flag

The task interface provides a flag button for reporting ineligible evaluation tasks, including missing components, severely malformed text, or source tables requiring expert knowledge.

  • Flag: Flag tasks with missing summaries or news articles, but do not flag empty or “None” table titles, section titles, or table-section text.These empty fields are explicitly acceptable.
  • Flag: Flag any task component containing severely malformed text, such as HTML artifacts or unformatted tables.
  • Flag: Flag source tables when proper attribution evaluation requires specialized scientific expertise, even if the table can be partially understood.The isotope example is flagged because evaluating its caption requires understanding scientific equations and terminology.

Appendix E: Annotator Interface for Conversational QA Tasks

The conversational QA annotator interface separates interpretability from attribution, revealing the source document only after an interpretable rating and conditioning justification requirements during training and pilot.

  • Interpretability stage: The interpretability stage hides the source document; if an item is not interpretable, attribution is skipped and the annotator proceeds to the next queued task.During training and pilot, justification appears only for tasks rated not interpretable.
  • Attribution stage: The attribution stage shows the source document, with justification required for every rating during training and pilot.

Appendix F: Annotator Interface for Summarization Tasks

The summarization annotator interface separates interpretation from attribution: annotators first assess the output with the source hidden, then evaluate attribution with the source shown when applicable.

  • The interpretability stage hides the source document and skips attribution when annotators rate a task as not interpretable.During training and pilot, a justification element appears only for tasks rated as not interpretable.
  • The interface therefore uses a conditional two-stage workflow in which attribution follows interpretation only for interpretable tasks.
  • The attribution stage shows the source document and requires justification for every rating during training and pilot.

Appendix G: Annotator Interface for Table-to-Text Tasks

The table-to-text annotator interface uses separate interpretability and attribution stages, with source information and justifications revealed under stage-specific conditions.

  • Interpretability stage: The interpretability stage hides the source table and description; uninterpretable ratings skip attribution, while training and pilot show justification only for those ratings.
  • Attribution stage: The attribution stage displays the source table and description, preserves highlighted ToTTo cells, and shows justifications for all ratings during training and pilot.
Loading 2112.12870v2…