Source-linked AI summary
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark
Nouha Dziri, Hannah Rashkin, Tal Linzen, David Reitter
TL;DR
Existing attribution metrics for knowledge-grounded dialogue require broader evaluation because prior comparisons use small datasets with few challenging cases. BEGIN collects human annotations for 12k responses from four systems trained on three corpora and evaluates overlap, embedding, QA, NLI, and adversarially trained metrics. Existing metrics rely on spurious correlations, fail to distinguish abstractive attributable from extractive unattributable responses, and degrade for longer knowledge sources.
Problem
Existing attribution metrics for knowledge-grounded dialogue require broader evaluation because prior comparisons use small datasets with few challenging cases.
Method
BEGIN collects human annotations for 12k responses from four systems trained on three corpora and evaluates overlap, embedding, QA, NLI, and adversarially trained metrics.
Results
Existing metrics rely on spurious correlations, fail to distinguish abstractive attributable from extractive unattributable responses, and degrade for longer knowledge sources.
Takeaways & Limitations
BEGIN provides a benchmark for developing more robust evaluation metrics for grounded dialogue response generation.
Takeaways & Limitations
BEGIN does not compare generated responses with gold responses because the gold responses may be unattributable.
Abstract
from arXiv · showhide
Knowledge-grounded dialogue systems powered by large language models often generate responses that, while fluent, are not attributable to a relevant source of information. Progress towards models that do not exhibit this issue requires evaluation metrics that can quantify its prevalence. To this end, we introduce the Benchmark for Evaluation of Grounded INteraction (BEGIN), comprised of 12k dialogue turns generated by neural dialogue systems trained on three knowledge-grounded dialogue corpora. We collect human annotations assessing the extent to which the models' responses can be attributed to the given background information. We then use BEGIN to analyze eight evaluation metrics. We find that these metrics rely on spurious correlations, do not reliably distinguish attributable abstractive responses from unattributable ones, and perform substantially worse when the knowledge source is longer. Our findings underscore the need for more sophisticated and robust evaluation metrics for knowledge-grounded dialogue. We make BEGIN publicly available at https://github.com/google/BEGIN-dataset.
1 Introduction
Knowledge-grounded dialogue systems can generate fluent responses containing unsupported statements, motivating reliable attribution evaluation. BEGIN addresses this gap with human-annotated data and broad metric analysis, finding that existing metrics rely on spurious correlations and fail on key response types and longer sources.
- Attributable responses must be connected to textual evidence supporting their entirety, whereas fluent systems often produce unsupported “hallucinations.”
- BEGIN evaluates whether current automatic metrics fulfill their intended purpose of measuring attribution in grounded dialogue.
- The benchmark classifies responses as fully attributable, not fully attributable, or generic, distinguishing harmless conversational acknowledgments from misleading unsupported content.
- BEGIN contains 12k human-annotated responses from four language-model-based systems trained on three knowledge-grounded dialogue corpora and evaluates overlap, embedding, QA, NLI, and adversarially trained metrics.
- All evaluated metrics rely substantially on spurious correlations, misidentify abstractive attributable and extractive unattributable responses, and degrade when knowledge sources are longer.
2 Task, Datasets and Models
BEGIN evaluates responses generated for knowledge-grounded dialogue, where systems should produce replies coherent with dialogue history and attributable to part of the supplied knowledge. The benchmark draws on language-model systems trained using established corpus splits.
- At each turn, the system receives dialogue history H and knowledge K_n, then generates a response coherent with H and attributable to a non-empty subset of K_n.
- Unlike conversational QA, the task permits diverse informative replies because the preceding utterance may be an open-ended statement rather than a direct question.
- BEGIN contains responses from language-model-based systems trained for this task, with training, development, and test portions used for training, tuning, and benchmark generation.
- The Wizard of Wikipedia dialogue corpus pairs a Wizard who provides topic information with an Apprentice in a knowledge-grounded conversation.
WoW
The Wizard of Wikipedia examples illustrate BEGIN’s response categories and the range of attribution errors produced by different systems. The benchmark includes multiple dialogue systems and corpus-based examples for comparison.
- A generic response asks for more information about Zuckerberg without adding knowledge-specific content.
- A DoHA response about television combines unsupported claims about Pokémon’s creation and episode count with evidence that only mentions television and narrators.
- Table 1 presents category examples from four models trained on three corpora, marking attributable information in green, non-informative text in blue, and hallucinations in red.
- The benchmark compares GPT2-base and T5-base with knowledge-grounded systems DoHA and CTRL-DIALOG, which use architectures or training designed for grounded dialogue.
3 Annotations
BEGIN annotates model-generated responses with a three-way attribution taxonomy and analyzes how response categories vary across models and training corpora. The resulting dataset exposes substantial unattributability in generated dialogue and supports systematic metric evaluation.
- 3 Annotations: Annotators classify responses as fully attributable, not attributable, or generic, separating evidence-supported information from unverifiable or non-informative language.
- 3 Annotations: The dataset contains 12,288 responses sampled across four models and three corpora, with at least 1,000 examples from each dataset-model pair.
- 3 Annotations: Each example is judged by three annotators using majority vote, while examples receiving three different labels are excluded.
- 3 Dataset Analysis: Overall, 70% of generated responses are unattributable; CTRL-DIALOG has the lowest proportion at 30.8%, while TOPICALCHAT-trained models have the highest proportion by corpus.
- 3 Dataset Analysis: The prevalence of unattributable responses is consistent with analyses reporting that more than 60% of ground-truth responses in the three corpora are unattributable to knowledge.
4 Evaluating Evaluation Metrics
The evaluation covers lexical, semantic, question-based, and inference-based metrics, including an adversarially trained classifier. Across analyses, metrics rely heavily on extractivity and perform poorly on abstractive responses and longer knowledge sources.
- Lexical and Semantic Metrics: Metrics generally score attributable responses higher, but overlap among categories prevents reliable mapping from scores to attribution labels.Generic responses can receive lower median scores than unattributable responses even though they are typically preferable in grounded dialogue.
- Q2: Q2 usually scores attributable responses higher but cannot distinguish generic from unattributable responses.Its clearer separation between attributable and unattributable responses appears only in abstractive cases.
- Inference-Based Classifiers: Intermediate MNLI fine-tuning improves RoBERTa by 4.1 F1, whereas DNLI intermediate fine-tuning shows no similar improvement.This gain is reported for the adversarially trained classifier setting.
- Inference-Based Classifiers: None of the models exceed 46% F1, while BEGIN-ADVERSARIAL outperforms classifiers trained on MNLI, DNLI, and AugWoW despite being smaller.Human annotations exceed 95% precision against the majority vote, leaving substantial room for improvement.
- Results: Unattributable extractive responses score higher on average than attributable abstractive responses across all metrics.The classifiers show the same pattern: attribution classification is better for extractive responses, while extractive unattributable responses are harder to detect.
- Robustness to Distribution Shift: Metrics rate responses from WOW higher than those from CMU-DOG and TOPICALCHAT, with TOPICALCHAT producing the worst classifier performance.RoBERTa-MNLI’s F1 decreases by 10 points on attributable TOPICALCHAT responses versus WOW, and longer knowledge sources receive poorer scores.
5 Related Work
Prior work has developed evaluation methods for natural language generation and grounded dialogue, but attribution assessment in dialogue remains comparatively underdeveloped. BEGIN extends this line by meta-evaluating automated metrics, including NLI-based approaches and adversarially augmented training data.
- The paper meta-evaluates automatic attribution metrics in knowledge-grounded responses, extending evaluation analyses previously conducted for other generation tasks.
- Evaluation research spans machine translation, data-to-text, summarization, and other NLG tasks, while grounded-response evaluation remains nascent.
- Existing dialogue evaluation studies measured properties such as engagingness and relevance but did not investigate response faithfulness.
- Prior grounded-dialogue work includes token-level hallucination critics, human attribution frameworks, and faithful benchmarks for information-seeking dialogue.
- The study also examines NLI-based attribution classifiers and adversarially augmented training data, building on prior NLI approaches to dialogue consistency and factuality.
6 Conclusion
BEGIN evaluates whether automatic metrics can assess attribution in knowledge-based dialogue. The metrics rely substantially on word overlap, mis-rank abstractive and generic responses, and degrade under distribution shift with longer knowledge sources.
- BEGIN shows that existing automatic metrics rely substantially on word overlap and fail to properly rank abstractive attributable and generic responses.
- The metrics also assign low scores to attributable responses grounded on long knowledge sources, revealing difficulty under distribution shift.
- The benchmark is intended to advance robust metric development for attribution in contemporary knowledge-based dialogue systems.
A BEGIN Annotation Protocol
BEGIN annotators classify generated responses according to whether their information is fully supported, unsupported, or too generic to be informative. Workers receive documents, conversational context, responses, and category examples with explanations.
- Workers label each generated response as fully attributable, not attributable, or too generic to be informative.
- Each worker receives a document, the previous conversational turn, a response generated by one of four systems, and examples explaining every category.
- The generic category covers utterances that are too bland or nonspecific to share new information.
- The unsupported-information category includes information that cannot be fully verified, including false, unverifiable, personal, or opinion-based content.
- The fully supported category requires that all information in the utterance be supported by the document.
B Implementations
The experiments implement or reproduce four dialogue systems using specified training procedures, publicly available code, or prior reported results. Training uses dataset-specific settings and, for DoHA, nucleus sampling.
- GPT2, T5: GPT2 and T5 use Hugging Face Transformers, Adam optimization, dropout, batch size 32, learning rate 6.25 × 10−5, and dataset-specific early stopping.
- CTRL-DIALOG: CTRL-DIALOG results are reproduced by following the training details reported by Rashkin et al. (2021b).
- DoHA: DoHA uses publicly available CMU-DOG code and a pretrained model, while models for WOW and TOPICALCHAT follow the authors’ procedure and select the best validation model.
- DoHA: DoHA generation uses nucleus sampling with p = 0.9.
C Model-Based Metrics
The study evaluates semantic similarity models using specified publicly available checkpoints and recommended configurations.
- BERTScore uses version 0.3.11 with the DeBERTaxl-MNLI model recommended at the time of investigation.