PaperlayerView exact revision on arXiv

Source-linked AI summary

Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods

Wei Li, Wenhao Wu, Moye Chen, Jiachen Liu, Xinyan Xiao, Hua Wu

arXiv:2203.05227v1cs.CL

TL;DR

Faithfulness failures remain a major obstacle for practical NLG because generated text can contain information inconsistent with its input or with world knowledge. This survey systematically analyzes the problem and organizes evaluation metrics and optimization methods across tasks. It finds that QA-based metrics generally perform best across meta-evaluation datasets, but their correlation with human judgments remains at most 0.6, leaving faithfulness evaluation open.

  • Problem

    Faithfulness problems, including inconsistent, hallucinated, or non-factual generated content, limit the credibility and practical usability of NLG systems.

    Source: §1.1 · p. 6, §1.2 · p. 6

  • Method

    The survey combines faithfulness analysis with unified taxonomies of evaluation metrics and optimization methods across NLG tasks.

    Source: §6 · pp. 37–38

  • Results

    QA-based metrics achieve better performance on most datasets and benchmarks, but their correlations with human evaluations are not more than 0.6.

    Source: §3.5 · p. 19

  • Takeaways & Limitations

    Faithfulness evaluation remains an open research question requiring further exploration across datasets, metrics, and NLG tasks.

    Source: §3.5 · p. 19

    Existing metrics often measure only aggregate scores, cannot distinguish error types or locate error spans, and are frequently task-specific.

    Source: §5.1 · p. 36

Abstract

from arXiv · show

Natural Language Generation (NLG) has made great progress in recent years due to the development of deep learning techniques such as pre-trained language models. This advancement has resulted in more fluent, coherent and even properties controllable (e.g. stylistic, sentiment, length etc.) generation, naturally leading to development in downstream tasks such as abstractive summarization, dialogue generation, machine translation, and data-to-text generation. However, the faithfulness problem that the generated text usually contains unfaithful or non-factual information has become the biggest challenge, which makes the performance of text generation unsatisfactory for practical applications in many real-world scenarios. Many studies on analysis, evaluation, and optimization methods for faithfulness problems have been proposed for various tasks, but have not been organized, compared and discussed in a combined manner. In this survey, we provide a systematic overview of the research progress on the faithfulness problem of NLG, including problem analysis, evaluation metrics and optimization methods. We organize the evaluation and optimization methods for different tasks into a unified taxonomy to facilitate comparison and learning across tasks. Several research trends are discussed further.

1 Introduction

NLG spans multiple input-output settings and has evolved from rule-based systems to neural and pretrained models, but faithfulness remains a major practical challenge. This survey organizes analyses, evaluation metrics, and optimization methods into a unified taxonomy across tasks.

1 Introduction

  • NLG tasks include text-to-text, data-to-text, and multimodality-to-text generation, with applications ranging from summarization and translation to table-to-text and image captioning.The survey also distinguishes open-ended generation, where outputs create new information, from non-open-ended generation, where inputs provide complete or excess output information.Source: §1 · p. 4

1.1 Developing of NLG

  • NLG development progressed through template-based, statistical-based, neural-based, and pretraining-based paradigms, trading rule-based reliability against neural fluency and informativeness.Template systems are constrained by rule diversity and generality, while later models retain faithfulness difficulties.Source: §1.1 · pp. 5–6
  • Pretrained NLG models improve fluency, informativeness, and controllability but remain prone to factual errors that contradict the input.Neural generation also faces faithfulness problems because probabilistic sampling over large vocabularies reduces controllability.Source: §1.1 · pp. 5–6

1.2 The Faithfulness Problem

  • About 30% of summaries produced by state-of-the-art abstractive systems have faithfulness issues, undermining the credibility and usability of summarization.The survey frames faithfulness as the biggest NLG challenge for practical applications.Source: §1.2 · p. 6
  • Faithfulness problems affect nearly all NLG tasks, including summarization, dialogue generation, machine translation, and table-to-text generation.For non-open-ended tasks, faithfulness concerns consistency with the input; for open-ended tasks, it concerns consistency with world knowledge or commonsense.Source: §1.2 · p. 7

1.3 Structure of This Survey

  • The survey analyzes faithfulness problems, including error categorization, manual annotation, evaluation and optimization challenges, causes, and relations with other generation properties.Its framework focuses mainly on factual consistency in non-open-ended tasks such as summarization, translation, knowledge-grounded dialogue, and data-to-text generation.Source: §1.3 · p. 7, §1.2 · p. 7
  • Evaluation methods are organized into entailment-based, QA-based, fact-based, and other metrics, while optimization methods are grouped into factual guidance, auxiliary tasks, post-editing, learning, constrained decoding, and other methods.The taxonomy supports comparison across task-specific and task-agnostic research.Source: §1.3 · p. 7

2 Problem Analysis

The section defines faithfulness errors, develops a hierarchical typology, and reviews their prevalence, evaluation difficulty, and causes across NLG.

2.1 Definition and Categorization

  • Faithfulness is defined at the span level: an output is unfaithful when a generated span is unsupported by the source, producing intrinsic or extrinsic errors.Intrinsic errors contradict the source; extrinsic errors are neither supported nor contradicted by it.Source: §2.1 · pp. 9–10
  • The hierarchical typology separates semantic-frame, discourse, and content-verifiable errors, including predicate, entity, circumstance, co-reference, discourse-link, out-of-article, and grammatical errors.It is intended to provide deeper error analysis than a binary faithful/unfaithful label.Source: §2.1 · pp. 10–11, Table 4 · p. 10
  • Hallucinated content may still be factual according to world knowledge, so extrinsic errors are further divided into factual and non-factual hallucinations.This distinction matters because some factual hallucinations can provide useful background information.Source: §2.1 · pp. 10–11

2.2 Challenges and Issues

  • Over 60% of summaries from all evaluated systems are unfaithful on XSum, while T5 and BART exceed 20% on CNN/DM.The dataset strongly affects observed faithfulness performance.Source: §2.2 · p. 11, Table 5 · p. 11
  • N-gram metrics such as BLEU, ROUGE, and METEOR correlate poorly with human factuality judgments, motivating dedicated faithfulness metrics.Table 6 reports Pearson and Spearman correlations on CNN/DM, XSum, and their combination.Source: §2.2 · p. 11, Table 6 · p. 12
  • Human faithfulness annotation is difficult: Fleiss κ is 0.58 for faithful versus unfaithful judgments and 0.39 for specific error types.Crowd–expert and inter-annotator agreement are relatively low, while ranking-based Best-Worst Scaling is reported as more reliable than rating-based annotation.Source: §2.2 · p. 11

2.3 Cause Analysis

  • Source–reference divergence, exposure bias, and poor text representation make models vulnerable to hallucinations and explain dataset-dependent faithfulness.XSum references contain 76.9% unfaithful content, whereas CNN/DM references have fewer hallucinations; teacher forcing and word-level likelihood do not directly reward faithfulness.Source: §2.3 · pp. 12–13

3 Automatic Evaluation Metrics

Automatic faithfulness evaluation supplements inadequate n-gram overlap metrics with entailment-based approaches that compare generated text against its source. These approaches range from sentence-level and fine-grained NLI to annotated and weakly supervised classifiers, but their effectiveness depends on task alignment and training data.

3 Automatic Evaluation Metrics

  • N-gram overlap metrics cannot evaluate faithfulness adequately, motivating four metric families: entailment-based, QA-based, fact-based, and other methods.The section focuses on entailment-based metrics after introducing this taxonomy.Source: §3 · p. 13

3.1 Entailment-based Metrics

  • Entailment-based metrics use an NLI model to score whether generated text is entailed by or conflicts with a source, with higher entailment indicating greater faithfulness.Most studies use entailment scores and evaluate correlations with human judgments or ranking-based downstream tasks.Source: §3.1 · p. 14, Figure 4 · p. 14
  • Out-of-the-box NLI models perform poorly for generation faithfulness, although RankNLI supports summary reranking and different NLI architectures affect ranking performance.RankNLI aggregates sentence-pair entailment scores between source and generated summaries.Source: §3.1 · p. 14
  • Sentence-level NLI is improved through task-specific training, including ANLI-based classifiers, CNN aggregation in SummaC, and direct paragraph-level annotation using roughly 500 manual samples.These approaches address the mismatch between sentence-level NLI and paragraph-level faithfulness judgments.Source: §3.1 · p. 14
  • Weakly supervised methods create synthetic positive and negative examples through heuristic edits, adversarial construction, or conversion of reading-comprehension data into long-premise NLI datasets.FactCCX additionally highlights source evidence spans, while SetenceNLI combines rule-based and generated samples.Source: §3.1 · p. 15
  • Fine-grained methods decompose entailment into dependency-arc predictions, while dialogue metrics construct NLI resources from relation triples or annotated dialogue and knowledge relations.DAE aggregates dependency-arc entailment scores; DialogNLI infers labels from relation triples to reduce annotation labor.Source: §3.1 · p. 15

3.2 QA-based Metrics

QA-based metrics evaluate faithfulness by converting generated information into questions, answering them from the source, and aligning source and target answers. Their performance depends on component choices, while QUALS simplifies the pipeline to reduce computation and memory.

3.2 QA-based Metrics

  • QA-based metrics use question generation and question answering to match source answers with information units extracted from generated text.The framework contains answer selection, question generation, source-based question answering, and answer-alignment scoring.Source: §3.2 · pp. 15–16
  • QAGS and FEQA extract n-grams or entities as target answers and combine BART-based question generation, BERT-based QA, and token-level F1 alignment.They are the earliest QA-based factuality metrics described in the section.Source: §3.2 · p. 16
  • QuestEval adds a recall-oriented procedure by generating question–answer pairs from the source and answering them from the generated text.This extends the basic generated-text-to-source evaluation direction.Source: §3.2 · p. 16
  • QUALS uses one QAGen model to jointly generate question–answer pairs and scores how likely the source is to generate the same pairs as the summary.Its score compares average log likelihoods over M question–answer pairs.Source: §3.2 · p. 16
  • QUALS substantially reduces the computational time and memory required by the original QAGS procedure.The simplification removes separate question-generation, question-answering, and alignment stages from the earlier pipeline.Source: §3.2 · p. 16
  • QAFactEval improves QA-based metric performance through carefully selected components: noun-phrase target units, filtered BART-QA2D questions, Electra-large QA, and LERC answer alignment.The study reports that component selection is critical and that the optimized configuration raises performance to a new level.Source: §3.2 · pp. 16–17
  • Q2 applies the QA-based framework to open-domain dialogue and uses an NLI entailment score to align answer spans.This extends QA-based factual consistency evaluation beyond summarization.Source: §3.2 · p. 17

3.3 Fact-based Metrics

Fact-based metrics evaluate faithfulness by comparing generated facts with source facts, using entities, n-grams, relation triples, or broader probability-based signals.

3.3 Fact-based Metrics

  • Fact-based metrics represent faithfulness through overlap between generated and source facts, including entities, n-grams, and relation triples.Source: §3.3 · p. 17

3.3.1 Entity-based

  • Entity-level inconsistency introduces unsupported named entities, whereas relation-level inconsistency combines supported entities with unsupported relations.Source: §3.3 · p. 17
  • EntityAlign measures entity faithfulness as precision, N(y ∩x)/N(y), matching generated named entities to source entities through n-gram overlap.Source: §3.3.1 · p. 17

3.3.2 Ngram-based

  • PARENT aligns generated and reference n-grams with table facts, using union-based precision and intersection-based recall before combining them into an F-score.Source: §3.3.2 · p. 17
  • PARENT-T focuses precision and recall on table entailment, rewarding table-supported n-grams and greater coverage of table information.Source: §3.3.2 · p. 18

3.3.3 Relation-based

  • TripleAlign uses fixed-schema relation extraction so extracted subject-relation-object triples can be compared more consistently than OpenIE triples.Source: §3.3.3 · p. 18

3.4 Other Metrics

  • Other metrics include BARTScore’s generation probabilities, TokenLevelCLS hallucination labels, CoCo’s counterfactual token probabilities, and ShannonScore’s information difference.Source: §3.4 · pp. 18–19

3.5 Meta Evaluation

Meta-evaluation benchmarks directly compare faithfulness metrics against human annotations, but results vary substantially across benchmarks and datasets.

3.5 Meta Evaluation

  • Benchmarks such as FRANK, SUMMAc, QAGS, FEQA, CoCo, and BEGIN compare metric correlations with human faithfulness judgments across generation tasks.Source: §3.5 · p. 19
  • QA-based metrics perform best on most datasets, yet their correlations with human evaluations remain below 0.6, leaving faithfulness evaluation unresolved.Source: §3.5 · p. 19

4 Optimization Methods

Faithfulness optimization methods span multiple NLG tasks and are organized into a unified taxonomy to support comparison across tasks.

4 Optimization Methods

  • Optimization methods cover abstractive summarization, dialogue generation, data-to-text generation, and machine translation, with many methods generalizable across tasks.Source: §4 · p. 19
  • The taxonomy includes factual guidance, auxiliary tasks, learning methods, post-editing, constrained decoding, and other approaches.Source: §4 · p. 19

4.1 Faithfulness in Abstractive Summarization

Abstractive summarization methods improve faithfulness through guidance, auxiliary objectives, contrastive learning, post-editing, constrained decoding, and verification-based selection.

4.1.1 Factual Guidance

  • Factual guidance supplies keywords, sentences, relations, or semantic graphs as extra inputs so summarizers focus on source information and reduce factual errors.Source: §4.1.1 · pp. 21–22

4.1.2 Auxiliary Tasks

  • Auxiliary-task methods use reinforcement learning, multi-task learning, or reranking to incorporate entailment, QA, entity, and factual-correctness signals.Source: §4.1.2 · pp. 22–23

4.1.3 Learning Methods

  • Contrastive learning separates factually consistent summaries from hard negative examples created through perturbations, entity changes, sentiment reversals, or altered factual spans.Source: §4.1.3 · pp. 23–25
  • CLIFF and CO2Sum combine contrastive objectives with standard training to distinguish factual summaries from inconsistent peers at summary, encoder, or decoder levels.Source: §4.1.3 · pp. 24–25

4.1.4 Post-Editing

  • SpanFact corrects entities iteratively using question-answering knowledge, correcting about 26% of unfaithful summaries while barely damaging otherwise correct summaries.Source: §4.1.4 · p. 26
  • Post-editing improves faithfulness while preserving informativeness, but remains an indirect rather than fundamental solution to factual inconsistency.Source: §4.1.4 · p. 26

4.1.5 Constrained Decoding

  • Constrained decoding enforces source-supported entities or phrases during beam search, while source-conditioned lexical bias encourages faithful token generation.Source: §4.1.5 · p. 26

4.1.6 Other Methods

  • Other methods verify or rerank candidate summaries using quantity support, discourse-based factuality scores, or prior-posterior token probabilities.Source: §4.1.6 · p. 26

4.2 Faithfulness in Dialogue Generation

Dialogue faithfulness concerns consistency with history, persona, and knowledge inputs. The survey organizes optimization methods into guidance, auxiliary tasks, learning, constrained decoding, post-editing, and other approaches.

4.2 Faithfulness in Dialogue Generation

  • Dialogue systems should remain faithful to history utterances and diverse background or knowledge inputs.Source: §4.2 · p. 27

4.2.1 Factual Guidance

  • Three guidance types support dialogue faithfulness: implicit representations, extracted source information, and retrieved open-domain knowledge.Source: §4.2.1 · pp. 27–28

4.2.2 Auxiliary Tasks

  • Entailment scores improve faithfulness either by reranking candidate responses or by serving as rewards in reinforcement learning.Source: §4.2.2 · pp. 28–29

4.2.3 Learning Methods

  • Unlikelihood training adds penalties for contradictory candidate sentences alongside maximum likelihood training to reduce factual inconsistency.Source: §4.2.3 · p. 29

4.2.4 Constrained Decoding

  • Constrained decoding enforces semantic consistency by requiring generated responses to satisfy structured input representations or logical checks.Source: §4.2.4 · p. 29

4.2.5 Post-Editing

  • Post-editing uses generate, delete, and rewrite stages to remove factual errors and replace them with correct contexts or knowledge-grounded entities.Source: §4.2.5 · pp. 29–30

4.2.6 Other Methods

  • Rational Speech Act modeling discourages contradictions by giving dialogue agents public self-consciousness across generation steps.Source: §4.2.6 · p. 30

4.3 Faithfulness in Machine Translation

Machine translation faithfulness is limited by omissions, unrelated fragments, hallucinations, and sensitivity to noisy or perturbed inputs. The survey covers auxiliary tasks, learning procedures, constrained decoding, and source-representation mechanisms.

4.3.1 Auxiliary Tasks

  • Faithfulness-enhanced NMT combines masked language modeling, word alignment, and contrastive max-margin objectives to improve source modeling and translation accuracy.Source: §4.3.1 · p. 30

4.3.2 Learning Methods

  • Minimum Risk Training substantially reduces hallucinations and makes large-beam search more stable by avoiding exposure bias.Source: §4.3.2 · pp. 30–31
  • Training robustness methods address hallucinations from noisy corpora, outliers, invalid pairs, and out-of-domain inputs through truncation, tilted risk, and filtering.Source: §4.3.2 · p. 31

4.3.3 Constrained Decoding

  • Lexically constrained decoding inserts required words or phrases while retaining the model distribution, improving faithfulness without changing model parameters or training data.Source: §4.3.3 · pp. 31–32
  • Post and Vilar’s lexically constrained algorithm has complexity O(1) in the number of constraints, compared with linear grid beam search and exponential constrained beam search.Source: §4.3.3 · p. 32

4.3.4 Other Methods

  • Other methods improve faithfulness by strengthening source coverage, global source representations, dynamic source control, or bag-of-words decoding constraints.Source: §4.3.4 · p. 32

4.4 Faithfulness in Data-to-Text Generation

Data-to-text systems remain vulnerable to unfaithful generation because of long-form generation and noisy training datasets. The survey organizes solutions around planning, auxiliary objectives, learning, constrained decoding, and data refinement.

4.4 Faithfulness in Data-to-Text Generation

  • Nearly two-thirds of WikiBio training instances contain unfaithful descriptions, while neural data-to-text systems show record-generation gaps against faithful templates.Source: §4.4 · p. 32

4.4.1 Factual Guidance

  • Two-stage planning improves faithfulness by selecting source records first and generating text while attending only to the selected data.Source: §4.4.1 · p. 32

4.4.2 Auxilary Tasks

  • Pseudo training plans covering target entities prevent hallucinated entities from affecting the plan-to-text stage despite noise in the original corpus.Source: §4.4.2 · pp. 32–33

4.4.3 Learning methods

  • Additional losses and reinforcement learning directly optimize table-text matching, entity alignment, information richness, or PARENT-based faithfulness.Source: §4.4.3 · p. 33

4.4.4 Constrained Decoding

  • Confidence-based decoding detects low-attention source-information tokens and skips tokens judged at higher risk of producing unfaithful text.Source: §4.4.4 · p. 33
  • Noise-aware prefixes and dependency-based alignment labels guide controllable decoding at increasingly fine-grained levels.Source: §4.4.4 · p. 33

4.4.5 Other Methods

  • Dataset refinement removes unaligned noise, and RotoWire-FG was created after observing that only 60% of RotoWire output content could be grounded in input records.Source: §4.4.5 · p. 33

4.5 Faithfulness in Other NLG Tasks

Other NLG tasks address faithfulness through structured knowledge, factuality detection, lexical constraints, and task-specific rewards. The surveyed applications include factual language modeling, image captioning, and radiology report generation.

4.5.1 Factual Language Model

  • KGLM dynamically grows a local knowledge graph and combines graph tokens with vocabulary tokens to generate accurate facts for rare entities.Source: §4.5.1 · pp. 33–34

4.5.2 Factuality Detection

  • Factuality detection includes eye-tracking ensembles, adversarial claim detection, graph reasoning, and controllable generation; Grover itself achieved 92% accuracy versus 73% for the best discriminator.Source: §4.5.2 · p. 34

4.5.3 Constrained Text Generation

  • Soft lexical constraints can lose keywords, particularly when many or weakly related keywords must be included, and attention or copying does not guarantee inclusion.Source: §4.5.3 · p. 35
  • Hard-constrained generation guarantees lexical inclusion but introduces architectural or computational costs, motivating progressive insertion and more efficient decoding methods.Source: §4.5.3 · p. 35

4.5.4 Image Caption

  • Image captioning models hallucinate objects, while standard caption metrics do not fully measure image relevance; constrained beam search forces selected visual tags into outputs.Source: §4.5.4 · p. 35, §4.5.3 · p. 35, Figure 13 · p. 36
  • Radiology report generation suffers from factual incompleteness and inconsistency, motivating entity-match and NLI-based rewards optimized with reinforcement learning.Source: §4.5.4 · pp. 35–36

5 Discussion

The discussion identifies gaps in current faithfulness evaluation and highlights reasoning-oriented directions for improving NLG faithfulness.

5.1 Fine-grained and General Evaluation

  • Current faithfulness metrics usually produce unified scores, cannot locate fine-grained error spans, and are often task-specific rather than general.The survey calls for evaluations that distinguish factual error types and work across NLG tasks.Source: §5.1 · pp. 36–37
  • Intrinsic and extrinsic factual errors should be evaluated separately because they use different references and have different consequences.Intrinsic errors reference the source text, whereas extrinsic errors additionally involve world knowledge; factual extrinsic hallucinations may be beneficial, unlike non-factual ones.Source: §5.1 · p. 36

5.2 Reasoning-based Optimization

  • Reasoning over source context or world knowledge is needed because many intrinsic errors arise from misunderstanding source facts.Large-scale NLU pre-training alone is described as insufficient for correctly understanding facts.Source: §5.2 · p. 37
  • Numerical reasoning is crucial because models treating dates, quantities, and scalars like ordinary tokens are vulnerable to numerical errors.The survey emphasizes this issue for quantity-heavy tasks such as table-to-text generation.Source: §5.2 · p. 37
  • Grounded language representations may improve faithfulness by strengthening document-level understanding and inference.The survey presents multimodal grounding and unified-modal models as promising directions for more expressive NLG representations.Source: §5.2 · p. 37
  • Causal inference is proposed as a fundamental direction because correlation-based language models may generalize poorly in out-of-distribution or long-tailed settings.The survey links the lack of causality to errors in model generalization and faithfulness.Source: §5.2 · p. 37

6 Conclusion

The survey unifies research on NLG faithfulness by combining analysis, evaluation, and optimization methods across tasks. It proposes taxonomies for evaluation and optimization and identifies future research directions from current challenges.

6 Conclusion

  • The survey systematically organizes faithfulness analysis, evaluation metrics, and optimization methods across different NLG tasks.Its unified categorization supports both task-specific and task-agnostic understanding.Source: §6 · p. 37
  • Evaluation metrics are grouped into entailment-based, QA-based, fact-based, and other types, while optimization methods are grouped into six methodological categories.The optimization categories are Factual Guidance, Auxiliary Task, Post-Editing, Learning Method, Constrained Decoding, and Others.Source: §6 · p. 37
  • The survey compares methods across tasks and proposes future directions based on challenges and the current status of faithfulness research.These directions are tied to the evaluation metrics and optimization methods reviewed in the survey.Source: §6 · pp. 37–38

Related papers

Loading 2203.05227v1…