PaperlayerView exact revision on arXiv

Source-linked AI summary

Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation

Ran Tian, Shashi Narayan, Thibault Sellam, Ankur P. Parikh

arXiv:1910.08684v3cs.CL

TL;DR

Encoder-decoder models can generate fluent text that is unfaithful to its source, a concern for data-to-text generation. The paper proposes confidence-guided decoding with a variational Bayes training framework, and reports improved faithfulness relative to existing state-of-the-art solutions.

  • Problem

    Encoder-decoder models can produce fluent text that is unsupported by the source, creating a faithfulness problem for data-to-text generation.

    Source: §1 · p. 1

  • Method

    The method scores source attention and whether words convey source information, then uses variational Bayes to train confident-subsequence sampling and the score parameters.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    The approach is considerably more faithful than existing state-of-the-art solutions by PARENT score and human evaluation, and reports strong results on WebNLG.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Faithfulness matters because occasional unfaithful outputs can limit the use of neural approaches in real-world systems.

    Source: §1 · p. 1

    The current confidence-score formulation specifically uses an RNN with attention.

    Source: §3 · p. 2

Abstract

from arXiv · show

We address the issue of hallucination in data-to-text generation, i.e., reducing the generation of text that is unsupported by the source. We conjecture that hallucination can be caused by an encoder-decoder model generating content phrases without attending to the source; so we propose a confidence score to ensure that the model attends to the source whenever necessary, as well as a variational Bayes training framework that can learn the score from data. Experiments on the WikiBio (Lebretet al., 2016) dataset show that our approach is more faithful to the source than existing state-of-the-art approaches, according to both PARENT score (Dhingra et al., 2019) and human evaluation. We also report strong results on the WebNLG (Gardent et al., 2017) dataset.

1 Introduction

The introduction frames hallucination as unsupported generated content in table-to-text tasks and motivates confidence-guided decoding that uses source attention when content conveys source information.

1 Introduction

  • The paper studies table-to-text generation, where structured tables provide a setting in which faithfulness can be objectively evaluated relatively easily.Source: §1 · p. 1
  • Source-reference divergence can encourage unsupported phrases, but hallucination also appears with cleaned references and is attributed to learned wrong correlations.Source: §1 · pp. 1–2
  • The method scores source attention and uses a language model to judge whether a word conveys source information, then trains the score through variational Bayes.Source: §1 · p. 2
  • The approach is considerably more faithful than existing state-of-the-art solutions by PARENT score and human evaluation, with strong results on WebNLG.Source: §1 · p. 2

2 Preliminaries

The preliminaries describe an encoder-decoder that models target sequences conditioned on source input, with attention and an optional copy mechanism.

2 Preliminaries

  • The encoder-decoder models the target sequence as a product of token probabilities conditioned on the source and preceding target tokens.Source: §2 · p. 2
  • An encoder maps source tokens to representations, while attention supplies a context vector and an RNN computes the decoder state.Source: §2 · p. 2
  • With a copy mechanism, the model mixes generation and source-copy probabilities using a generation probability and source-position attention weights.Source: §2 · p. 2

3 Modeling Confident Decoding

The method scores each token using source attention and a tailored language model, treating low confidence for source-bearing words as a possible hallucination signal. It learns confident target subsequences with a variational Bayes objective and also uses confidence at inference through calibration or a <null> token.

3 Modeling Confident Decoding

  • The confidence score combines source attention with a tailored language model: source-bearing tokens should be attended to, while template words should have high language-model probability.Low confidence for a source-bearing token generated without attention is treated as a possible hallucination signal.Source: §3 · pp. 2–3
  • Attention weights can sum to less than one, and decoder hidden states exclude source information, making the attention score measure source contribution to the context vector.Observed scores were 0.6–0.9 for source-supported tokens and 0.2–0.5 for templatic or hallucinated tokens; the modification did not impact generation quality.Source: §3 · p. 3
  • The tailored base language model down-weights inputs associated with high attention scores to reduce source-specific generation patterns; stop-gradient prevents its training from affecting the encoder-decoder through attention.Joint training without stop-gradient led most attention scores to fall below 0.5 in preliminary experiments.Source: §3 · p. 3

3.1 Training with Confident Sub-sequence Sampling

  • Training samples a latent keep-or-skip subsequence according to token confidence and uses a variational Bayes objective to learn its confidence scores.The encoder-decoder trains on sampled confident tokens, while the entropy term encourages higher confidence for tokens labeled skip; higher-likelihood subsequences eventually remain confident.Source: §3.1 · pp. 4–5
  • Because enumerating all target subsequences is exponential in target length, the expected objective is estimated by Monte Carlo sampling; the base language model is also trained on confident subsequences.Joint training on the confident subsequence implicitly biases the base language model toward confident generation patterns.Source: §3.1 · p. 5

3.2 Calibration and <null> Token

  • Calibration adds the confidence score to inference-time probability ranking; its trainable parameter converged to about 0.65 on WikiBio without sacrificing training perplexity.The calibrated family includes the original distribution as the κ = 0 case.Source: §3.2 · p. 5
  • The <null> strategy replaces low-confidence tokens during decoding and removes them after beam search; combined with length penalty, it increased recall while maintaining precision.Consecutive <null> tokens are shut out from the beam search.Source: §3.2 · p. 5

4 Experiments

Experiments evaluate confident decoding on WikiBio, where it improves faithfulness over corresponding baselines, and WebNLG, where the modeling enhancements do not degrade performance with clean references. WikiBio results also examine component contributions and sensitivity to source information.

4 Experiments

  • WikiBio and WebNLG test different source-reference divergence regimes: 62% of WikiBio examples exhibit divergence, while WebNLG has less.WikiBio is heuristically collected; WebNLG responses are human-generated.Source: §4 · p. 5

4.1 Results on WikiBio

  • Confident decoding clearly improves faithfulness over the Pointer-Generator and BERT-to-LSTM baselines, and achieves the highest PARENT precision and F1.Table 1 reports WikiBio test-set performance; the two fluency measures differ in whether they include sentences graded Mostly Fluent.Source: §4.1 · p. 6, Table 1 · p. 7
  • With a moderate null threshold, Conf-T2LSTM improves precision and recall over BERT-to-LSTM without sacrificing fluency; more aggressive settings can boost precision and recall at some fluency cost.Conf-PtGen without length penalty or null tokens tends to stop early when unconfident, reducing coverage; length penalty encourages longer generation.Source: §4.1 · pp. 6–7
  • All three ablated components improve PARENT precision, while calibration also improves PARENT recall and BLEU; the other techniques trade recall for precision.The tested components are confidence-score design, variational Bayes training with confident sub-sequence sampling, and calibration.Source: §4.1 · p. 7, Table 2 · p. 7
  • As source vectors are zeroed more often, Conf-T2LSTM predictions change more than BERT-to-LSTM predictions, although many changes are subtle to human eyes.At a source-vector zeroing probability of 0.1, over 65% of predictions change for both models; the experiment reports five decoding runs per noise level.Source: Figure 4 · p. 7, §4.1 · p. 8

4.2 Results on WebNLG

  • On WebNLG, the modeling enhancements do not degrade performance with clean references, and the models show strong performance against the cited OpenNMT baseline.The comparison uses the WebNLG v2.1 constrained split.Source: §4.2 · p. 8, Table 3 · p. 8

5 Discussion and Related Work

The discussion situates the method among approaches to improve generation accuracy, then contrasts its general prior over decoder attention and language modeling with methods relying on data removal or source-target alignment heuristics.

5 Discussion and Related Work

  • Removing unfaithful training examples is impractical for noisy WikiBio, where 62% of examples exhibit source-reference divergence.Matsumaru et al. (2020) report reduced hallucination after removing such instances; Kang and Hashimoto (2020) propose loss truncation to remove noise.Source: §5 · p. 8
  • Source-target alignment methods add loss terms, but their alignment quality depends on heuristics specific to the task and data.Wang et al. (2020) and Shen et al. (2020) use this approach.Source: §5 · p. 8
  • The paper's prior is presented as more general because it does not depend on source-specific data structure.The authors say it could be adapted to summarization or machine translation, or combined with other approaches.Source: §5 · p. 8
  • The approach is framed as complementary to decoder re-ranking and as a reconsideration of encoder-decoder attention and language modeling.Harkous et al. (2020) train a ranker on heuristically extracted faithful data-text pairs; Li and Rush (2020) use posterior regularization with surface-matching heuristics.Source: §5 · p. 8

A Generation Examples and Analysis of Hallucination

The examples show Pointer-Generator hallucinations that confident decoders often avoid, including unsupported details when table fields are absent and errors where corresponding fields are present. Omitting unsupported information can reduce fluency.

A Generation Examples and Analysis of Hallucination

  • Confident decoders omit details when table fields are missing, while Pointer-Generator invents them; some resulting sentences are less fluent.The first six examples include missing information, and the baseline makes up a middle name in one case.Source: A Generation Examples and Analysis of… · p. 12
  • For Richard Lloyd, the Pointer-Generator adds unsupported language-model content, while the confident models generate correctly.The confident outputs describe Lloyd as a British racing driver.Source: A Generation Examples and Analysis of… · pp. 12, 14
  • For Robert I. Marshall and Thomas Edwards, the baseline fails to generate correctly despite corresponding table fields, while confident outputs are more faithful.The authors note that these fields may not be simply copied.Source: A Generation Examples and Analysis of… · pp. 12, 14, 15

B Interplay between Attention Score, Base Language Model, and Confidence Score

The examples illustrate learned attention, language-model probability, and confidence scores. In Conf-PtGen, the scores detect tokens unsupported by table fields that are missing.

B Interplay between Attention Score, Base Language Model, and Confidence Score

C Human Evaluation Instructions

The human-evaluation instructions describe how annotators assess generated sentences against a table and reference, including which inferences count as faithful and how to mark supporting cells.

C Human Evaluation Instructions

1. How fluent is the sentence?

The evaluation guidance separates fluency from faithfulness: fluency concerns readability, while faithfulness concerns whether sentence content is supported by the table or background knowledge.

1. How fluent is the sentence?

  • Fluency is rated as Fluent, Mostly Fluent, or Not Fluent, based on naturalness, grammatical correctness, and how understandable the sentence is.Fluent text reads naturally and correctly; the lower categories allow errors or unnaturalness, with Not Fluent being difficult to understand.Source: §1 · p. 16
  • Evaluators also count how many table cells the writer used, then compare sentence content with the table and background knowledge to assess faithfulness.The examples use a table and background knowledge as the basis for these judgments.Source: §1 · p. 16
  • Faithfulness is judged by whether every part of a sentence is supported by the table or background knowledge, without unsupported additions.The guidance asks evaluators to compare the sentence against both sources.Source: §1 · p. 16
  • Faithfulness categories distinguish full support, rare cases of incomplete support, and information unsupported by the table or background knowledge.The labels are Faithful, Mostly Faithful, and Not Faithful.Source: §1 · p. 16
  • The example sentence identifies Alfred Angas Scott as a British motorcycle designer, alongside listed dates, birthplace, nationality, residence, and occupation.The example sentence variants describe him as British or European.Source: §1 · pp. 16–17

D Detailed Experiment Settings

The settings describe the model configurations and training/decoding choices, alongside example coverage judgments that contrast unsupported and supported information.

D Detailed Experiment Settings

  • The example coverage judgments classify the founder claim as not faithful, while the Europe claim is faithful because England is in Europe.Both examples report coverage 4.Source: D Detailed Experiment Settings · p. 17
  • The experiments use 12-layer Transformers with hidden size 768, filter size 3072, and 12 attention heads; the LSTM uses hidden size 256 and memory size 1024.BERT-to-BERT and BERT-to-LSTM use the multilingual uncased BERT-BASE checkpoint; BERT-to-BERT shares encoder and decoder parameters.Source: D Detailed Experiment Settings · p. 17
  • Training uses Adam, validation-loss early stopping, Monte Carlo sample count K=8, and beam size 8 for decoding.LSTM input dropout is 0.5 and memory RNN-dropout is 0.2; attention dropout is 0.2 for WikiBio and 0.1 for WebNLG.Source: D Detailed Experiment Settings · p. 17
Loading 1910.08684v3…