Source-linked AI summary
Reducing Quantity Hallucinations in Abstractive Summarization
Zheng Zhao, Shay B. Cohen, Bonnie Webber
TL;DR
Abstractive summarizers can hallucinate unsupported facts, while avoiding hallucinations through only general language would reduce informativeness. HERMAN verifies quantity entities in beam candidates and re-ranks summaries toward source-supported quantities, yielding higher ROUGE Precision and F1 and preliminary human preference. Its evaluation is limited by the XSum dataset, where some summary facts cannot be verified from the article.
Problem
Abstractive summarization systems generate fluent summaries containing unsupported hallucinated facts, including quantities that are important but may go unnoticed when incorrect.
Source: §1 · p. 2Method
HERMAN recognizes quantity entities, verifies their factual consistency with the source, and re-ranks beam candidate summaries whose quantities are supported.
Source: §1 · p. 2Results
Takeaways & Limitations
Quantity-focused verification can improve summary factuality while still encouraging the inclusion of quantity entities.
Source: §7 · p. 9HERMAN is limited by XSum cases where summary facts sometimes cannot be verified because the summary is simply the article’s first sentence.
Source: §7 · p. 9
Abstract
from arXiv · showhide
It is well-known that abstractive summaries are subject to hallucination---including material that is not supported by the original text. While summaries can be made hallucination-free by limiting them to general phrases, such summaries would fail to be very informative. Alternatively, one can try to avoid hallucinations by verifying that any specific entities in the summary appear in the original text in a similar context. This is the approach taken by our system, Herman. The system learns to recognize and verify quantity entities (dates, numbers, sums of money, etc.) in a beam-worth of abstractive summaries produced by state-of-the-art models, in order to up-rank those summaries whose quantity terms are supported by the original text. Experimental results demonstrate that the ROUGE scores of such up-ranked summaries have a higher Precision than summaries that have not been up-ranked, without a comparable loss in Recall, resulting in higher F$_1$. Preliminary human evaluation of up-ranked vs. original summaries shows people's preference for the former.
1 Introduction
Abstractive summarization can produce fluent but factually unsupported content, especially hallucinated quantities. HERMAN addresses this problem by verifying quantity consistency and re-ranking candidate summaries toward supported content.
- Examples in Table 1 show summaries transferring quantities from related source events, producing hallucinations such as attributing deaths and injuries from an earlier eruption to a new one.Source: Table 1 · p. 1
- Up to 30% of abstractive summaries contain hallucinated facts, raising concerns about their usefulness when summaries should combine general and specific information.Source: §1 · p. 2
- The approach is motivated by a broader trade-off: restricting summaries to general phrases can avoid hallucinations but would reduce informativeness.Source: §1 · p. 2
- HERMAN targets hallucinated dates, times, percentages, monetary values, measurements, ordinals, and cardinal numbers because quantities matter for factual consistency and may be difficult for readers to notice when wrong.Source: §1 · p. 2
- HERMAN learns to recognize quantities and verify their consistency with the source, then re-ranks beam candidates whose quantities are supported by that source.Source: §1 · p. 2
- Up-ranked summaries have higher ROUGE Precision and F1 than originals, while preliminary human evaluation finds them preferable.Source: §1 · p. 2
2 Related Work
Prior work documents frequent factual inconsistencies in abstractive summaries and develops methods for evaluating or reducing them. These studies also show that standard ROUGE scores are insufficient indicators of factual correctness.
- Around 30% of outputs in one study and 25% of summaries in another were reported to contain factual errors or inconsistencies.Source: §2 · p. 2
- ROUGE scores do not correlate with factual correctness, so ROUGE-based evaluation alone is insufficient for assessing summary factuality.Source: §2 · p. 2
- Kryscinski et al. (2019b) used weak supervision to train a model that evaluates factual consistency by transforming source-document claims and assigning binary labels.Source: §2 · p. 2
- Related work defines factual accuracy as Precision between source and generated claims represented as subject-relation-object triplets, and releases data for relation classification and fact extraction.Source: §2 · p. 3
- Other studies explored architectural and training or inference methods for factual inconsistency, while out-of-the-box NLI models transferred poorly to factual-correctness evaluation.Source: §2 · p. 3
3 Methodology
HERMAN verifies quantity entities in candidate abstractive summaries against the source, then uses those verification predictions to rerank beam-search outputs. Its model combines token-level labels, summary-level verification, and quantity-entity indicators to reduce hallucinated quantities.
Verification Model
- The verifier predicts whether summary tokens are verified, unverified, or other while also identifying quantity entities with BIO-formatted labels.The model produces a tag sequence Y and a binary summary label z, using quantity-location labels M to focus recognition on relevant entities.Source: §3 · p. 3, §3.1 · p. 4, §3.2 · p. 5
Dataset Generation
- HERMAN generates synthetic weakly supervised training data with verified summaries, quantity substitutions, token-level labels, and summary-level verification labels.Original summaries receive VERIFIED labels; quantity entities are replaced with same-type entities from the article to create UNVERIFIED examples.Source: §3 · p. 3, §3.1 · p. 4
Verification Model
- The verification architecture encodes the article, applies attention while processing the summary, and uses a CRF for token tags plus an MLP for the summary-level label.The same summary context vectors feed both the CRF tagger and binary classifier.Source: §3.2 · pp. 4–5, Figure 1 · p. 4
Training and Inference
- Training combines local token-level and global summary-level verification objectives, with α controlling the balance between their losses.At inference, the CRF uses Viterbi decoding to obtain the most probable tag sequence.Source: §3.3 · p. 5
- HERMAN scores k beam-search candidate summaries and reranks them using global or local verification to reduce hallucinated quantities.The approach is document-level, whereas the related Falke et al. (2019) method operates at the sentence level.Source: §3.4 · p. 5
4 Dataset
The experiments use XSum, a BBC-article dataset with professionally written one-sentence abstractive summaries and uneven quantity-entity coverage. HERMAN's training data is a filtered subset created by discarding instances that could not produce an UNVERIFIED summary.
XSum dataset
Quantity distribution
- 55% of XSum instances contain at least one quantity, with dates appearing in almost 30% of summaries and quantity entities in only 1%.Source: §4 · p. 5, Table 3 · p. 6
HERMAN dataset
- HERMAN's dataset contains 190,370 training, 10,594 validation, and 10,592 test instances after discarding unperturbable examples.Documents are truncated to 400 tokens and summaries to 90 tokens.Source: §4 · p. 6
5 Experiments
The experiments evaluate HERMAN with GloVe and BERT-based word-embedding initialization on the test set. They also specify shared model and optimization settings for these experiments.
Implementation
- The models use 256-dimensional hidden states, 100-dimensional word embeddings, and a 50k-word vocabulary.Source: §5 · p. 6
Optimization
- Training uses Adam with learning rate 0.001 and gradient clipping with a maximum gradient norm.Source: §5 · p. 6
Experimental evaluation
- Tables 4 and 5 report HERMAN test-set results using GloVe and BERT word embeddings, respectively.Source: Table 4 · p. 6, Table 5 · p. 6
6 Results
HERMAN re-ranks beam candidates using quantity-verification scores, improving ROUGE Precision and F1 despite lower Recall, while preliminary human evaluation favors the re-ranked summaries.
Verification Model
- 80.23 accuracy and 81.6 F1 are obtained with BERT, versus 80.12 accuracy and 80.94 F1 with GloVe, without a significant performance difference.Because BERT triples training time, the authors do not consider it justified.Source: §6 · p. 6
Automatic Evaluation
- HERMAN-GLOBAL achieves the best ROUGE-1/2/L Precision and F1 on XSum despite the strong shortest-summary baseline.The comparison covers BART, BERTSUM, and TCONVS2S; Table 6 reports the automatic evaluation setup and metrics.Source: §6 · p. 7, Table 6 · p. 7
Quantity Analysis
- HERMAN-GLOBAL increases the number of summaries containing quantities, while its ROUGE performance indicates that included quantity entities are also correct.It achieves the highest average quantity count for BERTSUM and TCONVS2S and is second for BART, close behind the overlap baseline.Source: §6 · p. 7
- TCONVS2S shows a 26% increase in summaries containing at least one quantity after re-ranking, while some quantity types decrease where data are underrepresented.Date or time quantities increase across systems, but percent in BERTSUM and quantity in TCONVS2S decrease slightly.Source: §6 · pp. 7–8, Table 7 · p. 8
Human Evaluation
- In preliminary human evaluation, subjects preferred the re-ranked summary in 12 of 19 unanimous trials and in 13 of 21 trials judged at least somewhat more faithful.All three subjects agreed with the authors in the 19 unanimous cases; some trials still favored the original or judged both summaries unfaithful.Source: §6 · pp. 8–9, Table 8 · p. 9
7 Conclusions
HERMAN recognizes and verifies quantities in abstractive summaries, then re-ranks candidates to reduce hallucinated quantities. Up-ranked summaries achieve higher ROUGE Precision and F1, while human evaluators find them more faithful.
7 Conclusions
- HERMAN recognizes and verifies quantities in generated summaries before re-ranking candidates supported by the source text.The approach addresses hallucinated quantities in abstractive summarization systems.Source: §7 · p. 9
- Higher ROUGE Precision and F1 for up-ranked summaries indicate fewer hallucinated quantities while retaining quantity inclusion.Human evaluation also found up-ranked summaries more faithful than directly generated summaries.Source: §7 · p. 9
- The approach is limited on XSum when factual information in a summary cannot be verified because the summary is simply the article’s first sentence.The authors propose incorporating verification into training and inference to improve factual correctness.Source: §7 · p. 9
A Supplementary Material
The appendix reports HERMAN’s training configuration, model size, embedding choices, and beam settings for the summarization systems supplying candidates.
A Supplementary Material
- The appendix provides HERMAN training details and experiment settings in addition to the main paper’s description.Source: A Supplementary Material · p. 13
- HERMAN has 19,424,661 parameters and requires approximately 1 hour per epoch with GloVe initialization or 3 hours with pre-trained base-uncased BERT.These timings use a single GeForce GTX 1080 Ti GPU with batch size 32.Source: A Supplementary Material · p. 13
- TCONVS2S, BERTSUM, and BART provide the summary beams that HERMAN re-ranks, using beam sizes 10, 5, and 6, respectively.BERTSUM uses the BERTSUMEXTABS abstractive variant selected for XSum performance.Source: A Supplementary Material · p. 13