Source-linked AI summary
Assessing The Factual Accuracy of Generated Text
Ben Goodrich, Vinay Rao, Mohammad Saleh, Peter J Liu
TL;DR
Generated-text evaluation often relies on local overlap metrics that do not directly measure factual accuracy. The paper proposes model-based factuality metrics, releases Wikipedia–Wikidata resources for fact extraction, and reports better agreement with human judgments than traditional metrics such as ROUGE.
Problem
Existing evaluation commonly uses ROUGE, BLEU, and perplexity, which measure locally constrained n-gram overlap rather than factual accuracy.
Method
The paper trains relation classifiers and Transformer-based end-to-end fact-extraction models using a large dataset created from Wikipedia and Wikidata, then evaluates factuality metrics on summarization.
Results
Model-based metrics are better correlated with human judgments of factual accuracy than traditional metrics like ROUGE on the evaluated summarization task.
Takeaways & Limitations
The proposed metric and released fact-extraction resources support standardized comparison of factual accuracy in generated text.
Takeaways & Limitations
The dataset uses only English Wikipedia sentences and Wikidata facts, biasing models toward Wikipedia’s neutral style and popular Wikidata relations.
Abstract
from arXiv · showhide
We propose a model-based metric to estimate the factual accuracy of generated text that is complementary to typical scoring schemes like ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BLEU (Bilingual Evaluation Understudy). We introduce and release a new large-scale dataset based on Wikipedia and Wikidata to train relation classifiers and end-to-end fact extraction models. The end-to-end models are shown to be able to extract complete sets of facts from datasets with full pages of text. We then analyse multiple models that estimate factual accuracy on a Wikipedia text summarization task, and show their efficacy compared to ROUGE and other model-free variants by conducting a human evaluation study.
1 INTRODUCTION
The paper proposes model-based metrics for factual accuracy in generated text, complementing overlap metrics, and develops fact-extraction resources and models for evaluating summarization.
- Motivation: Model-based metrics are proposed to evaluate factual accuracy alongside ROUGE, BLEU, and perplexity, which measure locally constrained n-gram overlap.The motivating distinction is between factual consistency and surface overlap.
- Metric: Factual accuracy is defined as precision between filtered fact tuples from the ground-truth and generated summaries.Filtering retains claims in generated text that can be verified or refuted by statements in the ground-truth text.
- Metric: A birth-year substitution can yield factual accuracy 0 while ROUGE-1 is 0.83, illustrating that n-gram overlap can remain high despite an incorrect claim.The example compares a ground-truth birth year of 1963 with a generated birth year of 1961.
- Resources: A large dataset based on Wikipedia and Wikidata, released with code and model weights, supports training relation classifiers and fact-extraction models.The dataset is introduced for fact tuple extraction and spans more relations and domains than previously used datasets, according to the paper.
- Fact extraction: A Transformer-based end-to-end model extracts structured relation tuples from full pages in one pass without separate named-entity recognition, coreference resolution, and relation-classification stages.This design avoids splitting extraction into multiple steps.
- Evaluation: Model-based metrics correlate better with human judgments of factual accuracy than traditional metrics such as ROUGE on the evaluated summarization task.The experiments compare proposed metrics against human evaluation and model-free metrics.
2 RELATED WORK AND MOTIVATION
Existing text-generation metrics largely measure n-gram overlap, while factual inaccuracies remain common in abstractive summaries. The paper motivates model-based factual-accuracy metrics and an end-to-end extraction approach using a larger, richer dataset.
- Existing evaluation: BLEU, ROUGE, METEOR, and related metrics evaluate text generation primarily through overlap-based or intrinsic content measures.These metrics are distinguished from extrinsic checks such as whether a summarized person's birth date is correct.
- Fact extraction: OpenIE extracts relation tuples without a fixed schema, making its outputs difficult to compare and unsuitable for logical reasoning.Its relations typically consist of the text linking two entities and do not leverage a knowledge base.
- Dataset and models: The proposed dataset is bigger, more diverse, and contains more relation types than prior data, while also supporting article-level training.The paper uses it to train models that extract facts directly from text.
- Dataset and models: The sequence-to-sequence approach outputs fact tuples in one pass without feature engineering, unlike multistage or hand-crafted end-to-end approaches.The motivation includes avoiding compounding errors across two-step extraction pipelines.
- Motivation: Approximately 17% of summaries in a large-scale experiment contained factual inaccuracies, including incorrect dates and locations.The paper contrasts these errors with the limited penalties imposed by cross-entropy or n-gram-based evaluation.
3 DATASET
The paper constructs a fact-extraction dataset through distant supervision from English Wikipedia and Wikidata. Wikipedia entities are paired with Wikidata relations to create positive and no-relation training examples.
- Dataset construction: The dataset is built entirely from the English Wikipedia corpus and the Wikidata knowledge base using distant supervision.The procedure follows an approach similar to Mintz et al.
- Dataset construction: For each Wikipedia article, the article topic is treated as the subject and other recognized entities as candidate objects.Named-entity recognition and coreference resolution are run before constructing entity pairs.
- Labels: Entity pairs linked by a Wikidata relation become positive examples, while pairs without such a relation receive the no-relation label r_0.Positive examples contain the corresponding subject–relation–object triplet; negatives use r_0.
4 MODEL-BASED METRICS
The paper defines factual accuracy by extracting structured relation tuples from ground-truth and generated summaries, using multistage classifiers, a binary relation classifier, or an end-to-end Transformer model. These approaches compare extracted facts while restricting evaluation to claims that can be assessed from the reference summary.
- Metric framework: Fact extraction converts an input text X into claims represented as relation tuples, which are then used to calculate factual accuracy.The metric operates on extracted facts from the ground-truth summary T and generated summary G.
- Relation classifier: The standard pipeline first recognizes named entities and resolves coreference, then classifies relations for entity pairs.The classifier evaluates sentences containing both entities, using SUBJ and OBJ markers as hints.
- Relation classifier: The relation classifier predicts a fixed-schema relation r_k for each entity pair and extracts tuples from both T and G.These tuples are compared using the factual-accuracy definition.
- End-to-end model: The end-to-end Transformer outputs sequences of structured fact tuples from text of sentence, paragraph, or article length.It uses a fixed Wikidata-based schema and avoids splitting extraction into named-entity recognition, coreference resolution, and relation-classification stages.
- End-to-end model: The end-to-end model is trained to emit subject, relation, and object fields separated into multiple fact tuples.For example, the Wikidata property P69 represents “educated at.”
- Binary relation classifier: The binary relation classifier predicts whether each entity pair has any relation, allowing multiple relation types between the same entities.It extracts tuples from both summaries and compares predicted labels only for entity pairs found in both.
5 MODEL-FREE METRICS
Model-free metrics provide interpretable alternatives to model-based evaluation, but they do not directly model factual relations. The paper uses ROUGE and OpenIE-based tuple precision as comparison metrics.
- Model-free evaluation: Model-free metrics are not susceptible to changes in training data and may be easier to interpret than model-based metrics.The paper presents them as comparison baselines for factual-accuracy evaluation.
- ROUGE: ROUGE is used to judge generated-text quality and correlates well with human judgments of overall linguistic quality.This correlation concerns overall linguistic quality rather than a direct factual-relation assessment.
- OpenIE: OpenIE extracts relation tuples without a specified schema, after which the paper computes precision between the extracted sets.The metric applies the same precision-style comparison used in the factual-accuracy definition.
6 MODEL EXPERIMENTS
The experiments compare fixed-schema relation classifiers with an end-to-end Transformer fact extractor. The end-to-end model produces structured fact tuples in one pass and achieves competitive performance against classifiers while supporting a simpler routine.
- The end-to-end model extracts entities and relations in one pass and can process text ranging from sentences to full articles.Its input is text of any length prefixed with a subject entity, and its Transformer encoder-decoder outputs structured facts.
- The end-to-end model’s exact-match fact-tuple evaluation covers entity identification, coreference resolution, and structured-output prediction.Precision and recall require predicted fact tuples to match ground-truth tuples exactly.
- The experiments report precision, recall, and F1 for classifiers and end-to-end models, including results on the ten most frequent relations.Table 3 also distinguishes a binary classifier and an E2E-Reduced variant that filters sentences without detected entities.
- The end-to-end model has competitive performance against relation classifiers while using a simple training and inference routine.
- The study evaluates fixed-schema relation classifiers and an end-to-end Transformer model for structured fact extraction.The classifier and end-to-end results are reported separately on the proposed dataset.
7 ERROR ANALYSIS OF MODEL PREDICTIONS
The error analysis examines false positives created by distant supervision, whose Wikidata-derived labels may not be entailed by the source sentence. Human evaluators assess whether extracted tuples are actually implied by Wikipedia text.
- Distant supervision labels entity pairs from Wikidata even when the containing sentence may not entail the assigned relation.This weak-labeling process can produce inaccurate estimates of true-positive rates for fact extraction models.
- Human evaluators judged facts marked false-positive by distant supervision as true only when both evaluators agreed the source text implied them.The evaluation used facts extracted from a random set of 30 Wikipedia articles.
- The analysis suggests that end-to-end models could benefit from a better labeling scheme.
- The error analysis reports the percentage of true facts inaccurately labeled wrong by the distant supervisor for the best end-to-end model and classifier.The end-to-end model is identified as the best Section 4.2 model, while Transformer-Sigmoid is the best Section 4.1 classifier.
8 EVALUATION OF 𝑓𝑎𝑐𝑡𝑎𝑐𝑐AS A METRIC
The paper evaluates factual-accuracy metrics against human judgments on generated Wikipedia summaries. Its end-to-end model has the best correlation on both an Actors subset and a random sample of Wikipedia categories, suggesting better generalization to generated text.
- The metric evaluation compares generated Wikipedia lead-section summaries with their real lead sections.The summaries are produced using a previously proposed Wikipedia summarization model.
- Human evaluators score only generated claims that are supported or refuted by the ground-truth text, excluding unverifiable claims.This comparison avoids using external knowledge.
- Inter-rater agreement was 0.6897 for the Actors subset and 0.7530 for the random Wikipedia subset.
- The end-to-end model has the best correlation with human factual-accuracy judgments on both the Actors subset and the random Wikipedia subset.The authors interpret this as evidence that it generalizes better to generated text.
9 CONCLUSION
The proposed metric indicates factual accuracy and agrees with human judgment on the evaluated datasets, while the released dataset remains limited in source coverage and linguistic diversity.
- The dataset is limited to English Wikipedia sentences and Wikidata facts, biasing models toward Wikipedia’s neutral style and popular fact types.
- Human evaluators compare ground-truth and generated text before rating factual accuracy on a five-point scale.
- Other writing styles, complex structures, negation, and alternate relation expressions may be difficult for the models to handle.
- The metric indicates generated-text factual accuracy and agrees with human judgment on the evaluated datasets.
- The end-to-end fact extraction model avoids compounding errors across subcomponents used in other fact-extraction pipelines.
A.1 Reproducibility
The authors release code, model weights, and a Wikipedia–Wikidata fact-extraction dataset, alongside examples comparing their structured extractors with OpenIE.
- Code, trained model weights, and a large Wikipedia–Wikidata fact-extraction dataset are released for reproducibility.
- The released implementation supports using end-to-end models as fact extractors and calculating factual accuracy.
- OpenIE produces many triplets that mostly cannot be used for reasoning, illustrating an advantage of structured fact extraction.
- Table 9 compares fact tuples extracted by OpenIE, the end-to-end model, and the classifier.