PaperlayerView exact revision on arXiv

Source-linked AI summary

FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation

Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi

arXiv:2305.14251v2cs.CLcs.AIcs.LG

TL;DR

Long-form LM generations mix supported and unsupported information, making binary factuality judgments inadequate and human validation costly. FACTSCORE decomposes generations into atomic facts and evaluates their support, combining human analysis with an automated retrieval-and-language-model estimator. The study reports substantial factual errors in commercial LMs and uses the estimator for large-scale comparisons, while noting that broader factuality requires attention to recall and abstention.

  • Problem

    Long-form LM generations contain mixtures of supported and unsupported information, making binary judgments inadequate while validating every piece is time-consuming and costly.

  • Method

    FACTSCORE decomposes generations into atomic facts and validates each against a knowledge source, with an estimator combining retrieval and language models.

  • Results

    Commercial LMs show substantial factual errors, while the automated estimator closely approximates human FACTSCORE and supports large-scale LM evaluation.

  • Takeaways & Limitations

    FACTSCORE provides a fine-grained factual-precision measure and enables automated evaluation at scale without manual human effort.

  • Takeaways & Limitations

    FACTSCORE measures factual precision but not factual recall, abstention frequency, or the average number of atomic facts, motivating a more holistic evaluation.

Abstract

from arXiv · show

Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate, and (2) human evaluation is time-consuming and costly. In this paper, we introduce FACTSCORE, a new evaluation that breaks a generation into a series of atomic facts and computes the percentage of atomic facts supported by a reliable knowledge source. We conduct an extensive human evaluation to obtain FACTSCOREs of people biographies generated by several state-of-the-art commercial LMs -- InstructGPT, ChatGPT, and the retrieval-augmented PerplexityAI -- and report new analysis demonstrating the need for such a fine-grained score (e.g., ChatGPT only achieves 58%). Since human evaluation is costly, we also introduce an automated model that estimates FACTSCORE using retrieval and a strong language model, with less than a 2% error rate. Finally, we use this automated metric to evaluate 6,500 generations from a new set of 13 recent LMs that would have cost $26K if evaluated by humans, with various findings: GPT-4 and ChatGPT are more factual than public models, and Vicuna and Alpaca are some of the best public models. FACTSCORE is available for public use via `pip install factscore`.

1 Introduction

FACTSCORE evaluates long-form LM factual precision by checking atomic facts individually, addressing mixed-supported generations and costly human validation. Human and automated evaluations reveal substantial factual errors while enabling large-scale assessment.

  • Long-form generations mix true and false information, making binary judgments inadequate, while validating every piece is costly.
  • FACTSCORE measures the percentage of atomic facts supported by a knowledge source after decomposing generations into short, single-information statements.
  • 42%, 58%, and 71% are the FACTSCOREs of InstructGPT, ChatGPT, and search-augmented PerplexityAI, respectively.
  • < 2% error is achieved by an automated estimator using retrieval and strong language models, enabling evaluation of new LMs without manual human effort.
  • FACTSCORE and annotated data are open-sourced for public use via pip install factscore.
  • Future work will extend FACTSCORE to broader generations and further improve the estimator.

2 Related Work

Related work addresses factual precision, fact verification, and model-based evaluation across task-specific and long-form generation settings. FACTSCORE builds on atomic decomposition and automated verification approaches in these areas.

  • Prior factual-precision research mainly studies task-specific models, short-answer question answering, or proxy metrics for longer generations.
  • Fact-verification research checks claims against large knowledge sources, with many methods assuming a single atomic claim.
  • Some verification methods decompose longer sentences or texts into atomic facts, an approach FACTSCORE takes inspiration from.
  • Model-based evaluation has been used to define automated scores, including summarization methods that assess consistency with source documents using QA or NLI.

3 FACTSCORE: Evaluating Factual Precision of Long-form Text Generation

FACTSCORE evaluates long-form factual precision by decomposing generations into atomic facts and checking support from a specified knowledge source. Human evaluation on biographies shows substantial errors, with precision varying by model, entity rarity, generation position, and error type.

  • 3 FACTSCORE: Evaluating Factual Precision of Long-form Text Generation: FACTSCORE evaluates factual precision by decomposing a generation into atomic facts and validating each against a given knowledge source.An atomic fact is a short sentence conveying one piece of information; each fact receives a binary support label.
  • 3.1 Definition: FACTSCORE treats support as relative to a trusted knowledge source and assumes atomic facts have equal importance and nonconflicting source information.The evaluation also assumes whether support holds is undebatable.
  • 3.3 Data: The study evaluates biographies against Wikipedia because biographies are objective, specific, broadly varied, and reasonably covered by a self-consistent source.The data collection samples 183 people entities from Wikidata with corresponding Wikipedia pages.
  • 3.4 Results: 42.5%, 58.3%, and 71.5% are the FACTSCOREs of InstructGPT, ChatGPT, and PerplexityAI, respectively.PerplexityAI uses a commercial search engine, yet its score remains below perfect copying from the correct Wikipedia page.
  • 3.4 Results: ChatGPT and PerplexityAI often abstain from answering, whereas InstructGPT rarely abstains.Abstention presumably improves the factual precision of the two models that use it more often.
  • 3.4 Results: PerplexityAI frequently produces irrelevant facts by copying search results, and its citations have little correlation with factual precision.36.0% of supported sentences and 37.6% of unsupported sentences have citations.
  • 3.4 Results: FACTSCORE decreases as entities become rarer, including a 50% relative drop for PerplexityAI at the atomic level and a 64% drop at the sentence level.The decline occurs across all evaluated language models, contrasting with reported retrieval robustness on short question answering.
  • 3.4 Results: FACTSCORE is significantly worse for facts appearing later in generations, partly because earlier information is more frequent in pretraining data and errors can propagate.This pattern suggests that short-answer evaluation may miss factual-precision problems in later text.

4 Estimating FACTSCORE for Automatic Evaluation

The paper develops an automated FACTSCORE estimator that decomposes generations into atomic facts, validates them against retrieved knowledge, and enables large-scale factuality evaluation without human annotation. Retrieval improves estimation, while combining retrieval-based language-model judgments with nonparametric probabilities improves robustness and preserves model rankings.

  • Motivation: Human evaluation costs $4 per generation, motivating an estimator that evaluates factual precision at scale.Each generation contains 26–41 atomic facts, making manual validation time-consuming.
  • Model: The estimator decomposes generations into atomic facts and validates each against a knowledge source using zero-shot language-model judgments.It compares conditional probabilities of True and False, or generated labels when logits are unavailable.
  • Model: Retrieve→LM supplies retrieved passages to the evaluator, while NP scores masked-token likelihoods and predicts factuality by thresholding.Retrieve→LM concatenates retrieved passages, the atomic fact, and a True-or-False prompt.
  • Model: Retrieve→LM + NP assigns Supported only when both ensemble components agree.This design reduces errors from language-model bias and distracting passages.
  • Results: Retrieval consistently outperforms no-context evaluation, while Retrieve→LM alone can overestimate FACTSCORE by up to 17%.For InstructGPT and ChatGPT, ensembling Retrieve→LM with NP reduces error substantially.
  • Results: The best estimator depends on the evaluated model: LLAMA+NP for InstructGPT and ChatGPT, and ChatGPT for PerplexityAI.Both evaluators preserve rankings, and their scores correlate at Pearson’s r = 0.99 across more than ten subjects.
  • Evaluation of New LMs: The estimator evaluates thousands of generations without human effort, revealing factuality differences across newly released language models.The study evaluates 10 recent models alongside earlier systems and human biographies.
  • Evaluation of New LMs: GPT-4 and ChatGPT are more factual than public models, while factual precision generally increases with model size within Alpaca and Vicuna.Among similarly sized public models, Alpaca and Vicuna are near 40%, compared with 30% for MPT-Chat and 17% for StableLM among 7B models.

5 Conclusion and Future Work

The paper introduces FACTSCORE for fine-grained factuality evaluation, first validating it through human assessments and then approximating it automatically. The estimator enables scalable evaluation, while the authors identify recall, estimator improvement, and factuality correction as future directions.

  • Conclusion: FACTSCORE computes the fraction of atomic facts in long-form LM generations that are supported by a knowledge source.The metric breaks each generation into atomic facts before validation.
  • Conclusion: Human evaluation finds substantial factual errors in commercial models, including a 58% FACTSCORE for ChatGPT.The evaluated systems are InstructGPT, ChatGPT, and search-augmented PerplexityAI.
  • Conclusion: The proposed retrieval-based estimator approximates human FACTSCOREs closely and supports automatic evaluation of newly released models.The paper applies it to 12 recent LMs in a case study that would have cost $65K to evaluate manually.
  • Future Work: Future work includes measuring factual recall, improving estimator accuracy, and using FACTSCORE to correct model generations.These directions extend evaluation beyond precision toward coverage and intervention.

Limitations

FACTSCORE is scoped mainly to objective, source-supported factual claims, and its estimator and evaluation dimensions remain incomplete. The paper identifies boundaries involving domain coverage, nuanced or conflicting evidence, human-written deception, and recall.

  • Scope: FACTSCORE experiments focus on people biographies evaluated against English Wikipedia, while broader domains receive only a proof of concept and are left for future study.The paper suggests news articles or scientific literature as alternative knowledge sources.
  • Assumptions: FACTSCORE is not applicable when facts are nuanced, open-ended, debatable, or drawn from sources whose texts frequently conflict.The paper also questions its suitability for nuanced human-written text containing intentional or implicit deception.
  • Estimator: The estimator closely approximates human judgments and rankings but is imperfect for individual judgments, with the best variant depending on human-likeness and linguistic complexity.Future work should examine how generation distributions affect estimator performance.
  • Beyond factual precision: FACTSCORE measures factual precision but not factual recall, so abstaining frequently or generating fewer facts can produce a higher score without being desirable.The authors leave holistic factuality evaluation for future work and recommend reporting abstention and average atomic-fact counts alongside FACTSCORE.

A.4 Examples in annotated data

The annotated data assigns atomic facts labels based on whether Wikipedia supports them, does not support them, or treats them as irrelevant. The examples also illustrate irrelevant content in PerplexityAI generations.

  • Annotation labels: Annotated atomic facts receive Supported, Not-supported, or Irrelevant labels according to Wikipedia evidence and relevance to the input prompt.Irrelevant facts include expansions that depend on earlier facts labeled Not-supported.
  • Examples: PerplexityAI produced irrelevant text in part because it often copied search results even when they were largely unrelated to the input prompt.The passage links this behavior to reported errors and irrelevance in generative search engines.

A.5 Qualitative Analysis

Disagreements often arise when evidence supports multiple interpretations, while Wikipedia coverage can understate support for rare people. The study reports high agreement overall and broad coverage for important information.

  • Disagreement cases: 70% of annotator disagreements reflected inherent debatability about whether a fact was supported by the knowledge source.Such cases involve multiple interpretations or uncertain inferences from evidence.
  • Disagreement cases: The Gerhard Fischer example shows how a widely associated occupation can conflict with a more precise account distinguishing invention from commercialization.The passage says Fischer commercialized the metal detector rather than inventing it.
  • Disagreement cases: The Chadwick Boseman example shows how a profession may be unsupported even when a related activity, producing one music video, is documented.This creates disagreement over whether the broader profession claim is supported.
  • Agreement: Annotator agreement was 91%, and the authors consider these ambiguous cases rare in the people-biography domain.They expect such cases to be more frequent in other domains.
  • Wikipedia coverage: Wikipedia may omit true facts about rare entities, but it had high coverage and mentioned most important information found from other web sources.A web check found 3 of 30 unsupported ChatGPT facts were supported elsewhere.

B.1 Implementation details

The estimator combines specified language models, nonparametric probabilities, and retrieved Wikipedia passages, while the paper also compares against Self-check LM. Figure 4 illustrates that F1MICRO and Error Rate can rank evaluators differently.

  • Evaluator models: The evaluation uses LLaMA 65B, Inst-LLaMA, and ChatGPT as evaluator language models, with BM25-based nonparametric probabilities and a threshold of 0.3.LLaMA 65B and Inst-LLaMA represent open models, while ChatGPT represents the commercial model.
  • Passage retrieval: Retrieval uses the large GTR dense retriever, restricts passages to the topic entity’s page, and sets k = 5 using English Wikipedia from 04/01/2023.The estimator was not sensitive to the retrieval system choice in reported ablations.
  • Baseline: Self-check LM validates atomic facts against multiple samples from the subject model and aggregates Supported judgments by majority vote.The baseline assumes the subject model is available during evaluation and produces nondeterministic outputs.
  • Metric comparison: Figure 4 shows that F1MICRO and Error Rate can rank two evaluators differently: Evaluator A leads on F1MICRO, whereas Evaluator B leads on ER.The figure is a warning that metric choice can change evaluator comparisons.

B.2 Segment-level vs. system-level evaluation

The paper separates evaluator quality at the individual-fact level from accuracy of aggregated FACTSCORE estimates, reporting both because they can diverge. Retrieval improves individual-fact validation, while nonparametric probabilities improve Retrieve→LM performance.

  • B.2 Segment-level vs. system-level evaluation: F1MICRO evaluates individual atomic-fact validation, whereas Error Rate evaluates aggregated FACTSCORE estimation.F1MICRO assumes oracle atomic facts and focuses on identifying Not-supported facts; Error Rate captures systematic over- or underestimation.
  • B.2 Segment-level vs. system-level evaluation: A high F1MICRO can coexist with a worse Error Rate when an evaluator consistently overestimates or underestimates factual precision.Conversely, an evaluator with lower individual-decision quality can achieve better aggregated estimation if it is not biased in either direction.
  • B.2 Segment-level vs. system-level evaluation: Self-check LM outperforms no-context LM by 4–11% on F1MICRO, but both significantly underperform retrieval-based methods.The passage contrasts these results with prior findings on datasets containing more frequent entities.
  • B.2 Segment-level vs. system-level evaluation: Adding NP improves Retrieve→LM by 2–9% on F1MICRO.The paper attributes this improvement to greater robustness against LM bias and distracting retrieved passages.
  • B.2 Segment-level vs. system-level evaluation: ChatGPT outperforms Inst-LLAMA 7B and LLAMA 65B as LMEVAL models, while retrieval remains critical across all evaluated models.NP helps LLAMA-based models but not ChatGPT, which is described as less affected by incorrect priors or distracting passages.
  • B.2 Segment-level vs. system-level evaluation: ChatGPT validates individual atomic facts better than LLAMA+NP, but its tendency to label Not-supported facts as Supported overestimates aggregated FACTSCORE.LLAMA+NP is not biased toward either overestimation or underestimation, producing better aggregated factual estimation in the reported comparison.

B.3 Ablations

The ablations examine prompting, retrieval, retrieval-system choice, estimator errors, sampling variance, and feasibility beyond biographies. Retrieval and True/False prompting are especially important, while broader-domain validation remains limited.

  • B.3 Ablations: True/False prompting significantly outperforms question-answer prompting across all methods.Generated questions are often vague or ambiguous, causing supported facts to be incorrectly labeled Not-supported when expected and obtained answers differ.
  • B.3 Ablations: All tested retrieval systems are equally good, so Retrieve→LM is not sensitive to whether it uses BM25, GTR Large, or GTR xLarge.The comparison uses Retrieve→LM with Inst-LLAMA 7B and evaluates F1MICRO.
  • B.3 Ablations: 70% of Retrieve→LM errors arise because retrieved passages provide no direct evidence for support or contradiction.The paper notes that these cases can require reading an entire page rather than a single passage.
  • B.3 Ablations: FACTSCORE estimates have low variance across random subsets and preserve rankings among 13 subjects in most cases.The analysis samples N={40,100,200} items from 500 and repeats the procedure M=100 times.
  • B.3 Ablations: The paper mainly evaluates people biographies using Wikipedia, leaving generalization to other prompts and domains for future work.The authors describe the NLP experiment as very small-scale and encourage broader evaluation at scale.
  • B.3 Ablations: A small NLP-domain study produced an Error Rate of 7.41, with human FACTSCORE 66.20 and model FACTSCORE 73.61.The study used 10 prompts about NLP papers and the ACL Anthology as its knowledge source.

C.3 Results

The editing experiments show that language models can make non-trivial corrections, with retrieval and atomic-fact labels producing progressively stronger performance. PerplexityAI outputs are hardest to edit.

  • C.3 Results: All editing models outperform trivial lower-bound baselines, including no-context ChatGPT editing its own outputs.ChatGPT achieves 6.8 EditCorr compared with 0.1 for a random-noise editor baseline.
  • C.3 Results: Retrieval boosts ChatGPT editing from 6.8 to 16.8 ErrCorr and from 4.0 to 9.5 SimAl.The paper hypothesizes that Wikipedia supplies information unavailable in ChatGPT’s internal knowledge for fine-grained editing.
  • C.3 Results: Atomic fact labels improve ChatGPT’s ErrCorr from 16.8 to 28.3, SimAl from 9.5 to 19.3, and ErrLoc from 43.9 to 63.5.The labels also reduce copying the input verbatim or claiming it has no errors from 37.3% to 3.9%.
  • C.3 Results: Editing success is highest for InstructGPT, followed by ChatGPT, and lowest for PerplexityAI.The paper hypothesizes that PerplexityAI’s search augmentation makes its remaining errors more subtle.
Loading 2305.14251v2…