PaperlayerView exact revision on arXiv

Source-linked AI summary

VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation

Yixiao Song, Yekyung Kim, Mohit Iyyer

arXiv:2406.19276v1cs.CL

TL;DR

Existing long-form factuality metrics assume that claims are verifiable, limiting their suitability for text that mixes verifiable and unverifiable content. VERISCORE evaluates verifiable claims and, across diverse tasks, GPT-4o has the highest average factual support; model performance varies by task and domain.

  • Problem

    Existing metrics assume long-form text can be decomposed into atomic, verifiable claims, although complex assertions may lose context and outputs may mix factual and unverifiable content.

    Source: §1 · p. 1

  • Method

    VERISCORE extracts claims describing a single event or state with necessary modifiers, uses inter-sentence context, and evaluates claims against Google Search evidence.

    Source: §1 · pp. 1–2, §2.1.2 · p. 3

  • Results

    GPT-4o generates the most factually supported text averaged across datasets, and LMs’ VERISCORE scores on different domains do not necessarily correlate well.

    Source: §1 · p. 2, §4.3 · p. 7

  • Takeaways & Limitations

    Comprehensive factuality evaluation requires multiple tasks because model factuality varies across domains.

    Source: §1 · p. 2, §4.3 · p. 7

    For extremely infactual responses, the claim extractor may still extract a small amount of unverifiable claims.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Existing metrics for evaluating the factuality of long-form text, such as FACTSCORE (Min et al., 2023) and SAFE (Wei et al., 2024), decompose an input text into "atomic claims" and verify each against a knowledge base like Wikipedia. These metrics are not suitable for most generation tasks because they assume that every claim is verifiable (i.e., can plausibly be proven true or false). We address this issue with VERISCORE, a metric for diverse long-form generation tasks that contain both verifiable and unverifiable content. VERISCORE can be effectively implemented with either closed or fine-tuned open-weight language models, and human evaluation confirms that VERISCORE's extracted claims are more sensible than those from competing methods across eight different long-form tasks. We use VERISCORE to evaluate generations from 16 different models across multiple long-form tasks and find that while GPT-4o is the best-performing model overall, open-weight models such as Mixtral-8x22 are closing the gap. We show that an LM's VERISCORE on one task (e.g., biography generation) does not necessarily correlate to its VERISCORE on a different task (e.g., long-form QA), highlighting the need for expanding factuality evaluation across tasks with varying fact density.

1 Introduction

Existing factuality metrics assume long-form text can be decomposed into atomic, verifiable claims, an assumption that fails for complex assertions and mixed factual and unverifiable content. VERISCORE extracts claims intended to be verifiable, uses inter-sentence context, and is evaluated across diverse tasks and models.

1 Introduction

  • Atomic-claim metrics assume all text can be decomposed into verifiable claims, but complex assertions may lose critical context and outputs can mix factual and unverifiable content.The authors say this limits FACTSCORE and SAFE to fact-dense, formulaic text such as biographies.Source: §1 · p. 1
  • VERISCORE extracts verifiable claims and uses inter-sentence context; its pipeline retrieves Google Search evidence for each claim and has an LLM verify it.The figure contrasts SAFE’s extraction of unverifiable claims with VERISCORE’s approach.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Human evaluators preferred VERISCORE’s claim extraction over SAFE’s 93% of the time, including for biography generation.Unlike FACTSCORE and SAFE, VERISCORE does not decompose everything and considers inter-sentence context during extraction.Source: §1 · p. 2
  • GPT-4o generates the most factually supported text averaged across eight diverse long-form datasets, while model factuality varies by task and domain.The authors also report that Google Search verification can fail to retrieve relevant documents for complex, lengthy assertions common in long-form QA.Source: §1 · p. 2

2 VERISCORE: an automatic factuality metric

VERISCORE evaluates factuality by extracting claims intended to be verifiable, retrieving search evidence, verifying claims, and scoring responses. Its extraction approach addresses limitations of existing methods by using context and excluding unverifiable content.

2.1.1 Issues with claim extraction in FACTSCORE and SAFE

  • FACTSCORE targets biographical atomic facts, while SAFE adds revision and relevance checks; the authors identify unresolved references, added cost, removed verifiable claims, and limited evaluation as issues.SAFE is evaluated only on FACTSCORE’s biography data; processing 100 claims without parallelization takes 35 minutes with GPT-4, and its revision and relevance prompts cost about $1.7 per 100 claims.Source: §2.1.1 · p. 2

2.1.2 VERISCORE’s extraction approach

  • VERISCORE defines a claim as describing one real-world event or state with necessary modifiers, and uses context to make extracted claims self-contained while excluding unverifiable content.Its human evaluation found that the method addresses unresolved referents, avoids claim revision and removal, and correctly avoids extracting claims from non-factual content.Source: §2.1.2 · p. 3

2.2 Evidence retrieval

  • For each claim, VERISCORE uses the claim as a Google Search query and retrieves up to 10 results as evidence.For QA tasks, the question is prepended to the sliding window; for non-QA paragraphs longer than five sentences, the first sentence is prepended.Source: §2.2 · p. 3

2.3 Claim verification

  • A claim is supported only when all its parts are supported without contradiction; it is contradicted if any part is contradicted, and otherwise may be inconclusive.Because few claims were directly contradicted in the human study, subsequent experiments combine contradicted and inconclusive claims as unsupported.Source: §2.3 · p. 3

2.4 Score calculation

  • VERISCORE uses F1@K to balance factual precision and recall, setting K to the median number of extracted facts among responses in each tested domain.K is the minimum number of factual claims a response must contain to achieve perfect recall; VERISCORE averages response scores within each domain.Source: §2.4 · p. 3

3 Validation of VERISCORE’s claim extraction and verification

The section validates VERISCORE’s extraction and verification through human evaluations and tests of claim density, search results, and automatic verification. Annotators preferred its extracted claims over SAFE’s, and the verification study found substantial human agreement while showing that many claims were not supported.

3 Validation of VERISCORE’s claim extraction and verification

  • VERISCORE’s extraction and verification were first developed with prompted closed LLMs, then open-weight LLMs were fine-tuned on GPT-4 and GPT-4o data to reduce cost.Source: §3 · p. 4

3.1 Human evaluation on claim extraction

  • SAFE’s claims were preferred only 26 times among 360 annotated items, with 19 preferences marginal; VERISCORE was significantly favored even for biographies.Three annotators fully agreed on 99 of 120 data points (Fleiss κ = 0.7662).Source: §3.1 · p. 4
  • Annotators flagged SAFE for extracting subjective or personal content and for over-decomposing text into overlapping claims that can disproportionately affect scores.Source: §3.1 · p. 4

3.2 VERISCORE’s claim extractor only extracts verifiable claims

  • WritingPrompts responses averaged a verifiable-claims-to-sentences ratio of 0.03 despite averaging 34 sentences, while fact-seeking domains had higher ratios.The authors interpret variation across domains as evidence that the extractor distinguishes verifiable from unverifiable content.Source: §3.2 · p. 4

3.3 Human evaluation on claim verification

  • On triple-annotated verification items, 82% had complete annotator agreement and 14% had two annotators in consensus (Fleiss κ = 0.7316).Source: §3.3 · p. 5
  • Only 55% of annotated claims were supported; 15 of 41 fully agreed items were inconclusive, including claims too general to verify or lacking evidence for their component connections.Source: §3.3 · p. 5
  • The first five search results were useful for over 30% of claims, peaking at 35.6% for the first result; the last result’s usefulness fell to 13.3%.Source: §3.3 · p. 5

3.4 Automatic verifier

  • Among the tested models, GPT-4o aligned best with human performance on claim verification, so its data was used to fine-tune an open claim verifier.Source: §3.4 · p. 6

4 Using VERISCORE to benchmark LM factuality

VERISCORE benchmarks factuality across long-form domains using fine-tuned open-weight models for affordable evaluation. Results show differences across models and tasks, while qualitative analyses identify limits in decomposing complex claims and verifying them from search results.

4.1 An open-weight VERISCORE pipeline

  • For affordable factuality evaluation, the fine-tuned VERISCORE pipeline uses open language models; on human-annotated data, Llama3 fine-tuned on GPT-4o data performs best, with verification F1 = 0.841.The few-shot pipeline generated 13,403 training data. In Appendix F, Mistral and GPT-4 extracted claims, with exact match 43.7% and ROUGE-L 0.801.Source: §4.1 · p. 6

4.2 Data domains and studied LMs

  • The benchmark spans six long-form fact-seeking domains and 16 LMs, with prompts ranging from highly fact-dense tasks to moderately factual ShareGPT.It also includes FreshBooks continuations from 20 nonfiction books published between 2023 and 2024, and uses 50 prompts per domain.Source: §4.2 · p. 6, §4 · p. 6

4.3 VERISCORE results

  • Closed models generally score higher, larger models tend to score better, and VERISCORE across dissimilar tasks does not necessarily correlate well.GPT-3.5-turbo-1106 is an exception to the model-family pattern: its shorter responses average 8.56 sentences versus 15.95 for GPT-3.5-turbo-0613, hurting recall; F1@K can penalize short outputs.Source: §4.3 · pp. 6–7, Figure 3 · p. 7
  • GPT-4o significantly outperforms every open LM, while Mixtral-8x22B-Inst-v0.1 is the best open LM in the reported VERISCORE benchmark.The table covers 50 responses per LM per dataset; bracketed values are K.Source: §4 · p. 6, §4.3 · p. 6, Table 3 · p. 7

4.4 Qualitative analysis

  • Decompositional factuality evaluation is limited for non-entity-centric and non-formulaic generations because claims may be long and search results may not support the expertise or reasoning needed.The benchmark discussion notes that complex claims can require extensive logical reasoning beyond snippets.Source: §4.4 · p. 7, §4.4.2 · p. 8

4.4.1 Claim complexity increases outside of entity-centric tasks

  • Claims outside entity-centric tasks are often longer than biography claims, and splitting a long claim may still fail to preserve or verify its full meaning.The sampled FACTSCORE claims average 7 words, compared with 12 words for GPT-4 ELI5 claims; the longest are 18 and 25 words, respectively.Source: §4.4.1 · p. 7

4.4.2 Google Search may be insufficient for complex claims

  • Search snippets may omit a claim’s parts or the connections between them, while some claims require extensive evidence assembled from many documents.The authors conclude that snippet-based reasoning makes it hard to determine whether an unsupported claim is hallucinated.Source: §4.4.2 · p. 8

5 Related work

Prior work detects factual errors at sentence or paragraph level and commonly decomposes long-form text for finer-grained evaluation. Retrieval is often used to supply the world knowledge such evaluation requires.

Factual error detection

  • Sentence-level error detection includes FEVER, FEVEROUS, and AVERITEC, while paragraph-level work tests error detection without locating errors.FEVEROUS and AVERITEC extend FEVER but remain limited to sentence-level facts.Source: §5 · p. 8

Long-form factuality evaluation

  • Long-form factuality work commonly decomposes text into shorter sentences or search queries, supporting error localization and fine-grained estimates of factuality.Retrieval supplies up-to-date world knowledge and can support post hoc citations and iterative improvement of generations’ factuality.Source: §5 · p. 8

6 Conclusion and future work

VERISCORE evaluates only verifiable claims, with human evaluations supporting its use for diverse long-form generation. Its open-weight implementation approaches the closed-weight version, while complex claims remain difficult to verify against search results.

Conclusion

  • VERISCORE focuses exclusively on verifiable claims and is validated by human evaluations for diverse long-form tasks containing verifiable and unverifiable content.The paper also releases closed- and open-weight implementations, with open-weight performance approaching the closed-weight version.Source: §6 · p. 8

Future work

  • Complex claims, especially those that are not entity-centric or formulaic, remain challenging to verify against search results.The authors identify improving this verification capability as future work.Source: §6 · p. 9

Limitations

The authors identify limitations in defining verifiable claims, runtime, creative-writing extraction, and fine-tuning hyperparameter search.

Limitations

  • The definition of verifiable claims remains a working definition rather than a formal one, particularly for sentences combining events and resulting states.The authors note that it can be difficult to determine whether such a sentence describes one state or multiple events and states.Source: Limitations · p. 9
  • Without parallelization, extracting claims from 400 GPT-4o responses takes about four hours on one RTX8000 GPU, and verifying 10,000 claims takes about two hours.The extractor uses a sliding window; the authors suggest that avoiding it could speed extraction.Source: Limitations · p. 9
  • For extremely infactual responses such as WritingPrompts, the extractor may still extract a small amount of unverifiable claims, though the authors do not consider this a major concern.Source: Limitations · p. 9
  • The authors did not exhaustively search fine-tuning hyperparameters for the open-source extractor and verifier, so better performance may be possible.They describe further search as resource-intensive and time-consuming.Source: Limitations · p. 9

Ethics Statement

The project aimed to reduce fine-tuning costs with LoRA; the annotation study was IRB-exempt and collected no personally identifiable information.

Ethics Statement

  • LoRA was used for efficient model fine-tuning to minimize computational cost.Annotators were paid $18 per hour, with bonuses for reasonable extra time.Source: Ethics Statement · p. 9
  • Annotators consented to scientific use and publication of their data, and no personally identifiable information was collected.The annotation review was exempted by an IRB.Source: Ethics Statement · p. 9

A Weaknesses in FACTSCORE and SAFE

The appendix details weaknesses in FACTSCORE and SAFE, formalizes claim-verification scenarios, and describes context-aware prompts for claim extraction.

A Weaknesses in FACTSCORE and SAFE

  • FACTSCORE’s extraction prompt is biography-focused, limiting its fit for other domains.SAFE adds only a brief task description to FACTSCORE’s prompt.Source: A Weaknesses in FACTSCORE and SAFE · p. 12
  • SAFE’s multistep pipeline revises vague references and screens claims, adding processing time and cost.FACTSCORE extracts claims sentence by sentence without context, but its predefined Wikipedia article setting avoids the reference-resolution need SAFE faces with Google Search.Source: A Weaknesses in FACTSCORE and SAFE · p. 12
  • 11% of 211 claims were removed by SAFE’s relevance check, and 58% of those removed claims were relevant.The remaining 42% were tautologies or not claims and should not have been extracted.Source: A Weaknesses in FACTSCORE and SAFE · p. 12
  • SAFE’s evaluation covered biography data despite 38 fact-seeking topics, and a human study found it fell short in less entity-centric domains.SAFE was applied to object-related prompts; six topics mostly contained biography questions.Source: A Weaknesses in FACTSCORE and SAFE · p. 12

B Formal definition of claim verification

  • Table 5 formalizes four claim-verification scenarios using support and contradiction predicates that are not simple negations of each other.The section points to this table as the formal definition of the scenarios.Source: B Formal definition of claim… · p. 12, Table 5 · p. 13

C Claim extraction prompts

  • The extraction prompts use a sliding sentence window and surrounding context to make claims self-contained and resolve references.QA inputs prepend the question; non-QA inputs prepend a paragraph’s first sentence when the paragraph exceeds five sentences.Source: C Claim extraction prompts · pp. 12–13

D Human evaluation of claim extractions by our prompt and SAFE

Annotators preferred the paper’s claim-extraction prompts over SAFE’s across eight domains; supplementary verification examples describe disagreement and inconclusive cases.

D Human evaluation of claim extractions by our prompt and SAFE

  • Three annotators evaluated the same 120 sampled data points, truncated at sentence boundaries to 300 words; the interface randomized the two claim lists.The sample included 15 points per dataset, with the first four datasets treated as non-QA and the others as QA; the interface also included source text, decisions, and justifications.Source: D Human evaluation of claim… · p. 13, Figure 4 · p. 14
  • SAFE’s claims were preferred only 26 times across 360 annotations, while the paper’s prompts were overwhelmingly preferred across all eight domains.Figure 5 distinguishes strong from slight preferences; counts aggregate three annotators.Source: D Human evaluation of claim… · p. 13, Figure 5 · p. 14

E.1 Detailed examples of human verification

  • Table 8 gives examples of claims labeled inconclusive by all annotators.The appendix presents these as detailed examples from the Section 3.3 human verification study.Source: §E.1 · p. 14

E.2 Reason of disagreement in human verification

  • Nine verification items lacked full agreement, attributed to annotator mistakes, interpretive differences, complex claims, or indirect evidence requiring reasoning.One annotator sometimes overlooked the connection between a claim and search results.Source: §E.2 · pp. 14–15

E.3 Verifying/falsifying inconclusive claims is hard

  • 42.2% of annotated claims were inconclusive; manual checks of 15 such cases found two claims too underspecified to verify.The authors also examined whether full web pages could resolve inconclusive cases.Source: §E.3 · p. 15
  • For the Angular example, search snippets omitted that Google maintains Angular, but the full page confirmed it.Only one claim in the example could be verified from the full page.Source: §E.3 · p. 15
  • Among the remaining 12 cases, only one claim was weakly contradicted, and its interpretation depended on what “as a battery discharges” meant.The article did not establish that reactions slow down if the phrase meant a battery continues to discharge.Source: §E.3 · p. 15

F Details of fine-tuning open-source models for claim extraction

The appendix reports fine-tuning details for claim extraction and verification, along with benchmark data scope and supplementary unsupported-claim examples.

F Details of fine-tuning open-source models for claim extraction

  • Claim-extraction training data combined existing datasets with newly generated model responses, sampled less heavily from Scruples because most examples were subjective.The data included 100 Scruples points, 200 from other datasets, and responses from 63 Biography prompts and 80 prompts from other datasets.Source: F Details of fine-tuning open-source… · p. 15
  • GPT-4 decomposed collected responses into sentence-marked claims, producing 99,592 input-output pairs after filtering and sampling.Short marked sentences and “No verifiable claim” outputs were selectively dropped; the data split was 95% training, 4% validation, and 1% test.Source: F Details of fine-tuning open-source… · p. 15
  • Llama3 and Mistral were fine-tuned for two epochs with LoRA, and the two-epoch Mistral checkpoint was selected for manual comparison with GPT-4.Mistral sometimes omitted small words, combined multiple facts into one claim, or extracted beyond the marked span.Source: F Details of fine-tuning open-source… · p. 16
  • Fine-tuned Mistral performed comparably to GPT-4 on claim extraction, with GPT-4 slightly better in the manual comparison.The two annotators fully agreed on 106 data points, with Cohen’s κ = 0.4320; Figure 6 presents their choices.Source: F Details of fine-tuning open-source… · p. 16, Figure 6 · p. 16

G Details of fine-tuning open-source models for claim verification

  • For claim verification, GPT-4o with ternary labels had the most balanced supported/unsupported performance, and one-epoch fine-tuned Llama3 was selected for further experiments.The ternary labels were converted to binary for generating fine-tuning data; unsupported training examples were triplicated.Source: G Details of fine-tuning open-source… · pp. 16–17

H Details of data domains and studied LLMs

  • The study used generations from the first 12 models to develop VERISCORE and all 16 models to benchmark it.The appendix points to Table 1 for datasets and Table 14 for studied language models.Source: H Details of data domains and studied… · p. 17

I Unsupported cases in VERISCORE outputs

J VERISCORE on WritingPrompts and FreshQA

FreshQA and WritingPrompts contain few verifiable claims, and response length affects their VERISCORE comparisons. WritingPrompts also tests claim extraction on fictional content.

J VERISCORE on WritingPrompts and FreshQA

  • 4 is the median verifiable-claim count in FreshQA, compared with K = 1 in WritingPrompts, where fictional premises yield few verifiable claims despite long generations.Source: J VERISCORE on WritingPrompts and… · p. 17
  • 82.61% of Gemma-2b-it claims and 71.15% of GPT-4o claims from WritingPrompts had potential to be verified or falsified.WritingPrompts VERISCORE is very low, but the authors report that the pipeline works effectively on fictional content while occasionally extracting unverifiable claims.Source: J VERISCORE on WritingPrompts and… · p. 17
  • 74.70% of GPT-4o’s extracted FreshQA claims were supported, versus 72.71% for Claude 3 Haiku, despite Claude 3 models’ higher overall VERISCORE.Claude 3 Haiku averages 5.5 claims per response versus GPT-4o’s 3.39; GPT-4o’s shorter responses contribute to its lower VERISCORE.Source: J VERISCORE on WritingPrompts and… · p. 17
  • Response length complicates fair VERISCORE comparisons: matching output lengths departs from end-user settings, while unnecessary length may elicit less factual later content (Min et al., 2023).Source: J VERISCORE on WritingPrompts and… · p. 17

K Detailed results of models’ VERISCORE

  • Appendix K supplies separate extraction prompts for QA and non-QA text, while Table 16 breaks down 16 models across eight domains using response length, precision, recall, and VERISCORE.The prompts seek fine-grained externally verifiable facts and exclude stories, hypotheticals, and subjective content.Source: K Detailed results of models’ VERISCORE · pp. 17–19, Table 16 · p. 27
Loading 2406.19276v1…