PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking the Evaluation of LLM Fact-Checkers

Ayana Niwa, Miyu Yamada, Kouta Nakayama, Masahiro Kaneko, Hisami Suzuki, Satoshi Sekine

arXiv:2610.30504v1cs.CL

TL;DR

Reliable evaluation of LLM fact-checkers lacks benchmarks that specify which claims are checked, the conditions for judging them, and eligible evidence. JSFactCheckBench makes these choices explicit in a Japanese benchmark, and experiments show that fact-checking performance varies across these conditions, including worse accuracy on claims directly answering the question.

  • Problem

    Existing benchmarks do not jointly control which claims are checked, the conditions under which they are judged, and which sources count as evidence.

    Source: §1 · p. 1

  • Method

    JSFactCheckBench evaluates 1,913 claims from LLM answers to questions about misleading information circulated on Japanese social media, with reference-time factuality and claim importance annotations grounded in preselected Web sources.

    Source: §1 · p. 1

  • Results

    All six LLMs verify direct-answer claims less accurately than supplementary claims; Reference macro-F1 ranges from 0.691–0.770, while Pages ranges from 0.458–0.705.

    Source: §1 · pp. 1–2, §3.3 · p. 8

  • Takeaways & Limitations

    Fact-checking ability is not captured by a single score under a single condition and must be evaluated from multiple aspects.

    Source: §4 · p. 9

    The reference time is fixed to the evidence retrieval date, so the benchmark cannot reproduce judgments for arbitrary past or future dates.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) are widely used to answer factual questions, but their answers can contain errors and must be fact-checked claim by claim. Yet the resulting score depends on three design decisions: which questions the claims answer, under what conditions each claim is judged, and which sources count as evidence. No existing benchmark specifies all three. We introduce JSFactCheckBench, a Japanese benchmark that specifies all three, with 1,913 claims from LLM answers to questions about misleading information that circulated on Japanese social media, each annotated for factuality at a reference time and for importance to the question, using evidence from preselected Web sources. Across six LLMs, all verify claims directly answering the question less accurately than supplementary claims, a gap that aggregate scores obscure. Scores also drop when models must locate the relevant content in longer or multiple sources and, for all four open models, when the surrounding question and response are added. These weaknesses are visible only because the benchmark records the conditions behind each label. A benchmark that leaves them implicit cannot reveal them.

1 Introduction

Reliable evaluation of claim-by-claim fact-checking depends on specifying which claims are judged, the conditions for judging them, and eligible evidence sources. JSFactCheckBench makes these choices explicit and shows that all six evaluated LLMs verify direct-answer claims less accurately than supplementary claims, a difference aggregate scores hide.

1 Introduction

  • LLM answers can contain false or misleading information, so automated fact-checking requires verifying claims individually and evaluating systems with reliable benchmarks.Source: §1 · p. 1
  • The benchmark makes explicit three design choices: which claims to verify, the conditions for judging them, and which sources count as evidence.These choices include real-world questions, reference time and claim importance, and controlled sources; no existing benchmark controls all three.Source: §1 · p. 1
  • JSFactCheckBench contains 1,913 claims from eleven LLMs answering questions about misleading information circulated on Japanese social media.Claims are annotated for factuality at a specified reference time and importance to the question, using evidence from preselected Web sources and judgments by three professional annotators.Source: §1 · p. 1
  • All six LLMs verify claims that directly answer the question less accurately than supplementary claims, a gap aggregate scores hide.Only a quarter of claims directly answer the question.Source: §1 · pp. 1–2

2 JSFactCheckBench

JSFactCheckBench evaluates individual claims from LLM answers to real-world Japanese questions, with factuality tied to a reference time and importance tied to the question. It uses controlled Web evidence and compares its design with prior benchmarks.

2.1 Dataset Design

  • Claims come from LLM answers to Japanese social-media misinformation questions, and each is judged at a specified time, for question importance, and against controlled Web evidence.Factuality labels distinguish Supported, Refuted, and NotEnoughInfo; importance labels distinguish Main, Sub, and Irrelevant claims.Source: §2.1 · pp. 2–3

2.2 Dataset Construction

  • The construction samples questions and model responses, decomposes responses into atomic claims, retrieves allowlisted evidence candidates, and has annotators label claims using those candidates.Three annotators label each target claim, with disagreements consolidated by stated precedence and majority-vote rules.Source: §2.2 · pp. 3–5
  • Independent relabeling of 50 sampled claims agreed with the original labels on 94% of factuality judgments and 96% of importance judgments.The relabelers received each claim and the passage used in its original judgment.Source: §2.2 · p. 5

2.3 Dataset Statistics

  • 1,913 claims from 170 responses to 85 questions form the dataset, averaging 11.3 claims per response.Responses were generated for sampled JSocialFact questions, and claims were extracted from them.Source: §2.2 · p. 3, §2.3 · p. 5
  • Among verifiable claims, 21% are Refuted; Sub claims make up 65.3% of the data and Main claims 25.2%.Most claims are supplementary rather than direct answers to questions.Source: §2.3 · p. 5, Table 3 · p. 5

2.4 Comparison with Existing Benchmarks

  • Compared datasets do not record a reference time for gold labels, while JSFactCheckBench manually annotates question-relative importance and uses an allowlist for evidence sources.It provides annotators the same stored candidates and excludes two fact-checking-site domains to reduce verdict leakage.Source: §2.4 · p. 6

3 Experiments

Experiments test how context, evidence presentation, and claim importance affect measured LLM fact-checking performance. Factuality classification is evaluated on 1,738 claims with six LLMs.

3.1 Task and Setup

  • The task predicts Supported, Refuted, or NotEnoughInfo for each claim; four open Japanese models and two commercial models are evaluated.Claims labeled Unevaluable or Unclear are excluded, leaving 1,738 claims.Source: §3.1 · p. 6

3.2 Input Settings

  • Twelve settings cross Claim, Time, or Q&R context with closed-book, Reference, Full, or Pages evidence access.Reference supplies the annotators’ quoted passage; Full supplies surrounding page text, while Pages presents candidate-page windows for source selection.Source: §3.2 · pp. 6–7

3.3 Results

  • From claims alone, open models score 0.444–0.490 and commercial models 0.549–0.600 in macro-F1, above the 0.313 uniform-random baseline.The authors characterize absolute performance using internal knowledge alone as low.Source: §3.3 · p. 7
  • Adding the question and response substantially reduces performance for all four open models across evidence conditions, while affecting the two commercial models comparatively little.With Reference evidence, LLM-jp-4 labels 93% of claims Supported in Q&R, compared with 54% without that context.Source: §3.3 · pp. 7–8
  • When models select among candidate pages, their macro-F1 range is 0.458–0.705, versus 0.691–0.770 when given the deciding passage.Reading full pages costs almost every model; choosing among pages further reduces scores for four of six, while Pages still beats closed-book for all six once reference time is stated.Source: §3.3 · p. 8
  • Stating the evidence-collection date changes macro-F1 by up to 0.063, significantly for three models in at least one evidence condition and not significantly for the other three.Answers to 46% of questions can change over time; the benchmark records a reference time for each label.Source: §3.3 · p. 8
  • Every model verifies Main claims less accurately than Sub claims, by 0.036–0.124, although aggregate accuracy obscures the gap.Among the four best models, aggregate accuracy differs by at most 0.005, yet Sonnet 5 scores 0.670 on Main claims versus 0.697–0.710 for the others.Source: §3.3 · p. 8, Table 7 · p. 8

4 Conclusion

JSFactCheckBench makes three design decisions behind fact-checking scores explicit in a Japanese benchmark of 1,913 claims. Experiments show that measured weaknesses differ across conditions, so no single score captures fact-checking ability.

4 Conclusion

  • JSFactCheckBench contains 1,913 claims from LLM answers and specifies which questions they answer, the conditions for judgment, and the evidence sources.Claims derive from questions about misleading information circulated on Japanese social media; labels include factuality at a reference time and importance to the question.Source: §4 · p. 8
  • Across six LLMs, the benchmark finds lower accuracy on claims directly answering the question, divergence when evidence must be found, and differing responses to reference time and surrounding context.The surrounding answer is associated with performance declines for the open models, while only some models respond to a stated reference time.Source: §4 · pp. 8–9
  • The benchmark’s results support evaluating fact-checking from multiple aspects rather than relying on a single score under one condition.Its recorded conditions expose distinct weaknesses across the six evaluated LLMs.Source: §4 · pp. 8–9

Limitations

The benchmark is limited by domain-level control of source quality, a fixed evidence-retrieval reference time, and its scale and Japanese-language scope.

Evidence source constraints

  • Source quality is controlled by domain, so quality differences among pages within the same domain are not modeled.Applying the design to another language requires locally grounded question sources and evidence-selection criteria.Source: Limitations · p. 9

Fixed reference time

  • Because labels use the evidence retrieval date, the benchmark cannot reproduce judgments for arbitrary past or future dates.For rapidly changing facts, gold labels may diverge from the current state; continued use requires periodic evidence and label updates.Source: Limitations · p. 9

Scale

  • The benchmark covers 85 questions and 1,913 Japanese claims, and small model differences should not be read as rankings.Claims were extracted from LLM responses and verified from scratch; the authors report statistical significance to show the dataset yields reliable results.Source: Limitations · p. 9

Ethics Statement

The dataset is intended for evaluating LLM fact-checking, with use constrained by its terms, time-bound labels, and consequential subject matter.

Data provenance and license

  • The dataset may be used, including commercially, to improve LLM safety, but redistribution of the dataset itself is not allowed.The release includes URLs and quoted passages rather than full page contents; copyright remains with the respective rights holders.Source: Ethics Statement · p. 9

Annotation work

  • Annotation was completed by three workers at a total contract cost of approximately JPY 2.3 million.This corresponds to about JPY 390 for each of 5,856 individual annotations.Source: Ethics Statement · p. 9

Intended use and misuse

  • The dataset is for evaluating LLM fact-checking, not as a factual knowledge source or sole basis for real-world decisions.Labels reflect evidence collected at a particular time and may not represent whether a claim remains true or false now; questions include medicine and nuclear energy.Source: Ethics Statement · p. 9

A Dataset Construction Details

The appendix details response sources, evidence selection and query generation, claim-level annotation procedures, and annotation effort.

A.1 Models Used for Response Generation

  • Table 8 lists the response-generating models and their IDs, while Figure 2 shows each model’s response count.The supplied passage identifies these materials but gives no model names or counts.Source: §A.1 · p. 12

A.2 Evidence-Link Selection

  • A pilot verification of 219 LLM responses prompted addition of the Companies and Wikipedia-reference evidence groups.URLs ending in .pdf were excluded from retrieval because documents spanning dozens of pages would impose excessive annotation cost.Source: §A.2 · p. 12

A.3 Query Generation

  • For each context-completed claim, an LLM generated a keyword query as a space-separated string rather than a question.The generation used Meta-Llama-3-8B-Instruct at temperature 0.3 with a 50-token limit and four fixed input-output examples.Source: §A.3 · p. 12

A.4 Annotation Guidelines

  • Each claim was judged for factuality and importance after annotators read the question and full response and inspected evidence candidates.Annotators received up to ten candidates per claim and recorded the labels, the evidence link and quoted passage, and annotation time; claims without candidates were skipped.Source: §A.4 · p. 12
  • Factuality labels distinguish Supported, Refuted, NotEnoughInfo, Unevaluable, and Unclear claims.Refuted claims were marked at the minimal erroneous span; independent errors were marked separately. Importance was defined relative to the question.Source: §A.4 · pp. 12–13
  • Evidence was limited to the linked page itself, excluding login- or paywalled pages; table evidence was recorded as header and cell value.For a negative existential claim with no statement of the asserted event in the candidate set, annotators labeled it Supported without an evidence link and noted the absence in a comment.Source: §A.4 · p. 13

A.5 Annotation Cost and Effort

  • Three native Japanese annotators labeled the 1,952 decomposed claims, with 19 duplicates and 20 Unclear claims removed to leave 1,913.Annotation took 294 person-hours, about three minutes per annotation and nine minutes per claim across the three annotators.Source: §A.5 · p. 13

A.6 Domain categories of evidence set

The appendix points to domain-level evidence-link counts and distributions of claim, question, and taxonomy statistics.

A.6 Domain categories of evidence set

  • Table 12 reports unique candidate and used evidence links by domain category.The caption identifies the comparison and grouping but does not provide the values.Source: §A.6 · pp. 13–14

A.7 Additional Statistics

  • Figure 4 shows the distribution of the number of claims per response.The supplied passages do not state the distribution’s values.Source: §A.7 · p. 14, Figure 4 · p. 14
  • Table 13 reports distributions for question stance and temporal stability.The supplied passage names the attributes but does not give their values.Source: §A.7 · p. 14
  • Tables 14–17 provide criteria for classifying questions along each taxonomy axis.The supplied passage does not specify the axes or criteria.Source: §A.7 · p. 14

B.1 Models Used for Evaluation

The evaluation uses six named models, including four open models and two additional models. The open models ran on a single NVIDIA A40 GPU (48 GB).

B.1 Models Used for Evaluation

  • The evaluated models include Llama-3.1-Swallow, Qwen3-Swallow, and LLM-jp-4.Their listed identifiers are tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3, tokyotech-llm/Qwen3-Swallow-8B-RL-v0.2, and llm-jp/llm-jp-4-8b-instruct.Source: §B.1 · p. 14
  • The evaluated models also include Mistral-Nemo-Japanese-Instruct-2408, GPT-5.1, and Sonnet 5.The listed identifiers are cyberagent/Mistral-Nemo-Japanese-Instruct-2408, gpt-5.1-2025-11-13, and us.anthropic.claude-sonnet-5 (Amazon).Source: §B.1 · p. 14
  • The four open models were run on a single NVIDIA A40 GPU (48 GB).Source: §B.1 · p. 14

B.2 Evaluation Prompt

The evaluation prompts vary which contextual components and evidence are provided to models. Evidence windows are tailored to the cited passage, while accompanying tables document label subtypes, annotation costs, evidence domains, and question classifications.

B.2 Evaluation Prompt

  • The prompt settings vary reference time, question and answer context, and evidence supplied as a quoted passage, cited page, or candidate pages.Under None, evidence is absent and the open-book instruction is replaced by a closed-book counterpart; the Question and Answer blocks and target-claim judgment instruction appear only under Q&R.Source: §B.2 · p. 14
  • Under Full, the cited page is truncated to a 6,000-character window centred on the annotators’ quoted passage; under Pages, candidate pages use 600-character windows.For Pages, the cited page’s window is centred on the quoted passage, ensuring the gold-label evidence is present.Source: §B.2 · pp. 14–15
  • Supplementary tables describe Unevaluable subtypes, annotation cost and effort, evidence domains, and question stance and temporal stability.Other tables specify the annotation schemes for question domains, geographic scope, question stance, and temporal stability.Source: Table 10 · p. 15, Table 11 · p. 15, Table 12 · p. 15
  • Figures 5–7 show representative prompt templates for Claim only, Claim + Time + Q&R + Reference, and Claim + Time + Q&R + Pages.The remaining settings differ only in which components are present.Source: §B.2 · p. 14, Figure 5 · p. 17, Figure 6 · p. 18

Related papers

Loading 2610.30504v1…