PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Large Language Models for Knowledge Graph Validation

Farzad Shami, Stefano Marchesin, Gianmaria Silvello

arXiv:2602.10748v1cs.DB

TL;DR

Knowledge-graph applications depend on factual accuracy, but existing validation methods and LLMs are not yet reliably suitable for large-scale fact checking. FactCheck benchmarks LLM validation using internal knowledge, retrieved evidence, and multi-model consensus, finding fluctuating and inconsistent performance that falls short of reliable real-world deployment.

  • Problem

    LLM suitability for knowledge-graph fact validation remains insufficiently established despite the importance of factual accuracy for reliable KGs.

  • Method

    FactCheck benchmarks LLMs for knowledge-graph fact validation across internal knowledge, external RAG evidence, and multi-model consensus.

  • Results

    LLMs remain unreliable for real-world KG validation; RAG yields fluctuating, inconsistently improved performance, and consensus does not consistently outperform individual models.

  • Takeaways & Limitations

    FactCheck underscores the difficulty of KG fact validation and the need for a dedicated benchmark to drive progress.

  • Takeaways & Limitations

    Infrastructure constraints included a 0.08% retrieval failure rate caused by network issues and regional restrictions.

Abstract

from arXiv · show

Knowledge Graphs (KGs) store structured factual knowledge by linking entities through relationships, crucial for many applications. These applications depend on the KG's factual accuracy, so verifying facts is essential, yet challenging. Expert manual verification is ideal but impractical on a large scale. Automated methods show promise but are not ready for real-world KGs. Large Language Models (LLMs) offer potential with their semantic understanding and knowledge access, yet their suitability and effectiveness for KG fact validation remain largely unexplored. In this paper, we introduce FactCheck, a benchmark designed to evaluate LLMs for KG fact validation across three key dimensions: (1) LLMs internal knowledge; (2) external evidence via Retrieval-Augmented Generation (RAG); and (3) aggregated knowledge employing a multi-model consensus strategy. We evaluated open-source and commercial LLMs on three diverse real-world KGs. FactCheck also includes a RAG dataset with 2+ million documents tailored for KG fact validation. Additionally, we offer an interactive exploration platform for analyzing verification decisions. The experimental analyses demonstrate that while LLMs yield promising results, they are still not sufficiently stable and reliable to be used in real-world KG validation scenarios. Integrating external evidence through RAG methods yields fluctuating performance, providing inconsistent improvements over more streamlined approaches -- at higher computational costs. Similarly, strategies based on multi-model consensus do not consistently outperform individual models, underscoring the lack of a one-fits-all solution. These findings further emphasize the need for a benchmark like FactCheck to systematically evaluate and drive progress on this difficult yet crucial task.

1 Introduction

FactCheck is introduced as a benchmark for evaluating LLMs on KG fact validation, motivated by the scalability limits of expert verification and the generalization shortcomings of existing automated methods. It assesses internal knowledge, RAG-based external evidence, and multi-model consensus across diverse datasets while tracking performance and resource use.

  • Motivation: KG fact validation is essential but difficult and costly because expert verification is time-consuming, while rule-based methods struggle to generalize across real-world facts.Expert annotation is described as the most reliable approach, but automated methods offer greater scalability and remain limited in coverage.
  • Motivation: LLMs offer potential for fact-checking through contextual extraction, semantic understanding, and extensive internal knowledge, but their factual reliability remains uncertain.The benchmark specifically examines whether LLMs can validate externally provided facts using only their internal knowledge.
  • Benchmark: FactCheck evaluates KG fact validation across three dimensions: LLM internal knowledge, external evidence through RAG, and synthesized knowledge from multiple models.Its validation pipeline converts structured triples into natural-language statements, checks them against reliable sources, and calculates accuracy scores.
  • Benchmark: The benchmark covers FactBench, YAGO, and DBpedia, and includes 2M+ documents for RAG-based KG fact validation.The datasets span everyday and domain-specific knowledge, while the RAG resource pairs questions with Google Search Engine Results Pages and supports reproducible retrieval experiments.
  • Benchmark: FactCheck combines model evaluation with computational-cost analysis, tracks inference time and token usage, and provides a web application for visual verification and error analysis.Predictions are compared with gold-standard labels to assess accuracy and reliability, while the application categorizes reasoning errors and exposes verification steps.
  • Findings: Experiments find that LLMs remain insufficiently reliable for real-world KG validation, while RAG and multi-model consensus produce inconsistent improvements over simpler approaches.The reported conclusions emphasize fluctuating RAG performance and the difficulty and inherent complexity of the validation task.

2 Related Work

Prior KG fact-checking methods either use internal KG paths or external evidence, but internal approaches inherit KG incompleteness and errors. FactCheck extends external-evidence research by evaluating LLMs across real-world KG datasets with Web SERP evidence and large-scale retrieved documents.

  • Fact-checking paradigms: KG fact-checking methods follow two paradigms: internal KG paths for supporting statements or external reference sources for supporting or conflicting evidence.Table 1 compares these internal KG-based and external evidence-based mechanisms.
  • Internal KG-Based Fact Checking: Internal methods including KStream, KLinker, COPPAL, and PredPath discover positive or negative evidential paths between entities and statements.KStream models the KG as a flow network, while KLinker discovers relational paths and PredPath uses both negative and positive paths.
  • Internal KG-Based Fact Checking: Because internal methods rely entirely on potentially erroneous or incomplete KGs, they cannot assess the accuracy of the KG itself.Some approaches are further restricted because they focus only on positive evidential paths.
  • External Evidence-Based Fact Checking: External methods such as DeFacto and Syed et al.’s approach validate KG triples using Web or static-corpus textual evidence retrieved for verbalized statements.DeFacto combines trustworthiness metrics with textual evidence, while Syed et al. retrieve similar corpus sentences through a search engine.
  • FactCheck’s position: FactCheck uses Web SERP evidence, provides LLM-based baselines, and evaluates three real-world KG datasets containing 13,530 facts with 2M+ retrieved documents.It targets LLM comparison with external evidence-driven solutions and is designed specifically for KG fact validation.
  • Related benchmarks and datasets: Unlike CRAG’s RAG benchmark and textual-claim tasks such as RumourEval and CheckThat!, FactCheck addresses KG fact validation with distinct evaluation goals.Existing KG verification datasets such as FactBench, YAGO, and DBpedia have seen minimal to no use with LLM-based approaches.

3 FactCheck

FactCheck benchmarks KG fact validation through internal-knowledge prompting, retrieval-augmented evidence, and multi-model consensus. Its methods combine direct and guided verification with structured retrieval and majority-based aggregation.

  • Benchmark strategies: FactCheck evaluates open-source and commercial LLMs through internal-knowledge, RAG, and multi-model consensus strategies addressing three research questions.The strategies respectively examine models’ internal knowledge, external evidence, and aggregated predictions.
  • Internal-knowledge verification: Direct Knowledge Assessment (DKA) uses a basic prompt to test fact verification from internal knowledge and serves as the baseline for comparing models and advanced strategies.DKA provides no further guidance beyond the direct prompt.
  • Internal-knowledge verification: Guided Iterative Verification (GIV) uses structured prompts, optional dataset constraints, re-prompting for non-conformant outputs, invalidation after repeated failures, and zero- or few-shot settings.The approach explicitly flags non-compliance before retrying.
  • Retrieval-augmented verification: RAG verifies each KG triple through triple transformation, question generation and ranking, document retrieval and filtering, and document processing and chunking.The pipeline augments LLMs with potentially supporting or refuting external evidence.
  • Retrieval-augmented verification: The RAG pipeline converts heterogeneous triples into readable sentences, generates multiple queries, retrieves English U.S.-localized results, excludes original KG sources, and ranks documents for chunking.Filtering removes documents originating from the KG’s source set to avoid circular verification and preserve evidence independence.
  • Multi-model consensus: The consensus strategy combines four LLM verdicts using simple majority voting, with tie-breaking by either a consistency-selected larger model or an independent commercial model.Consistency is measured by agreement with majority predictions across datasets; GPT-4o mini is an example independent judge.

4 Benchmark Construction

FactCheck is constructed from triples drawn from three real-world knowledge graphs and a retrieval-augmented generation dataset pairing KG-derived questions with Google search content. A mock API provides standardized access to pre-collected evidence for reproducible fact-validation experiments.

  • Dataset components: The benchmark combines triples from FactBench, YAGO, and DBpedia with content retrieved from Google SERPs.These two components support KG-based validation and external-evidence evaluation.
  • KG datasets: KG triples use snapshot-based semantics: a fact is true when supported by the source KG snapshot and false otherwise.FactBench focuses on English; YAGO contains 1,386 facts with gold standard accuracy μ=0.99, while DBpedia contains 9,934 triples with gold standard accuracy μ=0.85.
  • RAG dataset: The RAG dataset generates questions from transformed KG triples and pairs them with corresponding Google search results.It contains Q=130,820 questions generated for 13,530 facts, with k_q=10 intended questions per transformed triple.
  • RAG dataset: Question similarity spans three tiers: 45% high similarity, 34% medium similarity, and 21% low similarity.Overall, 79% of questions have at least moderate similarity to the transformed triple (δ≥0.40).
  • RAG dataset: The evidence corpus contains D=2,090,305 documents across 13,530 triples, with text coverage of 87%.Each triple links to a mean of μ_d_t=154.51 documents, while 13% of documents have empty text content.
  • Mock API: A web search-like mock API serves pre-collected search data, removing temporal variability and standardizing retrieval for reproducible benchmarking.Queries use each transformed triple and its top three generated questions, while sources directly related to the original fact are filtered to avoid circular verification.

5 Experimental Setup

The experiments used distinct computing environments for data retrieval, transformation, and LLM evaluation, with local deployment and monitoring infrastructure. The methodology also specified empirically selected RAG configurations and multi-model consensus scenarios involving larger model counterparts.

  • Computational Infrastructure: LLM prompting and evaluation ran on a Mac Studio with an Apple M2 Ultra, 24 cores, and 192 GB of unified memory.Google SERP retrieval used a Unix server with 2 CPU cores and 4 GB RAM, while triple transformation and question generation used an Apple M2 Max MacBook Pro with 32 GB RAM.
  • Implementation Framework: Open-source LLMs ran locally through Ollama, with Open-Telemetry tooling from OpenLIT monitoring token usage, inference time, vector databases, and GPUs.The monitoring setup was intended to track model behavior and resource usage across the LLM pipeline.
  • RAG Configuration: RAG pipeline parameters were selected through experiments comparing alternative configurations, with ablation-study results available in the GitHub repository.The finalized settings are reported in Table 4.
  • Multi-Model Consensus: Multi-model consensus used separate scenarios for higher-parameter open-source models and a commercial LLM.In the open-source scenario, models with the highest and lowest consistency scores were selected before replacing base versions with larger counterparts.
  • Multi-Model Consensus: The open-source consensus scenario scaled LLaMA3.1 from 8B to 70B, Gemma2 from 9B to 27B, Qwen2.5 from 7B to 14B, and Mistral from 7B to nemo:12B.These replacements followed consistency scoring across datasets.

6 Experimental Analysis

The experiments show that model and dataset characteristics strongly affect KG fact-validation performance: open-source models and few-shot prompting excel with internal knowledge, while RAG often improves accuracy but at substantial and variable computational cost. Multi-model consensus mainly stabilizes predictions, and no strategy consistently dominates across accuracy, robustness, and efficiency.

  • Internal Knowledge: Open-source Gemma2 and Mistral outperform commercial GPT-4o mini under internal-knowledge settings, while few-shot GIV-F generally improves performance over DKA and GIV-Z.On FactBench, Gemma2 reaches F1(T)=0.79 and F1(F)=0.76 in GIV-F; Mistral improves from 0.68 in DKA to 0.81 in GIV-Z.
  • Retrieval-Augmented Generation: RAG achieves the strongest results in most settings, but its gains vary by dataset, with substantial improvements on FactBench and YAGO and minimal or negative changes on Dbpedia.For FactBench, Qwen2.5 reaches F1(T)=0.89 with RAG versus 0.55 in DKA; GPT-4o mini improves by more than 25% in both F1 scores.
  • Multi-Model Consensus: Multi-model consensus stabilizes validation across internal-knowledge settings but does not consistently surpass individual models, and the tie-breaking model has little effect on final scores.The approach mitigates weaker-model effects, while majority voting appears more important than the specific arbitrator.
  • Computational Efficiency: Methodological complexity increases computational cost: DKA is fastest, RAG often costs sixfold or more than DKA, and consensus can be parallelized so latency is bounded by the slowest model.DKA runs from 0.21 to 0.30 seconds on FactBench, 0.19 to 0.31 seconds on YAGO, and 0.24 to 0.37 seconds on DBpedia; RAG latency is approximately 1.6–2.9 seconds.
  • Accuracy–Efficiency Trade-offs: The Pareto analysis positions DKA for latency-sensitive applications, RAG for stronger false-claim detection, and GIV-F with Gemma2 or Mistral as an intermediate accuracy–cost trade-off.DKA provides sub-second inference times below 0.3 seconds, whereas RAG trades approximately 1.6–2.9 seconds of latency for enhanced F1(F).

7 Qualitative Error Analysis

The error analysis uses LLM-generated explanations and document embeddings to cluster open-source model mistakes, identifying geographic and nationality inconsistencies as predominant. Model prediction overlaps further reveal shared knowledge, RAG-induced consensus, and prompting-dependent disagreement.

  • Error clustering: The semi-automated pipeline explains incorrect predictions with the same LLM, encodes explanations using cde-small-v1, and clusters them with UMAP and HDBSCAN.The analysis categorizes mistakes including role attribution, geographic/nationality, genre/classification, and identifier/biographical errors.
  • Error clustering: Geographic/Nationality Errors (E4) are the predominant challenge in FactCheck.On DBpedia, error rates decrease for common-knowledge partitions; Education and News are easier, while Architecture and Transportation remain more challenging.
  • Interactive analysis: The web-based platform enables interactive interpretation and visualization of the verification process and error analysis.The platform is available at factcheck.dei.unipd.it.
  • Model overlap: The largest prediction intersection generally contains facts correctly identified by all four models, especially in RAG, where shared evidence reduces variance.This agreement indicates that open-source LLMs share substantial internal knowledge and error profiles.
  • Model overlap: GIV-Z shrinks the all-model intersection from roughly 4,600 to about 3,200, whereas GIV-F raises it to over 5,200.GIV-Z produces stronger pairwise overlaps and more heterogeneous reasoning, while few-shot demonstrations in GIV-F restore stronger consensus.

8 Final Remarks

FactCheck is presented as a benchmark for systematic, reproducible evaluation of LLMs on KG fact validation. Its findings show promising but costly and constrained approaches, while motivating further work on retrieval, training, and logical reasoning.

  • Contributions: FactCheck benchmarks LLMs for KG fact validation across three real-world datasets: FactBench, YAGO, and DBpedia.It provides controlled, fine-grained analysis of internal knowledge, retrieval effectiveness, and multi-model interactions.
  • Key findings: Gemma2 achieves F1 scores up to 0.79 and 0.76 using internal knowledge alone, exceeding 0.89 when augmented with RAG.
  • Key findings: RAG improves performance across most settings but is roughly 10× slower than other methods.
  • Limitations: Dataset imbalance, schema diversity, retrieval failures, and hosted-model content filtering limit reliable deployment.The retrieval failure rate is 0.08%, with network issues and regional restrictions contributing to failures.
  • Contributions: FactCheck differs from prior benchmarks by exploiting KG structured semantics and supports transparent, collaborative progress toward scalable KG validation.
  • Future work: Future directions include fine-tuning or pretraining, hybrid structured–unstructured retrieval, and evaluating systems that use logical rules.
Loading 2602.10748v1…