Source-linked AI summary
CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era
Kaiwen Shi, Weixiang Sun, Zheyuan Zhang, Lichao Sun, Nitesh V. Chawla, Yanfang Ye
TL;DR
LLMs make plausible but nonexistent references a serious threat to citation integrity, while manual checking and existing automated systems face scale, robustness, and transparency problems. CiteAudit introduces an open benchmark and multi-agent verification framework spanning extraction, retrieval, memory reuse, scholarly search, and final judgment. Across generated and real-world settings, it reports a more reliable balance across accuracy, precision, recall, and F1 score than existing baselines, while manual verification remains difficult to scale.
Problem
LLMs can generate nonexistent references, while manual verification is difficult to scale and existing tools struggle with noisy formats and transparent standardized evaluation.
Method
CiteAudit combines an open benchmark with a multi-agent pipeline for extraction, memory reuse, web retrieval, scholarly search, and final citation judgment.
Results
The framework achieves a more reliable balance across accuracy, precision, recall, and F1 score than existing baselines in generated and real-world settings.
Takeaways & Limitations
CiteAudit provides practical infrastructure for building more accountable citation-verification tools in the LLM era.
Takeaways & Limitations
Manual citation verification is highly labor-intensive and difficult to scale the real-world dataset to a sufficiently large size.
Abstract
from arXiv · showhide
Scientific research relies on citation integrity, yet large language models (LLMs) have introduced a critical risk: fabricated references that appear plausible but correspond to no real publications. As manual verification becomes infeasible and existing automated tools remain fragile, we introduce CiteAudit, a comprehensive benchmark and detection framework for hallucinated citations. We design a multi-agent verification pipeline that decomposes citation checking into metadata extraction, memory lookup, web-based retrieval, and final judgment. To evaluate this, we construct a large-scale, human-validated dataset spanning diverse domains and hallucination types. Experiments demonstrate that our framework achieves superior verification performance over state-of-the-art LLMs and commercial baselines. Our work provides the necessary infrastructure to audit citations at scale and safeguard the trustworthiness of scholarly discourse. Code is available at https://github.com/shiiiikw/CiteAudit.
1 Introduction
LLMs create plausible but nonexistent references, intensifying citation-integrity risks that manual verification cannot feasibly address at publishing scale. CiteAudit responds with an open benchmark and multi-agent framework, reporting stronger balanced verification performance than existing baselines.
- Motivation: LLMs can automatically generate bibliographic entries with no counterpart in the scholarly record, creating a distinct risk beyond ordinary citation mistakes.Such hallucinated references can resemble legitimate academic works and compromise evidence assessment and publication reliability.
- Motivation: Rapidly expanding reference lists make thorough manual verification unrealistic for reviewers, editors, and co-authors.Automated tools are therefore needed, but existing systems often misfire on noisy reference formats and are frequently proprietary.
- Contributions: CiteAudit introduces a comprehensive benchmark and multi-agent framework for verifying citation existence and metadata consistency.Its agents extract structured metadata, reuse verified records, retrieve external evidence, query scholarly sources, and make a final real-or-fake judgment.
- Contributions: The benchmark is large-scale, standardized, human-validated, and spans diverse domains and citation types with unified evaluation protocols.The framework separates claim extraction, retrieval, matching, reasoning, and judgment under noisy and heterogeneous formats.
- Findings: The framework yields stronger accuracy and interpretability than existing baselines while addressing pervasive citation errors in state-of-the-art LLMs.The reported finding concerns citation-verification performance rather than a single benchmark metric.
2 Related Works
Prior work addresses hallucinated citations through retrieval and matching, but many systems remain opaque and difficult to compare. The paper positions an open, standardized benchmark as necessary for reproducible evaluation.
- Existing verification systems: Citation-verification systems parse citation strings and match them against external bibliographic databases to assess reference authenticity.The related-work passage describes retrieval-based checking without specifying a common evaluation standard.
- Agentic verification: Agentic systems extend standalone language models by using tools such as web search to acquire external evidence and support actionable decision-making.This provides the broader technical context for retrieval-enabled citation verification.
3 Benchmark
CiteAudit combines naturally occurring citation errors with controlled hallucinations to build a human-validated benchmark. Its construction uses a taxonomy, targeted perturbations, authoritative cross-checking, and human-in-the-loop verification.
- Benchmark scope: CiteAudit addresses the evaluation gap with a benchmark grounded in real hallucinations reported on OpenReview and a taxonomy of naturally occurring fake-citation types.The benchmark integrates observed academic-literature errors with controlled hallucinated references.
- Real-world data collection: Real-world citation entries are collected from scholarly manuscripts and records, then cross-checked against authoritative sources using title, authors, venue, year, and other metadata.This process targets representative citation entries from OpenReview, Google Scholar, arXiv, bioRxiv, and related public platforms.
- Real-world data collection: Manual verification produces a high-quality gold set of naturally occurring mistakes, including incorrect authors, venue mismatches, and nonexistent references, but is labor-intensive and difficult to scale.The limitation applies specifically to enlarging the real-world dataset through manual citation verification.
- Controlled data generation: Controlled hallucinated citations are generated by targeted edits to verified BibTeX references, guided by a principled taxonomy of hallucination types.The construction spans broad research areas and publication venues before applying systematic perturbations.
- Hallucination taxonomy: Title perturbations include keyword substitution, while author perturbations add nonexistent names, delete valid authors, alter name strings, or fabricate complete author lists.These edits preserve selected fields or structures to model realistic citation inconsistencies.
- Benchmark validation: χ2 = 5.6 × 10^-5 and p = 0.994 indicate no significant difference in GPTZero detection behavior between generated and real-world hallucinated citations.The result supports the fidelity of the generation framework in simulating observed citation-hallucination patterns.
- Annotation and verification: Benchmark entries undergo automated evidence retrieval followed by author-led cross-checking to assign high-confidence labels for real, hallucinated, or erroneous citations.This human-in-the-loop process combines web-grounded retrieval with detailed manual inspection.
4 Methodology
CiteAudit formalizes hallucinated-citation detection as multi-stage evidence verification coordinated by a hierarchical SOP and specialized agents. The pipeline extracts metadata, checks memory, retrieves web evidence, and escalates unresolved cases to authoritative scholarly sources.
- Problem formulation: CiteAudit models each citation as structured metadata and verifies field consistency against authoritative retrieved evidence.The metadata fields include title, authors, URL or identifier, venue, and publication year.
- Collaborative multi-agent pipeline: Five specialized agents divide verification into extraction, memory lookup, web search, judgment, and high-stakes scholarly validation.The Scholar Agent retrieves canonical records from repositories such as Google Scholar, publisher pages, DOI records, and preprint repositories.
- Collaborative multi-agent pipeline: A memory confidence score above τ = 0.92 immediately verifies a citation through a fast path, bypassing external retrieval.The fast path is intended to reduce redundant computation and latency for previously verified or closely matching references.
- Planning model: An LLM Controller and Planning Model route citation tasks through sequential and parallelizable SOP stages while allocating resources from a thread pool.The planning logic balances verification cost and accuracy, invoking expensive scholarly validation only for unresolved or ambiguous citations.
- Execution stages: The pipeline begins by converting visual and textual PDF content into structured citation metadata, which supplies the input for downstream auditing.Uncached citations then proceed to web-based content retrieval and consistency auditing.
5 Experiments
CiteAudit combines specialized agents, benchmark evaluations, and qualitative case studies to verify citation authenticity and metadata consistency. Across controlled and real-world tests, it balances hallucination detection with preservation of genuine references, while exposing practical cost and retrieval-grounding trade-offs.
- Implementation: The framework combines multimodal citation extraction, external evidence retrieval, memory reuse, and model-based judgment for citation verification.Gemini 3 Flash handles planning and final judgment, while Qwen3-VL-235B A22 performs extraction and Mem0 supports persistent citation memory.
- Experimental Setup: The evaluation uses classification metrics, batch runtime, and input/output costs to assess verification quality and deployment feasibility.The benchmark includes 3,586 real-world references and 2,500 controlled hallucinated references, alongside a real-world test set of 2,889 authentic and 467 naturally occurring hallucinated citations.
- Generated Benchmark: On the generated benchmark, CiteAudit achieves the highest accuracy and F1 score while maintaining high precision and recall.The framework is reported to balance detection of fabricated references with preservation of genuine citations better than the compared systems.
- Generated Benchmark: The framework offers a favorable trade-off between verification quality, cost, and latency rather than minimizing runtime or API price alone.It achieves the strongest overall classification performance on the generated benchmark while using substantially cheaper inference than several proprietary high-end models.
- Real-World Benchmark: On the real-world evaluation, CiteAudit achieves the highest accuracy, precision, recall, and F1 score, with F1 exceeding the second-best system by 0.267 absolute points.The real-world setting contains ambiguity, noise, and incomplete metadata, and the relative model behavior remains consistent with the controlled benchmark.
- Additional Analysis and Case Studies: Diagnostic analyses find that proprietary systems do not reliably perform verifiable external retrieval, whereas CiteAudit provides traceable evidence and structured metadata mismatch reports.Case studies show detection of title and author discrepancies even when citations refer to real papers and baseline tools accept them as valid.
6 Conclusion
Hallucinated citations threaten research integrity as LLMs enter scientific workflows. CiteAudit combines standardized evaluation with multi-agent verification and demonstrates reliable citation checking in representative cases.
- Hallucinated citations pose a growing threat to research integrity as LLMs become integrated into scientific writing and peer review.
- The benchmark is open, standardized, and scalable, covering controlled perturbations and naturally occurring citation errors.
- The multi-agent framework combines planning, extraction, retrieval, memory-based reuse, scholarly search, and final judgment.
- The framework achieves a more reliable balance across accuracy, precision, recall, and F1 score than commercial and open-source baselines in generated and real-world settings.
A Citation Hallucination Taxonomy
The paper proposes a structured taxonomy of citation hallucinations to support systematic and reproducible evaluation. It organizes errors by bibliographic inconsistency type and severity, including subtle and severe patterns produced by LLMs.
- The taxonomy categorizes citation hallucinations by the type and severity of bibliographic inconsistencies observed in real scholarly manuscripts.
- It is designed to capture both subtle and severe hallucination patterns produced by large language models.
- The taxonomy supports systematic and reproducible evaluation of citation verification systems.
A.1 Definition of Citation Hallucination
A citation hallucination is a plausible-looking bibliographic reference that cannot be matched to a valid scholarly record. The taxonomy covers title, author, metadata, and compound inconsistencies, including cases that remain structurally complete but bibliographically invalid.
- Definition: A citation is hallucinated when no authoritative scholarly source satisfies all essential fields in its title, authors, identifier, venue, and year metadata tuple.
- Title-level hallucinations: Title-level hallucinations include incorrect or fabricated titles with otherwise plausible metadata.
- Title-level hallucinations: Representative title subclasses are keyword substitution, paraphrased fabrication, and topic-conditioned synthesis.
- Author-level hallucinations: Author-level hallucinations include author addition, deletion, name perturbation, and fully fabricated authorship.
- Metadata-level hallucinations: Metadata-level hallucinations involve venue, year, DOI, arXiv ID, or other persistent-identifier inconsistencies.
- Compound hallucinations: Compound hallucinations span two or more metadata fields and represent the most severe form of citation fabrication.
A.6 Relation to Real-World Citation Errors
The benchmark distinguishes harmful hallucinations from benign citation noise. Minor formatting differences remain acceptable when the core bibliographic identity is verifiable.
- Minor formatting variations, missing page numbers, and capitalization differences are not hallucinations when the core bibliographic identity remains verifiable.
- This distinction targets genuinely harmful hallucinations rather than benign citation noise and supports fair, realistic evaluation.
B Dataset Construction and Statistics
The dataset is built from verified real citations and controlled metadata perturbations that generate plausible hallucinated references while preserving valid citation formatting.
- The benchmark begins with verified citations collected from public bibliographic repositories and open-access scholarly sources, structured in BibTeX and checked across essential metadata fields.
- These verified citations provide both the real-reference portion of the benchmark and seed references for generating hallucinated variants.
- Hallucinated references are created through controlled perturbations that modify one or more metadata fields while preserving plausible, valid citation structure.
- The generated hallucinations cover title errors, author errors, and metadata errors through operations such as keyword substitution, paraphrasing, synthesis, and author-list perturbation.
- All hallucinated citations retain valid BibTeX structure and formatting, preventing detection based on trivial syntactic irregularities.
B.3 Dataset Composition and Balance
The final benchmark combines 2,500 hallucinated citations with 3,586 verified real citations and uses an intentionally realistic, non-class-balanced composition.
- 2,500 hallucinated citations and 3,586 verified real citations form the 6,086-instance test set.
- The hallucinated portion is distributed across title-level, author-level, and metadata-level errors, while real references remain unperturbed.
- The dataset is not class-balanced because genuine references are more frequent than fabricated ones in the intended realistic evaluation setting.
- Its controlled hallucination distribution supports targeted robustness analysis and tests whether systems detect fabricated references without over-rejecting valid citations.
B.4 Quality Control and Validation
CiteAudit validates labels through retrieval, manual metadata checks, multi-author review, and explicit agent roles that separate routing from evidence-based judgment.
- Dataset validation: A hallucinated citation is retained only when retrieval and human validation find no authoritative scholarly record matching its perturbed metadata.
- Dataset validation: This validation preserves realistic citation form while ensuring hallucinated instances contain verifiable bibliographic inconsistencies rather than formatting artifacts.
- Annotation procedure: Annotators represent citations as structured metadata tuples and assess scholarly-record validity and consistency against authoritative sources.
- Annotation procedure: Manual verification includes authoritative search, metadata cross-checking, existence validation, and hallucination confirmation.
- Labeling standard: Minor formatting differences are accepted, whereas inconsistencies in essential fields such as titles, authorship, venues, or identifiers produce hallucinated labels.
- Quality control: At least two authors independently review every citation, resolve disagreements by consensus, and exclude cases lacking consensus.
- Quality control: A random subset is re-checked after initial labeling to help prevent systematic bias and maintain consistent criteria.
- Agent responsibilities: The Planning Agent routes citations through memory, web, and scholar verification but does not judge correctness.