PaperlayerView exact revision on arXiv

Source-linked AI summary

HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

Tzu-Ling Lin, Dong-Ting Yao, Teng-Fang Hsiao, Wei-Chih Chen, Hong-Han Shuai

arXiv:2609.03580v1cs.AIcs.CL

TL;DR

Prior work lacked a dedicated benchmark for verifying hallucinated claims in scientific peer reviews against source papers. HalluPeer builds a taxonomy-driven, paper-grounded benchmark, and its evaluations find that existing verifiers struggle while domain-specific fine-tuning improves performance.

  • Problem

    Prior work lacked a dedicated benchmark for detecting, categorizing, and localizing hallucinated review claims grounded in submitted papers, despite review hallucination being a realistic risk.

    Source: §2.2 · p. 3

  • Method

    HalluPeer defines hallucination as paper-grounded verification and constructs aligned paper, human-review, and injected-review triples using a hierarchical taxonomy and aspect-aware injection pipeline.

    Source: §1 · pp. 1–2, §7 · p. 9

  • Results

    Across detection, classification, and localization, general verifiers struggle to distinguish unsupported claims from critique; domain-specific fine-tuning improves performance, and authentic reviews exhibit HalluPeer-defined patterns.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Scientific review auditing requires specialized, source-aware verification, although source attribution remains a challenge for trustworthy AI-assisted peer review.

    Source: §1 · p. 2, §7 · p. 9

    Naturally occurring review hallucinations are sparse, and broader sampling could improve distributional coverage at the cost of noisier supervision.

    Source: §8 · pp. 9–10

Abstract

from arXiv · show

The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but unsupported claims that undermine review reliability. Existing hallucination benchmarks are not designed for peer review, where verification requires grounding claims in long, technical papers. We introduce HalluPeer, a benchmark for detecting hallucinations in scientific peer reviews, providing aligned triples of paper content, human-written reviews, and hallucination-injected reviews, annotated for detection, classification, and localization. Our pipeline induces a peer-review-specific hallucination taxonomy, identifies review contexts, and injects hallucinations with automated filtering. Experiments on 12K papers and 38K reviews show that existing detectors struggle to separate hallucinations from legitimate critique, while evaluation on authentic reviews demonstrates that HalluPeer-defined hallucination patterns occur in real peer reviews, highlighting the critical need for source-aware verification. Our project page can be found in https://github.com/Lin-TzuLing/HalluPeer.git

1 Introduction

HalluPeer frames scientific-review hallucination detection as paper-grounded verification and provides a taxonomy-driven benchmark for evaluating it. The introduction reports that existing verifiers struggle with technical grounding and legitimate critique, while specialized evaluation can detect errors in authentic reviews.

1 Introduction

  • Peer-review pressure is rising as submissions to major NLP conferences grew from approximately 3.4K in 2020 to over 8K in 2025, while fluent but unsupported review claims can mislead editorial decisions.The introduction describes LLM-assisted reviewing as an emerging response to this pressure and identifies hallucination detection as essential for reliability.Source: §1 · p. 1
  • Review verification requires synthesizing evidence across long technical papers while separating factual claims from subjective critique, needs that existing hallucination datasets rarely support with grounded evidence and fine-grained types.The introduction identifies wrong numbers, fabricated comparisons, and unsupported attribution as distinct error types.Source: §1 · p. 1
  • HalluPeer defines a hallucinated review claim as unsupported by or incorrect with respect to the submitted paper, separating factual grounding from tone and subjective judgment.The benchmark aligns paper content, human-written reviews, and hallucination-injected reviews, with annotations for detection, type classification, and localization.Source: §1 · pp. 1–2
  • Its construction induces a peer-review-specific hierarchical taxonomy, matches hallucination templates to aspect-compatible review contexts, and uses constrained LLM editing followed by automated filtering.The pipeline segments human reviews into sentences, assigns aspect tags, and preserves review style during injection.Source: §1 · p. 2
  • General verifiers confuse unsupported claims with legitimate critique, and LLM judges struggle with technical grounding; HalluPeer-trained detection recovers all expert-annotated authentic-review hallucinations, though source attribution remains challenging.Cross-generator, cross-venue, and authentic-review evaluations show robustness beyond the synthetic setting, while domain-specific fine-tuning substantially improves performance.Source: §1 · p. 2

2 Related Work

Existing hallucination resources do not capture verification across long technical papers, and LLM-generated peer reviews can contain unsupported claims despite the growing use of LLMs in review tasks.

2.1 LLM Hallucination Detection

2.2 LLMs in Automated Peer Review

3 Taxonomy of Review Hallucinations

HalluPeer develops a hierarchical, review-specific taxonomy with operational definitions that support controllable hallucination injection and fine-grained evaluation labels.

3 Taxonomy of Review Hallucinations

  • General-domain distinctions such as factual versus faithfulness are too coarse for peer reviews, where unsupported claims can involve paper-specific numbers or attribution errors.The taxonomy refines these categories into fine-grained review hallucination types with operational instructions for injection and evaluation.Source: §3 · p. 3

3.1 Top-Down Taxonomy Construction

  • The taxonomy is a rooted tree whose leaf descriptions specify how hallucinations appear in review text, serving as injection instructions and fine-grained evaluation labels.It begins with nine coarse anchors: Number, Entity, False Concatenation, Attribution Failure, Overgeneralization, Reasoning Error, Hyperbole, Temporal, and Context-based Meaning Error.Source: §3.1 · p. 3

3.2 LLM-Guided Recursive Decomposition

  • LLM proposers recursively expand concepts and define child types; a node becomes a leaf at the maximum depth or when no valid subdivision remains.EXPANDM generates child concepts, while DESCRIBEM provides operational definitions of how each child manifests in review text.Source: §3.2 · pp. 3–4
  • Generated sibling categories are prompted to be minimally overlapping yet collectively cover their parent, grounded in concrete review scenarios, and kept at comparable granularity.The proposer returns an empty list when a parent is already atomic at the current depth.Source: §3.2 · p. 4

3.3 Ensemble and Expert Refinement

  • Multi-model consensus and expert review refined the taxonomy to 265 nodes, after overlap filtering reduced 86,800 candidate pairs to 434 high-overlap pairs.The workflow generates trees with multiple LLMs, filters for cross-model agreement, identifies global overlaps, and has experts decide which concepts to merge or keep distinct.Source: §3.3 · p. 4, Figure 3 · p. 4

4 HalluPeer Construction Pipeline

HalluPeer constructs review-grounded hallucinations by selecting and aspect-labeling source reviews, matching sentences to applicable taxonomy concepts, and filtering generated outputs for quality.

4.1 Data Collection

  • ICLR and NeurIPS reviews are collected, aspect-labeled, and conservatively filtered for high-confidence alignment with their meta-reviews before injection.Aspect labels constrain which hallucination types fit each sentence, following prior review-aspect analysis (Lu et al., 2025); meta-reviews serve as a proxy for salient review points, not factual ground truth.Source: §4.1 · p. 4

4.2 Injection Template Construction

  • Templates pair review sentences with fine-grained taxonomy concepts, after screening for applicable concepts and filtering for natural fit to the sentence and aspect.The initial screening narrows the search—for example, Number concepts require explicit numeric values—before CHECKM filters incompatible templates.Source: §4.2 · p. 5

4.3 Automated Injection Pipeline

  • INJECTM generates fluent, coherent hallucinated sentences from feasible templates, and VERIFYM discards outputs judged semantically equivalent to their originals.The pipeline provides sentence-level control over hallucination types and broad taxonomy coverage; verification is post-generation quality control, distinct from the template-level applicability check.Source: §4.3 · p. 5

5 Experimental Results

Domain-specific fine-tuning produced strong results across detection, hallucination-type classification, and localization; the detectors also transferred across venues and generators and recovered authentic reviewer hallucinations.

5.1 Task Formulation

  • The benchmark evaluates hallucination presence, type classification, and span localization using review- and sentence-level detection and categorization.Detection reports Accuracy, Precision, Recall, F1, and MCC; classification reports Macro-F1 and Micro-F1; localization reports Token-F1, Exact Match Span-F1, and Overlap Span-F1.Source: §5.1 · p. 5, §5.3 · p. 6

5.2 Baselines

  • Detection baselines include specialized verifiers, prompting-based LLMs, and instruction-tuned LLMs; type classification and localization compare prompting-based and instruction-tuned LLMs.Specialized verifiers and RA-LLM use BM25-retrieved evidence; other prompting LLMs are evaluated with full-paper and LCS variants.Source: §5.2 · pp. 5–6

5.4 Hallucination Detection (Task 1)

  • Qwen3-32B fine-tuned achieved F1 of 0.90/0.91 at review/sentence level and sentence-level MCC of 0.87, the strongest overall detection results.Specialized verifiers had review-level MCC scores of 0.03 or lower, while RA-LLM (KR) led sentence-level prompting methods with MCC 0.61 and Accuracy 0.82.Source: §5.4 · p. 6, Table 1 · p. 7

5.5 Hallucination Type Classification (Task 2)

  • Fine-tuning raised challenging-category F1 to 0.72/0.87 for Hyperbole, 0.48/0.87 for Temporal, and 0.56/0.82 for Context-based Meaning Error.Zero-shot prompting struggled with semantically complex categories and review-level classification; lexical-cue categories such as Entity and Number performed relatively better.Source: Table 2 · p. 7, §5.5 · p. 7

5.6 Hallucination Localization (Task 3)

  • Fine-tuned Qwen3-32B reached 0.91 Token-F1 and 0.86 Exact Span-F1, outperforming prompting-based approaches across localization metrics.For GPT-5.2, Overlap Span-F1 was 0.58 versus 0.46 Exact Span-F1, illustrating the difficulty of exact boundary detection.Source: §5.6 · pp. 7–8, Table 3 · p. 8

5.7 Cross-Venue Transferability

  • Qwen3-32B transferred across NeurIPS and ICLR, with review/sentence detection F1 of 0.90/0.91 and 0.86/0.93 in the two directions.Cross-venue localization also remained strong: NeurIPS-to-ICLR achieved 0.91 Token-F1 and 0.88 Exact Span-F1, while the reverse achieved 0.92 and 0.87.Source: §5.7 · pp. 8–9

5.8 Cross-Generation Ablation

  • Across unseen hallucination generators, Task 1 F1 shifted by only 0.01–0.02 on average, Task 2 shifts were generally within 0.05, and Task 3 Token-F1 stayed within 0.02.Detectors were trained exclusively on Qwen3-32B-injected data; the taxonomy and construction protocol were unchanged for the Mistral-Small-3.1 and Llama-3.3-70B test sets.Source: §5.8 · p. 9

5.9 Evaluation on Authentic Reviews

  • On 1,161 independently annotated NeurIPS reviews containing 20 genuine hallucinations, Qwen3-32B recovered all 20 at TPR 100.0% and FPR 22.1%.The detector was trained exclusively on synthetic HalluPeer ICLR 2024 data and had no access to the authentic annotations.Source: §5.9 · p. 9

6 Alignment with Real Review Errors

A case study of 13,803 human-written NeurIPS 2024 reviews found that authentic review errors align with HalluPeer’s taxonomy. It also exposed false positives when reviewers quote unsupported claims from submitted papers, underscoring the need for source-aware attribution.

Hallucination Types in Authentic Reviews

  • Manual inspection found that errors in authentic NeurIPS 2024 reviews align with HalluPeer’s taxonomy of hallucination patterns.A fine-tuned detector flagged potential hallucinations among 13,803 reviews, and researchers manually inspected 200 flagged instances.Source: §6 · p. 9

False Positives from Paper Claim Quotation

  • Reviewer quotations of unsupported paper claims can trigger false positives, so verification must distinguish reviewer errors from propagated claims and valid criticism.The quoted statements may originate in the submitted paper rather than from the reviewer.Source: §6 · p. 9

7 Conclusion

HalluPeer benchmarks hallucination detection in peer reviews as paper-grounded, long-context verification, supported by a hierarchical taxonomy and aspect-aware injection pipeline. Experiments find that existing verifiers struggle to separate unsupported claims from legitimate critique, while authentic reviews exhibit HalluPeer-defined patterns and source-aware attribution remains challenging.

7 Conclusion

  • HalluPeer frames peer-review auditing as paper-grounded verification that requires long-context reasoning over manuscripts.It is a benchmark for hallucination detection in peer reviews.Source: §7 · p. 9
  • HalluPeer supports systematic evaluation with a hierarchical taxonomy and an aspect-aware pipeline for injecting realistic hallucinated reviews.Source: §7 · p. 9
  • Existing verifiers struggle to distinguish unsupported claims from legitimate scientific critique, while domain-specific finetuning substantially improves performance; authentic reviews show these patterns, but source-aware attribution remains challenging.The authentic-review evaluations provide evidence that HalluPeer-defined hallucination patterns occur in real peer reviews.Source: §7 · p. 9

8 Limitations

HalluPeer is a controlled, synthetic benchmark rather than a model of the full distribution of real-world review hallucinations. Its scope is limited by domain-specific data, a factual-grounding definition, model-guided taxonomy construction, and selective review sampling.

Limitations

  • Naturally occurring review hallucinations are sparse and costly to annotate, so HalluPeer covers plausible error patterns in a controlled framework rather than modeling their real-world distribution.The framework is intended to support research on automated peer-review auditing and review-quality tracking.Source: §8 · pp. 9–10
  • Although aspect-conditioning and style preservation aim to simulate realistic errors, injected hallucinations may not capture subtle drift-based errors or complex reasoning failures in real reviews.The dataset therefore remains synthetic, and localized editing may not reproduce more complex naturally occurring errors.Source: §8 · p. 10
  • Because the data come from computer science conferences on OpenReview, HalluPeer’s taxonomy and detectors may not generalize zero-shot to other disciplines without adaptation.Reviewing norms, claim structures, and evidence densities vary across scientific disciplines.Source: §8 · p. 10
  • Hallucination is defined as factual inconsistency with submission content; subjective issues such as unfair novelty judgments, tone, and critiques of future work are outside scope.Those aspects require evaluation frameworks beyond factual grounding.Source: §8 · p. 10
  • Because the taxonomy comes from LLM-guided decomposition, it may inherit proposer-model biases and represent a model-centric, noncanonical, nonexhaustive categorization.Those biases could affect how hallucination types are grouped or defined.Source: §8 · p. 10
  • Meta-review alignment favors reviews whose main points appear in final assessments, trading broader distributional coverage for more reliable labels and less noisy supervision.Broader sampling could improve coverage but increase supervision noise and the risk of including unreliable reviews.Source: §8 · p. 10

9 Ethical Considerations · A Statistics

HalluPeer’s authors identify dual-use risks in hallucination injection and emphasize that synthetic reviews were used only for offline research, with AI serving as a diagnostic aid rather than an autonomous reviewer.

9 Ethical Considerations

  • The hallucination-injection pipeline could be misused to generate more convincing hallucinated reviews.The authors restrict the taxonomy and generation scripts to defensive research purposes as a mitigation.Source: §9 · p. 10
  • No hallucinated reviews were submitted to real venues or used in editorial decisions.Human-written reviews were synthetically corrupted to construct negative samples.Source: §9 · p. 10
  • The dataset is intended solely for offline training and evaluation.Source: §9 · p. 10
  • The authors advocate human-in-the-loop review systems, with AI as a diagnostic aid rather than an autonomous decision-maker.Source: §9 · p. 10

A.1 HalluPeer Dataset Statistic … B.2 Review Aspect Tagging

HalluPeer combines a large, controllably injected review dataset with a hierarchical taxonomy of review-specific hallucinations. The appendices describe the taxonomy’s structure and examples, the construction models, and zero-shot review-aspect tagging.

A.1 HalluPeer Dataset Statistic

  • 12,000 papers, 38,063 reviews, and over 1.02M review sentences make up the dataset, with more than 10.1M hallucination templates and an average of 9.86 templates per sentence.The data uniformly samples 1,200 papers from each ICLR and NeurIPS venue-year pair; injection counts can be adjusted to construct positive and negative instances.Source: §A.1 · p. 12, Table 4 · p. 13

A.2 HalluPeer Taxonomy Statistics

  • The taxonomy has 265 total nodes and 205 leaf nodes; 200 of its leaves (approximately 98%) are at Depth 3, and its maximum depth is 3.Source: §A.2 · p. 12

A.3 HalluPeer Taxonomy Examples

  • The taxonomy recursively decomposes nine broad hallucination categories into domain-specific subtypes and operational leaf instructions, such as Number to Year Discrepancy to Citation Year Substitution.The categories follow prior work on hallucination categorization; other examples include Year Discrepancy subtypes Event Year Fabrication and Citation Year Substitution.Source: §A.3 · p. 12, Figure 5 · p. 14

B.1 Model Configuration

  • Qwen3-32B serves as the backbone for review filtering, feasibility checking, hallucination injection, and semantic verification.Taxonomy expansion and concept-description generation use Qwen3-32B, Llama-3.3-70B, and Mistral-Small-3.1-24B; generation uses temperature 0 with extended reasoning disabled.Source: §B.1 · pp. 12–13

B.2 Review Aspect Tagging

  • Review-aspect tagging uses Llama-3.3-70B in a zero-shot procedure that categorizes each review sentence into predefined aspects.The model uses temperature 0.0 and a maximum generation length of 512 tokens.Source: §B.2 · pp. 13–14

C Baseline Implementation … E.1 Hallucination Detection (Task 1)

The appendix specifies specialized and general-purpose hallucination-detection baselines and defines evaluation metrics for detection, type classification, and localization.

C Baseline Implementation

  • The specialized baselines are HHEM-2.1-Open, True-NLI, seNtLI-style retrieval-then-NLI (Schuster et al., 2022), and claim-triplet RefChecker (Hu et al., 2024).True-NLI uses the maximum entailment probability across candidate paper chunks; specialized frameworks use thresholds selected to maximize training-split F1 and fixed for testing.Source: C Baseline Implementation · p. 14
  • Prompting baselines include zero-shot LLMs and a Qwen3-32B retrieval-augmented judge using three evidence-oriented strategies (Li et al., 2023).The tested zero-shot models are Qwen3-32B, Llama-3.3-70B, GPT-OSS-20B, GPT-OSS-120B, Mistral-Small-3.1-24B, RootSignals-Judge-Llama-70B, and GPT-5.2; the strategies are Knowledge Retrieval, Chain-of-Thought, and Sample Contrast.Source: C Baseline Implementation · p. 15
  • Instruction-tuned baselines fine-tune three Qwen models with Unsloth, 4-bit NF4 quantization, and QLoRA adapters (r = 16, alpha = 32, dropout = 0.05).Adapters cover attention and MLP linear modules; training uses 8-bit AdamW, a 2 × 10^-4 peak learning rate, cosine decay, 5% warmup, and bfloat16 precision.Source: C Baseline Implementation · p. 15

D.1 Task 1: Hallucination Detection

  • For Task 1, Accuracy, Precision, Recall, and F1 use standard definitions, with MCC added to summarize performance under imbalanced hallucination labels.The appendix presents these as detection metrics among the measures defined for the three tasks.Source: §D.1 · p. 15, D Evaluation Metrics · p. 15

D.2 Task 2: Hallucination Type Classification

  • Task 2 evaluation uses Macro-F1 and Micro-F1 on ground-truth hallucinated sentences, respectively weighting hallucination types equally and instances by class prevalence.The appendix identifies these among its metrics for the three tasks.Source: §D.2 · p. 15, D Evaluation Metrics · p. 15

D.3 Task 3: Hallucination Localization

  • Task 3 reports token-F1 and span-level Exact Match and Partial Span-F1, with partial matches requiring at least one token of overlap.Spans are aligned at the token level, converted to BIO tags for token-F1, and greedily matched to avoid double-counting.Source: §D.3 · p. 15, D Evaluation Metrics · p. 15

E.1.1 Ablation Study of RA-LLM … G.1 Setup

The appendix reports additional detection, classification, localization, and transfer results, details taxonomy refinement, and evaluates detectors trained on synthetic data against authentic reviews.

E.1.1 Ablation Study of RA-LLM

  • RA-LLM (KR) is strongest among prompting variants: it reaches sentence-level MCC 0.61 and Accuracy 0.82 with BM25, while Contrast review-level MCC falls from 0.26 with No-Retrieval to 0.15 with BM25.Filtered evidence can constrain the global context needed for CoT and Contrast reasoning; few-shot demonstrations make KR more resilient to fragmented evidence.Source: §E.1.1 · p. 16, Table 8 · p. 16

E.1.2 In-Domain Results on ICLR 2024

  • On ICLR 2024, specialized verifiers remain near-random at review level (MCC ≤0.04), while fine-tuned Qwen3-32B achieves sentence-level F1 0.94 and review-level F1 tied at 0.89.RA-LLM (KR) has sentence-level MCC 0.59; even compact fine-tuned Qwen2.5-3B surpasses all zero-shot baselines.Source: §E.1.2 · p. 17, Table 9 · p. 17

E.2.1 Per-label F1 Results; E.2.2 In-Domain Results on ICLR 2024

  • For HalluPeer (ICLR 2024), Table 11 presents overall F1 results and Table 12 presents per-label F1 results, respectively.Source: §E.2.2 · p. 17

E.3.1 In-Domain Results on ICLR 2024

  • On ICLR localization, fine-tuned Qwen3-32B achieves Token-F1 0.93 and Exact Span-F1 0.90, while zero-shot performance remains limited and Overlap and Exact Span-F1 diverge.Source: §E.3.1 · p. 17

E.4 Cross-Venue Transferability

  • The appendix provides full quantitative results in Tables 14–16 corresponding to the cross-venue transfer analysis in Section 5.7.Source: §E.4 · p. 17

E.5 Cross-Generation Ablation

  • Across unseen generators, detection F1 shifts by only 0.01–0.02 for Mistral and is non-negative for Llama; classification shifts generally stay within 0.05, and localization Token-F1 stays within 0.02 of base.The review-level classification Micro-F1 improves by up to +0.066 under the Llama injector; fine-tuned Qwen3-32B’s transfer suggests it captures semantic inconsistencies rather than superficial artifacts.Source: §E.5 · pp. 17–18

F Details of the Taxonomy Generation and Refinement Pipeline; F.1 Stage 1: Multi-Model Taxonomy Ensemble; F.2 Stage 2: Global Overlap Identification; F.3 Stage 3: Human Expert Validation

G Evaluation on Authentic Reviews; G.1 Setup

  • For authentic-review evaluation, two annotators found 20 hallucinations among 1,161 NeurIPS 2024 reviews (about 1.7%); detectors trained only on synthetic ICLR data were tested on these reviews without access to their annotations.Annotation was independent of detector outputs and measured both true-positive and false-positive behavior.Source: §G.1 · p. 20, G Evaluation on Authentic Reviews · p. 20

G.2 Results and Analysis … J Prompt Example

On authentic reviews, fine-tuned Qwen3-32B detected all 20 annotated hallucinations while narrowing the review pool for screening. The appendix details the taxonomy, template, and verification procedures used to construct HalluPeer’s injection pipeline.

G.2 Results and Analysis

  • Fine-tuning improved both Qwen detectors’ TPR and FPR, while Claude Opus 4.7 reached TPR = 85.0% at FPR = 0.3%.Qwen3-32B changed from TPR = 95.0% and FPR = 29.5% to 100.0% and 22.1%; Qwen2.5-7B changed from 70.0% and 34.0% to 80.0% and 23.1%. The authors say simultaneous recall gains and FPR reductions are not explained by a simple threshold shift.Source: §G.2 · p. 20

H Annotation Protocol for Authentic Review Analysis

  • Two annotators independently inspected 200 flagged instances from 13,803 NeurIPS 2024 reviews, then resolved candidate or uncertain cases with a senior expert to reach consensus on 11 validated cases.The 200 instances were selected consecutively; annotators checked whether each was a genuine hallucination and whether its predicted type matched the manual label.Source: H Annotation Protocol for Authentic… · p. 20

I.1 Algorithm – Hallucination Taxonomy Construction

  • The taxonomy algorithm recursively decomposes predefined first-level concepts into child concepts, assigning descriptions and stopping at maximum depth or when a concept has no children.The resulting tree’s leaf nodes define the fine-grained hallucination types used for injection.Source: §I.1 · pp. 20–21

I.2 Algorithm – Hallucination Injection Template Construction

  • Template construction screens each review sentence against first-level anchor concepts, builds candidates from compatible anchors’ leaf types, then filters them with an LLM compatibility check.The check removes templates whose hallucination type cannot naturally map onto the target sentence.Source: §I.2 · p. 21

I.3 Algorithm – Automated Hallucination Injection Pipeline with Semantic Verification

  • The injection pipeline rewrites sentences using feasible templates and retains a candidate only when post-hoc verification judges it semantically different from the original.Equivalent rewrites are discarded as failed injections.Source: §I.3 · pp. 21–22

G.2 Results and Analysis

  • Fine-tuned Qwen3-32B recovered all 20 authentic hallucinations (TPR = 100.0%, FPR = 22.1%), reducing 1,161 reviews to 272 candidates (Table 20).The paper frames recall as the primary measure given hallucinations’ low natural prevalence, with FPR indicating the reviewer-level screening burden.Source: §G.2 · p. 20, Table 20 · p. 22

J.1–J.4 Prompt Examples

  • Appendix prompts cover taxonomy generation and overlap filtering, coarse category screening, and template construction.The prompt examples are identified in Figures 7, 8, 10, and 11.Source: §J.1 · p. 22, §J.2 · p. 22, §J.3 · p. 22

J.5–J.15 Prompt Examples

  • Further prompt examples cover review filtering, aspect tagging, template feasibility and injection, post-hoc verification, and task-specific evaluation prompts.The feasibility prompt requires operational applicability, the leaf type’s semantic prerequisites, aspect grounding, and a realistic edit; the appendix also provides prompts for RA-LLM baselines and general-purpose evaluation on detection, classification, and localization.Source: §J.5 · p. 22, §J.6 · p. 22, §J.7 · p. 22

Related papers

Loading 2609.03580v1…