Source-linked AI summary
Detecting and Repairing Hallucinations in Retrieval-Augmented Generation
Sai Krishna Reddy Mulakkayala, Niki van Stein, Aske Plaat
TL;DR
Detection alone leaves faulty answers unchanged, and little evidence compares what repair action should follow. This paper evaluates claim-level deletion, source replacement, and rewriting on RAGTruth, finding that all improve judged grounding but occupy different points on a grounding-preservation trade-off rather than a quality ranking.
Problem
Research has focused on detecting unsupported statements, while little evidence compares the subsequent repair choices that determine what happens to flagged answers.
Method
The study extracts individual factual claims from flagged answers in RAGTruth, checks them against retrieved sources, and compares deletion, source replacement, and rewriting.
Results
All three repair strategies improve judged grounding over unchanged answers, but removal minimizes hallucination while generative rewriting retains the most claims and text.
Takeaways & Limitations
Repair strategies should be reported as grounding-preservation operating points, because assessing usefulness requires evidence beyond grounding and retention metrics.
Takeaways & Limitations
The evaluation does not establish which condition is better for readers because no human or automatic quality evaluation was completed.
Abstract
from arXiv · showhide
Language models increasingly answer questions by consulting retrieved documents rather than memory alone, a design now common in search assistants and enterprise knowledge tools. Grounding a model in retrieved text reduces unsupported statements but does not eliminate them, and a reader cannot tell a grounded sentence from an invented one. Most research on this problem stops at detection, yet flagging a faulty answer changes nothing for the person reading it, and little is known about which action should follow. Using RAGTruth, a benchmark whose unsupported passages are annotated by hand, we split each flagged answer into individual factual claims, check each against the retrieved source, and compare leaving the answer untouched with three repair strategies of increasing richness: deleting an unsupported claim, replacing it with source text, and rewriting it. Three language models from different families judge the 916 repaired answers. Every strategy reduces the proportion of answers judged to contain unsupported content, and all three judges agree on the ordering. Deletion achieves the largest reduction while retaining least of the original answer, at 64.3% of the text, whereas rewriting retains 80.1% and reduces least. Repair is not confined to faulty answers: 83.5% of answers annotated clean are edited too. The strategies occupy different points on a grounding preservation trade-off rather than forming a quality ranking, and choosing between them needs evidence about answer usefulness that automatic metrics cannot supply.
1 Introduction
RAG reduces unsupported content but does not make invented details identifiable, while prior work has largely emphasized detection rather than what follows a flag. This paper studies claim-level repair choices and their grounding, retention, and evaluation reliability.
- RAG can reduce invented content without eliminating it, leaving readers unable to distinguish source-grounded sentences from unsupported ones.
- Prior research has focused on detecting unsupported statements, while evidence remains limited on which repair action should follow.
- The pipeline decomposes flagged answers into atomic factual claims, checks them against retrieved evidence, and removes, extracts, or rewrites unsupported claims.
- The study compares detector stability, repair strategies, judged hallucination, and claim- and text-retention outcomes.
- The contributions include a five-seed detector comparison, a controlled four-condition repair study, verifier validation, and judge-reliability analysis.
2 Related Work
Related work establishes RAGTruth as a benchmark for response-level detection and span localization, while positioning this paper as a study of repair action sets and evaluation reliability.
- RAGTruth provides word-level hallucination annotations over realistic RAG outputs, enabling response-level detection and span localization.
- Existing detectors include LettuceDetect and MiniCheck, but this paper does not propose a new detector or claim a direct state-of-the-art comparison.
- Prior work on verification and self-correction motivates explicit verifier and repair stages, while this study examines how enlarging the repair action set changes grounding and retention.
- QLoRA enables parameter-efficient detector scaling, and multiple judge families are used because LLM-generated labels are not human labels.
3 Method
The method routes flagged answers through claim decomposition, source verification, and nested repair actions, then evaluates complete repaired answers with three language-model judges.
- A detector produces response-level labels and hallucination spans from a source passage and generated response.
- A deterministic splitter creates atomic claims, TF–IDF retrieves source evidence, and a prompted verifier assigns each claim a support probability.
- The pipeline evaluates unchanged answers alongside three nested repair action sets in a four-condition paired design.
- Claims below P(support) = 0.50 enter repair, using one shared routing threshold.
- Removal drops unsupported claims; extractive repair substitutes an accepted source fragment; generative repair rewrites claims with extractive fallback.
- All judges label hallucination spans for complete answers, while retention measures surviving material rather than completeness, helpfulness, or fluency.
4 Experimental Setup
Experiments use RAGTruth with fixed detector and repair protocols, five seeds per detector size, and paired evaluation across unchanged and repaired answers.
- The detector uses RAGTruth’s 2,700-response test split, while repair evaluates 916 flagged responses containing 7,926 atomic claims.
- Measured against human annotations, the selection step has precision 80.8% and recall 78.5%.
- The repair population, threshold, prompts, policies, outcomes, and five primary contrasts were fixed before outcome computation.
- The detector comparison holds model family, context length, epochs, data split, and QLoRA protocol constant across 7B and 13B configurations.
- Five training seeds are used for each detector size, but matching seed numbers do not create comparable initializations across architectures.
- Table 1 reports response-level F1 across five seeds per model size and separately summarizes all five runs and four usable runs after inference failures.
5 Detection Results
The detection results show substantial instability across training runs and no reliable model-size advantage, while verifier ranking is stronger than its operating-threshold precision. The repair experiment also inherits predictions from the best of five detector runs.
- Detector stability: Five-seed evaluation exposes unusable generations and makes all-runs detector means unsuitable as capability estimates.One run in five at each size failed to produce usable generations, mixing inference-failure rates with capability measurements.
- Detector stability: Among usable runs, the 13B detector averages 78.75 F1 versus 74.83 for 7B, but the 3.93-point gap is not statistically reliable.The 95% confidence interval is −5.52 to 13.37, including zero, with Welch’s test p = 0.284.
- Detector stability: The repair experiment uses predictions from the best of five detector runs, so its repaired population depends on a selection that a median run would change.The paper reports that median-run selection would produce a different, probably noisier population.
- Verifier validation: The verifier is evaluated against human span annotations using claim alignment and overlap-based gold-unsupported labels.A claim is gold-unsupported when its character span overlaps an annotated hallucination span in the same response.
- Verifier validation: At threshold 0.50, verifier ranking quality reaches ROC AUC 0.881, while precision is 0.401 at recall 0.858.The threshold flags 2,584 claims, including 1,035 gold-unsupported and 1,549 non-unsupported claims.
7 Repair Results
Across three judges, all repair strategies reduce judged hallucination, but richer repairs preserve more content while reducing fewer hallucination labels. The results therefore define a grounding–preservation trade-off rather than a single quality ranking, with task mix and verifier threshold affecting interpretation.
- Judged Hallucination: All three repair strategies lower the proportion of answers judged hallucinatory relative to the untouched control under all three judges.Every record is evaluated under all four conditions, and the judges agree on the ordering of conditions.
- Judged Hallucination: The generative condition is labelled hallucinatory for 65.2% of answers by Qwen but 97.7% by Gemma.Absolute hallucination estimates differ substantially even though judges order the conditions identically.
- Judged Hallucination: Extractive repair exceeds removal-only by 7.53 points under Qwen, 3.82 under Llama and 1.42 under Gemma.Generative repair also exceeds extractive repair under Qwen and Gemma, while the 1.42-point Llama difference is not significant.
- Behaviour by Task Type: On the primary judge, removal-only reductions are 48.0 and 48.5 percentage points for summarisation and question answering, versus 33.9 for data-to-text.Data-to-text supplies 63.1% of the intervention population, so aggregate figures are closer to that task.
- Repair of Answers Annotated as Clean: The pipeline edits 83.5% of answers annotated clean and 98.1% annotated hallucinatory, so reductions on clean answers may reflect judge disagreement rather than corrected errors.The data cannot establish whether these edits remove missed unsupported content or damage sound answers.
- Grounding–Preservation Trade-off: Removal-only reduces judged hallucination most, while generative repair retains 80.1% of text and produces the smallest reduction.The repair arms occupy different points on a consistent grounding–preservation trade-off.
8 Judge Reliability
The judges agree on the direction and ordering of repair effects, but their absolute estimates and pairwise agreement are weak. This supports the qualitative trade-off more strongly than any single numerical result.
- All judges agree that every repair arm improves on the original and rank the arms in the same order.
- Pairwise Cohen’s κ ranges from 0.072 to 0.481 across conditions, with no pair reaching 0.60.
- Judge-estimated effect magnitudes differ by up to approximately a factor of nine.
- Gemma labels more than 94% of every condition as hallucinatory, producing much smaller absolute changes than Qwen or Llama.
- The results support the direction and ordering of the trade-off more strongly than any single numerical estimate.
9 Discussion
The discussion finds no universally best repair strategy: removal, extraction, and rewriting trade grounding against retained content. It also shows that verifier breadth and detector variability complicate interpretation.
- Among usable runs, the 13B detector exceeds the 7B detector by 3.93 F1 points, but the difference is not statistically reliable.
- Under this training recipe, run-level variation is at least as important as nominal parameter count.
- All three action sets improve judged grounding, but no repair strategy dominates both grounding and retention outcomes.
- Removal-only yields the lowest hallucination rate, while generative repair retains the most claims and text.
- The verifier ranks unsupported claims well, but its low precision flags many claims that human annotations do not mark as unsupported.
10 Limitations and Future Work
The study’s conclusions are bounded by its benchmark, selection procedure, detector experiments, and automatic evaluation. Future work should add human assessment, replication, and explicit trade-off optimization.
- All results come from a single benchmark, so generalization to other domains, retrievers, generators, and evidence distributions is not established.
- The detector comparison has only five runs per model size, with one inference failure per size and a correspondingly wide confidence interval.
- The repair evaluation covers detector-flagged answers only, missing 203 human-annotated hallucinated answers and reporting no full-test-split effect.
- The verifier’s fixed threshold has precision 0.401 at recall 0.858, so many claims entering repair are not gold-unsupported.
- Retention measures surviving material rather than completeness, helpfulness, coherence, or fluency, and no answer-quality evaluation was completed.
- A blinded multi-annotator study should measure factual support, completeness, helpfulness, fluency, and overall preference for the same outputs.
11 Conclusion
The paper evaluates claim-level repair for hallucinations in retrieval-augmented generation. Repair improves automatically judged grounding, but strategies represent different grounding–preservation operating points rather than a simple quality ranking.
- Across five training seeds per model size, the 13B detector exceeds the 7B detector by 3.93 F1 points without statistical reliability.
- The experiment does not establish a consistent benefit from increasing detector size.
- On 916 detector-flagged answers, all three repair strategies reduce automatically judged hallucination relative to the original under three judge families.
- Removal-only produces the largest reduction, whereas generative repair retains the most content.
- The methods occupy different points on a grounding–preservation trade-off rather than forming a simple quality ranking.
- A repair pipeline can look better on hallucination metrics by deleting more answer content, so grounding and usefulness require separate assessment.
A Reproducibility Details
The study uses QLoRA-fine-tuned Llama-2 detectors and quantized Qwen-based verification and repair components, evaluated with three untuned language-model judges.
- Detector training: Both detectors are 7B and 13B Llama-2 base models fine-tuned with QLoRA on frozen 4-bit NF4-quantised bases.Training used LoRA rank 32, alpha 64, a learning rate of 5 × 10−4, three epochs, 2,048-token context, and effective batch size 16.
- Detector training: Detector training used five seeds, development-data checkpoint selection, and one test evaluation per trained run.Seeds were 13, 21, 42, 84, and 100; bootstrap resampling used seed 42.
- Verification and repair: A 4-bit quantized Qwen-2.5-32B verifier routed claims using a support threshold fixed at 0.50.The Qwen-2.5-7B repair planner was invoked on every eligible claim in the generative arm.
- Evaluation: Three judges—Qwen-2.5-32B, Llama-3.1-70B, and Gemma-2-27b-it—used the same prompt without per-judge tuning and decoded greedily at temperature 0.Detector training used one A100 40 GB; verifier, rewriter, and judge inference used one or two A100 GPUs.