PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions

1 Introduction

• LLMs can generate false, biased, or misleading claims, and their outputs are often not grounded in sources.The paper identifies these as weaknesses accompanying LLM capabilities.Source: §1 · p. 1
• Prompted editing has a steep compute-quality tradeoff, while fine-tuned editors are limited to domains with adequate training data.Post-hoc editing retrieves evidence to attribute a claim, then edits factual errors for consistency with that evidence.Source: §1 · p. 1
• PURR’s data pipeline summarizes topically related evidence into a claim, corrupts it, then trains a compact editor to denoise it using evidence.Unlike prior approaches described here, it does not assume paired relevant evidence or edit data for training.Source: §1 · p. 1
• PURR’s editor trained on challenging corruptions surpasses the same LLM used for prompted editing across multiple datasets.The paper distinguishes this from distillation, where the student generally under-performs the teacher and a challenging distillation set is vital.Source: §1 · p. 2
• Across all benchmarks, PURR improves attribution over more expensive LLM-prompted editors while running orders of magnitude faster.PURR is a fine-tuned fusion-in-decoder T5 model trained on denoising data.Source: §1 · p. 2

2.1 Task Overview

• Editing for Attribution produces an evidence report grounding a statement and a revised statement that fixes inaccuracies conflicting with that evidence.The revised statement may remain unchanged if the editor judges it cannot be further edited to improve attribution.Source: §2.1 · p. 2, §2.4 · p. 3

2.2 Evaluation Metrics

• The evaluation balances attribution of the revised statement with preservation of the original using their harmonic mean, F1AP.Attribution measures entailment by evidence; preservation measures how much information changes from the original statement.Source: §2.2 · pp. 2–3

2.3 Evaluation Sets

• The test sets cover Natural Questions, StrategyQA, and QReCC, with 150 test instances per dataset generated from strong LLM outputs.Validation sets are used for tuning, and results are reported on the test sets.Source: §2.3 · p. 3

2.4 Baselines

• EFEC fine-tunes a T5 editor on FEVER, while RARR uses iterative few-shot prompting to check evidence for contradictions and revise claims.RARR’s repeated, computationally expensive large-model calls improve attribution and preservation; EFEC often improves attribution at preservation’s expense.Source: §2.4 · p. 3

2.5 Generating the Attribution Report

• The attribution report selects up to five evidence snippets to maximize the sum of each query’s best relevance score to any evidence.The relevance scores are produced by a cross-encoder.Source: §2.5 · p. 3

3.1 Overview of PURR at Inference Time

• At inference, PURR uses fusion-in-decoder to aggregate evidence and a distilled T5-large model to generate research queries without prompting an LLM.The evidence is incorporated simultaneously rather than concatenated, and FiD is described as allowing aggregation beyond context-length limits.Source: §3.1 · p. 4

3.2 Creating Training Data via Noising

• PURR training creates clean statements by summarizing up to four retrieved gold evidence passages, then prompts an LLM to corrupt them for denoising.The pipeline starts with domain queries, retrieves and chunks web pages, and separates relevant gold passages from hard negatives; the clean summaries are reported as highly faithful upon manual inspection.Source: §3.2 · p. 4
• The corruption prompts cover hallucination-like errors including incorrect dates and entities, semantic-role errors, and quantification errors.PURR is trained to maximize P(y|x, E), where x is the corruption, y the clean statement, and E the evidence set.Source: §3.2 · p. 5

3.3 Dataset Statistics and Training Details

• Starting from roughly 6,000 seed queries, the authors generated an edit dataset of 6,000 instances; training pairs each corrupted statement with four evidence snippets, supplementing with negatives when gold evidence is insufficient.They reserve 10% for validation; adding negative evidence during training helped PURR ignore irrelevant evidence at inference.Source: §3.3 · p. 5

4.1 Primary Quantitative Results

• Across all three tasks, PURR reaches the state of the art on F1AP by improving attribution with generally smaller edits, and its size makes it significantly more efficient.Source: §4.1 · p. 5

4.2 Breaking Down the Numbers

• The breakdown defines huge edits by preservation below 0.5, bad edits by attribution dropping over 0.1, unnecessary edits as bad edits when initial attribution exceeds 0.9, and good edits by attribution gain over 0.3 with preservation above 0.7.Source: §4.2 · pp. 5–6
• On 150 Natural Questions test instances, PURR almost never makes huge edits, avoids harmful edits to near-perfect claims, and makes more good edits than baselines.The breakdown contrasts this balance with EFEC’s large edits and RARR’s occasional over-editing.Source: §4.2 · p. 6, Figure 2 · p. 6

4.3 Qualitative Analysis

• PURR is strong at fixing entity and numerical hallucinations and longer spans, while fusion-in-decoder supports edits combining multiple evidence pieces.Source: §4.3 · p. 6
• Challenging distractors in evidence can lead PURR to erroneous edits; the authors propose stronger training corruptions to address this case.Source: §4.3 · p. 6
• Among 20 low-attribution Natural Questions cases, roughly 80% were linked to incomplete question coverage or failure to find the best evidence; the authors consider editing a relatively small issue.Source: §4.3 · pp. 6–7

4.4 Inference Speed and Cost Comparisons of Fine-tuned vs Prompted Editors

• 40 seconds per statement on dozens of TPUs for RARR compares with approximately 2 seconds on a 12GB Titan-RTX GPU for PURR; generating PURR’s training set cost under $300.Source: §4.4 · p. 7

5 Related Work

• RARR and Rethinking-with-Retrieval use few-shot prompting, while FRUIT and EFEC use fine-tuned editors with Wikipedia training data; PURR’s denoising approach avoids that domain constraint.PEER also includes factuality editing, but its primary focus is collaborative writing.Source: §5 · p. 7

Improving Trust in Large Language Models

• The authors propose applying their denoising strategy to compact editors for other undesirable generation behaviors, including protected-information regurgitation and hateful content.The related work also notes input sensitivity as a safety concern.Source: Improving Trust in Large Language Models · pp. 7–8
• PURR’s approach implicitly distills information from an LLM while producing challenging training instances.The section contrasts this with related pipelines that use LLM outputs for distillation, filtering, or evaluation-set generation.Source: Improving Trust in Large Language Models · p. 8

• Prompted editing is computationally costly, while training compact editors is hindered by scarce training data.Source: §6 · p. 8
• PURR uses an entirely unsupervised process in which LLMs corrupt text and compact editors learn to denoise the resulting faux hallucinations.Source: §6 · p. 8
• PURR improves attribution over prompted editors across various datasets while executing orders of magnitude faster.Source: §6 · p. 8

• Figure 3 presents a zero-shot prompt for summarizing multiple documents.Source: Figure 3 · p. 12
• The summarization prompt accepts multiple source texts separated by newlines as its {text} input.Source: Figure 3 · p. 12
• Figure 4’s few-shot corruption prompt uses chain-of-thought to vary how many information pieces are corrupted and what kinds are changed.Source: Figure 4 · p. 12