Source-linked AI summary
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, Scott Wen-tau Yih
TL;DR
Language models may underweight supplied context, causing unfaithful or hallucinated outputs, especially when context conflicts with pretrained knowledge. The paper introduces context-aware decoding, which contrasts context-conditioned and context-free probabilities without additional training. CAD improves summarization faithfulness across model families and substantially helps resolve knowledge conflicts, although performance can decline when evaluation data appeared in instruction tuning.
Problem
Language models can insufficiently attend to supplied context, producing hallucinations and failing to override outdated prior knowledge when the context contradicts it.
Method
Context-aware decoding contrasts output probabilities with and without context to downweight prior knowledge and promote context-supported generation without additional training.
Results
CAD improves faithfulness across diverse language models and summarization tasks, including a 14.3% factuality gain for LLaMA and a 2.9x improvement on a knowledge-conflict QA dataset.
Takeaways & Limitations
CAD provides more reliable and factual outputs across language models of varied sizes and can override prior knowledge when trusted contextual information conflicts with it.
Takeaways & Limitations
A slight decline occurred on one NQ dataset because it had been included in FLAN-T5’s instruction-finetuning data.
Abstract
from arXiv · showhide
Language models (LMs) often struggle to pay enough attention to the input context, and generate texts that are unfaithful or contain hallucinations. To mitigate this issue, we present context-aware decoding (CAD), which follows a contrastive output distribution that amplifies the difference between the output probabilities when a model is used with and without context. Our experiments show that CAD, without additional training, significantly improves the faithfulness of different LM families, including OPT, GPT, LLaMA and FLAN-T5 for summarization tasks (e.g., 14.3% gain for LLaMA in factuality metrics). Furthermore, CAD is particularly effective in overriding a model's prior knowledge when it contradicts the provided context, leading to substantial improvements in tasks where resolving the knowledge conflict is essential.
1 Introduction
Language models must balance learned prior knowledge with information supplied in context, but may underweight new or contradictory evidence. Context-aware decoding addresses this by amplifying context-sensitive probabilities, improving faithfulness across models and especially in knowledge-conflict tasks.
- Language models combine pretrained prior knowledge with context knowledge supplied in the input, but their balance during generation remains unresolved.
- Insufficient attention to newly introduced context can produce summaries containing facts absent from the input document.
- Contradictory context may fail to override outdated model knowledge, as LLaMA predicts two World Cups despite a document stating Argentina won three.
- Context-aware decoding amplifies probability differences with versus without context, downweighting prior knowledge without additional training.
- CAD improves faithfulness across OPT, GPT-Neo, LLaMA, and FLAN summarization models, including a 14.3% gain in summary factuality metrics for LLaMA-30B.
- CAD is especially effective on knowledge-conflict tasks, producing a 2.9x improvement for LLaMA-30B and larger gains as model size increases.
2 Method
CAD contrasts context-conditioned and context-free model outputs to reduce reliance on prior knowledge when context is unfamiliar or conflicting. It normalizes a PMI-weighted product-of-experts distribution and samples from the adjusted probabilities.
- CAD addresses cases where out-of-distribution or conflicting context is underweighted and the model instead relies on encoded prior knowledge.
- The method factors out prior knowledge contrastively by comparing output probabilities conditioned on context with those conditioned only on the query and prefix.
- CAD forms a product-of-experts distribution combining the original output probability with context-generation PMI weighted by α, favoring context-supported outputs.
- The unnormalized expression is normalized across possible next tokens before sampling from the adjusted distribution.
- Larger α increases adjustment strength, while α = 0 recovers regular decoding; standard sampling strategies can then operate on the adjusted distribution.
- CAD is a contrastive ensemble of logits from context-conditioned and context-free model evaluations.
3 Experimental Setup
The evaluation tests CAD with prompting on context-dependent summarization and knowledge-conflict tasks across pretrained and instruction-finetuned language models. Experiments use standard task-specific datasets, metrics, baselines, and fixed adjustment settings.
- The experiments evaluate whether language models read and reason over contexts while producing outputs faithful to those contexts.
- Summarization: Summarization uses CNN-DM and XSUM, with ROUGE-L for quality and BERT-Precision plus FactKB for factual consistency.
- Knowledge Conflicts: Knowledge-conflict evaluation uses MemoTrap, which tests memorization traps by changing proverb endings, and NQ-Swap, a question-answering dataset.
- Task Inputs: Across datasets, CAD is intended to upweight source documents in XSUM and NQ-Swap and instructions in MemoTrap.
- Models: Models include OPT, GPT-Neo, LLaMA, and instruction-finetuned FLAN-T5 variants spanning approximately 2.7B to 30B parameters.
- Settings: The adjustment uses α = 0.5 for summarization and α = 1 for knowledge-conflict tasks, while CAD retains the baseline sampling strategies.
4 Results
CAD improves summary quality and factuality across diverse language models and substantially helps resolve knowledge conflicts. Its gains remain consistent across model sizes in CNN-DM and increase with size on conflict-focused tasks.
- Main Results: CAD outperforms regular decoding across eight models on CNN-DM and XSUM for both summary quality and factuality.The evaluated metrics are ROUGE-L, factKB, and BERT-P.
- Main Results: 21% increase in ROUGE-L, 14.3% increase in factKB, and 7.8% increase in BERT-P result from applying CAD to LLaMA-30B on CNN-DM.
- Main Results: CAD significantly outperforms regular decoding on knowledge-conflict datasets, except for a minor FLAN-T5 decrease on non-conflict NQ.The reported exception is attributed to FLAN-T5 having seen NQ during instruction-finetuning.
- Main Results: 54.4% improvement on MemoTrap and 128% on NQ-SWAP occur for GPT-Neo 20B with CAD.These gains are reported for knowledge-conflict datasets.
- Analysis: CAD’s gain stays consistent across OPT sizes on CNN-DM but increases with model size on Memotrap and NQSWAP.The pattern suggests larger models rely more on prior knowledge in knowledge-conflict tasks.
- Analysis: λ = 0.5 consistently provides robust improvements across three datasets, while larger adjustment values further improve knowledge-conflict tasks.The adjustment level α controls how strongly CAD changes the original next-token distribution.
5 Related Work
Prior work studies hallucination reduction, knowledge conflicts, and contrastive decoding, but context addition alone may not override parametric knowledge. CAD addresses this gap with a decoding strategy applicable across language models.
- Summarization Factuality: Summarization research has pursued factual consistency through source-document fact triples, factuality objectives, and post-editing correction.
- Knowledge Conflicts: Knowledge-conflict settings evaluate whether models follow updated contextual information instead of outdated parametric knowledge.This setting is especially relevant to retrieval-augmented language models using external documents.
- Knowledge Conflicts: Existing prompting-based approaches are limited to large instruction-finetuned models, whereas CAD is presented as a decoding strategy applicable to any LM.
- Contrastive Decoding Methods: Contrastive decoding methods compare differently conditioned model outputs to improve generation properties such as coherence or diversity.CAD applies the same contrastive intuition to knowledge-conflict scenarios.
6 Conclusion
The paper presents CAD as a simple inference-time method for improving models’ attention to supplied context over learned prior knowledge. Across summarization and knowledge-conflict tasks, CAD produces more reliable and factual outputs across language models of various sizes.
- CAD downweights output probabilities associated with prior knowledge to promote attention to contextual information.
- CAD is evaluated on summarization and knowledge conflicts tasks, both requiring strong attention to context.
- CAD provides more reliable and factual outputs across different language models of various sizes.