PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparators

Dingkang Yang, Dongling Xiao, Jinjie Wei, Mingcheng Li, Zhaoyu Chen, Ke Li, Lihua Zhang

arXiv:2408.12325v5cs.CL

TL;DR

LLMs can produce plausible but unfaithful claims, while hallucinations vary across tasks and existing model-editing approaches may affect internal knowledge. CDT uses truthful and hallucinatory comparators with instruction-guided experts during decoding; the paper reports improved factuality and performance across multiple downstream tasks.

  • Problem

    LLMs can generate plausible claims that contradict verifiable facts or user instructions, while hallucinations take varied forms across tasks and existing parameter or representation interventions carry knowledge-related concerns.

    Source: Introduction · p. 1

  • Method

    CDT builds truthful and hallucinatory comparators from multi-task fine-tuning samples and uses instruction-prototype-guided experts to contrast predictions during decoding.

    Source: Introduction · p. 2, Comparator-driven Decoding-Time… · p. 3

  • Results

    Across multiple NLP benchmarks, CDT improves response factuality and performance, achieving the best results on most hallucination-recognition metrics.

    Source: Introduction · p. 2, Comparison with State-of-the-Art Methods · p. 5, Conclusion · p. 8

  • Takeaways & Limitations

    CDT provides decoding-time attribute control while preserving target LLMs’ internal knowledge, fluency, and diversity.

    Source: Introduction · p. 2

    CDT slightly increases inference time because it requires two forward propagations, and it cannot create knowledge beyond the model’s internal knowledge.

    Source: Limitations · p. 12

Abstract

from arXiv · show

Despite their remarkable capabilities, Large Language Models (LLMs) are prone to generate responses that contradict verifiable facts, i.e., unfaithful hallucination content. Existing efforts generally focus on optimizing model parameters or editing semantic representations, which compromise the internal factual knowledge of target LLMs. In addition, hallucinations typically exhibit multifaceted patterns in downstream tasks, limiting the model's holistic performance across tasks. In this paper, we propose a Comparator-driven Decoding-Time (CDT) framework to alleviate the response hallucination. Firstly, we construct hallucinatory and truthful comparators with multi-task fine-tuning samples. In this case, we present an instruction prototype-guided mixture of experts strategy to enhance the ability of the corresponding comparators to capture different hallucination or truthfulness patterns in distinct task instructions. CDT constrains next-token predictions to factuality-robust distributions by contrasting the logit differences between the target LLMs and these comparators. Systematic experiments on multiple downstream tasks show that our framework can significantly improve the model performance and response factuality.

Introduction

LLMs can produce plausible but unfaithful claims, and hallucinations vary across tasks and can co-occur. The paper proposes decoding-time comparators and instruction-guided experts to address these patterns while preserving target-model knowledge.

Introduction

  • LLMs generate plausible but non-factual claims that conflict with real-world facts or user instructions, limiting their credibility in applications.The introduction notes that hallucinations can take different forms across tasks, and multiple patterns can occur in one response.Source: Introduction · pp. 1–2, Figure 1 · p. 1
  • Existing approaches optimize model parameters or edit semantic representations, but secondary training is labor-intensive and representation interventions may interfere with internal factual knowledge.The paper also identifies unintended knowledge infusion and spurious generation patterns as concerns with secondary training.Source: Introduction · pp. 1–2
  • CDT contrasts target-model predictions with truthful and hallucinatory comparators during decoding, using instruction-prototype-guided experts to capture task-specific patterns.The framework uses multi-task samples to construct the comparators and logit-distribution integration to guide next-token prediction.Source: Introduction · p. 2
  • The authors present CDT as decoding-time attribute control that does not compromise target models’ internal knowledge and maintains generation fluency and diversity.The approach uses LoRA-based supervised fine-tuning to build the comparators and a perturbation mechanism to strengthen the truthful comparator’s factual knowledge mastery and mitigate potential overfitting.Source: Introduction · p. 2
  • The paper reports extensive experiments across multiple NLP benchmarks and describes the framework as broadly applicable and effective.It also presents the comparator approach as plug-and-play across model structures and task types.Source: Introduction · p. 2

Related Work

Prior work addresses hallucination through retrieval, data construction, task migration, model optimization, representation editing, and decoding-time intervention. The paper positions CDT as an inference-time approach intended to handle diverse patterns across tasks.

Related Work

  • Hallucination mitigation includes retrieval-augmented generation, high-quality data construction, and synthetic task migration, alongside parameter optimization and semantic-representation editing.The paper notes that existing methods may undermine LLMs’ inherent knowledge and cause performance bottlenecks on out-of-attention tasks.Source: Related Work · p. 2
  • Parameter-focused approaches include preference alignment, while representation-editing approaches activate attention heads or latent features associated with truthfulness.The section gives DPO-based factual alignment and Truth Forest’s multidimensional probing as examples.Source: Related Work · p. 2
  • Earlier decoding-time methods address fluency, toxicity, or reasoning, but the paper says they can struggle with multifaceted hallucinations and task-type sensitivity.CDT is presented as using hallucination- and truthfulness-aware comparators to adapt across distinct tasks.Source: Related Work · p. 2

Methodology

CDT combines target-model predictions with hallucinatory and truthful comparators during decoding to steer next-token distributions toward factuality. The comparators are trained on multi-task hallucinated/factual instruction pairs, with prototype-guided experts intended to capture task-specific patterns.

Comparator-driven Decoding-Time Framework

  • CDT combines target-model predictions with hallucinatory and truthful comparator distributions to steer next-token predictions toward factuality.The truthful comparator favors factual tokens while the hallucinatory comparator models non-factual attributes.Source: Comparator-driven Decoding-Time… · p. 3
  • An adaptive plausibility constraint limits penalties to a target-model-confidence-based subset of the vocabulary, retaining high-probability tokens to preserve response integrity.Larger δ values retain only higher-probability tokens.Source: Comparator-driven Decoding-Time… · p. 3

Hallucinatory/Truthful Comparator Formalization

  • Hallucinatory and truthful comparators are fine-tuned from base models on multi-task instructions and corresponding hallucinated or factual responses.The tasks include question-answering, knowledge-grounded dialogue, summarization, and general tasks.Source: Hallucinatory/Truthful Comparator… · p. 3
  • To reduce overfitting and mundane faithfulness in the truthful comparator, training perturbs embeddings using hallucinated responses before updating on factual responses.The authors describe the adversarial procedure as promoting more effective factual semantic learning.Source: Hallucinatory/Truthful Comparator… · p. 3

Instruction Prototype-guided Mixture of Experts

  • Instruction prototypes learned from multi-task data guide multiple LoRA experts to capture distinct hallucination or truthfulness patterns for different instructions.Instruction features are extracted from the target LLM's last-layer hidden state and compressed with principal component analysis.Source: Instruction Prototype-guided Mixture… · pp. 3–4
  • Prototype-informed gating activates appropriate experts for an input, using Euclidean distance to select its prototype and added noise to regularize expert load balancing.The routing gate combines the input and prototype representations; expert outputs are integrated through soft routing.Source: Instruction Prototype-guided Mixture… · p. 4

Experiments

Experiments evaluate CDT across hallucination recognition, truthfulness, summarization, and generation tasks, then test its design choices, model extensibility, and example outputs.

Datasets and Evaluation Metrics

  • Evaluation spans six datasets, including hallucination-recognition, TruthfulQA, summarization, and open-ended-generation tasks, with task-specific factuality and quality metrics.KINGHT-Judge uses 616 dialog samples; Alpaca-Judge uses 600 responses, and TruthfulQA contains 817 questions across 38 categories.Source: Datasets and Evaluation Metrics · pp. 4–5

Implementation Details

  • The main experiments use LLaMA-2 models, comparators trained on 30K HaluEval instructions plus 10K annotated Alpaca samples, and defaults of 32 prototypes and four LoRA experts.Experiments use eight Nvidia A800 GPUs.Source: Implementation Details · p. 5

Comparison with State-of-the-Art Methods

Ablation Studies

  • Ablations find complementary comparator contributions, benefits from PME and each SFT data type, and a necessary role for PAT; four experts and 32 prototypes perform best among tested settings.Two experts underperform on XSUM and Alpaca-Gen, while excess experts converge in gains; fewer prototypes perform poorly and too many are redundant.Source: Ablation Studies · pp. 6–7, Table 5 · p. 6

Extensibility Analysis

  • Across eight LLMs, CDT improves TruthfulQA multiple-choice results, averaging 14.25% improvement across six vanilla baselines.Absolute gains are 12.95% for Baichuan2-7B-Chat and 15.6% for Mistral-7B-Instruct.Source: Extensibility Analysis · p. 7, Figure 4 · p. 7

Qualitative Analysis

  • In a shared question-answering prompt, CDT corrects fabricated claims spanning inference and factuality hallucinations that remain in the vanilla baseline and ICD outputs.The examples include diamonds described as fossilized tree sap and an age claim of 5 and 8 billion years.Source: Table 6 · p. 7, Qualitative Analysis · p. 8

Conclusion

CDT uses hallucinatory and truthful comparators during decoding to address multifaceted hallucinations. Its instruction prototype-guided mixture-of-experts strategy supports more factual responses across downstream tasks by controlling next-token prediction.

Conclusion

  • CDT uses hallucinatory and truthful comparators during decoding to address multifaceted hallucinations in target LLMs.The framework controls next-token prediction to facilitate more factual responses across multiple downstream tasks.Source: Conclusion · p. 8
  • An instruction prototype-guided mixture-of-experts strategy equips comparators to master different hallucination and truthfulness patterns.Source: Conclusion · p. 8
  • CDT controls next-token prediction to facilitate more factual responses across multiple downstream tasks.Source: Conclusion · p. 8

Training Configuration Details

The comparators are trained through LoRA-based supervised fine-tuning from LLaMA2-7B-base, with different training durations for hallucinatory and truthful comparators. Inference configurations vary by dataset and were selected by grid search.

Training Configuration Details

SFT Dataset Acquisition

The hallucinatory comparator draws on multi-task hallucinated instruction-response pairs, while the truthful comparator uses the same instructions with factual responses. Factual responses are expanded from vanilla outputs and manually checked for noise or misleading information.

SFT Dataset Acquisition

  • The hallucinatory comparator uses 30k HaluEval pairs across question answering, knowledge-grounded dialogue, and summarization, spanning three hallucination patterns.The patterns are unverifiable, non-factual, and irrelevant hallucinations.Source: SFT Dataset Acquisition · p. 10
  • The training data also includes 10K held-out Alpaca samples with GPT-4-generated hallucinated responses, kept separate from test data.Source: SFT Dataset Acquisition · p. 10
  • The truthful comparator uses the same instructions with factual responses, expanded from vanilla outputs and manually checked for noise or misleading information.Source: SFT Dataset Acquisition · p. 10

Model Response Comparison

CDT corrects misinformation in qualitative comparisons on knowledge-grounded dialogue and text summarization. The examples contrast its outputs with hallucinations from the vanilla baseline and remaining errors from ICD.

Model Response Comparison

  • CDT removes potential hallucination patterns in the knowledge-grounded dialogue example, producing a more authentic and informative response than the baseline and ICD.Source: Model Response Comparison · p. 10
  • In dialogue, the baseline misidentifies “Valhalla Valkyrie” as a movie starring James Bond, while ICD substitutes Keira Knightley despite avoiding an extrinsic-hard hallucination.Source: Model Response Comparison · p. 10
  • In summarization, CDT corrects baseline misinformation, while ICD still misstates “multiple stab wounds” as “whack death.”The baseline also turns “multiple stabs” into “one-off stabbing” and fabricates past violent behavior.Source: Model Response Comparison · pp. 10–11

Additional Ablation Studies

The ablations compare prototype-learning algorithms and examine the adaptive plausibility constraint (APC). GMM performs more robustly on multi-task instructions, while APC improves text-generation performance without affecting TruthfulQA results.

Additional Ablation Studies

  • GMM yields consistently higher gains than K-Means++ and Mean-Shift across four tasks, supporting its use for instruction prototype learning.The authors attribute this to GMM's handling of complex distributions and ambiguous prototype boundaries through probabilistic confidence estimates.Source: Additional Ablation Studies · p. 11
  • APC consistently improves text-generation performance while preserving common sense, grammar, and response logicality.TruthfulQA results are unaffected because the default setting scores by probability without APC.Source: Additional Ablation Studies · p. 11

Expert Routing Selection

The routing and prototype visualizations indicate that comparators select specialized experts and task-relevant instruction prototypes. The framework slightly increases inference time and may be bottlenecked when tasks require knowledge beyond the model's internal knowledge.

Expert Routing Selection

  • Experts specialize by task: truthful Expert 1 favors open-ended generation, while hallucinatory Expert 4 favors hallucination judgment.Expert 4 in the truthful comparator favors TruthfulQA, and Expert 3 favors summarization in both comparators.Source: Expert Routing Selection · p. 11

Instruction Prototype Selection

  • Truthful prototypes 4 and 7 chiefly support TruthfulQA, while hallucinatory prototypes 9, 18, and 25 capture XSUM hallucination patterns.The latter include faithful and intrinsic hallucinations, guiding CDT to weaken erroneous information generation.Source: Instruction Prototype Selection · p. 12, Figure 7 · p. 12

Limitations

  • CDT slightly increases inference time and may face bottlenecks when situations exceed the model's internal knowledge.The framework requires two forward propagations and cannot create new knowledge.Source: Limitations · p. 12

Related papers

Loading 2408.12325v5…