PaperlayerView exact revision on arXiv

Source-linked AI summary

SFAD: Speculative Factuality-Aware Decoding

Guanqiao Chen, Di Wang, Lijie Hu

arXiv:2609.00796v2cs.CL

TL;DR

Contextual faithfulness is difficult to improve without sacrificing inference efficiency because existing decoding and alignment approaches add substantial computation. SFAD trains a context-faithful draft model with ConFide and DPO, then uses confidence-weighted friction detection and selective logit steering during speculation. It substantially improves faithfulness while delivering a 2.48× speedup.

  • Problem

    Existing methods for improving contextual faithfulness can double decoding cost or require extensive compute and large-scale preference data.

  • Method

    SFAD trains a context-faithful draft model on ConFide via Direct Preference Optimization and adaptively steers target logits when Epistemic Friction detects potential hallucinations.

  • Results

    2.48× speedup accompanies substantial faithfulness improvements across diverse benchmarks, approaching the performance of models 5× larger.

  • Takeaways & Limitations

    SFAD simultaneously improves contextual faithfulness and speculative-decoding efficiency by selectively correcting the target distribution while preserving standard speculation otherwise.

  • Takeaways & Limitations

    SFAD requires a domain-aligned DPO-trained draft model, and its friction threshold may require tuning for out-of-distribution domains.

Abstract

from arXiv · show

As one of the most critical challenges in large language models, contextual faithfulness directly determines their reliability in knowledge-intensive applications. This task is particularly challenging as it requires balancing factual consistency with generation efficiency. Contrastive decoding methods require dual forward passes (with and without context) to compare model outputs, doubling inference computational overhead, while post-training alignment demands extensive reinforcement learning with substantial computational overhead. To address this challenge, we present SFAD, a speculative decoding framework that enhances contextual faithfulness without inference degradation. We first construct ConFide, a preference dataset with fine-grained atomic perturbations, to train a context-faithful draft model via Direct Preference Optimization. During inference, Epistemic Friction detects potential hallucinations by quantifying distributional tension weighted by specialist certainty. When friction exceeds the threshold, Asymmetric Logit Steering refines the target distribution through residual-based logit injection; otherwise, standard speculation proceeds. Extensive experiments demonstrate that SFAD substantially improves faithfulness while achieving $2.48\times$ speedup, offering a practical solution for efficient LLMs.

1 Introduction

LLMs often prioritize static parametric knowledge over provided context, causing hallucinations and undermining contextual faithfulness. SFAD addresses this tension by combining a context-faithful draft model with adaptive correction while preserving speculative-decoding efficiency.

  • Static parametric knowledge can become outdated or incomplete, while knowledge conflicts cause LLMs to favor internal knowledge over task-specific context.
  • Contrastive decoding requires dual forward passes, effectively doubling computational cost and halving generation speed.
  • Post-training alignment methods typically require extensive compute and large-scale preference data, motivating efficient faithfulness-enhancement approaches.
  • SFAD trains a context-faithful draft model on ConFide and ConFiQA via Direct Preference Optimization to detect and correct hallucinations during speculative decoding.ConFide uses atomic decomposition and controllable perturbations to create hard negatives.
  • SFAD uses Epistemic Friction to detect potential hallucinations and Asymmetric Logit Steering to selectively refine target distributions; otherwise, standard speculation proceeds.This adaptive mechanism maintains the 2 ∼3× speedup of speculative decoding.
  • 2.48× speedup is achieved alongside substantial faithfulness improvements across diverse benchmarks with minimal computational overhead.The framework is reported to approach the performance of models 5× larger.

2 Related Work

Prior work addresses hallucination mitigation, speculative-decoding efficiency, and specialized inference settings, but SFAD targets hallucination mitigation within speculative decoding.

  • Hallucinations include factuality errors that contradict real-world facts and faithfulness errors that conflict with provided inputs or grounding documents.
  • Mitigation strategies span training data curation and grounding, confidence estimation, knowledge retrieval, and inference-time editing.
  • Speculative decoding accelerates inference by verifying tokens generated by a smaller draft model against a target LLM.
  • Speculative-decoding research has explored architectural extensions, tree-based verification, sparse MoE acceleration, reinforcement-learning enhancements, long-context retrieval, multi-sample inference, quantized caching, and safety alignment.
  • SFAD differs from performance-driven or safety-centric speculative-decoding work by targeting hallucination mitigation.

3 Aligning Draft Models for Context-Faithfulness via ConFide

ConFide aligns draft models with contextual faithfulness by pairing faithful responses with fluent negatives containing fine-grained atomic hallucinations. Direct Preference Optimization then encourages probability mass on context-verifiable tokens.

  • ConFide diversifies negative error patterns and positive stylistic variations to improve draft-model generalization across hallucination types.
  • The pipeline uses atomic fact decomposition and controllable perturbations to produce high-quality contrastive pairs.
  • The source data comprise fact-intensive LLM-AggreFact and CG2C samples suitable for atomic decomposition.
  • Each response is decomposed into minimal verifiable atomic facts before preference construction.
  • Entity swaps, numerical distortions, and relation inversions transform atomic facts into corrupted negative samples.The operators are denoted τent, τnum, and τrel, respectively.
  • Positive samples are paraphrased when faithful or corrected by GPT-4o when unfaithful, ensuring genuine contextual alignment rather than surface-level patterns.
  • Pairing refined faithful responses with perturbed responses yields preference data containing fluent, fine-grained hallucinations as negatives.
  • Direct Preference Optimization maximizes the preference margin without an explicit reward model and encourages probability on tokens verifiable against context.

4 SFAD: Speculative Factuality-Aware Decoding

SFAD dynamically separates token verification from distribution correction, using a specialist draft model to detect factual conflicts and selectively steer the target model. Low-friction steps retain standard speculative decoding, while high-friction steps apply plausibility-masked logit correction.

  • SFAD decouples token verification from distribution correction through a dynamic inference framework using generalist and specialist models.At each step, both models produce logits conditioned on the current prefix, and intervention occurs when the specialist detects potential factual inconsistencies.
  • Epistemic Friction measures Jensen-Shannon distributional tension weighted by specialist certainty to detect confident hallucinations.High friction requires both substantial model disagreement and high specialist confidence.
  • A shifted-sigmoid gate controls steering intensity, switching toward standard speculation below the friction threshold and steering mode above it.The threshold τ determines the transition, while β controls its sharpness.
  • The Contextual Plausibility Mask restricts specialist intervention to linguistically plausible tokens under the current target-model prefix.The plausible set retains vocabulary items whose specialist probability reaches an η-scaled fraction of the specialist maximum.
  • Asymmetric logit steering injects factuality-aware corrections through ReLU-based residuals while preserving the generalist’s linguistic fluency.The plausibility indicator acts as a safety guard, and the hybrid policy samples steered logits on the steering path or uses standard speculative verification on the fast path.

5 Experiments

Experiments evaluate SFAD across context-intensive tasks using Qwen3-1.7B as the DPO-aligned draft model and Qwen3-14B as the target. Results report consistent metric improvements across benchmarks while preserving efficient generation and evidentiary grounding.

  • Models and Configurations: SFAD uses Qwen3-1.7B as the DPO-fine-tuned draft model and Qwen3-14B as the target model.Its approximately 36K-sample DPO dataset combines complementary data sources.
  • Evaluation Tasks and Datasets: Experiments span Foundation QA, summarization, and Long-Form QA across diverse context-intensive datasets.The reported evaluation includes factual retrieval and abstractive faithfulness tasks.
  • Main Results: SFAD demonstrates consistent metric improvements across all evaluated benchmarks while optimizing task accuracy and inference efficiency.The framework is designed to avoid the redundant forward passes that slow decoding-level interventions.
  • Main Results: Table 1 compares Foundation QA performance and relative latency against the Qwen3-14B vanilla baseline.Relative latency is defined as inference time relative to that baseline.
  • Main Results: Table 2 compares summarization performance on XSum and TofuEval, emphasizing factuality and inference overhead.The caption states that SFAD reduces inference overhead while achieving superior factuality.
  • Main Results: SFAD maintains evidentiary grounding and long-range coherence while narrowing the performance gap between Qwen3-14B and the 5× larger Llama-3.1-70B reference model.The paper attributes these results to prioritizing context-aligned tokens during generation.

6 Analysis

SFAD’s analyses show that ConFide-trained DPO drafting improves contextual faithfulness, while selective friction-triggered steering balances factuality and efficiency. The framework also preserves general capabilities and transfers across model families.

  • 6.1 Effectiveness of ConFide and DPO: ConFide+DPO suppresses parametric bias more effectively than ConFide+SFT and consistently outperforms ConFiQA+DPO across knowledge-conflict metrics.DPO shifts the draft model toward contextual evidence, while atomic perturbations provide finer-grained signals for identifying hallucinated completions.
  • 6.2 Latency-Faithfulness Trade-off Analysis: 0.52×–0.82× ATGA characterizes decoding-level baselines, whose dual forward passes impose severe latency penalties despite moderate faithfulness.These methods compare contextualized and baseline inputs, adding token-generation overhead.
  • 6.2 Latency-Faithfulness Trade-off Analysis: 2.18× speedup is achieved by vanilla speculative decoding, but its faithfulness falls below decoding baselines because its unaligned draft relies on parametric priors.Standard speculative decoding verifies tokens without ensuring that draft proposals are grounded in the provided evidence.
  • 6.2 Latency-Faithfulness Trade-off Analysis: 2.48× speedup and an 85.2 faithfulness score place SFAD at the reported Pareto-optimal balance between contextual faithfulness and inference efficiency.The same analysis reports that SFAD exceeds the best baseline by 21.7 points while maintaining 2.9× faster inference.
  • 6.3 Epistemic Friction Analysis: Detecting Confident Hallucinations: DJS alone would over-trigger on stylistic variation, whereas low specialist certainty suppresses friction and avoids intervention.When DJS is 0.52–0.58 and κt is 0.28–0.32, friction remains 0.15–0.17; a confident hallucination with DJS ≈0.89 and κt ≈0.97 yields friction ∼0.86 and triggers steering.
  • 6.4 Quantifying Logit Steering Impact: 62.45% faithful-token probability is reached by SFAD, compared with 18.91% for standard speculative decoding, while selective steering retains 2.48× speedup and 85.2 faithfulness.The method modifies target logits only when distributional conflict and specialist conviction jointly exceed the trigger condition.
  • 6.5 SFAD Preserves General Utility: 91.27% on GSM8K and negligible Just-Eval degradation indicate that SFAD maintains general utility when no knowledge conflict requires steering.With Llama-3.1-8B as target, SFAD also consistently improves faithfulness while maintaining inference efficiency.

7 Conclusion

SFAD unifies contextual faithfulness enhancement with speculative acceleration by using a context-faithful drafter as a factuality sentinel. ConFide training and selective target-logit steering enable simultaneous faithfulness and speed gains.

  • 7 Conclusion: SFAD uses a ConFide-trained, context-faithful draft model to detect hallucinations through epistemic friction during speculative decoding.The framework selectively steers target logits when conflicts are detected while preserving speculative efficiency.
  • 7 Conclusion: Atomic-level hallucination perturbations train the drafter, while selective logit steering corrects the target distribution only after conflict detection.This design supports simultaneous speedup and contextual-faithfulness gains.

Limitations

SFAD’s deployment requires additional draft-model preparation and threshold calibration beyond standard speculative decoding.

  • Limitations: SFAD requires a domain-aligned DPO-trained draft model, adding data-construction overhead, and its friction threshold may need tuning for out-of-distribution domains.These requirements constrain direct deployment relative to standard speculative decoding.

A.1 Role of Contextual Plausibility Mask (η)

The Contextual Plausibility Mask limits steering to corrections compatible with the target model’s linguistic manifold. Ablation results show that removing it increases factual alignment but damages linguistic quality.

  • A.1 Role of Contextual Plausibility Mask (η): η = 0 removes the linguistic safety guard, producing the highest AlignScore of 87.49 but reducing ROUGE-L by 1.54 and BERT-P by 2.72 points.Without CPM, correct entities can be injected with syntactically mismatched usage.
  • A.1 Role of Contextual Plausibility Mask (η): η = 0.1 filters linguistically dissonant corrections and supports competitive fluency and factuality relative to the 70B Frontier Model.The mask acts as a safety guard by restricting steering to the target model’s plausible manifold.

A.2 Sensitivity Analysis of Friction Threshold

The friction threshold τ controls how often SFAD invokes logit steering, trading faithfulness against latency. The default τ = 0.5 resolves most hallucinations while retaining substantial speculative-decoding speed.

  • Threshold trade-off: Increasing τ makes SFAD intervene more selectively, reducing the Steering Ratio monotonically.Steering Ratio is the percentage of generated tokens processed through the logit-steering path.
  • Threshold trade-off: At τ = 0.1, frequent intervention maximizes faithfulness but incurs a latency penalty from additional target-model logit computations.
  • Threshold trade-off: At τ = 0.9, SFAD mostly uses the Fast Path, maximizing speed but failing to resolve knowledge conflicts.
  • Default setting: At τ = 0.5, SFAD surpasses standard speculative decoding by 46.7 points while achieving 2.48× speedup.
  • Fusion ablation: The asymmetric steering ablation compares fusion operators while holding adaptive gating and the Contextual Plausibility Mask constant.The evaluated alternatives include weighted averaging, subtractive contrast, and SFAD’s asymmetric operator.
  • Fusion ablation: SFAD’s ReLU-based injection preserves target-model fluency by boosting only tokens where the specialist has higher confidence.Linear Sum and Interpolation reduce ROUGE-L, while Subtractive Contrast can suppress valid tokens.

B Generalization Across Model Families

SFAD’s decoding mechanism generalizes across model families and evaluated tasks, improving faithfulness while preserving its latency advantage. Its design combines adaptive conflict detection, selective steering, and speculative verification.

  • Cross-family results: Across summarization and long-form QA, SFAD improves faithfulness while preserving the intended latency advantage on another target-model family.The reported cross-family results are provided in Tables 11 and 12.
  • Cross-family results: SFAD’s learned factuality-aware draft model and friction-triggered steering provide a portable correction signal across model families.
  • Inference mechanism: When friction exceeds τ, SFAD steers the target distribution; otherwise, it samples and verifies draft tokens with standard rejection sampling.
  • Framework components: ConFide creates preference pairs by decomposing responses into atomic facts, perturbing them, and refining faithful alternatives.Perturbations include entity swaps, numerical changes, and relation inversion; positives are paraphrased or teacher-corrected.
  • Inference mechanism: The Contextual Plausibility Mask constrains steering to the generalist distribution’s support, limiting corrections to plausible tokens.
  • Theoretical mechanism: Asymmetric steering amplifies faithful candidates while leaving lower-confidence surface patterns unboosted.The faithful token has higher specialist than generalist logit, whereas ReLU sets the competing correction to zero.
  • Theoretical mechanism: SFAD preserves support and avoids the zero-probability trap that can suppress valid tokens in subtractive contrastive methods.
  • Mode switching: The hybrid policy invokes steering when expected hallucination-risk reduction outweighs marginal computational cost.

E.7 Summary of Theoretical Guarantees

SFAD’s theoretical analysis presents formal guarantees for linguistic safety, factuality amplification, support preservation, and adaptive efficiency. Together, these results characterize the framework as a mathematically grounded safeguarding approach.

  • Overview: The theoretical analysis establishes SFAD as a mathematically grounded framework for verifiable generation.
  • Guarantees: Theorem 1 guarantees that SFAD’s intervention preserves the linguistic manifold.
  • Guarantees: Proposition 1 ensures that SFAD resolves distributional conflicts in favor of faithful tokens.
  • Guarantees: Theorem 2 protects against numerical instability and suppression of valid tokens.
  • Guarantees: Theorem 4 justifies SFAD’s adaptive switching between steering and fast speculative decoding for deployment.
  • Conclusion: Collectively, the guarantees formalize SFAD as a safe-by-design decoding framework for factual integrity.
Loading 2609.00796v2…