PaperlayerView exact revision on arXiv

Source-linked AI summary

DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Models

Xin Zhang, Yili Wang, Yue Tan, Xin He, Yanyu Qian, Yixin Liu, Yi Chang, Shirui Pan, Xin Wang

arXiv:2608.14632v1cs.CLcs.AI

TL;DR

D-LLM hallucination detectors often compress denoising trajectories and miss their full token-step structure. DeMTS models these trajectories as multivariate time series over learnable latent variables, and experiments show consistent gains across backbones, benchmarks, and settings.

  • Problem

    Existing D-LLM hallucination detectors may miss uncertainty patterns involving inconsistent token convergence and cross-token fault propagation by compressing denoising trajectories.

  • Method

    DeMTS formulates denoising trajectories as multivariate time series over learnable latent variables and dynamically models temporal evolution with inter-variable dependencies.

  • Results

    DeMTS consistently outperforms baselines across two D-LLM backbones, three benchmarks, and multiple generation settings while remaining robust, transferable, and efficient.

  • Takeaways & Limitations

    Preserving denoising trajectories’ two-dimensional token-step structure supports hallucination detection across model backbones, benchmarks, and generation settings.

  • Takeaways & Limitations

    Applying standard multivariate time-series methods is non-trivial because token-position variables may change semantic roles across samples and during denoising.

Abstract

from arXiv · show

Diffusion large language models (D-LLMs) have emerged as a promising paradigm for text generation. However, similar to autoregressive LLMs, D-LLMs remain vulnerable to hallucinations, where fluent outputs may contain factually incorrect or unsupported content. Although existing hallucination detection methods for D-LLMs attempt to leverage uncertainty trajectories of the denoising process to better identify hallucination signals, they typically compress the trajectories along either the temporal or token dimension, overlooking the useful information encoded in the complete two-dimensional token-step structure. Consequently, they may fail to capture hallucination-relevant patterns, such as inconsistent convergence and cross-token fault propagation, leading to suboptimal detection performance. To bridge this gap, we propose a D-LLM hallucination detection framework that formulates the Denoising trajectories as Multivariate Time Series over learnable latent variables (DeMTS for short). DeMTS employs a trajectory-preserving token-to-variable assignment module to convert token signals into stable latent variables. Based on these variables, we propose dynamic multivariate temporal modeling to progressively integrate inter-variable dependency modeling with temporal encoding for hallucination prediction. Extensive experiments on two D-LLMs backbones and three benchmarks demonstrate that DeMTS outperforms existing hallucination detection methods while maintaining strong robustness, efficiency, and cross-task transferability.

Introduction

D-LLMs remain vulnerable to hallucinations whose signals can emerge in denoising trajectories through inconsistent refinement and cross-token propagation. DeMTS preserves the full token-step structure by modeling trajectories as multivariate time series over stable latent variables, jointly capturing dependencies and temporal evolution.

  • Motivation: D-LLM hallucination cues may appear during denoising as uncertainty rebound, inconsistent refinement, and cross-token propagation, making output-level evidence insufficient.An unstable or incorrect token state can cause contextually connected tokens’ entropy trajectories to deviate together from a coherent factual path.
  • Motivation: Compressing denoising trajectories along only the step or token dimension can overlook token-step uncertainty patterns relevant to hallucination detection.The introduction identifies inconsistent convergence and propagative cross-token patterns as requiring explicit preservation and modeling of the full two-dimensional structure.
  • Problem Formulation: DeMTS formulates D-LLM hallucination detection as multivariate time series learning, with denoising steps as the temporal axis and token-associated uncertainty signals as interacting variables.This formulation is intended to preserve the two-dimensional token-step trajectory structure.
  • Method: DeMTS uses trajectory-preserving token-to-variable assignment to transform semantically unstable token signals into stable latent variables while retaining local denoising variations.The assignment considers uncertainty states, contextual semantics, and positional structure, with an additional trajectory-preservation constraint.
  • Method: DeMTS jointly models dynamic inter-variable dependencies and variable-wise temporal evolution for hallucination prediction.Its Dynamic Multivariate Temporal Modeling module integrates relational and temporal patterns over the constructed variables.
  • Results: DeMTS consistently outperforms baselines across two D-LLM backbones and three benchmarks while remaining robust, transferable, and efficient.The experiments assess the framework across multiple backbones and benchmarks and report these accompanying properties.

Related Works

Prior hallucination detection methods for D-LLMs exploit denoising-trajectory uncertainty by selecting steps, tokens, or partially connected variables. DeMTS addresses their limitations by converting changing token signals into stable latent variables for temporal and inter-variable modeling.

  • Hallucination Detection in D-LLMs: D-LLM denoising provides trajectory-level uncertainty evidence for hallucination detection.The iterative denoising process supplies rich uncertainty signals across generation trajectories.
  • Hallucination Detection in D-LLMs: Existing methods select or reweight informative denoising stages, while others focus on token-oriented evidence or channel-partial variable interactions.These approaches seek useful trajectory evidence while balancing modeling scope between temporal and variable dimensions.
  • Hallucination Detection in D-LLMs: Channel-partial strategies assume stable variable identities, but D-LLM token positions can change semantic roles during generation.This mismatch limits direct temporal and inter-variable modeling over token signals.
  • Hallucination Detection in D-LLMs: DeMTS converts changing token signals into stable latent variables for temporal and inter-variable modeling.The conversion is designed to support modeling when token positions do not retain stable identities across samples and time.

Preliminary

D-LLMs generate fixed-length responses through iterative denoising from highly masked sequences, progressively reconstructing tokens until the final response. DeMTS uses token-level uncertainty trajectories, contextual embeddings, and hallucination labels to formulate trajectory-based binary classification.

  • Diffusion Large Language Models: D-LLMs iteratively denoise a highly masked, fixed-length sequence to progressively reconstruct the final response.The intermediate response is indexed by denoising step t, with reconstruction proceeding from r(T) to r(0).
  • Diffusion Large Language Models: At each denoising step, the model predicts a vocabulary distribution for every token position, whose entropy forms the uncertainty trajectory H across token and temporal axes.H ∈ R^{N×(T+1)} records uncertainty over sequence positions and denoising steps.
  • Diffusion Large Language Models: Contextual token embeddings E complement H as additional representations collected across positions and denoising steps.The embeddings are collected in E ∈ R^{N×(T+1)×d_h}, while H supplies the primary detection signal.

Methodology

DeMTS represents D-LLM denoising trajectories as multivariate time series, preserving temporal evolution and cross-token relationships through stable latent variables. It uses T2V assignment and DMTM to model evolving temporal dynamics and inter-variable dependencies for hallucination prediction.

  • Framework Overview: DeMTS takes token-step uncertainty trajectories as input and jointly preserves temporal evolution with cross-token relationships.This formulation avoids compressing the trajectory along only one dimension.
  • Trajectory-Preserving Token-to-Variable Assignment: T2V maps schedule-biased and semantically unstable token signals into K latent variables with more consistent identities.It normalizes uncertainty within denoising steps and reorganizes position-indexed trajectories into stable variable-indexed trajectories.
  • Trajectory-Preserving Token-to-Variable Assignment: T2V combines normalized uncertainty, contextual token embeddings, and positional structure to assign token signals to latent variables.The joint representation encodes uncertainty and semantics, while multi-frequency Fourier features capture global ordering and local neighborhood relations; a content-adaptive gate regulates structural contributions.
  • Preservation-Constrained Variable Construction: Assignment-weighted aggregation constructs latent-variable trajectories while preservation constraints retain both the magnitude and temporal direction of token dynamics.The Amplitude Preservation Loss addresses amplitude smoothing caused by aggregation weakening local uncertainty changes.
  • Dynamic Multivariate Temporal Modeling: DMTM progressively integrates inter-variable dependency modeling with variable-wise temporal encoding for hallucination prediction.Stage-adaptive relation fusion limits noisy early-step interactions and strengthens information exchange as variables stabilize, while attention weights capture step-specific cross-variable dependencies.

Experiments

Experiments evaluate DeMTS across three QA benchmarks, two D-LLM backbones, and two generation-length settings. Results and analyses compare output-level, latent-based, and trajectory-structure detectors while examining latent-variable resolution and assignment stability.

  • Datasets: The evaluation covers TriviaQA, HotpotQA, and CommonsenseQA using complete denoising trajectories from LLaDA-8B-Instruct and Dream-7B-Instruct.The benchmarks target open-domain recall, multi-clue factual integration, and everyday-knowledge reasoning, respectively.
  • Performance Comparison: DeMTS consistently outperforms output-based baselines across different backbones, benchmarks, and generation-length settings.The comparison is presented in Table 1 across two D-LLM backbones, three QA benchmarks, and two generation-length settings.
  • Performance Comparison: DeMTS outperforms latent-based detectors in most cases, indicating that static representation signals do not capture evolving temporal dependencies in hallucination formation.Latent-based methods use richer model-side signals than output-level statistics but still underperform DeMTS in most cases.
  • Performance Comparison: DeMTS achieves the best performance when preserving the multivariate structure of denoising trajectories, compared with TraceDet and DynHD.This comparison tests whether the complete trajectory structure provides additional hallucination-detection evidence.
  • Latent-variable Analysis: A moderate number of latent variables K best balances information preservation and variable stability, whereas small K overcompresses signals and large K creates fragmented, redundant variables.Figure 3a varies K to assess variable resolution, and the accompanying visualization examines assignment patterns across samples.

Conclusion … B Algorithm

DeMTS preserves the two-dimensional structure of D-LLM denoising trajectories and converts changing token signals into stable latent variables for temporal and inter-variable modeling. The surrounding discussion motivates this design through trajectory-level evidence, multivariate time-series principles, and the framework’s overall optimization workflow.

  • Conclusion: DeMTS preserves the two-dimensional token-step structure of denoising trajectories to model how hallucination-related uncertainty evolves and propagates during generation.The framework is evaluated across multiple D-LLMs, benchmarks, and generation settings.
  • A.1 Hallucination Detection in D-LLMs: D-LLM iterative denoising exposes intermediate predictions, uncertainty estimates, and hidden states that provide trajectory-level evidence for hallucination detection.Factual errors may emerge during refinement rather than only in the final output.
  • A.1 Hallucination Detection in D-LLMs: Step-oriented methods select informative denoising sub-traces or intermediate hidden evidence, showing that denoising stages contribute unequally to hallucination detection.TraceDet formulates denoising as a decoding trace, while HIVE uses selected intermediate evidence for verification.
  • A.1 Hallucination Detection in D-LLMs: Dynamics-oriented methods model uncertainty evolution or temporal token relations, but existing approaches do not fully preserve the denoising trajectory’s structure.DynHD measures deviations from expected dynamics, while TDGNet constructs temporal dynamic graphs.
  • A.2 Multivariate Time-Series Modeling: Multivariate time-series methods model temporal patterns across variables and their dependencies, using channel-independent or channel-dependent strategies.PatchTST models variables separately to preserve variable-specific temporal patterns and reduce interference from noisy correlations.
  • A.2 Multivariate Time-Series Modeling: Denoising steps form the temporal axis and token signals form the raw variable axis, but token positions lack stable semantic identities across samples and time.Directly treating token positions as variables can therefore produce unstable temporal patterns and unreliable dependencies.
  • B Algorithm: DeMTS converts changing token signals into stable latent variables, then models their temporal evolution and stage-adaptive inter-variable interactions.Algorithm 1 normalizes the uncertainty trajectory, computes uncertainty-semantic and positional representations, assigns latent variables, constructs their trajectory, and applies dynamic temporal modeling.

C Computational Complexity

DeMTS has inference complexity TNKd + TK^2d + KT^2d, excluding the original denoising process and lightweight components. Its efficiency comes from performing relational and temporal modeling over K compact latent variables rather than N token positions.

  • T2V: T2V incurs O(TN) for SAUN normalization and O(TNKd) for token-to-variable assignment and aggregation.T denotes denoising steps, N token positions, K latent variables, and d representation dimension.
  • DMTM: DMTM costs O(TK^2d) for per-step latent-variable interactions and O(KT^2d) for temporal self-attention.SAIVI models interactions among K latent variables at each denoising step, while VTDM models T-step dependencies for each latent variable.
  • Overall complexity: The overall inference-time complexity is TNKd + TK^2d + KT^2d.The reported costs exclude the original D-LLM denoising process, token-wise transformations, and lightweight classifier.
  • Training complexity: Training adds O(TNKd) for trajectory preservation, matching the assignment and aggregation terms asymptotically.The added constraint aggregates d-dimensional token-level variations into latent-variable variations.
  • Efficiency advantage: Direct token-indexed axial attention would require O(TN^2d + NT^2d), whereas DeMTS models relational and temporal dependencies in K latent-variable trajectories.DeMTS first transforms the token-indexed trajectory into compact latent-variable trajectories.

D Experiments · D.1 Baselines

DeMTS is evaluated against hallucination detectors spanning output-level signals, latent representations, and denoising trajectories. These baselines test whether final outputs, static hidden states, or process-level evidence suffice for hallucination detection in D-LLMs.

  • D.1 Baselines: The baseline comparison covers output-based, latent-based, and trajectory-based hallucination detectors.The categories respectively use generated-response signals, internal representations, and intermediate denoising information.
  • D.1 Baselines: Output-based methods estimate hallucination risk from model likelihood, token-level uncertainty, or consistency among sampled responses.They assess whether final-output signals are sufficient for detection.
  • D.1 Baselines: Perplexity treats generated-response likelihood as confidence, considering lower-likelihood responses less reliable.This baseline uses likelihood directly as an output-level signal.
  • D.1 Baselines: LN-Entropy uses length-normalized token-level entropy to estimate predictive uncertainty while reducing bias from differing output lengths.Semantic Entropy instead samples multiple responses and groups them according to semantic information described in the passage.
  • D.1 Baselines: Latent-based methods use internal LLM representations to capture factuality-related signals beyond surface outputs.They test whether static hidden-state features can detect hallucinations without explicitly modeling denoising trajectories.
  • D.1 Baselines: EigenScore measures semantic consistency through the spectral structure of hidden-state covariance matrices, while TSV learns a latent-space steering vector.TSV is intended to improve separation between truthful and hallucinated responses.
  • D.1 Baselines: Trajectory-based methods leverage intermediate information from iterative denoising and provide the most direct comparison with DeMTS.They use process-level evidence rather than relying only on final outputs or static representations.
  • D.1 Baselines: TraceDet formulates the D-LLM denoising process as a decoding trace and selects hallucination-relevant sub-traces from intermediate denoising states.It is designed specifically for D-LLMs.

D.2 Implementation Details

Experiments evaluate DeMTS across three QA benchmarks using two D-LLM backbones and controlled denoising trajectories. The detector uses latent-variable modeling, specified optimization settings, and validation-AUROC checkpoint selection.

  • Experimental Setup: Experiments use TriviaQA, HotpotQA, and CommonsenseQA with LLaDA-8B-Instruct and Dream-7B-Instruct, recording token-wise predictive entropy and token IDs over 64 or 128 denoising steps.Each dataset uses 1,600 training examples plus 200 validation and 200 test instances, with random seed 42.
  • Model and Training: The detector uses hidden dimension 64, constructs K=32,48 latent variables, and trains with Cross Entropy plus trajectory-preservation loss using AdamW.AdamW uses learning rate 2 × 10−4, weight decay 0.1, and batch size 8; training runs at most 100 epochs with patience 20.
  • Evaluation and Implementation: AUROC is the primary evaluation metric, and the checkpoint with highest validation AUROC is selected for testing; experiments use PyTorch 2.6.0 on 4 NVIDIA L40 GPUs.All experimental hyperparameter settings are reported in Table 5.

D.3 Sensitivity Analysis of Trajectory Preservation Losses

The sensitivity analysis examines amplitude and direction trajectory-preservation losses, finding stable detector performance across broad loss-weight ranges but lower performance when either loss is removed or minimally weighted.

  • Loss sensitivity: The analysis evaluates the amplitude consistency loss Lamp and direction consistency loss Ldir.Figure 5 presents the sensitivity analysis for both trajectory-preservation losses.
  • Loss sensitivity: The detector maintains stable performance across a broad range of loss weights.This indicates that the trajectory-preservation objective is not overly sensitive to specific hyperparameter choices.
  • Loss sensitivity: Performance decreases when either trajectory-preservation loss is removed or assigned a very small weight.The result suggests that preserving both amplitude and direction information contributes to detector performance.

D.4 Complete Ablation Results · E Prompt Templates

Complete ablations show that DeMTS’s gains arise from integrating schedule normalization, stable latent-variable construction, trajectory-preserving regularization, and dynamic multivariate temporal modeling. Prompt templates tailor generation and hallucination annotation to each benchmark while enforcing consistent answer extraction.

  • D.4 Complete Ablation Results: Ablations across both D-LLM backbones, datasets, and generation lengths show consistent trends, supporting the robustness and generality of DeMTS’s design.The complete results further verify the effectiveness of each component.
  • D.4 Complete Ablation Results: Replacing schedule-normalized trajectories with raw uncertainty signals consistently reduces performance, highlighting the importance of suppressing schedule-induced global trends.
  • D.4 Complete Ablation Results: Using token positions directly as variables degrades performance, confirming the need for semantically stable latent variables before time-series modeling.
  • D.4 Complete Ablation Results: Removing Lamp or Ldir weakens performance, indicating that amplitude consistency and direction consistency preserve fine-grained denoising dynamics during token-to-variable assignment.
  • D.4 Complete Ablation Results: Removing inter-variable interaction or temporal evolution modeling consistently degrades performance, while overall gains reflect the integration of multiple complementary components.The integrated design includes schedule normalization, stable latent-variable construction, trajectory-preserving regularization, and dynamic multivariate temporal modeling.
  • E.1 D-LLM Response Generation: TriviaQA prompts request concise factual answers, HotpotQA prompts require answers based only on provided context, and CommonsenseQA prompts require selecting one option without explanations.All generated answers are enclosed in <answer></answer> tags for consistent extraction and subsequent hallucination annotation.
  • E.2 Automatic Hallucination Annotation: For TriviaQA and HotpotQA, Qwen3-8B maps yes to hallucinated label 1 and no to nonhallucinated label 0, whereas CommonsenseQA uses rule-based matching.
  • E.2 Automatic Hallucination Annotation: The annotation instructions classify outputs using fluency, relevance, support, exact-match, semantic-match, and unknown-answer rules, with final judgments returned as yes or no inside answer tags.Supported or semantically equivalent answers are treated as not hallucinated, including appropriate unknown responses when references indicate unanswerability.
Loading 2608.14632v1…