Conditional Denoising Diffusion for Sequential Recommendation

1 INTRODUCTION

• Noisy interactions bias rankings, especially for short sequences and cold-start items, limiting attention-based recommendation methods.Random clicks can degrade performance despite the effectiveness of attention-based approaches.Source: §1 · p. 1
• GANs face unstable optimization, whereas VAEs suffer posterior collapse and over-smoothed generations in sparse sequential recommendation.Over-smoothing produces similar candidate-item scores and inaccurate relative positions.Source: §1 · pp. 1–2
• Denoising diffusion divides learning and generation into fine-grained steps, helping address posterior collapse and over-smoothing while retaining stable VAE-like optimization.The diffuser gradually adds noise, and the denoiser removes it step-by-step to learn intermediate transitions.Source: §1 · p. 2
• Traditional diffusion is difficult to apply because recommendation uses learnable item representations, requires autoregressive conditioning, and must predict preferences rather than reconstruct inputs.Unconditional generation from Gaussian noise can worsen representation collapse in discrete recommendation settings.Source: §1 · p. 2
• CDDRec combines a sequence encoder, cross-attentive conditional denoising decoder, step-wise diffuser, cross-divergence loss, and contrastive optimization.The decoder conditions denoising on encoded interactions, while the losses target high-fidelity representations, user preferences, and prevention of collapse.Source: §1 · p. 2
• CDDRec substantially improves all metrics across four datasets, supporting its effectiveness for sequential recommendation.The paper also reports ablation studies examining each key design.Source: §1 · p. 2

2.1 Generative Models for Sequential Recommendation

• VAE-based recommendation methods encode interaction sequences into latent representations and reconstruct them, with later work adding autoregression, self-attention, or contrastive learning.ContrastVAE uses contrastELBO from a multi-view perspective, while other methods use self-attention or autoregressive factorization.Source: §2.1 · pp. 2–3
• GAN-based recommendation methods jointly train generators and discriminators to produce sequence representations and assess their factor-specific rationality.Adversarial training has also been incorporated into VAEs to obtain more diverse and informative item representations.Source: §2.1 · p. 3

2.2 Denoise Diffusion models for discrete tasks

• Diffusion models have been applied to images, audio, and discrete text generation through multi-step denoising, including non-autoregressive and partially conditioned variants.SUNDAE, Diffusion-LM, and DiffSeq illustrate different strategies for reconstructing corrupted discrete sequences.Source: §2.2 · p. 3
• Existing discrete diffusion methods are not directly suitable for sequential recommendation because historical and target interactions overlap, while item rankings and user preferences may remain unmodeled.TimeGrad introduces conditional autoregressive diffusion over RNN latent states, providing a closer precedent.Source: §2.2 · p. 3

3 PRELIMINARY

• DDMs use multiple latent variables and progressively restore corrupted data, unlike VAEs, which use a single latent variable and reconstruct in one step.The multi-stage reverse process is intended to simplify learning and support high-fidelity generation.Source: §3 · p. 3
• The diffusion phase adds small Gaussian noise at each Markov-chain step according to a variance schedule, producing progressively corrupted states.The schedule parameter β_t controls the amount of noise introduced at each stage.Source: §3 · p. 3
• The denoising phase learns to reverse diffusion step by step, typically by minimizing KL divergence between the true and learned reverse transitions.The simplified objective samples diffusion steps and minimizes the corresponding denoising term.Source: §3 · pp. 3–4
• In the denoising objective, ε_θ removes Gaussian noise added to x_0 at step t, while ε denotes the Gaussian noise.This parameterization trains the denoiser to recover the original input through the reverse process.Source: §3 · p. 4
• Standard DDMs can collapse in discrete recommendation because item representations are randomly initialized and learnable, unlike fixed informative image features.Sparsity, cold-start items, and long-tail distributions further exacerbate collapse.Source: §3 · p. 4
• CDDRec therefore generates target-item representations in T steps conditioned on historical interactions rather than starting from uncontrollable Gaussian noise.This preserves the autoregressive next-item prediction objective of sequential recommendation.Source: §3 · p. 4

4 METHODOLOGY

• CDDRec encodes historical interactions, conditionally denoises target sequence embeddings, and progressively corrupts them with a step-wise diffuser for autoregressive next-item generation.The encoder supplies conditioned sequence representations, while the diffuser creates step-specific corrupted targets for reconstruction.Source: §4 · p. 4, §4.4 · p. 6

4.1 Sequence Encoder

• The self-attentive sequence encoder represents historical item interactions with positional embeddings and future-item masking to support autoregressive prediction.Item and position embeddings are combined before self-attention, and attention masks prevent access to future interactions.Source: §4.1 · p. 4

4.2 Cross-attentive Conditional Denoising Decoder

• The cross-attentive decoder directly conditions every denoising step on encoded historical interactions and the step indicator, avoiding a long denoising dependence chain.It preserves the full sequence representation rather than only the final position and uses the step indicator as the cross-attention query.Source: §4.2 · p. 5, Figure 1 · p. 5

4.3 Step-wise Diffuser

• The step-wise diffuser incrementally adds Gaussian noise according to a predefined schedule, producing corrupted target embeddings for each denoising step.The corrupted targets are sampled with the reparameterization trick and reconstructed by the conditional decoder.Source: §4 · p. 4, §4.3 · p. 5

4.4 Optimization

• The optimization paradigm uses cross-divergence to make predicted embeddings closer to targets than unrelated items, thereby preserving recommendation ranking rather than collapsing item representations.Negative targets are sampled from items absent from the user’s historical interactions.Source: §4.4 · p. 6
• A multi-view contrastive objective encourages consistent target-item representations under noisy or augmented interaction sequences.Augmented views are created through random cropping, shuffling, and masking, while InfoNCE maximizes agreement with the original view.Source: §4.4 · p. 6
• Losses are evaluated at every diffusion step and rescaled by the step indicator because later steps contain progressively more noise and less information.The explicit per-step calculation avoids relying on randomly sampled step indicators during optimization.Source: Figure 1 · p. 5, §4.4 · p. 6

5.1 Overall Experiments

• CDDRec consistently outperforms other models across four datasets, with Recall@1 gains of 20.98%, 16.67%, 17.59%, and 18.42% on Office, Beauty, Tools, and Toys, respectively.The reported gains are versus the second-best models; improvements on ranking-position metrics such as NDCG@5, NDCG@10, and MRR are also described as substantial.Source: §5.1 · p. 7, Table 2 · p. 8

5.2 Ablation Study

• The cross-divergence loss prevents collapse and adds ranking capacity, while combining in-view and cross-view contrastive losses improves performance more than either component alone.MSE variants show performance drops of 88.88% and 91.42%, and removing in-view InfoNCE makes cross-view InfoNCE detrimental.Source: §5.2 · pp. 7–8, Table 3 · p. 9
• Diffusion and denoising generally improve performance, with denoising more important on Office and Toys and diffusion more important on Beauty and Tools.Replacing either sampling process with its ablated alternative lowers performance relative to CDDRec.Source: §5.2 · p. 8, Table 4 · p. 9
• The conditional denoising decoder improves MRR consistently when integrated with GRU4Rec, FMLPRec, and SASRec across the four datasets.The result supports compatibility with existing sequence encoders and positive contributions beyond their original performance.Source: §5.2 · p. 8, Figure 2 · p. 9

5.3 Hyperparameter Sensitivity

• Optimal diffusion steps are 10, 15, 25, and 30 for Office, Tools, Toys, and Beauty, while optimal maximum noise levels are 0.04, 0.06, 0.08, and 0.1, respectively.The paper relates these settings to dataset item counts and the combined influence of sequence length and item count.Source: §5.3 · p. 9, Figure 3 · p. 10, Figure 4 · p. 10

5.4 Study the effect on Length and Frequency

• CDDRec performs especially well on short sequences and low-frequency target items, and its ranking predictions are less smooth than ContrastVAE’s, becoming clearer across denoising steps.Short-sequence gains matter because 95% of Office sequences contain fewer than 30 interactions; Avg.Change increases as denoising progresses from T=20 toward T=0.Source: §5.4 · p. 10, §5.5 · p. 10, Figure 5 · p. 11

• CDDRec outperforms state-of-the-art methods in comprehensive experiments.Source: §6 · p. 11