Source-linked AI summary
Diffusion Augmentation for Sequential Recommendation
Qidong Liu, Fan Yan, Xiangyu Zhao, Zhaocheng Du, Huifeng Guo, Ruiming Tang, Feng Tian
TL;DR
Sparse interactions and weaker recommendations for users with short histories motivate DiffuASR, which adapts diffusion generation to sequential recommendation using an SU-Net and two guidance strategies. Across the evaluated settings, the authors report that it outperforms competing augmentation methods on three sequential recommendation models.
Problem
Sequential recommendation faces sparse interaction records and often gives users with few interactions worse recommendations, while existing augmentation can require complex training or produce poor-quality interactions.
Source: §1 · pp. 1–2Method
DiffuASR adapts diffusion to generate augmented item sequences, using an SU-Net for sequence information and classifier-guided and classifier-free strategies to align generated items with user intent.
Source: §1 · p. 2Results
DiffuASR's two variants consistently outperform competing augmentation methods on all three sequential recommendation models, according to the reported comparison.
Source: §4.2 · p. 7Takeaways & Limitations
Abstract
from arXiv · showhide
Sequential recommendation (SRS) has become the technical foundation in many applications recently, which aims to recommend the next item based on the user's historical interactions. However, sequential recommendation often faces the problem of data sparsity, which widely exists in recommender systems. Besides, most users only interact with a few items, but existing SRS models often underperform these users. Such a problem, named the long-tail user problem, is still to be resolved. Data augmentation is a distinct way to alleviate these two problems, but they often need fabricated training strategies or are hindered by poor-quality generated interactions. To address these problems, we propose a Diffusion Augmentation for Sequential Recommendation (DiffuASR) for a higher quality generation. The augmented dataset by DiffuASR can be used to train the sequential recommendation models directly, free from complex training procedures. To make the best of the generation ability of the diffusion model, we first propose a diffusion-based pseudo sequence generation framework to fill the gap between image and sequence generation. Then, a sequential U-Net is designed to adapt the diffusion noise prediction model U-Net to the discrete sequence generation task. At last, we develop two guide strategies to assimilate the preference between generated and origin sequences. To validate the proposed DiffuASR, we conduct extensive experiments on three real-world datasets with three sequential recommendation models. The experimental results illustrate the effectiveness of DiffuASR. As far as we know, DiffuASR is one pioneer that introduce the diffusion model to the recommendation.
1 INTRODUCTION
Sequential recommendation faces sparse interaction records and weaker recommendations for users with few interactions. DiffuASR uses diffusion-based sequence generation, a sequential U-Net, and two guidance strategies to produce higher-quality augmentation, which the authors evaluate across datasets and recommendation models.
1 INTRODUCTION
- Data sparsity and poor recommendations for users with few interactions are two stated challenges for sequential recommendation.The paper calls users with short histories long-tail users and notes that most users in its Beauty example have fewer than 10 interactions.Source: §1 · pp. 1–2
- Existing sequence-level augmentation may require complex training and is not tailored to long-tail users, while item-level augmentation can be low quality and drift from original sequences.The paper attributes the item-level risk to iterative generation based on a model with relatively low top-1 accuracy.Source: §1 · p. 2
- DiffuASR combines a pseudo-sequence diffusion framework, an SU-Net for embedding sequences, and classifier-guided and classifier-free strategies to align generated items with user intent.The framework addresses the gap between discrete item sequences and the continuous-valued matrices used by general diffusion models.Source: §1 · p. 2
- The authors report that experiments on three real-world datasets and three sequential recommendation models show DiffuASR's effectiveness and generality.These experiments are presented as evidence for the proposed augmentation method.Source: §1 · p. 2
2 PRELIMINARY
The preliminary section defines next-item prediction and pre-order sequence augmentation, then reviews the forward-noising and reverse-denoising processes of diffusion models.
2.1 Problem Definition and Notations
- Sequential recommendation predicts the next item from a user's interaction history, and the paper's augmentation prepends M generated items to the raw sequence.The augmented items form a pre-order sequence, denoted S_aug, which is joined to the original interaction sequence.Source: §2.1 · pp. 2–3
2.2 Diffusion Model
- The diffusion forward process gradually adds Gaussian noise to data samples, with the number of steps and variance schedule controlling the noise.A reparameterization trick is used because directly learning the variance β_t is difficult.Source: §2.2 · p. 3
- The reverse process recovers a sample from noise through a Markov chain, and the model's noise predictor is typically based on a U-Net.The diffusion objective is expressed as a KL-divergence form and also has a simplified formulation.Source: §2.2 · p. 3
3 DIFFUASR
Section 3 outlines DiffuASR’s framework, SU-Net architecture, guidance strategies, and training and augmentation process.
3 DIFFUASR
- The section presents DiffuASR’s framework, SU-Net architecture, guidance strategies, and train-and-augment process.It previews the framework in Section 3.1, SU-Net in Section 3.2, guidance in Section 3.3, and the train-and-augment process in Section 3.4.Source: §3 · pp. 3–4
- Figure 4 depicts the architecture of the proposed SU-Net.Source: Figure 4 · p. 4
3.1 Framework
DiffuASR bridges discrete item sequences and continuous diffusion inputs, then recovers and guides augmented sequences to reflect users’ histories.
3.1 Framework
- The framework generates discrete-valued augmented sequences through embedding and rounding procedures, with guidance intended to align them with raw-sequence distributions.Source: §3.1 · p. 4
3.1.1 Forward Process.
- DiffuASR embeds discrete item identities into a continuous matrix, adds Gaussian noise to reach a Gaussian distribution, and uses a linear variance schedule.Source: §3.1.1 · p. 4
3.1.2 Reverse Process.
- The reverse process predicts and removes noise with SU-Net, then rounds recovered embeddings to vocabulary items by cosine similarity.Cosine similarity is used because generated embeddings can have different lengths from original item embeddings.Source: §3.1.2 · p. 4
3.1.3 Guide Procedure.
- The guide procedure conditions noise prediction on the average embedding of the user’s historical interactions and adds two further guidance strategies.Source: §3.1.3 · p. 4
3.2 SU-Net
SU-Net adapts image-oriented U-Net processing to preserve sequence information in embedding sequences while predicting diffusion noise.
3.2 SU-Net
- SU-Net treats sequence positions as channels and reshapes each embedding into a matrix, converting the sequence into a multichannel input for convolution.A 64-dimensional embedding is reshaped to an 8 × 8 matrix.Source: §3.2 · p. 4
- Its U-Net-like architecture applies convolutional residual blocks, an attention-equipped middle block, and down- and up-sampling with copied features concatenated during up-sampling.Source: §3.2 · pp. 4–5
- The output is reshaped back into embedding vectors, while the diffusion step and conditional vector are combined and injected into residual blocks to control noise prediction.Source: §3.2 · p. 5
3.3 Guide Strategy
DiffuASR guides reverse diffusion toward items aligned with users’ historical preferences through classifier-based and classifier-free strategies.
3.3 Guide Strategy
- The guidance strategies aim to align generated items with preferences in raw interaction sequences, since unrelated items can be noisy interactions that harm recommendation.Source: §3.3 · p. 5
3.3.1 Classifier Guide Strategy.
- Classifier guidance treats a pretrained sequential recommendation model as the classifier, using the first raw-sequence item as the next-item target for the augmented prefix.The guidance uses binary cross-entropy rather than the cross-entropy loss in [5].Source: §3.3.1 · p. 5
3.3.2 Classifier Free Strategy.
- Classifier-free guidance is offered as an alternative because classifier guidance requires a pretrained classifier and incurs time and computation.The guidance scale and padding vector are used in the classifier-free strategy.Source: §3.3.2 · p. 5
- Algorithm 1 selects sequences with n_u > M and adds each generated sequence alongside its original sequence to the augmented dataset.Source: §3.3.2 · p. 5
3.4 Train and Augment
DiffuASR trains an SU-Net on pre-order items from sufficiently long sequences, then uses guided reverse diffusion to generate items that are prepended to each raw sequence.
3.4 Train and Augment
- Only raw sequences longer than M train DiffuASR, with their pre-order items serving as augmentation targets.The first M items in each eligible sequence are used for diffusion training.Source: §3.4 · p. 5
- Forward and reverse diffusion update SU-Net parameters until the model converges.The training process uses the first M items of each sequence.Source: §3.4 · p. 5
- Guided reverse diffusion generates a pre-order sequence that is prepended to each raw sequence to form the augmented dataset.The resulting augmented dataset can be used directly to train sequential recommendation models.Source: §3.4 · p. 5
4 EXPERIMENT
Experiments test DiffuASR against augmentation baselines on three datasets and three sequential recommendation models. DiffuASR consistently outperforms competing augmentation methods across the models.
4 EXPERIMENT
- The experiments assess baseline comparisons, generality across SRS models, long-tail user performance, augmented-item counts, and diffusion hyperparameters.Source: §4 · p. 6
4.1.1 Dataset.
- The evaluation uses Yelp, Amazon Beauty, and Steam, with implicit interactions and users with fewer than three interactions removed.The last interaction is the test item and the penultimate interaction is used for validation.Source: §4.1.1 · p. 6
4.1.2 Baselines.
- Baselines include no augmentation, random item and sequence sampling, and ASReP; the study uses reverse SASRec as the augmentor without extra training tricks.ASReP's pretrain-finetune scheme is compatible only with SASRec.Source: §4.1.2 · p. 6
4.1.3 Backbone Recommendation Models.
- The augmentation methods are paired with Bert4Rec, SASRec, and S3Rec as backbone recommendation models.Source: §4.1.3 · p. 6
4.1.4 Implementation Details.
- Hyperparameters are selected using validation HR@10, test items are excluded during augmentation, and results average three runs with different seeds.The number of diffusion steps is fixed at 1,000.Source: §4.1.4 · p. 6
4.1.5 Evaluation Metrics.
- Evaluation uses HR@10 and NDCG@10 with one ground-truth item and 100 sampled negatives per test user.Reported results are averaged over all test users.Source: §4.1.5 · p. 6
4.2 Overall Performance Comparison (RQ1)
- DiffuASR (CG) and DiffuASR (CF) consistently outperform competing augmentation methods across all three SRS models.Augmentation yields more gains on Beauty than Steam; Bert4Rec benefits only from DiffuASR, while ASReP performs better among baselines on SASRec.Source: Table 2 · p. 7, §4.2 · p. 7
4.3 Analysis of Different User Groups (RQ2)
DiffuASR’s user-group performance is compared across interaction lengths, while further analyses examine augmented-item count and diffusion hyperparameters. The results favor appropriate augmentation settings and show less harm to experienced users.
4.3 Analysis of Different User Groups (RQ2)
- DiffuASR better alleviates the long-tail user problem and avoids as much harm to experienced users as other augmentation methods.Figure 5 groups Yelp users by interaction length: short [3, 5], medium (5, 20], and long (>20).Source: §4.3 · p. 7, Figure 5 · p. 8
4.4 Analysis of Augmented Number (RQ3)
- Performance rises as M grows from 4 to 10, then falls beyond 10, even below no augmentation; larger M leaves fewer DiffuASR training sequences and risks underfitting.Figure 6 reports the augmented-item-count experiment on Yelp with Bert4Rec.Source: Figure 6 · p. 8, §4.4 · p. 8
4.5 Hyper-parameter Analysis (RQ4)
- Small increases in γ improve performance as generated items better match raw-sequence intent, but continued increases sharply reduce performance and resemble Random-Seq.Figure 7 shows performance trends as γ changes.Source: Figure 7 · p. 8, §4.5 · p. 9
- The linear β schedule performs best for both classifier-guide and classifier-free variants, so the authors use it across experiments.Table 3 reports scheduler results on Yelp with Bert4Rec.Source: §4.5 · p. 8, Table 3 · p. 9
5 RELATED WORKS
Related work places sequential recommendation among approaches that model historical interactions and describes augmentation as one response to data sparsity. It also distinguishes DiffuASR’s augmentation use of diffusion from prior diffusion-based recommendation methods.
Sequential Recommendation
- Sequential recommendation models historical records to predict the next item, evolving from RNNs and CNNs toward self-attention and combinations with side information.The section also identifies work on popularity bias, efficiency, and noise.Source: §5 · p. 9
- Data augmentation and contrastive learning address data sparsity; sequence-level augmentation may require fabricated training procedures, while item-level methods can suffer from poor augmentation quality.ASReP generates pre-order items iteratively, and BiCAT adds a forward constraint and self-knowledge distillation.Source: §5 · p. 9
Diffusion Model
- DiffuRec and DiffRec use diffusion to generate target items, whereas DiffuASR uses diffusion for data augmentation.The section also notes diffusion work on image generation, sampling speed, and controlled generation.Source: §5 · p. 9
6 CONCLUSION
The conclusion presents DiffuASR as a diffusion-based augmentation method for sequential recommendation. It combines a sequence-aware noise-prediction model and guidance strategies, and reports effectiveness across datasets and recommendation models.
6 CONCLUSION
- DiffuASR adapts diffusion to item-sequence generation and uses a Sequential U-Net to capture sequence information while predicting added noise.Two guidance strategies aim to generate items more aligned with preferences in the raw sequence.Source: §6 · p. 9
- Experiments across three real-world datasets and three sequential recommendation models show the reported effectiveness and generality of DiffuASR.Source: §6 · p. 9
- The experiments find that the number of augmented items substantially affects performance.Source: §6 · p. 9