PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Item-based Collaborative Structures via Noise Rescheduling in Diffusion for Generative Recommendation

Jiaqi Wang, Tianying Liu, Heng Chang, Jihong Guan, Wengen Li, Shuigeng Zhou

arXiv:2608.23400v1cs.IRcs.AI

TL;DR

Generative recommendation’s discrete diffusion models often omit item-based collaborative information in both semantic-ID representations and uniform denoising schedules. ANR-DiffRec injects co-occurrence priors and adaptively reschedules denoising using recoverability and behavior-aware dependencies, with benchmark evaluations reporting consistent improvements over generative baselines.

  • Problem

    Existing discrete diffusion recommenders overlook item-based collaborative relationships because semantic-focused representations lack collaborative priors and uniform denoising ignores token-level structural dependencies.

  • Method

    ANR-DiffRec fuses item co-occurrence structure with textual features for semantic-ID generation and jointly uses local recoverability and behavior-aware dependencies for adaptive denoising.

  • Results

    ANR-DiffRec consistently and significantly outperforms state-of-the-art autoregressive and diffusion-based generative recommendation models across five diverse real-world benchmarks.

  • Takeaways & Limitations

    Aligning the denoising process with collaborative signals in recommendation data significantly improves generative performance.

  • Takeaways & Limitations

    The current approach does not fully capture user-specific behavioral dependencies from static structural context, and future work targets multimodal information.

Abstract

from arXiv · show

Discrete Diffusion Models (DDMs) have recently been introduced to recommendation systems, modeling user history as a token generation process via iterative denoising. However, while effective at capturing user-level sequential patterns, these methods often fail to explicitly integrate item-based collaborative filtering information, a critical component for accurate recommendation. This deficiency manifests in two key aspects: (1) the item representation is often semantic-focused, lacking collaborative priors for diffusion training; and (2) the denoising process employs a uniform noise schedule, treating all tokens indiscriminately and ignoring item-level adaptive structural dependencies. To bridge this gap, we propose ANR-DiffRec, a unified framework designed to encode item-based collaborative structures into discrete diffusion for generative recommendation. First, we explicitly incorporate an item co-occurrence matrix to guide semantic ID generation, providing a structured collaborative prior for discrete diffusion training. Second, we introduce an item-based adaptive noise rescheduling mechanism that dynamically adjusts denoising weights according to both local contextual recoverability and behavior-aware item dependencies. Specifically, the proposed strategy jointly models intra-item structural context and inter-item collaborative signals, enabling structure-aware denoising during diffusion training. Extensive experiments on multiple benchmarks demonstrate that our method consistently outperforms state-of-the-art generative recommendation models. Code: https://github.com/CalmaQi/ANR-DiffRec.

1 Introduction

ANR-DiffRec addresses the omission of item-based collaborative structure in generative recommendation by enriching semantic IDs and adapting diffusion denoising to token-specific context and behavior dependencies. Experiments across five real-world benchmarks report consistent, significant gains over autoregressive and diffusion-based generative recommenders.

  • Motivation: Existing discrete diffusion recommenders model user-level sequential patterns but largely overlook intrinsic item-based collaborative relationships.This creates a disconnect between generic generation and recommendation’s item-structural requirements.
  • Motivation: Semantic-only item tokenization misses collaborative co-occurrences, leaving diffusion to infer item associations from sparse sequence contexts.The proposed representation instead combines textual semantic features with item-item co-occurrence structure before hierarchical discretization.
  • Motivation: Uniform noise schedules assign identical denoising weights despite uneven information density and differences in token recoverability.Tokens supported by neighboring items or related positions within an SID are treated the same as context-isolated tokens.
  • Proposed Framework: ANR-DiffRec constructs co-occurrence-guided SIDs by factorizing item interaction co-occurrences and fusing the resulting embeddings with pretrained textual features.The resulting token space captures semantic similarity and implicit transition patterns, providing more informative diffusion conditions.
  • Proposed Framework: Its adaptive scheduler combines local recoverability estimation with behavior-aware dependency modeling to assign larger weights when structural and behavioral context is informative.Local recoverability uses intra-item and inter-item context, while behavior-aware modeling uses attention-based interaction modeling over user behavior sequences.
  • Results: ANR-DiffRec consistently and significantly outperforms state-of-the-art autoregressive and diffusion-based generative recommendation models across five diverse real-world benchmarks.The paper’s contributions include dynamic learning of item-to-item collaborative structures and an explicit co-occurrence-guided semantic ID mechanism.

2 Related Works

Generative recommendation uses semantic IDs to frame recommendation as sequence generation, while discrete diffusion replaces autoregressive decoding with bidirectional masked-token refinement. ANR-DiffRec extends this foundation with collaborative structural priors and adaptive denoising tailored to item-based dependencies.

  • Generative Recommendation and Semantic IDs: Generative recommendation reframes next-item prediction as sequence-to-sequence generation over compressed discrete semantic IDs.Methods such as residual quantization and hierarchical clustering convert items into token sequences.
  • Autoregressive Models: Autoregressive recommenders suffer from unidirectional context and error propagation caused by strict left-to-right hierarchical token generation.Early coarse-token errors can propagate to later tokens and produce irrelevant item retrievals.
  • Discrete Diffusion Models: Discrete diffusion avoids autoregressive ordering by reconstructing masked tokens bidirectionally from partially corrupted sequences.This allows the model to use global user history during non-autoregressive refinement.
  • Discrete Diffusion Models: Existing recommendation diffusion methods use uniform masking and do not distinguish easy-to-recover from difficult tokens.They therefore lack structure-aware guidance for capturing collaborative filtering signals.
  • Generative Recommendation and Semantic IDs: Semantic-ID systems support item generation but typically emphasize content semantics rather than structured collaborative item dependencies.This limits explicit modeling of inter-item relationships under noisy contexts.
  • Positioning of ANR-DiffRec: ANR-DiffRec adapts discrete diffusion to collaborative SIDs by combining dynamic attention with structural priors in item-based adaptive noise rescheduling.This design specifically targets the intra-item and inter-item structures present in recommendation semantic IDs.

3 Preliminaries and Background

The recommendation task estimates a user’s next item from chronologically ordered interaction history represented as concatenated semantic-ID tokens. Discrete diffusion performs this generation through iterative bidirectional denoising and confidence-based token refinement.

  • Problem Formulation: For user u, the task is to estimate the probability distribution of the subsequent item v_next given chronological history H_u.The history is an ordered sequence of interacted items.
  • Problem Formulation: Each item is represented by a length-L semantic ID containing discrete tokens derived from a quantization codebook.A user history is flattened by concatenating the semantic IDs of interacted items.
  • Discrete Diffusion Modeling: Discrete diffusion generates the next-item semantic ID through a non-autoregressive reverse denoising trajectory from a fully corrupted state.Inference starts with an all-[MASK] target and progressively recovers clean tokens over T steps.
  • Discrete Diffusion Modeling: At each denoising step, a Transformer encoder predicts token distributions for all masked positions.The process iteratively updates the partially recovered semantic ID.
  • Discrete Diffusion Modeling: The inference procedure keeps the top-n_t tokens by prediction confidence and re-masks the remaining L−n_t positions for the next step.This confidence-based refinement produces the input x_t−1 from the current prediction.

4 Methodology

ANR-DiffRec integrates item-based collaborative structure into discrete diffusion through collaborative-aware semantic IDs, adaptive noise rescheduling, and constrained inference. The framework combines semantic and co-occurrence information, weights denoising using structural and behavioral dependencies, and preserves valid generated SIDs.

  • 4 Methodology: ANR-DiffRec comprises collaborative-guided SID generation, item-based adaptive noise rescheduling, and constrained discrete diffusion inference.The framework uses fused item representations, a Transformer-based denoising process, beam search, and SID validity constraints.
  • 4.1 Item Co-occurrence-Guided SID Generation: Item representations concatenate semantic embeddings with collaborative features derived from an item co-occurrence matrix and truncated singular value decomposition.The co-occurrence matrix counts item co-occurrences in user interaction sequences, while truncated SVD extracts low-dimensional collaborative features.
  • 4.1 Item Co-occurrence-Guided SID Generation: The fused representation preserves both semantic similarity and item collaborative proximity, supplying a collaborative prior for discrete diffusion training.Collaborative signals are incorporated before residual-quantized K-Means generates discrete semantic IDs.
  • 4.1.2 RQ-KMeans-based SID Generation: RQ-KMeans applies multi-stage residual vector quantization so each item is represented as a sequence of L discrete semantic-ID tokens.Each stage quantizes the current residual, and subsequent stages recursively encode what remains.
  • 4.2 Item-based Discrete Diffusion Training: During diffusion training, tokens are independently masked and the model reconstructs masked tokens from noisy sequences using user-history and next-item masking strategies.User-history masking models contextual dependencies across the history, while next-item masking predicts the target item conditioned on observed history.
  • 4.2.2 Item-based Adaptive Noise Rescheduling: Adaptive denoising weights combine local contextual recoverability with behavior-aware item dependencies, assigning larger weights to richly supported or strongly correlated tokens.The mechanism combines static intra-item and inter-item structural context with item-level attention over behavioral representations; sparse-context tokens receive smaller weights.
  • 4.3 Constrained Discrete Diffusion Inference: At inference, the model progressively denoises a fully masked target SID with beam search while enforcing an SID-validity constraint against candidate real items.Beam search retains the top B partially denoised candidates by cumulative log-likelihood, and candidate items remain consistent with revealed tokens.

5.1 Experimental Setup

The evaluation uses five real-world datasets, leave-one-out ranking metrics, and comparisons against 16 representative baselines. ANR-DiffRec combines semantic and collaborative item features before discrete SID generation.

  • Datasets: Five datasets span Amazon product domains, MovieLens, and Steam, covering varied scales and recommendation domains.The Amazon subsets include Scientific, Musical Instruments, and Video Games.
  • Data Split: The leave-one-out protocol reserves the final interaction for testing, the penultimate interaction for validation, and earlier interactions for training.Historical interactions are chronologically ordered.
  • Metrics: Performance is measured with Recall@k and NDCG@k for k∈{1, 5, 10}.NDCG@1 has the same numerical value as Recall@1.
  • Baselines: The study compares ANR-DiffRec with 16 baselines spanning item-ID, semantic-ID, autoregressive, and discrete-diffusion recommendation models.Examples include SASRec, BERT4Rec, TIGER, RPG, DiffGRM, and LLaDA-Rec.
  • Model Configuration: Items combine textual metadata with a 64-dimensional SVD embedding of the item co-occurrence matrix before RQ-KMeans discretization into semantic IDs.RQ-KMeans uses four levels and a codebook size of 256 per level.
  • Model Configuration: The diffusion model is a bidirectional Transformer encoder using four layers on most datasets and six layers on the larger Game dataset.The encoder has eight attention heads and 256-dimensional hidden states.

5.2 Main Results

ANR-DiffRec consistently achieves the strongest reported recommendation performance across datasets and evaluation metrics, outperforming competitive generative baselines. Its advantages extend across sequence lengths and item-popularity groups, with especially clear gains for mid-frequency items.

  • Overall Performance: ANR-DiffRec achieves the best performance across all datasets and evaluation metrics, significantly surpassing LLaDA-Rec across most metrics.On Scientific, the paper specifically reports a substantial Recall@1 improvement.
  • Overall Performance: Discrete-diffusion models outperform autoregressive generators among semantic-ID approaches, while collaborative structure further improves item relationships and transition patterns.The paper attributes the broader advantage to bidirectional attention and reports additional gains from item co-occurrence modeling.
  • Sequence Length: ANR-DiffRec consistently outperforms RPG and LLaDA-Rec across short, medium, long, and extremely long user-history groups.RPG declines on sequences longer than 20, whereas diffusion models are described as more resilient.
  • Item Popularity: ANR-DiffRec outperforms LLaDA-Rec across head, mid, and tail item groups, with the clearest gains on mid-frequency items.Tail improvements are smaller because sparse items provide limited behavioral evidence, although Instruments still shows better tail performance than LLaDA-Rec.
  • Item Popularity: The item co-occurrence-guided construction supports more balanced representations across the popularity spectrum rather than overfitting to extreme-tail patterns.The paper suggests this construction alleviates representation isolation for infrequent items.

5.3 Ablation Study

Ablations identify adaptive noise rescheduling as the strongest contributor to ANR-DiffRec, while item co-occurrence information and complementary dependency signals provide additional benefits. Inter-item context contributes more to recoverability than intra-item dependencies.

  • Effect of Noise Rescheduling: Removing adaptive noise rescheduling causes the largest performance drop, including an NDCG@1 decrease exceeding 15% on Scientific.The authors characterize rescheduling as the core engine for learning item-based collaborative filtering signals.
  • Effect of Item Co-occurrence: Removing item co-occurrence information reduces performance, showing that collaborative signals complement semantic features for modeling users’ structural behavior.The full model combines textual semantics with collaborative information in SID generation.
  • Component Synergy: Removing both modules produces the lowest performance, indicating that adaptive scheduling and collaborative SID construction are mutually reinforcing.The joint ablation is worse than removing either contribution alone.
  • Dependency Modeling: Local contextual recoverability is the primary denoising guide, while behavior-aware dependency modeling supplies complementary information beyond static structural priors.Removing behavior-aware dependency modeling also causes a noticeable, though smaller, degradation.
  • Dependency Modeling: Inter-item contextual modeling contributes more to recoverability than intra-item dependencies, which mainly refine semantic consistency within each SID.Both dependency types provide useful information, but their effects are unequal.
  • Collaborative Signal Construction: SVD and LightGCN both improve over removing collaborative information and perform comparably, whereas Node2Vec performs noticeably worse.The results suggest first-order co-occurrence already provides a strong inductive bias in this diffusion setting.

5.4 Hyper-parameter Analysis

The hyper-parameter analysis finds 64 dimensions to be the preferred collaborative-embedding size, balancing representational capacity and noise suppression. Performance remains robust across the tested geometric-distribution sharpness values.

  • Collaborative Embedding Dimension: Performance improves from 32 to 64 collaborative-embedding dimensions, then offers negligible gains or slight degradation at 128 and 256.The authors therefore set the embedding dimension to 64.
  • Collaborative Embedding Dimension: The 64-dimensional embedding balances representation capacity and noise suppression without overfitting sparse co-occurrence patterns.This setting is used to capture complex structural dependencies between items.

5.5 Efficiency Analysis

ANR-DiffRec maintains Transformer-level asymptotic training complexity while adding item-based adaptive noise rescheduling with low training and inference overhead. Its co-occurrence-guided SID generation also reduces the cost of SID construction compared with RQ-VAE-based generation.

  • Model training complexity: ANR-DiffRec remains bounded by O(M^2d), matching the asymptotic complexity class of Transformer-based baselines.The rescheduling module adds O(M^2) structural computation and reuses Transformer attention scores.
  • SID generation: Co-occurrence-guided SID generation incurs only a fraction of RQ-VAE-based pretraining cost.Its costs include co-occurrence-matrix construction, SVD decomposition, and RQ-KMeans clustering.
  • Model training complexity: Around 1% additional training overhead is introduced by item-based adaptive noise rescheduling compared with LLaDA-Rec.The reported comparison concerns model training in Scientific.
  • Inference efficiency: At Beam=50, per-user inference latency is 10.17 ms for ANR-DiffRec versus 10.06 ms for LLaDA-Rec.Inference time scales linearly with beam size, while the two models remain nearly identical across tested configurations.
  • Inference efficiency: ANR-DiffRec introduces negligible inference overhead compared with LLaDA-Rec while achieving superior recommendation performance.The conclusion is drawn from the reported constrained diffusion inference comparison.

5.6 Case Studies

Case studies evaluate whether co-occurrence features make item embeddings and SIDs more collaborative. Across sampled items, the method improves retrieval of ground-truth co-occurring neighbors, with especially large gains in SID space.

  • Experimental setup: The case study compares Top-20 neighbors from continuous embedding similarity and discrete SID overlap against ground-truth co-occurrence neighbors.Three randomly sampled items are evaluated using the embedding and SID spaces.
  • Results: Co-occurrence feature integration produces consistent improvements across all sampled cases.The comparison uses the number of ground-truth co-occurring items retrieved in the Top-20 lists.
  • SID space: For Item #6278, SID retrieval triples the number of co-occurring neighbors, increasing from 4 to an unstated higher value.The supplied passage states the tripling but does not provide the ending count.
  • SID space: The quantizer guided by co-occurrence features successfully encodes collaborative information into SIDs.The evaluation reports gains in co-occurring hits after feature integration.

6 Conclusion

ANR-DiffRec integrates item-based collaborative structures into discrete diffusion through co-occurrence-guided SID generation and adaptive noise rescheduling. Experiments across benchmarks support its effectiveness, while future work targets multimodal integration.

  • Conclusion: ANR-DiffRec integrates item-based collaborative structures into discrete diffusion for generative recommendation.The framework’s central contribution is structure-aware denoising guided by item contexts.
  • Conclusion: Its adaptive noise rescheduling adjusts noise levels according to intra-item and inter-item contexts.This provides structure-aware guidance during denoising.
  • Conclusion: Co-occurrence-guided SID generation injects item co-occurrence priors into diffusion training.The mechanism complements adaptive noise rescheduling.
  • Conclusion: Empirical results across various benchmarks confirm the effectiveness of aligning denoising with collaborative signals.The conclusion states that this alignment significantly improves generative performance.
  • Future work: Future work aims to integrate multimodal information into generative recommendation models.This is identified as a direction for further advancement.
Loading 2608.23400v1…