PaperlayerView exact revision on arXiv

Source-linked AI summary

MGDiff: Multi-Interest Sequence Recommendation with Masking GNN-Guided Diffusion

Wenjing Xiao, Hao Ding

arXiv:2609.01619v1cs.IR

TL;DR

Diffusion-based sequence recommenders face semantic distortion from sparse, random interactions and popularity bias in generated recommendations. MGDiff addresses these issues with Dual-layer Semantic Guidance and Popularity-Aware Guidance, and its experiments show stronger performance than baseline methods across multiple datasets. The combined components also improve performance over variants that omit either module.

  • Problem

    Diffusion-based sequence recommenders are limited by semantic distortion from sparse, random interactions and endogenous popularity bias.

  • Method

    MGDiff combines a Weight-adaptive Masking GNN and dynamically routed multi-interest Mixture-of-Experts network in Dual-layer Semantic Guidance, plus Popularity-Aware Guidance.

  • Results

    MGDiff consistently outperforms state-of-the-art baselines across four real-world datasets, while combining DSG and PAG outperforms variants omitting either module.

  • Takeaways & Limitations

    The framework provides a combined approach for more precise guidance and recommendations with reduced popularity bias and broader item exposure.

Abstract

from arXiv · show

We propose a novel Multi-Interest Sequence Recommendation Framework with \underline{M}asking \underline{G}NN-Guided \underline{Diff}usion Model (MGDiff), designed to generate accurate, bias-free user interest information during the diffusion process. First, we propose a semantics-enhanced Dual-layer Semantic Guidance (DSG) framework, which decomposes guidance into two synergistic stages: extracting latent item semantics and decoupling multidimensional user intent. We design a Weight-adaptive Masking Graph Neural Network reconstructs missing links to uncover deep item relationships beyond superficial co-occurrence, while a Dynamic Multi-Expert Network projects user preferences into distinct semantic subspaces to suppress irrelevant interference. This hierarchical design yields structured guidance that significantly improves the generation accuracy of diffusion models. Second, We propose a Popularity-Aware Guidance (PAG) mechanism that performs spatial geometric adjustments on the outputs of diffusion models: by using item popularity as a differentiable adjustment signal to recalibrate similarity metrics, we enable DMs to generate diverse recommendations free from popularity bias. Finally, we compare MGDiff with multiple baseline models across four widely used datasets, demonstrating its superior performance and validating its effectiveness.

I. INTRODUCTION

MGDiff addresses semantic distortion and popularity bias in diffusion-based sequence recommendation by combining hierarchical semantic guidance with popularity-aware adjustment. Experiments across real-world datasets support its effectiveness against existing baselines.

  • Motivation: Existing diffusion-based sequence recommenders denoise target items using interaction sequences, but guidance can be distorted by sparse and random interactions.Such distortion may treat incidental co-purchases as meaningful relationships.
  • Method: MGDiff introduces Dual-layer Semantic Guidance to extract latent item relationships and decouple multidimensional user intent.The lower layer structures item semantics, while the upper layer generates refined guidance signals for diffusion models.
  • Method: A Weight-adaptive Masking GNN and dynamic routing multi-interest Mixture-of-Experts network generate precise, stable diffusion guidance.The framework extracts latent structures while modeling distinct user interests.
  • Method: Popularity-Aware Guidance targets endogenous generation bias, including low exposure for long-tail items and homogenized recommendations.The mechanism is designed to improve item discriminability.
  • Evaluation: MGDiff consistently outperforms state-of-the-art baseline models in experiments on four real-world datasets.The reported experiments compare MGDiff with multiple traditional and diffusion-based recommendation models.

II. METHOD

The method represents interaction sequences as item embeddings and enriches them with relational information through Dual-layer Semantic Guidance. A refined user representation is then used to predict the next item matching the user’s interests.

  • Sequence Representation: A user interaction sequence Q contains chronologically ordered items before the ground-truth next item vL.The sequence is encoded as X = [x1, x2, ..., xL−1], where each item embedding has dimension d.
  • Prediction: The resulting user representation is used to predict the next item that best matches the user’s interests.This connects sequence encoding and relational modeling to the next-item prediction objective.
  • Dual-layer Semantic Guidance: Dual-layer Semantic Guidance combines item-relationship learning with user-interest refinement to produce a richer user representation.The Weight-adaptive Masking GNN produces item embeddings G, which are fused with the initial sequence encoding X into H.

B. Dual-layer Semantic Guidance

The DSG module combines a weight-adaptive masking GNN for latent item relationships with a dynamic multi-expert network for robust, disentangled guidance. These components progressively expose harder graph links, specialize expert routing, and combine refined representations for diffusion guidance.

  • Framework overview: DSG comprises a weight-adaptive masking GNN and a multi-interest expert network that collaboratively generate guidance for diffusion models.The GNN extracts item structure, while the expert network models user interests through dynamic routing.
  • Weight-adaptive Masking GNN: The graph connects adjacent items in interaction sequences, with edge weights counting observed transitions between items.Items are graph nodes, and transition counts determine edge weights.
  • Weight-adaptive Masking GNN: High-weight edges are masked first, then masking shifts toward a uniform distribution to expose increasingly difficult low-weight relationships.This curriculum-style schedule moves from easy-to-reconstruct popular-item links toward challenging long-tail links.
  • Weight-adaptive Masking GNN: Masked edges are removed to form a corrupted graph, whose node representations are propagated through GNN layers to produce item embeddings G.The resulting embeddings preserve relational information learned from the masked graph.
  • Dynamic Multi-Experts Network: The expert network fuses encoder embeddings X with GNN embeddings G, then uses Gumbel-Softmax routing to assign tokens to experts.Temperature annealing sharpens assignments, while a rising threshold enforces progressively sparser expert activation.
  • Dynamic Multi-Experts Network: As training progresses, decreasing temperature and increasing threshold reduce activated experts per token, promoting progressive specialization before producing final guidance for diffusion models.The final user embedding Z is combined with the initial embeddings X, with λq controlling vector-injection strength.

III. EXPERIMENT

The experiment evaluates MGDiff against state-of-the-art models, examines DSG and PAG improvements, tests recommendation diversity, and studies hyper-parameter effects.

  • Research questions: The experiments compare MGDiff with state-of-the-art models to assess overall performance.This addresses research question RQ1.
  • Research questions: Ablation-oriented analysis evaluates the improvements contributed by DSG and PAG.This addresses RQ2.
  • Research questions: The study tests whether recommendation diversity meets users’ diverse needs.This addresses RQ3.
  • Research questions: Hyper-parameter analysis examines how parameter choices affect MGDiff’s performance.This addresses RQ4.

A. Experimental Settings

MGDiff is evaluated using four widely used real-world sequence-recommendation datasets, spanning product reviews, movie ratings, and video-game information.

  • Datasets: Four real-world datasets are used to validate MGDiff’s effectiveness in sequence recommendation.The datasets are Amazon Beauty, Amazon Toys, MovieLens-1M, and Steam.
  • Datasets: Amazon Beauty and Amazon Toys contain nearly 20 years of Amazon user reviews for beauty products and toys.
  • Datasets: MovieLens-1M contains 1 million movie ratings from 6,000 users on 4,000 movies.
  • Datasets: Steam provides video-game information including users’ playing time, prices, categories, and related attributes.

1) Baseline:

The baseline set covers traditional sequential models, multi-interest representation methods, and generative or contrastive approaches, including several diffusion-based recommenders.

  • Traditional recommendation models: Traditional baselines include GRU4Rec, SASRec, and BERT4Rec for sequential recommendation.They use recurrent or Transformer-based sequence modeling, including masked item prediction in BERT4Rec.
  • Multi-interest representation models: ComiRec and TiMiRec represent multi-interest modeling baselines using dynamic routing, attention, and target-item supervision.
  • Generative and contrastive learning models: ACVAE and CL4SRec provide generative and contrastive-learning comparisons based on variational autoencoding, adversarial training, sequence augmentation, and consistency regularization.
  • Generative and contrastive learning models: DreamRec and DiffuRec are pioneering diffusion-based recommenders for next-item generation.
  • Generative and contrastive learning models: DiQDiff extends diffusion recommendation with quantile-guided diffusion optimized through contrastive divergence maximization.

B. Overall Performance (RQ1)

Across three real-world datasets, MGDiff is compared with sequential, interest-based, and diffusion-based baselines. The results support MGDiff’s strategies for reducing semantic distortion and popularity bias.

  • Model comparison: Diffusion-based baselines achieve the strongest performance among the compared existing methods.The paper attributes this to modeling conditional item distributions rather than relying on discriminative point estimation.
  • Model comparison: Traditional sequential models generally underperform because fixed user vectors struggle to capture complex, multi-faceted user intents.
  • Model comparison: Interest-based methods outperform traditional sequential models but remain limited by deterministic representations.
  • MGDiff: MGDiff’s Weight-adaptive Masking GNN and Popularity-Aware Guidance strategies mitigate semantic distortion and bias issues limiting existing diffusion models.The paper reports that these strategies lead to more precise and robust recommendations.

C. Ablation Study (RQ2)

An ablation study evaluates the DSG and PAG modules individually and jointly, while additional analysis examines performance for sparse users and cold items.

  • Module ablation: The ablation compares Base, w/o DSG, and w/o PAG variants to assess the contributions of the two modules.Base excludes both modules; the other variants omit DSG or PAG respectively.
  • Module ablation: On all datasets, both w/o DSG and w/o PAG consistently outperform Base, demonstrating the feasibility of each individual design.
  • Module ablation: MGDiff achieves the highest performance among the three variants, indicating that DSG and PAG are stackable configurations.
  • Hard-subset analysis: On the Toys dataset, MGDiff consistently outperforms baselines for sparse users and cold items.Sparse users are the shortest 20% of interaction sequences, while cold items are in the bottom 20% popularity percentile.
  • Hard-subset analysis: The hard-subset results highlight MGDiff’s performance in capturing user intent from scarce data and recommending diverse, long-tail items.

D. Visualization (RQ3)

The visualization analysis examines embedding diversity across model variants. Structured guidance further increases diffusion-generated embedding diversity while preserving some naturally convergent user-interest clusters.

  • MGDiff and w/o DSG produce more diverse embeddings, although some clusters remain because real-world user interests can naturally converge.
  • The visualization compares scenarios involving little interaction history and long-tail-item targets.
  • Adding structured guidance during diffusion generation further enhances embedding diversity.The comparison between MGDiff and w/o DSG, and between Base and w/o PAG, supports the contribution of the guidance components.

E. Long-tail Performance Analysis

MGDiff is evaluated for popularity bias and long-tail recommendation performance on the Toys and Beauty datasets. It improves both debiasing indicators and recommendation outcomes for users focused on long-tail items.

  • MGDiff achieves lower ARP and Gini coefficients on Toys and Beauty than other methods, indicating reduced over-recommendation of popular items.
  • MGDiff improves coverage and Tail@20, broadening recommendations and increasing exposure opportunities for long-tail items.
  • MGDiff performs best on H@20 for users whose interactions primarily involve long-tail items.This result indicates improved recommendation accuracy for users with niche preferences.
  • The experiments consistently demonstrate that MGDiff mitigates popularity bias while optimizing recommendation quality for long-tail recommendation scenarios.
  • The analysis includes performance comparisons with different numbers of experts.

IV. CONCLUSION

MGDiff addresses semantic distortion and endogenous generation bias in diffusion-based sequence recommendation through hierarchical semantic guidance and popularity-aware adjustment. Extensive real-world experiments show consistent superiority over state-of-the-art baselines.

  • MGDiff combines a Weight-adaptive Masking GNN with dynamically routed Mixture-of-Experts guidance to refine user interests from broad exploration to specialization.
  • Popularity-Aware Guidance incorporates popularity as a differentiable signal in contrastive learning to improve item discriminability and balance accuracy with diversity.
  • Extensive experiments on real-world datasets demonstrate that MGDiff consistently outperforms state-of-the-art baselines.
Loading 2609.01619v1…