PaperlayerView exact revision on arXiv

Source-linked AI summary

DiffKG: Knowledge Graph Diffusion Model for Recommendation

Yangqin Jiang, Yuhao Yang, Lianghao Xia, Chao Huang

arXiv:2312.16890v1cs.IR

TL;DR

Knowledge-aware recommendation must handle sparse interactions and noisy or task-irrelevant KG information. DiffKG combines diffusion-based KG generation, collaborative knowledge graph convolution, and diffusion-enhanced augmentation; experiments across benchmark datasets report superiority over competitive baselines.

  • Problem

    Knowledge-aware recommendation is challenged by sparse interactions and knowledge graphs containing long-tail, noisy, or task-irrelevant information.

  • Method

    DiffKG integrates a generative knowledge graph diffusion model with data augmentation and collaborative knowledge graph convolution guided by user-item interactions.

  • Results

    Extensive experiments across diverse benchmark datasets report that DiffKG outperforms baseline approaches and improves recommendations with task-relevant KG signals.

  • Takeaways & Limitations

    DiffKG provides a framework for extracting high-quality, task-relevant signals from noisy knowledge graphs while aligning item semantics with collaborative relation modeling.

Abstract

from arXiv · show

Knowledge Graphs (KGs) have emerged as invaluable resources for enriching recommendation systems by providing a wealth of factual information and capturing semantic relationships among items. Leveraging KGs can significantly enhance recommendation performance. However, not all relations within a KG are equally relevant or beneficial for the target recommendation task. In fact, certain item-entity connections may introduce noise or lack informative value, thus potentially misleading our understanding of user preferences. To bridge this research gap, we propose a novel knowledge graph diffusion model for recommendation, referred to as DiffKG. Our framework integrates a generative diffusion model with a data augmentation paradigm, enabling robust knowledge graph representation learning. This integration facilitates a better alignment between knowledge-aware item semantics and collaborative relation modeling. Moreover, we introduce a collaborative knowledge graph convolution mechanism that incorporates collaborative signals reflecting user-item interaction patterns, guiding the knowledge graph diffusion process. We conduct extensive experiments on three publicly available datasets, consistently demonstrating the superiority of our DiffKG compared to various competitive baselines. We provide the source code repository of our proposed DiffKG model at the following link: https://github.com/HKUDS/DiffKG.

1 INTRODUCTION

Recommendation systems use knowledge graphs to enrich collaborative filtering, but noisy, sparse, and irrelevant KG connections can undermine task-specific modeling. DiffKG addresses this by combining diffusion-based KG generation, task-relevant signal extraction, collaborative guidance, and data augmentation.

  • Motivation: Sparse user-item interactions hinder recommendation performance, motivating knowledge-aware recommendation methods that incorporate item information from knowledge graphs.Existing approaches include embedding-based, path-based, and graph neural network methods for modeling semantic and high-order information.
  • Challenges: Knowledge-aware recommendation depends on high-quality knowledge graphs, yet long-tail entities and topic-irrelevant item-entity connections introduce noise.Contrastive-learning methods address some KG issues but may rely on random augmentation or intuitive cross-view information.
  • Proposed framework: DiffKG progressively corrupts and reconstructs knowledge graphs through a diffusion process to generate task-relevant structures.The framework also introduces a KG filter for irrelevant or erroneous data and uses flexible neural networks for iterative reverse generation.
  • Proposed framework: DiffKG integrates a generative diffusion model with KG learning to align knowledge-aware item semantics with collaborative relation modeling.The model is designed to distill high-quality signals from noisy knowledge graphs and enhance collaborative filtering with task-relevant item knowledge.
  • Reported outcome: Extensive evaluations across diverse benchmark datasets report performance gains over baseline models while addressing data noise and scarcity.The paper presents these evaluations as evidence for DiffKG’s effectiveness in knowledge-aware recommendation.

2 PRELIMINARIES

The preliminaries represent collaborative behavior with a binary user-item interaction graph and external item attributes with a knowledge graph. KG-enhanced recommendation learns a model from both graphs to predict user-item interaction likelihood.

  • User-Item Interaction Graph: The user-item interaction graph represents collaborative signals between users and items with binary interaction indicators.An indicator of 1 denotes an observed interaction, while 0 denotes its absence.
  • Knowledge Graph: The knowledge graph organizes external item attributes through entities and relationships represented as head-relation-tail triplets.Each triplet characterizes semantic relatedness between a head entity and a tail entity.
  • KG-Enhanced Recommendation: KG-enhanced recommendation trains a parameterized recommender from the user-item interaction graph and associated knowledge graph.The model predicts the likelihood that user u interacts with item i.

3 THE PROPOSED DIFFKG FRAMEWORK

The DiffKG framework combines heterogeneous knowledge aggregation, a knowledge graph diffusion model, and diffusion-enhanced data augmentation. Together, these components capture diverse KG relationships and provide higher-quality KG information for recommendation.

  • Framework architecture: DiffKG contains heterogeneous knowledge aggregation, knowledge graph diffusion, and KG diffusion-enhanced data augmentation modules.The modules are presented as the framework’s technical components and overall architecture.
  • Framework architecture: The framework uses these components to capture diverse knowledge-graph relationships and ensure high-quality KG information for recommendation enhancement.The stated design connects KG processing with the goal of improving recommendation quality.

3.1 Heterogeneous Knowledge Aggregation

Heterogeneous knowledge aggregation models diverse KG relations with relation-aware embeddings and attention-based representations. The module uses neighboring entities, normalization, and dropout while relation-specific attention captures item-entity semantics.

  • Relation-aware embedding: A relation-aware knowledge embedding layer captures diverse relationships in heterogeneous knowledge graphs.The layer is inspired by graph-attention mechanisms and uses parameterized attention to represent entity- and relation-dependent context.
  • Knowledge aggregation: Knowledge aggregation combines each item with neighboring entities across relation types using item and entity embeddings, dropout, normalization, and attentive relevance.The attentive relevance is entity- and relation-specific during aggregation.
  • Framework overview: Figure 1 presents the overall DiffKG framework, whose architecture includes the heterogeneous knowledge aggregation module alongside diffusion and augmentation components.The figure caption identifies it as the overall framework of the proposed model.
  • Relation-aware embedding: The attention mechanism captures distinct semantics of relationships between items and entities through a parameterized weight matrix and LeakyReLU transformation.The model customizes the transformation to input item and entity representations.

3.2 KG-enhanced Data Augmentation

DiffKG replaces simplistic knowledge-graph augmentation with a generative process that reconstructs recommendation-relevant subgraphs. It then combines the original and generated KG views with collaborative filtering and contrastive learning.

  • Random augmentation and simplistic cross-view contrasts can introduce unwanted information and overlook irrelevant knowledge-graph content.
  • DiffKG generates a subgraph G′_k containing relationships relevant to the downstream recommendation task.The generated subgraph is reconstructed from the original knowledge graph.
  • The encoding pipeline applies heterogeneous knowledge aggregation to G_k and G′_k, then uses graph-based collaborative filtering to refine item representations.The two KG views provide item feature vectors while preserving KG semantics before collaborative propagation.
  • Embeddings of the same node across the two knowledge-enhanced views form positive pairs, while embeddings of different nodes form negative pairs.The contrastive objective maximizes agreement for positive pairs and minimizes agreement for negative pairs.
  • The self-supervised objective combines user-side and item-side contrastive losses.The combined loss is denoted L_cl.

3.3 Diffusion with Knowledge Graph

DiffKG progressively corrupts knowledge-graph relations with Gaussian noise and learns to reverse that corruption to reconstruct a recommendation-relevant graph. Collaborative knowledge graph convolution injects user-item signals so denoising reflects downstream user preferences.

  • 3.3 Diffusion with Knowledge Graph: DiffKG learns a recommendation-relevant subgraph by identifying true item-entity relationships after diffusion-based corruption and iterative denoising.The forward process adds noise to relations, while reverse learning recovers the original relations.
  • 3.3.1 Noise Diffusion Process.: The KG is represented as an adjacency matrix whose binary entries indicate whether an item is related to an entity.This representation provides the input structure for the diffusion process.
  • 3.3.1 Noise Diffusion Process.: The forward process initializes χ_0 as the original adjacency matrix and adds Gaussian noise over T Markov-chain steps.The noise scale at step t is controlled by β_t, and χ_T approaches a standard Gaussian as T increases.
  • 3.3.1 Noise Diffusion Process.: A linear noise scheduler controls 1−ᾱ_t using s, α_low, and α_up to regulate noise addition.s controls noise scales, while α_low and α_up bound the added noises.
  • 3.3.1 Noise Diffusion Process.: The reverse process uses neural networks to remove noise from χ_t and reconstruct χ_{t−1}, generating relations from χ_T through iterative denoising.The networks parameterized by θ produce the Gaussian mean and covariance for the denoising transition.
  • 3.3.2 Optimization of KG Diffusion Process.: The ELBO objective measures recovery of χ_0 and regulates approximation of reverse transitions across diffusion steps.The model predicts χ_0 from χ_t and t using an MLP, while uniformly sampling t to optimize L_elbo.
  • 3.3.3 Knowledge Graph Generation with Diffusion Model.: DiffKG avoids pure-noise reverse initialization by corrupting the original KG first and then applying deterministic reverse denoising.The method reconstructs G′_k from the predicted χ_0 and selects the top k relations for each item.
  • 3.3.3 Knowledge Graph Generation with Diffusion Model.: CKGC aggregates user-item interaction data to guide KG diffusion toward relationships pertinent to downstream recommendation.It incorporates user-preference signals into the denoised knowledge graph and bridges KG denoising with recommendation.

3.4 The Learning Process of DiffKG

DiffKG jointly trains recommendation and knowledge-graph diffusion objectives, balancing diffusion losses while combining Bayesian personalized ranking with contrastive learning and regularization.

  • Training objectives: DiffKG training has separate recommendation-task and KG-diffusion components optimized jointly.KG diffusion uses ELBO and CKGC losses optimized simultaneously.
  • Recommendation task: The recommendation objective combines Bayesian personalized ranking loss with the previously introduced contrastive loss.BPR is defined over user-positive-item and user-negative-item triples.
  • Recommendation task: Training triples pair observed user-item interactions with unobserved interactions formed from the user-item Cartesian product excluding observed pairs.The learnable parameters are Θ, while λ1 and λ2 weight contrastive loss and L2 regularization.

4 EXPERIMENTS

The experiments evaluate DiffKG across three public recommendation datasets using full-rank Recall@20 and NDCG@20 against diverse baselines. DiffKG outperforms all baselines, while component analyses support the roles of task-relevant diffusion and collaborative knowledge graph convolution.

  • Experimental setup: Experiments address overall baseline comparison, component and hyperparameter effects, and DiffKG’s behavior across three public datasets.The datasets are Last-FM, MIND, and Alibaba-iFashion, representing music, news, and e-commerce scenarios.
  • Experimental setup: Evaluation uses full-rank Recall@20 and NDCG@20 to avoid bias from negative sampling.N is set to 20 for both top-N recommendation metrics.
  • Baselines: DiffKG is compared with baselines spanning collaborative filtering, embedding-based, GNN-based, and self-supervised knowledge-aware recommendation methods.The baseline families include BPR, NeuMF, CKE, KTUP, KGNN-LS, KGCN, KGAT, KGIN, MCCLK, and KGCL.
  • RQ1: Overall Performance Comparison: DiffKG consistently outperforms all baseline approaches across the evaluated datasets and metrics.The authors attribute this result to diffusion-generated knowledge graphs containing task-specific entity relationships, which improve data augmentation.
  • RQ1: Overall Performance Comparison: Knowledge-aware recommenders outperform traditional BPR and NeuMF, while DiffKG’s gap over KGAT, KGIN, and KGCL indicates the impact of irrelevant KG relations.KGCL augments the user-item interaction matrix, whereas DiffKG generates a task-related knowledge graph through diffusion.

4.3 RQ2: Ablation Study

The ablation study evaluates DiffKG’s contrastive learning, diffusion, and collaborative convolution components, while further analyses examine hyperparameters, sparsity, and KG noise. Results consistently support the contribution of these modules and the model’s robustness to challenging recommendation conditions.

  • Key Module Ablation: Removing KG-enhanced contrastive learning significantly degrades performance across all cases.
  • Key Module Ablation: Removing the diffusion model worsens results, with larger drops on Last-FM and MIND indicating more noise in those knowledge graphs.
  • Key Module Ablation: Removing collaborative knowledge graph convolution degrades performance across all cases because it excludes user collaborative knowledge from diffusion training.
  • Hyperparameter Analysis: On MIND, λ1 = 1 and τ = 1 produce the best performance, while T = 5 balances diffusion performance and computation.
  • Sparsity and KG Noise: DiffKG outperforms KGCL under sparse data and significantly improves recommendations for long-tail items, while achieving the strongest performance among knowledge-aware models.
  • Sparsity and KG Noise: With 10% noisy triplets, DiffKG has the lowest average performance decrease, indicating stronger filtering of irrelevant KG information.
  • Case Study: In a news case study, KG diffusion replaces recommendations about unrelated entities with articles more relevant to the Star Wars topic.

5 RELATED WORK

Related work spans knowledge-aware recommendation, self-supervised data augmentation, and diffusion probabilistic models. These lines of research motivate combining knowledge-graph representations, additional supervision, and corruption-reconstruction processes for recommendation.

  • Knowledge-Aware Recommendation: Knowledge-aware recommendation methods include embedding-based, path-based, and GNN-based approaches that extract information from knowledge graphs.
  • Knowledge-Aware Recommendation: GNN-based methods such as KGCN, KGAT, and KGIN combine paradigms to aggregate or weight knowledge-graph neighbors for item representations.
  • Self-Supervised Data Augmentation: Data augmentation with self-supervised learning addresses recommendation sparsity by extracting additional supervision signals from raw data.
  • Self-Supervised Data Augmentation: Contrastive methods generate augmented user or item views and train models to distinguish positive from negative pairs.
  • Self-Supervised Data Augmentation: Masking and reconstruction methods hide parts of interactions or items and train models to predict the missing elements.
  • Diffusion Models: Diffusion probabilistic models learn to recover original data from perturbed data and have been applied in vision and text generation.

6 CONCLUSION

DiffKG uses task-specific item knowledge and a generative diffusion model to align knowledge-aware item semantics with collaborative modeling. Extensive benchmark evaluations report improved performance and effective handling of noisy data.

  • DiffKG extracts high-quality, task-specific signals from noisy knowledge graphs for knowledge-aware recommendation.
  • The framework integrates generative diffusion with knowledge-graph learning to align item semantics and collaborative relation modeling.
  • Extensive evaluations on diverse benchmark datasets report significant performance improvements over baseline models.
  • DiffKG effectively addresses noisy data that can impede recommender-system accuracy.
Loading 2312.16890v1…