Source-linked AI summary
Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
Ding Zou, Wei Wei, Xian-Ling Mao, Ziyang Wang, Minghui Qiu, Feida Zhu, Xin Cao
TL;DR
Knowledge-aware recommendation suffers from sparse supervised signals, motivating self-supervised learning that can exploit information in the data itself. MCCLK combines three graph views with multi-level cross-view contrastive learning and a kNN semantic graph, and experiments show advantages on three benchmark datasets.
Problem
Knowledge-aware recommendation models face sparse supervised signals because real recommendation applications provide limited training labels.
Method
MCCLK performs local- and global-level cross-view contrastive learning over collaborative, semantic, and structural views, including a kNN item-item semantic graph.
Results
MCCLK outperforms state-of-the-art models on three benchmark datasets.
Takeaways & Limitations
MCCLK improves user and item representation learning by jointly exploiting collaborative, semantic, and structural information in a self-supervised framework.
Abstract
from arXiv · showhide
Knowledge graph (KG) plays an increasingly important role in recommender systems. Recently, graph neural networks (GNNs) based model has gradually become the theme of knowledge-aware recommendation (KGR). However, there is a natural deficiency for GNN-based KGR models, that is, the sparse supervised signal problem, which may make their actual performance drop to some extent. Inspired by the recent success of contrastive learning in mining supervised signals from data itself, in this paper, we focus on exploring the contrastive learning in KG-aware recommendation and propose a novel multi-level cross-view contrastive learning mechanism, named MCCLK. Different from traditional contrastive learning methods which generate two graph views by uniform data augmentation schemes such as corruption or dropping, we comprehensively consider three different graph views for KG-aware recommendation, including global-level structural view, local-level collaborative and semantic views. Specifically, we consider the user-item graph as a collaborative view, the item-entity graph as a semantic view, and the user-item-entity graph as a structural view. MCCLK hence performs contrastive learning across three views on both local and global levels, mining comprehensive graph feature and structure information in a self-supervised manner. Besides, in semantic view, a k-Nearest-Neighbor (kNN) item-item semantic graph construction module is proposed, to capture the important item-item semantic relation which is usually ignored by previous work. Extensive experiments conducted on three benchmark datasets show the superior performance of our proposed method over the state-of-the-arts. The implementations are available at: https://github.com/CCIIPLab/MCCLK.
1 INTRODUCTION
MCCLK addresses sparse supervision in knowledge-aware recommendation by combining three graph views with multi-level cross-view contrastive learning. It adds a kNN semantic graph and reports advantages on three benchmark datasets.
- Knowledge graphs provide item-side facts and connections that complement historical user-item behavior for knowledge-aware recommendation.
- GNN-based and related KGR methods model user, item, and entity connections, but supervised training still relies on sparse interactions.
- Contrastive learning is used to obtain auxiliary supervision from unlabeled data by contrasting positive and negative samples.
- MCCLK uses local collaborative and semantic views plus a global structural view, contrasting local views with each other and local views with the global view.The user-item graph is collaborative, the item-entity graph is semantic, and the user-item-entity graph is structural.
- MCCLK introduces a kNN item-item semantic graph to capture semantic relations often neglected by previous work.
- Experiments on three benchmark datasets demonstrate advantages in representation learning and overall performance over state-of-the-art models.
2 RELATED WORK
Prior KGR work uses embeddings, paths, or GNN aggregation to incorporate knowledge-graph information, but these approaches commonly depend on sparse supervised interactions. MCCLK instead explores self-supervised contrastive learning for node representation learning.
- Embedding-based methods preprocess knowledge graphs with KGE models and incorporate entity and relation embeddings into recommendation.
- Path-based methods enrich recommendation by modeling item connections through meta-paths, meta-graphs, or multi-hop relation paths.
- GNN-based methods aggregate multi-hop neighbors to capture node features and graph structure, including long-range connectivity.
- These approaches generally use supervised training and rely on sparse original interactions, whereas MCCLK extracts supervisory signals from the data itself.
- Contrastive-learning methods learn node representations by contrasting positive and negative pairs across local, global, or multiple structural graph views.
3 PROBLEM FORMULATION
The problem formulation combines implicit user-item interactions with an item-aligned heterogeneous knowledge graph. The task is to predict how likely each user is to adopt an item.
- Users and items are represented as sets U and V, while the interaction matrix Y records implicit feedback.
- An interaction value y_uv = 1 indicates that user u engaged with item v through behavior such as clicking or purchasing.
- The knowledge graph stores item-related facts as heterogeneous triples of entities and relations.Each triple contains a head, relation, and tail, with entities and relations drawn from E and R.
- Item-entity alignments connect recommendation items to knowledge-graph entities, allowing the KG to profile items and complement interaction data.
- Given Y and G, knowledge-aware recommendation learns a function that predicts how likely a user is to adopt an item.
4 METHODOLOGY
MCCLK generates collaborative, semantic, and structural graph views, then applies cross-view learning across local and global levels. Its semantic view explicitly models item-item similarity with a relation-aware kNN graph.
- MCCLK comprises multi-view generation, including global structural and local collaborative and semantic graph views.
- The user-item graph captures collaborative item relationships, while the original user-item-entity graph preserves long-range structural connectivity.
- The semantic view constructs a kNN item-item graph whose edge weights represent semantic similarity between items.A zero entry indicates that no semantic link exists between the corresponding items.
- Relation-aware aggregation encodes neighboring entities and relations into item representations before computing item-item similarity.
- kNN sparsification removes computationally demanding, potentially noisy, and unimportant edges from the fully connected item graph.
- After the three views are acquired, MCCLK performs local- and global-level contrastive learning across them.
4.2 Local-level Contrastive Learning
MCCLK learns complementary item representations from collaborative and semantic local-level views, then contrasts the views to supervise discriminative embeddings.
- Collaborative View Encoder: The collaborative view models item-user-item co-occurrences with Light-GCN to capture collaborative signals from user-item interactions.Light-GCN recursively aggregates messages for K layers without feature transformation.
- Semantic View Encoder: The semantic view applies Light-GCN to an item-item semantic graph constructed from item-entity affiliations.This view injects item-item affinities into item representations.
- Local-level Contrastive Optimization: The two view-specific embeddings are mapped through a one-hidden-layer MLP before contrastive loss calculation.The MLP uses trainable parameters and an ELU nonlinearity.
- Local-level Contrastive Optimization: For each node, the same node across views forms a positive pair, while other nodes in both views form negative pairs.Negative samples include intra-view and inter-view nodes.
- Local-level Contrastive Optimization: The local contrastive objective uses cosine similarity and a temperature parameter, with negatives drawn from intra-view and inter-view sources.This establishes cross-view supervision between the collaborative and semantic representations.
4.3 Global-level Contrastive Learning
MCCLK adds a global structural view that captures long-range user-item-entity connectivity and contrasts it with local-level views.
- Global-level Contrastive Learning: The global view addresses structural information spanning user-item and item-entity graphs, including long-range connectivity.It complements the feature information revealed by local-level views.
- Structural View Encoder: A path-aware GNN aggregates neighboring information while preserving path information across multiple layers.The encoder represents paths such as user-interact-item-relation-entity.
- Structural View Encoder: Relation-aware attention weights are used during structural aggregation to weight each relation and entity.Item and entity representations retain relational signals propagated from their multi-hop neighbors.
- Structural View Encoder: The structural encoder sums representations across layers to obtain global node representations.The neighbor set includes neighboring entities and the item itself.
- Global-level Contrastive Optimization: Global and local representations are mapped into a contrastive-loss space and optimized with the same positive and negative sampling strategy.The global-level contrastive loss supervises the two levels to learn comprehensive representations.
4.4 Model Prediction
MCCLK combines the learned collaborative and global representations for users and items, then predicts their matching score by inner product.
- Model Prediction: After multi-layer aggregation and multi-level contrastive optimization, MCCLK obtains multiple representations for each user and item.The representations include collaborative and global-level embeddings.
- Model Prediction: The final user and item representations are formed by summing and concatenating the learned representations.Their matching score is then predicted through an inner product.
- Model Prediction: MCCLK uses the resulting user-item matching score for recommendation prediction.The prediction is based on the combined representation rather than a single view.
4.5 Multi-task Training
MCCLK jointly trains the recommendation and self-supervised objectives, using pairwise BPR loss alongside global- and local-level contrastive losses.
- Multi-task Training: Pairwise BPR loss reconstructs historical interactions by ranking observed items above unobserved items for each user.The training data contains observed interactions O+ and unobserved counterparts O−.
- Multi-task Training: The overall objective combines the global- and local-level contrastive losses with BPR loss and L2 regularization.Hyperparameters control the local-global contrastive ratio, contrastive-loss weight, and regularization term.
5 EXPERIMENT
Experiments on three public datasets evaluate MCCLK against diverse recommender baselines, test its components, and examine aggregation and loss-weight settings. MCCLK consistently achieves the strongest reported performance, while both contrastive levels and carefully chosen depths contribute to results.
- Experimental Setup: Experiments use Book-Crossing, MovieLens-1M, and Last.FM, which vary in domain, size, and sparsity.The evaluation includes CTR prediction with AUC and F1, and top-K recommendation with Recall@K.
- Experimental Setup: MCCLK is compared with CF-based, embedding-based, path-based, and GNN-based recommender methods.Baselines include BPRMF, CKE, RippleNet, PER, KGCN, KGNN-LS, KGAT, and KGIN.
- Performance Comparison (RQ1): MCCLK consistently outperforms all baselines across three datasets on all reported measures.Its AUC improvements over the strongest baselines are 3.11% on Book, 1.61% on Movie, and 2.77% on Music.
- Ablation Studies (RQ2): Removing global-level contrastive learning significantly degrades performance, while removing local-level contrastive learning is least competitive in most cases.The local-level variant is especially weak on the Book and Movie datasets.
- Hyper-parameter Analysis (RQ3): Semantic-view aggregation performs best at L=1, 1, and 2 on Book, Movie, and Music, respectively.The findings indicate that one or two hops suffice for aggregating item-item semantic neighbors.
- Hyper-parameter Analysis (RQ3): Structural-view aggregation uses L′=2 as a proper distance, while α=1 performs worst and β=0.1 gives the best model performance.Further structural layers introduce noise, and the results support using both local- and global-level contrastive losses.
(a) MCCLK
Figure 5 visualizes item-embedding representations on Book-Crossing, while Figure 6 examines the local-level contrastive-loss weight. MCCLK produces more diverse embeddings than comparison models, and removing either contrastive level makes embeddings less distinguishable.
- Ablation analysis: Figure 6 reports the impact of the local-level contrastive loss weight 𝛼.The supplied figure description identifies the parameter studied but does not report a specific trend or optimal value.
- Representation visualization: Figure 5 compares MCCLK, two ablated variants, KGIN, and RippleNet using 2D visualizations of learned item embeddings on Book-Crossing.The embeddings are regularized before visualization.
- Representation visualization: MCCLK embeddings have a more diverse distribution than KGIN and RippleNet, whose embeddings are mixed and concentrated in a narrow cone.The reported comparison associates MCCLK’s distribution with representing different node feature information and alleviating representation degeneration.
- Ablation analysis: Removing either local-level or global-level contrastive loss makes the learned embeddings more indistinguishable.The comparison supports the effectiveness and robustness of the multi-level cross-view contrastive learning mechanism for representation learning.
6 CONCLUSION
MCCLK improves user and item representation learning for knowledge-aware recommendation through three graph views and multi-level cross-view contrastive learning. Its semantic view additionally constructs a kNN item-item graph to capture item similarities often overlooked by prior work.
- Conclusion: MCCLK improves user and item representation learning in a self-supervised manner for knowledge-aware recommendation.
- Conclusion: MCCLK uses global structural, local collaborative, and local semantic views, including an explicit kNN item-item semantic graph.The graph is intended to mine item-item semantic similarity that previous work rarely notices.
- Conclusion: Multi-level cross-view contrastive learning combines feature and structural information to learn discriminative representations.