Source-linked AI summary
Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation
Hongwei Wang, Fuzheng Zhang, Miao Zhao, Wenjie Li, Xing Xie, Minyi Guo
TL;DR
Recommendation based on collaborative filtering faces sparsity and cold-start problems. MKR uses end-to-end multi-task learning, linking recommendation with knowledge graph embedding through cross&compress units. Across movie, book, music, and news scenarios, it reports substantial gains over strong baselines and maintains decent performance in sparse settings.
Problem
Collaborative filtering suffers from sparse user-item interactions and cold-start problems, motivating side information for recommender systems.
Method
MKR jointly trains recommendation and knowledge graph embedding tasks, connecting them with cross&compress units that transfer latent features and model high-order item-entity interactions.
Results
MKR achieves substantial gains over state-of-the-art baselines across movie, book, music, and news recommendation, including 11.6% average movie AUC and 66.4% average book Recall@10 improvements.
Takeaways & Limitations
Knowledge graph embedding can serve as an explicit multi-task constraint for recommendation, while shared features mutually benefit both recommendation and KGE tasks.
Takeaways & Limitations
The implemented framework leaves investigation of other neural networks and alternative KGE methods to future work.
Abstract
from arXiv · showhide
Collaborative filtering often suffers from sparsity and cold start problems in real recommendation scenarios, therefore, researchers and engineers usually use side information to address the issues and improve the performance of recommender systems. In this paper, we consider knowledge graphs as the source of side information. We propose MKR, a Multi-task feature learning approach for Knowledge graph enhanced Recommendation. MKR is a deep end-to-end framework that utilizes knowledge graph embedding task to assist recommendation task. The two tasks are associated by cross&compress units, which automatically share latent features and learn high-order interactions between items in recommender systems and entities in the knowledge graph. We prove that cross&compress units have sufficient capability of polynomial approximation, and show that MKR is a generalized framework over several representative methods of recommender systems and multi-task learning. Through extensive experiments on real-world datasets, we demonstrate that MKR achieves substantial gains in movie, book, music, and news recommendation, over state-of-the-art baselines. MKR is also shown to be able to maintain a decent performance even if user-item interactions are sparse.
1 INTRODUCTION
Collaborative filtering is limited by sparse interactions and cold-start conditions, motivating knowledge graphs as side information for recommendation. MKR addresses these limitations with end-to-end multi-task learning that connects recommendation and knowledge graph embedding through cross&compress units.
- Collaborative filtering commonly suffers from sparse user-item interactions and the cold-start problem.
- Knowledge graphs provide side information containing facts and item-related connections, but their high dimensionality and heterogeneity require embedding.
- Prior KG-aware methods rely on manually designed meta-paths, precomputed embeddings, or restricted side information, limiting generic or end-to-end recommendation.
- MKR jointly learns recommendation and knowledge graph embedding, using cross&compress units to share latent features and model high-order item-entity interactions.
- 11.6% AUC improvements on average for movies and 66.4% Recall@10 improvements on average for books were reported over state-of-the-art baselines.
- MKR formulates knowledge graph embedding as an explicit multi-task constraint that regularizes recommendation rather than treating the problem solely as transfer learning or cross-domain recommendation.
2 OUR APPROACH
MKR jointly trains recommendation and knowledge graph embedding modules, linking them through cross&compress units that transfer features between associated items and entities. The framework defines the problem, modules, cross-feature mechanism, and multi-task optimization procedure.
- 2.1 Problem Formulation: MKR predicts unseen user–item engagement from an implicit-feedback matrix and a knowledge graph of entity–relation–entity triples.Items may correspond to one or more knowledge-graph entities.
- 2.2 Framework: The framework combines recommendation, KGE, and cross&compress modules to extract features and jointly support prediction and knowledge-graph learning.The recommendation module uses MLPs, while the KGE module processes heads and relations through nonlinear layers and cross&compress units.
- 2.3 Cross&compress Unit: Cross operation forms a d × d matrix of item–entity feature interactions, while compress operation projects it back into item and entity representations.The unit outputs vectors for the next layer and adaptively adjusts knowledge-transfer weights between tasks.
- 2.3 Cross&compress Unit: Cross&compress units are restricted to low-level layers because higher-level sharing risks negative transfer and mixes features without explicit item–entity association.Higher layers become more task-specific as representations transform from general to specific.
- 2.6 Learning Algorithm: MKR trains recommendation and KGE objectives with negative sampling, balancing their losses alongside L2 regularization to reduce overfitting.The recommendation term uses cross-entropy; the KGE term raises scores for true triples and lowers scores for false triples.
3 THEORETICAL ANALYSIS
The analysis establishes the expressive capacity of cross&compress units and positions MKR as a generalized framework encompassing representative recommendation and multi-task models. It also identifies how restricted versions lose approximation capacity or alter feature sharing.
- 3.1 Polynomial Approximation: Cross&compress units can approximate item–entity feature interactions to exponentially increasing degree as network depth grows.The analysis uses polynomial approximation arguments based on the Weierstrass theorem.
- 3.1 Polynomial Approximation: Theorem 1 shows that MKR automatically models sufficiently high-order combinatorial features of items and entities.The result concerns theoretical polynomial approximation capacity, not guarantees of actual task performance.
- 3.2 Generalized Framework: MKR provides a unified view in which several recommendation and multi-task models are restricted versions of, or theoretically related to, the framework.The comparison includes factorization machines, deep&cross networks, and cross-stitch networks.
- 3.2.1 Factorization Machines: A one-layer cross&compress unit is conceptually similar to a factorization machine but factorizes item–entity weights as sums of scalars rather than vector dot products.This design reduces parameters and is described as increasing model robustness.
- 3.2.2 Deep&Cross Network: Under stated restrictions, the cross&compress unit is conceptually equivalent to a DCN layer, but its approximation degree is O(l) instead of the original unit’s O(2^l).The restricted version sets wVV_l = 1 and constrains the entity representation in the second term.
- 3.2.3 Cross-stitch Networks: MKR can reproduce cross-stitch-style task sharing while using vector dot products as finer-grained transfer weights, which can also act as feature-dependent attention.Setting transfer weights to zero yields task-specific behavior; larger values produce more shared representations.
4 EXPERIMENTS
The experiments evaluate MKR across four recommendation scenarios using real-world datasets, knowledge graphs, and multiple baselines. Evaluation includes CTR prediction and top-K recommendation.
- Datasets: MKR is evaluated on movie, book, music, and news recommendation using four real-world datasets.The datasets are MovieLens-1M, Book-Crossing, Last.FM, and Bing-News.
- Data processing: Explicit feedback datasets are converted into implicit feedback with positive interactions and sampled negative items.MovieLens-1M uses a positive-rating threshold of 4, while Book-Crossing and Last.FM use no threshold because of sparsity.
- Knowledge graphs: The experiments construct each dataset’s knowledge graph from Microsoft Satori triples filtered by confidence and item matching.MovieLens-1M and Book-Crossing additionally use relation-specific filtering to reduce knowledge-graph size.
- Baselines: MKR is compared with PER, CKE, DKN, and other recommendation baselines under reported or default hyper-parameter settings.PER uses manually designed meta-path features, while CKE combines collaborative filtering with structural knowledge.
4.3 Experiments setup
The setup combines standard recommendation evaluation with an empirical test of structural correspondence between recommender-system items and knowledge-graph entities. The experiments examine whether shared proximity patterns support cross-task transfer.
- Experimental protocol: Experiments use a 6:2:2 training, validation, and test split, optimize hyperparameters on validation AUC, and average results over three runs.MKR uses K = 1, an inner-product fRS, and λ2 = 10^-6 across the reported datasets.
- Empirical study: The empirical study samples one million MovieLens-1M item pairs and groups them by the number of common raters.For each group, it measures the average number of common knowledge-graph neighbors.
- Empirical study: Items with more common raters tend to share more common knowledge-graph neighbors, and the reverse-direction analysis shows the same positive correlation.The authors interpret these findings as similar proximity structure in the recommender system and knowledge graph.
4.5 Results
MKR generally outperforms the evaluated recommendation methods across four scenarios and retains stronger performance as interactions become sparse. Additional analyses support the value of cross&compress units and bidirectional task sharing.
- Comparison with baselines: MKR achieves the best overall performance among the evaluated methods across movie, book, music, and news recommendation.Its average Accuracy gains are 11.6%, 11.5%, 12.7%, and 8.7% for movie, book, music, and news, respectively.
- Comparison with baselines: RippleNet outperforms MKR on MovieLens-1M but performs worse than MKR on Book-Crossing, Last.FM, and Bing-News.The authors associate this pattern with RippleNet’s greater sensitivity to interaction density.
- Comparison with MKR variants: MKR outperforms MKR-1L, MKR-DCN, and MKR-stitch, supporting high-order feature interactions and fine-grained knowledge transfer.MKR-1L corresponds to factorization machines, MKR-DCN to deep&cross networks, and MKR-stitch to cross-stitch networks.
- Sparse scenarios: When MovieLens-1M training data is reduced to 10%, MKR’s AUC decreases by only 5.3%.The comparison varies the training-set ratio while keeping validation and test sets fixed.
- Cross-task effects: Jointly training recommendation and KGE reduces KGE tail-vector prediction error by 1.9%–6.4%.The result supports shared features that benefit both tasks in MKR.
4.6 Parameter Sensitivity
MKR’s performance depends on balanced KGE training frequency and embedding dimension, while increasing KG coverage improves sparse-news recommendation results.
- KG size: 13.6% and 11.8%: AUC and Accuracy improve as the KG ratio increases from 0.1 to 1.0 in three Bing-News scenarios.The authors attribute this effect to the extreme sparsity of Bing-News.
- RS training frequency: MKR performs best when the KGE module is trained every t = 5 iterations.Higher frequency can mislead the objective, whereas lower frequency may underuse transferred KG knowledge.
- Embedding dimension: Performance initially improves with embedding dimension because larger embeddings encode more useful information.This improvement does not continue indefinitely.
- Embedding dimension: Performance drops at excessively large embedding dimensions because additional dimensions may introduce noise into subsequent prediction.
5 RELATED WORK
Related work uses knowledge graphs, multi-task learning, and deep models to improve recommendation, but existing KG approaches can depend on manually designed relational features. MKR instead frames recommendation and KGE as connected asymmetric tasks.
- Knowledge graph methods: Knowledge graph embedding maps entities and relations into low-dimensional vectors while preserving graph structural information.KGE methods include translational distance and semantic matching models.
- Multi-task learning: Multi-task learning jointly uses information from related tasks to improve their generalization performance compared with separate learning.The passage identifies feature learning, low-rank, task clustering, task relation learning, and decomposition approaches.
- MKR’s positioning: MKR is an asymmetric multi-task framework that uses the connection between recommendation and KG tasks, which are trained at different frequencies.
- Deep recommendation: Deep recommender systems use neural networks either to process raw user or item features or to model recommendation interactions.The related-work discussion includes models that extract dense features from textual input for collaborative filtering.
6 CONCLUSIONS AND FUTURE WORK
MKR combines recommendation and KGE modules through cross&compress units that transfer latent information and model item–entity interactions. Experiments across four recommendation scenarios report superiority over strong baselines, while future work targets broader architectural and KGE choices.
- Conclusion: MKR combines recommendation and KGE modules, using cross&compress units to transfer knowledge between item and entity representations.Both modules use multiple nonlinear layers to model user-item and head-relation interactions.
- Conclusion: The cross&compress units automatically learn high-order interactions between item and entity features because recommendation items and KG entities are connected.
- Conclusion: Experiments in movie, book, music, and news recommendation demonstrate MKR’s significant superiority over strong baselines and the efficacy of KG usage.
- Future work: Future work will investigate other neural networks, such as CNNs, and incorporate additional KGE methods by redesigning the cross&compress unit.
A Proof of Theorem 1
The theorem proof establishes the cross&compress unit’s polynomial expressive capacity by induction. At each layer, multiplying maximal-degree terms doubles the relevant degree growth pattern.
- Base case: The proof begins with the base case l = 1, where maximal-degree cross terms in v1 and e1 have exponent sums equal to 1 = 2^(1−1).
- Induction hypothesis: The induction hypothesis assumes maximal-degree terms x and y in the relevant norms have exponent sums 2^(l−1) for both v and e.
- Induction step: For the induction step, the maximal-degree term in the next layer is the cross term xy.
- Induction step: The degree of xy becomes 2^(l+1−1) for both v and e, completing the stated induction pattern.
B Proof of Proposition 1
This section indicates that the proof of Proposition 1 refers to Theorem 1 in Appendix A, while proofs for Propositions 2 and 3 are omitted as straightforward.
- The proof of Proposition 1 is connected to the proof of Theorem 1 in Appendix A.
- The proofs of Propositions 2 and 3 are omitted because they are considered straightforward.