Source-linked AI summary
DeepCF: A Unified Framework of Representation Learning and Matching Function Learning in Recommender System
Zhi-Hong Deng, Ling Huang, Chang-Dong Wang, Jian-Huang Lai, Philip S. Yu
TL;DR
Implicit-feedback recommendation must match users and items despite a semantic gap and incomplete negative evidence. DeepCF combines representation learning with matching-function learning, and its CFNet implementation targets both nonlinear matching and efficient low-rank relation learning. The framework is evaluated on four real-world datasets, while training samples negatives from unobserved interactions and uses binary cross-entropy.
Problem
Representation-based collaborative filtering has limited dot-product expressiveness, while matching-function learning is weaker at capturing low-rank relations; implicit feedback also lacks reliable negative instances.
Method
DeepCF combines representation learning and matching-function learning; its CFNet implementation uses a vanilla MLP for implicit-feedback collaborative filtering.
Results
Experiments on four real-world datasets demonstrate the effectiveness and rationality of the DeepCF framework.
Takeaways & Limitations
DeepCF is designed to provide both flexibility in learning complex matching functions and efficiency in learning low-rank user–item relations.
Takeaways & Limitations
The paper uses uniformly sampled unobserved interactions as negative instances for implicit-feedback learning.
Abstract
from arXiv · showhide
In general, recommendation can be viewed as a matching problem, i.e., match proper items for proper users. However, due to the huge semantic gap between users and items, it's almost impossible to directly match users and items in their initial representation spaces. To solve this problem, many methods have been studied, which can be generally categorized into two types, i.e., representation learning-based CF methods and matching function learning-based CF methods. Representation learning-based CF methods try to map users and items into a common representation space. In this case, the higher similarity between a user and an item in that space implies they match better. Matching function learning-based CF methods try to directly learn the complex matching function that maps user-item pairs to matching scores. Although both methods are well developed, they suffer from two fundamental flaws, i.e., the limited expressiveness of dot product and the weakness in capturing low-rank relations respectively. To this end, we propose a general framework named DeepCF, short for Deep Collaborative Filtering, to combine the strengths of the two types of methods and overcome such flaws. Extensive experiments on four publicly available datasets demonstrate the effectiveness of the proposed DeepCF framework.
Introduction
DeepCF combines representation learning and matching-function learning to address their complementary weaknesses in collaborative filtering. Its CFNet implementation uses deep models to learn both complex matching functions and low-rank user–item relations, with experiments on four real-world datasets.
- Motivation: Collaborative filtering methods either map users and items into a common representation space or directly learn user–item matching functions.Representation-based methods use similarity in the learned space, while matching-function methods map pairs to matching scores.
- Motivation: Representation learning uses dot products that have limited expressiveness, whereas matching-function learning is weaker at capturing low-rank relations.These complementary weaknesses motivate combining the two approaches.
- DeepCF Framework: DeepCF incorporates representation learning and matching-function learning by obtaining different representations for each input user–item pair.The framework is designed to combine the distinct advantages of the two method types.
- DeepCF Framework: CFNet implements DeepCF with a vanilla MLP, providing flexibility for complex matching functions while efficiently learning low-rank user–item relations.The framework abandons the traditional Deep+Shallow pattern and uses deep models for implicit-feedback collaborative filtering.
- Evaluation: Experiments on four real-world datasets demonstrate the effectiveness and rationality of the DeepCF framework.
Related Work
Prior collaborative-filtering research includes representation-learning methods that compare users and items in a common space and matching-function methods that learn nonlinear pairwise scores. DeepCF is presented as a framework that combines their strengths while addressing dot-product expressiveness and low-rank-relation weaknesses.
- Representation Learning: Matrix factorization maps users and items into a common representation space where they can be compared directly.Deep representation-learning approaches such as AutoRec, CDAE, and DMF extend this line of work.
- Matching Function Learning: Matching-function methods replace or augment dot products with neural architectures that learn complex relations from user and item inputs.Examples include NeuMF, NNCF, ConvNCF, Wide&Deep, DeepFM, and NFM.
- Matching Function Learning: For pure collaborative filtering without auxiliary data, the paper mainly discusses NeuMF and compares it with DeepCF.
- DeepCF Motivation: Both method types are effective, but representation learning has limited dot-product expressiveness and matching-function learning is weak at capturing low-rank relations.DeepCF is proposed to combine their strengths and overcome these weaknesses.
Preliminaries
The paper formulates implicit-feedback recommendation as probabilistic interaction and matching-score prediction under the One-Class Collaborative Filtering setting. It models match probabilities and learns model parameters with binary cross-entropy using observed interactions and sampled unobserved interactions.
- Problem Statement: Implicit feedback is represented by a binary user–item interaction matrix, where observed interactions are 1 and unobserved interactions are 0.An unobserved interaction does not necessarily indicate that a user dislikes an item.
- Problem Statement: The absence of real negative feedback makes implicit-feedback recommendation a One-Class Collaborative Filtering problem.The paper samples negative instances uniformly from unobserved interactions.
- Problem Statement: The model treats each interaction as Bernoulli and predicts p_ui, the probability that user u is matched by item i.This transforms binary interaction prediction into matching-score prediction for ranking.
- Learning the Model: A model-based predictor maps users, items, and parameters Θ to a predicted interaction probability through ˆy_ui = f(u, i|Θ).The function and parameter-estimation procedure are the two central modeling questions.
- Learning the Model: The paper uses point-wise likelihood estimation with sampled negatives and minimizes binary cross-entropy for implicit feedback.Squared loss is rejected because binary interactions do not satisfy its normal-error assumption.
The Proposed Framework
The proposed DeepCF framework combines representation learning and matching function learning to address their complementary limitations. Its CFNet model learns user-item representations through MLP-based components, fuses their predictive vectors, and is trained for implicit-feedback recommendation.
- Framework overview: DeepCF combines representation learning and matching function learning within one framework for collaborative filtering.The framework is presented as a unified approach to overcome the shortages of the two method types.
- Framework overview: Representation learning maps users and items into a common space, while matching functions compute scores from their learned representations.Representation-based methods commonly use simple operations such as dot product or cosine similarity, whereas matching-function methods learn more complex mappings.
- Representation learning: CFNet-rl uses MLPs to learn user and item latent representations, then applies element-wise product and a parametric neural layer for matching.The neural matching layer preserves low-rank modeling while allowing different latent dimensions to have different importance and enabling nonlinear mapping.
- Matching function learning: CFNet-ml uses linear embedding layers, concatenates user and item representations, and applies an MLP to learn the matching function.Concatenation preserves information from the preceding layer while allowing the MLP to model the user-item mapping.
- Fusion and learning: DeepCF fuses the predictive vectors from CFNet-rl and CFNet-ml into a joint representation to obtain the final CFNet model.The two predictive vectors are learned from different perspectives, and their concatenation is intended to produce a stronger, more robust joint representation.
- Fusion and learning: The model is trained with binary cross-entropy and mini-batch Adam, using uniformly sampled negative instances from unobserved interactions.The reported training setup uses batch size 256 and learning rate 0.001; pre-training can initialize the two-component fusion model.
Experiments
The experiments evaluate DeepCF and CFNet on four public datasets using ranking metrics, compare against established baselines, and examine pre-training and hyperparameter sensitivity.
- Datasets and Evaluation: Experiments evaluate the models on MovieLens 1M, LastFM, Amazon music, and Amazon toys datasets.The evaluation uses leave-one-out testing with 100 sampled unobserved interactions per user, reporting HR and NDCG.
- Baselines: The comparison includes popularity-based, matrix-factorization, representation-learning, and matching-function-learning methods, including NeuMF.NeuMF is the most closely related baseline but uses a deep+shallow architecture and user/item IDs as inputs.
- Pre-training: CFNet with pre-training outperforms CFNet without pre-training in all cases.The authors attribute this to pre-training enabling CFNet-rl and CFNet-ml to learn features from different perspectives.
- Negative Sampling Ratio: Sampling one or two negative instances is insufficient, while the best HR@10 occurs at ratio 3 and the best NDCG@10 at ratio 6.Overall, the optimal negative sampling ratio is around 3 to 7; larger ratios increase training time and degrade performance.
- Predictive Factors: CFNet achieves its best performance with 64 predictive factors on most datasets, except AMusic, where 16 factors perform best.The authors associate more predictive factors with greater model capacity and representation ability.
Conclusion and Future Work
The paper concludes that DeepCF can fuse representation learning and matching function learning, with CFNet as an effective MLP implementation. The framework is extensible to other component types and future settings.
- Conclusion: DeepCF fuses representation learning-based and matching function learning-based collaborative filtering methods.The framework is presented as simple but effective, with CFNet serving as its MLP implementation.
- Conclusion: The framework is intended to provide flexibility for complex matching functions and efficiency in learning low-rank user-item relations.The conclusion describes these as complementary strengths of incorporating both method types.
- Future Work: DeepCF can integrate different representation-learning and matching-function-learning methods beyond the MLP components used in this paper.This extensibility is identified as a supported property of the framework.
- Future Work: Future work includes using auxiliary data, exploring aggregation methods beyond element-wise product and concatenation, and applying pairwise loss.The authors also state that DeepCF is suitable for other data-mining tasks involving relations between two kinds of objects.