Source-linked AI summary
Outer Product-based Neural Collaborative Filtering
Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, Tat-Seng Chua
TL;DR
The paper addresses limitations in modeling correlations among embedding dimensions in collaborative filtering. It proposes ONCF, which forms an outer-product interaction map and uses CNNs to learn high-order correlations; experiments report that ConvNCF outperforms state-of-the-art methods in top-k recommendation. The authors also note constraints around the BPR objective and regularization or maxnorm requirements.
Problem
Matrix factorization and common MLP-based recommender designs provide limited or unreliable modeling of correlations among embedding dimensions.
Method
ONCF applies an outer product to user and item embeddings to create a K × K interaction map, then ConvNCF uses convolutional layers to learn high-order correlations.
Results
ConvNCF outperforms state-of-the-art methods in top-k recommendation on two real-world datasets.
Takeaways & Limitations
The interaction map supports CNN-based learning of high-order embedding-dimension correlations for collaborative filtering.
Takeaways & Limitations
With the BPR objective, ONCF requires L2 regularization or a maxnorm constraint on its weight vector to avoid a trivial scaling solution; other pairwise objectives remain future work.
Abstract
from arXiv · showhide
In this work, we contribute a new multi-layer neural network architecture named ONCF to perform collaborative filtering. The idea is to use an outer product to explicitly model the pairwise correlations between the dimensions of the embedding space. In contrast to existing neural recommender models that combine user embedding and item embedding via a simple concatenation or element-wise product, our proposal of using outer product above the embedding layer results in a two-dimensional interaction map that is more expressive and semantically plausible. Above the interaction map obtained by outer product, we propose to employ a convolutional neural network to learn high-order correlations among embedding dimensions. Extensive experiments on two public implicit feedback data demonstrate the effectiveness of our proposed ONCF framework, in particular, the positive effect of using outer product to model the correlations between embedding dimensions in the low level of multi-layer neural recommender model. The experiment codes are available at: https://github.com/duxy-me/ConvNCF
1 Introduction
Collaborative filtering relies heavily on matrix factorization, but fixed inner products and common MLP designs may fail to model correlations among embedding dimensions. ONCF addresses this by using an outer-product interaction map and CNN-based high-order modeling, with experiments supporting its effectiveness.
- Motivation: Matrix factorization models user–item interactions with a fixed inner product that assumes embedding dimensions are independent and equally contributive.The paper identifies this assumption as impractical for complex real-world feedback data.
- Motivation: MLP-based models over concatenated or element-wise product embeddings provide no practical guarantee that dimension correlations are effectively captured.Although MLPs are theoretically universal approximators, the paper highlights optimization concerns in learning these correlations.
- Proposed approach: ONCF uses an outer product above the embedding layer to explicitly capture pairwise correlations between embedding dimensions.The resulting K × K interaction map includes the inner-product signal on its diagonal and additional pairwise correlations.
- Proposed approach: ConvNCF applies CNN layers to the interaction map to learn high-order correlations hierarchically from local to global patterns.The 2D matrix format makes CNN-based interaction modeling feasible.
- Evaluation: Extensive experiments on two public implicit-feedback datasets demonstrate the effectiveness and rationality of ONCF methods.The introduction presents these experiments as evidence for the proposed framework.
- Contribution: The paper identifies CNN-based interaction-function learning as a novel direction for neural collaborative filtering.It describes this as the first work using CNNs to learn the interaction function between user and item embeddings.
2 Proposed Methods
ONCF models user–item interactions by forming a two-dimensional outer-product interaction map, then applies configurable hidden layers to predict personalized ranking scores. ConvNCF uses CNN layers above this map to learn high-order correlations with fewer parameters than an MLP.
- ONCF framework: ONCF forms a K × K interaction map from the outer product of user and item embeddings.Each map element captures the product of one user-embedding dimension and one item-embedding dimension.
- ONCF framework: The interaction map includes MF’s diagonal interactions while also modeling correlations between different embedding dimensions.This provides more interaction signal than MF and retains correlations that concatenation does not model.
- ONCF framework: Hidden layers transform the interaction map into an output vector, which the prediction layer projects to the user–item score.The hidden-layer function can be any matrix-to-vector function, while w re-weights the resulting interaction signal.
- Learning ONCF for Personalized Ranking: ONCF is trained for personalized ranking with Bayesian Personalized Ranking, which favors observed interactions over unobserved ones.The objective optimizes relative interaction orders rather than absolute prediction scores.
- Convolutional NCF: Flattening the interaction map for an MLP creates a parameter-heavy design, whereas ConvNCF uses locally connected CNN layers to learn high-order correlations with fewer parameters.The paper motivates CNNs as enabling deeper models while reducing the parameter burden associated with MLPs.
- Convolutional NCF: Convolutional layers produce feature maps from the interaction map and ultimately reduce the representation to a 1 × 1 × 32 tensor projected to the final score.The first layer operates on the interaction map, and subsequent layers operate on 3D feature-map tensors.
3 Experiments
Experiments on Yelp and Gowalla evaluate ConvNCF against recommendation baselines and test the effects of outer products, CNN layers, and feature-map counts. ConvNCF generally performs best, while outer products and CNNs provide favorable performance and stability.
- Experimental Settings: Experiments use Yelp and Gowalla, with one held-out latest interaction paired with 999 unrated items per user for testing.The evaluation reports Hit Ratio and Normalized Discounted Cumulative Gain.
- Performance Comparison: ConvNCF achieves the best performance in general and obtains high improvements over state-of-the-art methods on the two datasets.The evaluation uses top-k recommendation performance for k ∈ {5, 10, 20}; RI denotes average improvement over a baseline, and asterisks mark statistically significant improvements at p < 0.05.
- Efficacy of Outer Product and CNN: JRL consistently outperforms MLP by a large margin on both datasets, supporting the usefulness of explicitly modeling embedding-dimension correlations for subsequent hidden layers.The passage also notes practical difficulty training MLP effectively despite its theoretical representation capacity.
- Efficacy of Outer Product and CNN: ConvNCF outperforms GMF, JRL, and MLP by a large margin across epochs on both datasets when comparing NDCG@10.GMF and JRL use element-wise products, MLP uses concatenation, and ConvNCF uses an outer product.
- Efficacy of Outer Product and CNN: ConvNCF substantially outperforms ONCF-mlp while using fewer parameters and shows more stable performance across epochs.ONCF-mlp uses a 3-layer MLP over the flattened interaction map, whereas ConvNCF uses a 6-layer CNN.
- Hyperparameter Study: Different feature-map counts on Yelp produce steadily increasing curves that eventually reach similar performance, with only slight convergence differences.The authors interpret this as evidence of strong expressiveness and generalization without overfitting from substantially increasing parameters.
4 Conclusion
The conclusion presents ONCF as an outer-product framework that creates a semantic-rich interaction map, with ConvNCF using convolutional layers to learn higher-order correlations. On two real-world datasets, ConvNCF outperforms state-of-the-art methods in top-k recommendation.
- 4 Conclusion: ONCF uses an outer product above the embedding layer to create a semantic-rich interaction map encoding pairwise correlations between embedding dimensions.The interaction map facilitates learning higher-order correlations in subsequent deep layers.
- 4 Conclusion: ConvNCF instantiates ONCF with multiple convolution layers above the interaction map to learn the interaction function.The conclusion identifies future exploration of advanced CNN architectures and content-based recommendation as extensions.
- 4 Conclusion: ConvNCF outperforms state-of-the-art methods in top-k recommendation on two real-world datasets.This is presented as evidence supporting the proposed framework and model.