PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Collaborative Filtering

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, Tat-Seng Chua

arXiv:1708.05031v2cs.IR

TL;DR

Collaborative filtering commonly models user–item interactions with a fixed inner product, limiting its ability to capture complex relationships. NCF replaces that interaction function with neural architectures, and experiments on two real-world datasets show significant improvements over state-of-the-art methods.

  • Problem

    Fixed inner-product interactions can hinder matrix factorization’s ability to model complex user–item relationships, while deep neural networks remain relatively underexplored for recommendation.

  • Method

    NCF models user and item latent features with neural architectures, including an MLP that learns flexible nonlinear interaction functions; matrix factorization is a specialization.

  • Results

    NeuMF achieves the best performance on both datasets, improving over eALS and BPR by average relative margins of 4.5% and 4.9%, respectively.

  • Takeaways & Limitations

    NCF provides a generic framework for developing deep-learning collaborative-filtering models that complements shallow recommendation methods.

  • Takeaways & Limitations

    Implicit-feedback data provide noisy preference signals because unobserved interactions may represent missing data rather than negative feedback.

Abstract

from arXiv · show

In recent years, deep neural networks have yielded immense success on speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks on recommender systems has received relatively less scrutiny. In this work, we strive to develop techniques based on neural networks to tackle the key problem in recommendation -- collaborative filtering -- on the basis of implicit feedback. Although some recent work has employed deep learning for recommendation, they primarily used it to model auxiliary information, such as textual descriptions of items and acoustic features of musics. When it comes to model the key factor in collaborative filtering -- the interaction between user and item features, they still resorted to matrix factorization and applied an inner product on the latent features of users and items. By replacing the inner product with a neural architecture that can learn an arbitrary function from data, we present a general framework named NCF, short for Neural network-based Collaborative Filtering. NCF is generic and can express and generalize matrix factorization under its framework. To supercharge NCF modelling with non-linearities, we propose to leverage a multi-layer perceptron to learn the user-item interaction function. Extensive experiments on two real-world datasets show significant improvements of our proposed NCF framework over the state-of-the-art methods. Empirical evidence shows that using deeper layers of neural networks offers better recommendation performance.

1. INTRODUCTION

The paper introduces Neural network-based Collaborative Filtering (NCF) to learn user–item interaction functions for recommendation from implicit feedback. It replaces MF’s inner product with neural architectures, including multilayer perceptrons, and evaluates the resulting framework on two real-world datasets.

  • Motivation: Collaborative filtering models users’ item preferences from past interactions such as ratings and clicks, helping recommender systems alleviate information overload.The approach is widely used in e-commerce, online news, and social media services.
  • Limitations of MF: Matrix factorization is the de facto latent-factor approach, but its simple inner product can hinder collaborative-filtering performance.Prior work enhanced MF with neighbor models, topic models, and factorization machines.
  • Research Gap: Prior neural recommendation methods mainly modeled auxiliary information while still using MF and inner products for the core user–item collaborative effect.Examples include textual item descriptions, music audio features, and image visual content.
  • Problem Setting: The paper formalizes neural collaborative filtering for implicit feedback, which is easier to collect automatically than ratings but more challenging to utilize.Implicit signals include watching videos, purchasing products, and clicking items.
  • Contributions: NCF provides a neural architecture for modeling latent user and item features and their interaction in collaborative filtering.The framework is designed specifically to learn the interaction function from data.
  • Contributions: MF is a specialization of NCF, while a multi-layer perceptron introduces substantial non-linearities into NCF modeling.The paper also tests NCF approaches on two real-world datasets to demonstrate their effectiveness.

2. PRELIMINARIES

This section formalizes implicit-feedback recommendation and reviews matrix factorization (MF), emphasizing noisy signals, ranking-based prediction, and MF’s limited expressiveness from its fixed inner product.

  • Implicit Feedback: Implicit feedback provides noisy preference signals because observed interactions indicate interest, whereas unobserved entries may simply reflect missing awareness.A value of 1 does not necessarily mean the user likes an item, and 0 does not necessarily mean dislike.
  • Problem Formulation: Recommendation with implicit feedback estimates scores for unobserved user–item entries and ranks items using an interaction function ˆyui = f(u, i|Θ).The model parameters Θ are learned through objective functions such as pointwise or pairwise losses.
  • Neural Collaborative Filtering: NCF addresses MF’s interaction-function limitation by parameterizing f with neural networks, supporting both pointwise and pairwise learning.The framework learns the interaction function directly rather than relying on a simple fixed inner product.
  • Matrix Factorization: MF represents users and items with latent vectors and estimates their interaction using an inner product.MF combines latent dimensions linearly with equal weights, making it a linear model of latent factors.
  • Matrix Factorization: MF’s fixed inner product can fail to recover complex user–item relationships in a low-dimensional latent space, producing incorrect similarity rankings.Increasing the number of latent factors may address the limitation but can harm generalization through overfitting, especially with sparse data.

3. NEURAL COLLABORATIVE FILTERING

NCF models user–item interactions with a neural architecture for implicit-feedback recommendation, providing a probabilistic binary-classification treatment that includes matrix factorization as a special case. Its instantiations use MLPs and a flexible NeuMF fusion of GMF and MLP pathways to capture nonlinear interactions.

  • NCF framework: NCF uses multilayer neural architectures to map user and item representations to prediction scores, with embeddings projecting sparse inputs into dense latent vectors.Inputs can use one-hot user and item identities in pure collaborative filtering, while the generic representation can also incorporate context or content features.
  • Probabilistic learning: For implicit feedback, NCF treats observed and unobserved interactions as binary labels and optimizes binary cross-entropy, or log loss, using stochastic gradient descent.A Logistic or Probit output activation constrains prediction scores to [0, 1] so they can represent relevance likelihoods.
  • Generalized matrix factorization: Matrix factorization is recovered when NCF uses element-wise latent-vector products, an identity output activation, and uniform output weights.Learning the output weights or using a nonlinear output generalizes MF, yielding GMF with sigmoid activation and learned weights under log loss.
  • MLP architecture: The MLP pathway adds hidden layers to concatenated user and item latent features, enabling interactions that simple vector concatenation cannot model.MLP layers may use sigmoid, tanh, or ReLU activations, and the implemented tower structure halves layer width at each successive higher layer.
  • NeuMF: NeuMF combines GMF’s linearity with MLP’s nonlinearity by learning separate embeddings and concatenating the pathways’ last hidden layers.Separate embeddings address the limitation that shared GMF and MLP embeddings must have the same size and may prevent an optimal ensemble.

4. EXPERIMENTS

Experiments on MovieLens and Pinterest evaluated NCF methods against established implicit-feedback baselines using leave-one-out recommendation. NeuMF performed best, while pointwise log loss, moderate negative sampling, and deeper MLPs improved or enabled effective interaction modeling.

  • Experimental settings: Experiments used MovieLens and Pinterest, with MovieLens converted from explicit ratings into implicit binary feedback and Pinterest filtered to users with at least 20 interactions.The filtered Pinterest subset contains 55,187 users and 1,500,809 interactions.
  • RQ1: Comparative performance: NeuMF achieved the best performance on both datasets, outperforming eALS and BPR by average relative improvements of 4.5% and 4.9%, respectively.The authors attribute this to fusing linear MF and nonlinear MLP models; NeuMF also remained stronger on Pinterest with factor size 8 than eALS and BPR with factor size 64.
  • RQ1: Comparative performance: NeuMF showed consistent, statistically significant improvements over other methods across Top-K positions from 1 to 10, with all improvements significant at p < 0.01.ItemKNN underperformed model-based methods, while ItemPop performed worst, emphasizing the value of personalized user modeling.
  • RQ2: Optimization and negative sampling: NCF treated implicit-feedback recommendation as binary classification and optimized a probabilistic model with pointwise log loss, whose negative-sampling ratio could be flexibly controlled.The optimal sampling ratio was around 3 to 6 on both datasets, while ratios above 7 on Pinterest reduced performance.
  • RQ3: Network depth: Deeper MLP hidden layers improved interaction modeling, whereas MLP-0 performed weakly and no better than the non-personalized ItemPop baseline.The results support transforming concatenated user and item latent vectors with hidden layers rather than directly projecting them to predictions.

5. RELATED WORK

Related work has increasingly shifted from explicit ratings toward implicit-feedback item recommendation. Prior neural approaches mainly used deep networks for auxiliary information or feature-based recommendation, whereas this work applies them directly to user–item interactions in pure collaborative filtering.

  • Implicit-feedback collaborative filtering is typically formulated as item recommendation, aiming to recommend users a short list of items.
  • Earlier neural recommendation systems modeled explicit ratings with Restricted Boltzmann Machines and later used autoencoders to reconstruct users’ historical ratings.
  • For implicit feedback, recent deep-learning methods primarily model auxiliary information—including item text, music acoustics, cross-domain behavior, and knowledge bases—then integrate learned features with matrix factorization.
  • Neural Tensor Networks learn interactions between entities, while NeuMF is presented as more flexible and generic by allowing its matrix-factorization and multilayer-perceptron components to learn different sets.
  • Unlike Wide & Deep’s feature-focused App recommendation, this work targets deep neural networks for pure collaborative filtering and direct modeling of user–item interactions.

6. CONCLUSION AND FUTURE WORK

The paper presents NCF as a general framework for neural collaborative filtering, instantiated by GMF, MLP, and NeuMF. Future work will investigate pairwise learning, auxiliary information, group personalization, and multimedia recommendation.

  • Contributions: NCF provides a simple, generic framework for neural collaborative filtering, including GMF, MLP, and NeuMF as distinct user–item interaction models.The framework is intended to guide development beyond the models presented and complement mainstream shallow collaborative-filtering methods.
  • Future Work: Future work will study pairwise learners for NCF models and incorporate auxiliary information such as user reviews, knowledge bases, and temporal signals.These extensions target additional information beyond the core collaborative-filtering interaction.
  • Future Work: The authors also plan to develop recommender systems for groups of users to support decision-making in social groups.Existing personalization models have primarily focused on individuals.
Loading 1708.05031v2…