Source-linked AI summary
Hybrid Recommender System based on Autoencoders
Florian Strub, Romaric Gaudel, Jérémie Mary
TL;DR
The paper addresses matrix completion for sparse user–item ratings, where neural networks must handle extensive missing data and cold-start users or items. It develops an autoencoder collaborative-filtering framework with a missing-data training process and integrated side information. The experiments report strong collaborative-filtering performance, while side information has greater impact for cold users and items than in averages across all users and items.
Problem
Matrix completion must infer unknown ratings from incomplete user–item matrices, while autoencoders face the additional challenge of missing values and cold-start users or items.
Method
The paper uses autoencoders with a training process for incomplete input and target vectors, and integrates ratings with side information in one network.
Results
The framework beats state-of-the-art collaborative-filtering results, remains robust to data density, and side information matters more for users or items with few ratings.
Takeaways & Limitations
Integrated side information is especially useful for cold-start users and items, although its average test-error improvement across all users and items is slight.
Takeaways & Limitations
The single-autoencoder framework is presented as mathematically equivalent to a mixture of autoencoders, with its advantage framed as greater flexibility and restored neural-network optimization benefits.
Abstract
from arXiv · showhide
A standard model for Recommender Systems is the Matrix Completion setting: given partially known matrix of ratings given by users (rows) to items (columns), infer the unknown ratings. In the last decades, few attempts where done to handle that objective with Neural Networks, but recently an architecture based on Autoencoders proved to be a promising approach. In current paper, we enhanced that architecture (i) by using a loss function adapted to input data with missing values, and (ii) by incorporating side information. The experiments demonstrate that while side information only slightly improve the test error averaged on all users/items, it has more impact on cold users/items.
1 Introduction
Recommender systems predict users’ ratings for unseen items, commonly by factorizing incomplete rating matrices. Neural-network approaches face the challenge of missing values, while this paper proposes an autoencoder framework with missing-data training and integrated side information.
- Motivation: Collaborative filtering combines a user’s feedback with other users’ ratings to predict ratings for unseen items.Matrix factorization is the most successful collaborative-filtering approach described in the passage.
- Motivation: Matrix factorization learns user and item representations while completing the incomplete rating matrix.
- Challenges: Neural networks have seen few applications to collaborative filtering because the input contains missing values.Deep-learning successes have mainly involved fully observable data.
- Contribution: The paper introduces a missing-data training loss and integrates side information into a single autoencoder network.The framework is presented as scalable, robust, and intended to address cold-start settings.
2 State of the art
Matrix factorization completes sparse rating matrices, while autoencoders provide a nonlinear formulation that reconstructs missing ratings from incomplete user or item vectors. The main challenges are extreme sparsity, efficient training, and integrating heterogeneous side information for cold-start cases.
- Matrix Factorization: Matrix factorization approximates the incomplete rating matrix with a low-rank matrix whose unknown entries provide predictions.The factorization is written as bR = UVT, with user and item factors of rank k.
- Autoencoders: Autoencoders are unsupervised networks trained to reconstruct their input using a backpropagated squared-error loss.A one-hidden-layer network uses input x, weight matrices, biases, and a nonlinear transfer function.
- Autoencoder CF: In collaborative filtering, autoencoders receive incomplete user rows or item columns and output vectors predicting missing ratings.This produces a nonlinear low-rank approximation of the rating matrix.
- Challenges: Collaborative-filtering tasks are difficult for autoencoders because missing values affect both inputs and targets, often exceeding 95% sparsity.Prior industrial-data analysis cited in the passage used only 5% missing values.
- Challenges: Autorec assigns one autoencoder per sample, but shared weights hinder efficient computation and prevent several gradient-optimization techniques.The paper instead introduces a single-autoencoder method mathematically equivalent to a mixture of autoencoders.
- Side Information: Prior work had not trained one neural network end to end on both ratings and heterogeneous side information.The paper integrates side information into its collaborative-filtering system to address cold start.
3 End-to-End Collaborative Filtering with Autoencoders
The paper develops an end-to-end autoencoder approach for collaborative filtering that handles incomplete ratings with a specialized training process and incorporates side information to address cold-start cases.
- Autoencoder-based collaborative filtering: The model predicts full user or item rating vectors from incomplete vectors, using separate U-CFN and I-CFN autoencoders.U-CFN predicts missing ratings for users, while I-CFN predicts missing ratings for items.
- Handling incomplete data: Missing inputs are set to zero, and errors for unknown ratings are excluded so no error is back-propagated for missing values.Actual zero ratings remain eligible for error propagation.
- Handling incomplete data: Corrupting training inputs and weighting prediction errors by α more heavily than reconstruction errors by β focuses learning on recovering missing ratings.The loss also includes regularization over the full matrix of network weights.
- Integrating side information: Side information is appended to incomplete inputs and injected into every autoencoder layer, including the intermediate and final hidden layers.This design enables end-to-end training while partly recovering the error function used in classic hybrid systems.
- Integrating side information: Appending side information to inputs supplies information when no ratings are available, while intermediate-layer injection enhances internal representations.The final-layer injection partially retrieves the error function of classic hybrid systems.
4 Experiments
Experiments evaluate CFN on MovieLens and Douban against matrix-completion and autoencoder baselines, examining accuracy, side information, loss design, density robustness, and tractability. CFN performs competitively, with side information especially useful for items with few ratings and robustness across training densities.
- Experimental setting: Experiments evaluate CFN on MovieLens and Douban datasets against five matrix-completion algorithms.MovieLens includes item and user side information, while Douban provides user side information.
- Side information: Side information has limited average RMSE impact but improves predictions most for items with fewer ratings.Average estimates are biased toward well-rated users and items; conditional RMSE analysis shows larger gains for cold items.
- Loss design: The denoising autoencoder loss improves RMSE on large datasets when balanced, but the reconstruction criterion remains necessary.Setting β = 0 cannot produce an efficient representation, while the denoising loss provides additional benefit.
- Model design: Removing non-linearity yields a MovieLens-10M RMSE of 0.8151 ± 1.4e-3, substantially worse than the classic non-linear I-CFN.The comparison keeps α, β, masking ratio, and hidden-neuron count constant while tuning learning rates and weight decay.
- Robustness and tractability: CFN remains robust to changes in training density and can refine predictions for new ratings without retraining for new users or items.The density experiment keeps hyperparameters fixed after optimization on MovieLens-10M (90%/10%); GPU computation also enables training within minutes.
5 Conclusion
The paper consolidates autoencoder best practices for collaborative filtering, linking autoencoders to matrix factorization while addressing incomplete ratings and cold-start side information.
- Autoencoders are connected to matrix factorization for matrix completion in collaborative filtering.
- The approach combines modern training techniques and scalable code to defeat state-of-the-art methods.
- Side information is integrated directly into autoencoders rather than through two separate systems.