Source-linked AI summary
VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback
Ruining He, Julian McAuley
TL;DR
Existing personalized ranking methods often ignore item appearance and suffer from cold-start issues. VBPR incorporates image-derived visual factors into scalable matrix-factorization ranking, outperforming BPR-MF by over 12% for all items and more than 28% for cold-start items.
Problem
Personalized ranking methods largely omit visual item features, while sparse feedback creates cold-start issues that limit latent-factor estimation.
Method
VBPR integrates pre-trained deep-network image representations with matrix factorization and learns visual and latent preference dimensions using Bayesian Personalized Ranking.
Results
Over 12% average improvement over BPR-MF for all items and more than 28% for cold start demonstrate VBPR’s effectiveness on the reported ranking task.
Takeaways & Limitations
Visual features can improve personalized ranking and alleviate cold-start issues across multiple large real-world implicit-feedback datasets.
Takeaways & Limitations
The paper leaves temporal dynamics and explicit-feedback settings for future work.
Abstract
from arXiv · showhide
Modern recommender systems model people and items by discovering or `teasing apart' the underlying dimensions that encode the properties of items and users' preferences toward them. Critically, such dimensions are uncovered based on user feedback, often in implicit form (such as purchase histories, browsing logs, etc.); in addition, some recommender systems make use of side information, such as product attributes, temporal information, or review text. However one important feature that is typically ignored by existing personalized recommendation and ranking methods is the visual appearance of the items being considered. In this paper we propose a scalable factorization model to incorporate visual signals into predictors of people's opinions, which we apply to a selection of large, real-world datasets. We make use of visual features extracted from product images using (pre-trained) deep networks, on top of which we learn an additional layer that uncovers the visual dimensions that best explain the variation in people's feedback. This not only leads to significantly more accurate personalized ranking methods, but also helps to alleviate cold start issues, and qualitatively to analyze the visual dimensions that influence people's opinions.
Introduction
The introduction identifies visual appearance as a neglected source of information in personalized recommendation and motivates scalable models that learn visual preferences from implicit feedback. The proposed approach incorporates visual signals into personalized ranking to improve performance, alleviate item cold start, and explain recommendations through visual dimensions.
- Matrix Factorization methods uncover latent dimensions from feedback but suffer from cold start issues because real-world datasets are sparse.
- Visual personalized ranking: Existing recommender systems typically neglect the visual appearance of items, motivating its incorporation into preference predictors.
- The paper develops scalable models that learn visual dimensions relevant to product selection from visual features for personalized ranking on implicit-feedback datasets.
- The visual models aim to alleviate cold start issues, explain recommendations using visual signals, and produce personalized rankings.
- The approach incorporates visual signals into opinion predictors while scaling to large datasets and uses a Bayesian Personalized Ranking training procedure.
- Experiments show significant performance improvements on real-world datasets such as Amazon clothing, especially for item cold start problems.
Related Work
Prior recommendation work includes matrix-factorization, point-wise, pairwise, and content-based or hybrid models using non-visual information. The paper distinguishes its contribution by incorporating visual signals into preference models and uncovering visual dimensions, building on advances in fashion retrieval and transferable Deep CNN features.
- Matrix Factorization: Matrix Factorization methods uncover latent user and item dimensions and underpin many state-of-the-art recommendation approaches.For implicit-feedback ranking, traditional MF methods face ambiguity in interpreting non-observed feedback.
- Point-wise Methods: Point-wise methods treat non-observed feedback as negative to some degree, predicting affinity scores through regression before ranking items.Hu, Koren, and Volinsky (2008) model differing confidence levels, whereas Pan et al. (2008) sample non-observed feedback as negative instances.
- Pairwise Methods: Pairwise methods assume positive feedback should be more preferable than non-observed feedback and directly optimize ranking.Rendle et al. (2009) proposed the generalized Bayesian Personalized Ranking framework, including BPR-MF with matrix factorization as the underlying predictor.
- Content-based and Hybrid Models: Existing content-based and hybrid models use text, context, taxonomies, and user demographics, but the cited work had not incorporated visual signals into preference models.The paper’s distinction is also uncovering visual dimensions that influence users’ preferences.
- Visual Features: Prior visual methods addressed in-style image retrieval, fashionability prediction, accessory-style consistency, and related image-query tasks.Separately, high-level Deep CNN features had succeeded in object detection, photographic style annotation, and aesthetic quality categorization, with transfer learning enabling reuse across datasets.
VBPR: Visual Bayesian Personalized Ranking
VBPR learns personalized rankings from implicit feedback by jointly modeling visual and latent item dimensions. It embeds high-dimensional CNN image features into a compact visual space and trains the resulting predictor with pairwise Bayesian Personalized Ranking.
- Task: VBPR ranks unseen items for each user using positive implicit feedback and one image per item.The task is defined over users’ observed positive item sets and unobserved candidate items.
- Model: The model partitions rating dimensions into visual and latent non-visual factors to address cold items with few observations.Visual factors provide an auxiliary signal where latent dimensions are difficult to estimate from sparse feedback.
- Visual representation: A shared embedding matrix E maps each item’s 4096-dimensional CNN feature vector into a much lower-dimensional visual rating space.The projected dimensions represent visual rating facets, while sharing E across items reduces the number of parameters.
- Training: VBPR trains on triples (u, i, j) containing a user, a positively observed item, and a non-observed item using stochastic gradient ascent under BPR.The visual parameters are updated alongside the non-visual parameters, with an additional regularization hyperparameter λE for E.
- Efficiency: O(K + D) is the total time complexity for updating one sampled triple, where K is the latent dimension and D is the visual dimension.Updating the user visual factors and embedding costs O(D), while the CNN feature dimension F is fixed to 4096.
Experiments
Experiments evaluate VBPR on Amazon and Tradesy datasets using implicit feedback, product-image CNN features, and AUC test ranking. VBPR improves over BPR-MF, especially for cold-start items, while combining collaborative and visual signals to outperform baselines in most cases.
- Datasets: The experiments use Amazon Women’s Clothing, Men’s Clothing, and Cell Phones & Accessories, plus a new Tradesy second-hand clothing dataset, with one image per item and implicit positive feedback.Amazon feedback comes from review histories; Tradesy combines purchase histories and thumbs-up, creating an inherently cold-start setting because of one-off trades.
- Evaluation: Each user contributes a random validation item and test item, with remaining interactions for training; performance is reported as test-set AUC using validation-selected hyperparameters.Evaluation uses the Area Under the ROC curve and reports performance on the test set T for the best validation-set V hyperparameters.
- Baselines: The comparison includes random, Most Popular, MM-MF, BPR-MF, WRMF, and Image-based Recommendation, with equal MF dimensions and a fifty-fifty visual/non-visual split in VBPR.BPR-MF is described as state-of-the-art personalized ranking for implicit feedback, while IBR retrieves stylistically similar items from a learned visual space.
- Results: Over 12%: VBPR’s average improvement over BPR-MF for all items; more than 28% for cold-start items, demonstrating benefits from incorporating CNN features.Cold-start items have fewer than five positive training instances and comprise around 60% of the Amazon test sets and 80% of Tradesy’s.
- Results: VBPR outperforms all baselines in most cases and shows particularly large improvements on Tradesy, while IBR is stronger than MF methods in cold start but weaker for warm start.IBR’s cold-start advantage reflects MF difficulty learning meaningful factors, whereas its warm-start disadvantage reflects not using historical user feedback.
5. Visual features
VBPR maps items into a low-dimensional visual space where similarly evaluated styles are nearby. On Women’s Clothing, the learned embedding captures transitions across subcategories despite CNN features being pretrained on another dataset, demonstrating their expressive power.
- Visual space: VBPR maps items to a low-dimensional visual space, placing similarly styled items near one another according to users’ evaluations.The space is visualized for Women’s Clothing in Figure 4.
- Visual space: The embedding learns a visual transition across different subcategories despite features extracted from a CNN pretrained on another dataset.This observation confirms the expressive power of the extracted visual features.
Conclusion & Future Work
The paper proposes a scalable method that incorporates product-image visual features into matrix factorization to uncover visual dimensions influencing behavior. Future work will add temporal dynamics for drifting fashion tastes and examine explicit-feedback settings.
- Conclusion: The proposed scalable method incorporates visual features extracted from product images into matrix factorization.It is designed for personalized ranking tasks on implicit-feedback datasets.
- Conclusion: The model aims to uncover the visual dimensions that most influence people’s behavior.
- Future Work: Future work will extend the model with temporal dynamics to account for fashion tastes drifting over time and investigate explicit-feedback settings.