Source-linked AI summary
Scalable Recommendation with Poisson Factorization
Prem Gopalan, Jake M. Hofman, David M. Blei
TL;DR
Recommendation from sparse user behavior requires modeling both limited user attention and large, mostly unobserved user/item matrices. The paper develops Bayesian Poisson factorization with scalable variational inference over observed entries, and reports significantly better recommendations across multiple large real-world data sets than competing methods.
Problem
Recommendation systems must learn from large, sparse matrices containing explicit ratings or implicit consumption for only a small subset of items.
Method
Bayesian Poisson factorization models finite user resources and iterates over only observed entries, with scalable variational inference for latent user and item representations.
Results
Poisson factorization significantly outperforms existing recommendation methods across movie, music, scientific-article, and newspaper user-behavior data sets.
Takeaways & Limitations
Poisson factorization provides efficient, high-quality recommendations for both explicit rating data and implicit behavior data without ad hoc modifications.
Takeaways & Limitations
The evaluated variational inference algorithm is intended for massive data but future work identifies stochastic variational inference for data sets larger than those studied.
Abstract
from arXiv · showhide
We develop a Bayesian Poisson matrix factorization model for forming recommendations from sparse user behavior data. These data are large user/item matrices where each user has provided feedback on only a small subset of items, either explicitly (e.g., through star ratings) or implicitly (e.g., through views or purchases). In contrast to traditional matrix factorization approaches, Poisson factorization implicitly models each user's limited attention to consume items. Moreover, because of the mathematical form of the Poisson likelihood, the model needs only to explicitly consider the observed entries in the matrix, leading to both scalable computation and good predictive performance. We develop a variational inference algorithm for approximate posterior inference that scales up to massive data sets. This is an efficient algorithm that iterates over the observed entries and adjusts an approximate posterior over the user/item representations. We apply our method to large real-world user data containing users rating movies, users listening to songs, and users reading scientific papers. In all these settings, Bayesian Poisson factorization outperforms state-of-the-art matrix factorization methods.
1. INTRODUCTION
The paper develops Poisson factorization for recommendation, modeling heterogeneous user interests and finite consumption resources while exploiting sparse behavior data for scalable inference. Across explicit and implicit feedback settings, it reports better performance than existing matrix-factorization methods.
- Motivation and contribution: Poisson factorization models users’ heterogeneous interests, item types, and finite resources for consuming items.The model represents user preferences and item attributes with sparse, non-negative latent vectors and uses a Poisson likelihood.
- Model and recommendation process: The inference problem estimates latent item attributes and user preferences from observed behavior, then uses them to recommend new content.A user’s observed movies infer a preference vector whose components indicate interest in corresponding latent attributes.
- Empirical findings: PF and HPF outperform existing methods across large data sets involving movie watching, music listening, and scientific-article reading.The introduction also describes evaluation on explicit ratings and implicit consumption data.
- Motivation and contribution: Finite user resources help distinguish unwatched items from items a user is unlikely to prefer.The model can be viewed as first allocating a viewing budget and then spending it on items of interest.
- Scalability: Poisson factorization needs to iterate only over viewed items, including in implicit positive-only data.This follows from the mathematical form of the Poisson distribution and lets the method exploit sparse behavior matrices.
2. RELATED WORK
Related work connects Poisson factorization to nonnegative and probabilistic matrix-factorization models, while contrasting its sparse-data computation with approaches that modify or sample implicit feedback.
- Poisson-model predecessors: Poisson factorization is rooted in nonnegative matrix factorization, whose objective corresponds to a factorized Poisson likelihood.Earlier work derived NMF updates as expectation-maximization for Poisson maximum likelihood.
- Poisson-model predecessors: GaP and PFM extend Poisson models with Gamma priors and use point-estimation or approximate-MAP procedures for user and item representations.GaP places a Gamma prior on user weights, while PFM adds one for item weights.
- Related applications: Poisson factorization has also been studied for source separation and community detection, but those data and derivations differ from the user-data setting here.The paper presents its auxiliary-variable derivation as more direct for user behavior data.
- Implicit-feedback methods: Implicit-feedback methods include neighborhood models, weighting positive examples, and sampling informative negative examples.These approaches address the challenges of modeling unobserved items and the selection bias in provided ratings.
- Implicit-feedback methods: Poisson factorization avoids such special adjustments and scales linearly with the number of observed ratings.This directly exploits the sparsity of user behavior data.
3. POISSON RECOMMENDATION
Poisson factorization models sparse user behavior with non-negative latent user preferences and item attributes, while accounting for users’ varied activity and items’ varied popularity. Its hierarchical structure supports sparse factors, realistic long-tailed behavior, scalable inference, and improved representation of observed data over classical matrix factorization.
- Model: Poisson factorization represents each user and item with sparse, non-negative latent vectors, modeling observations with a Poisson distribution parameterized by their inner product.Gamma priors encourage sparse representations, while hierarchical priors capture variation in user activity and item popularity.
- Model: Hierarchical Poisson factorization models users’ finite consumption resources through user activity and item popularity parameters.The model’s generative process includes separate user activity and item popularity variables, with Bayesian Poisson Factorization fixing their rate hyperparameters to a common pair.
- Statistical properties: HPF captures long-tailed distributions of user activity and item popularity, whereas classical matrix factorization does not.Most users consume few items while a small number of tail users consume thousands; the same long-tail pattern occurs for item popularity.
- Statistical properties: Posterior predictive checks on Netflix data show that HPF captures the empirical distribution of user activity much more closely than classical matrix factorization.The comparison simulates user behavior from fitted posterior estimates and evaluates the resulting distribution against observed data.
- Statistical properties: HPF downweights unconsumed items because an absence can reflect either disinterest or limited future activity, whereas consumption directly indicates interest.This gives consumed user/item pairs more influence than unconsumed pairs when fitting preferences.
- Inference: Mean-field variational inference estimates latent preferences and attributes using an algorithm that scales to millions of users and hundreds of thousands of items on a single CPU.The inference procedure uses coordinate ascent and exploits the sparse matrix by iterating only over observed behavior entries.
4. EMPIRICAL STUDY
The empirical study evaluates HPF and BPF across large, diverse user-behavior data sets and finds that they outperform competing recommendation methods. The evaluation covers sparse implicit and explicit feedback, with performance measured through precision and recall at 20 recommendations and across user activity levels.
- Data sets and methods: HPF and BPF give significantly better recommendations than competing methods across music, movie, scientific-article, and newspaper user data.The study includes large-scale data sets spanning both implicit and explicit user behavior.
- Data sets and methods: The evaluation spans data sets differing in scale, sparsity, feedback type, and behavioral meaning, including 1 million music users, 480,000 movie users, and 80,000 scientific-article users.Mendeley, Echo Nest, and New York Times data are sparse, while Netflix contains explicit star ratings and denser observations.
- Computational considerations: Traditional matrix factorization requires explicit zero or sampled negative examples for implicit feedback, whereas HPF, BPF, and LDA use only non-zero observed ratings.The study also reports that fully Bayesian probabilistic matrix factorization and BPR failed to scale to the evaluated data sets.
- Predictive performance: HPF and BPF outperform other methods on every data set in normalized mean precision at 20 recommendations, by as much as 8 percentage points.The comparison includes NMF, LDA, and biased matrix factorization, while competing methods vary substantially across data sets.
- Predictive performance: Poisson factorization outperforms other methods for users at all activity levels, including both lightly active and heavily active users.Figure 5 compares mean precision and recall differences to HPF at 20 recommendations across user-activity percentiles.
5. CONCLUSION
Poisson factorization generates high-quality recommendations across movie views and scientific article libraries, outperforming existing methods on explicit and implicit behavior data. Its algorithms scale to massive data while modeling heterogeneous user and item activity; future work targets cold-start, nonparametric components, and larger datasets.
- Poisson factorization significantly outperforms existing recommendation methods on both explicit rating data and implicit behavior data.
- The method generates high-quality recommendations across data sets ranging from movie views to scientific article libraries.
- Its algorithms scale to massive data and capture heterogeneity in user and item activity and popularity.
- Future work includes cold-start recommendations using text, Bayesian nonparametric inference for latent components, and stochastic variational inference for larger datasets.
APPENDIX
The appendix derives variational inference for HPF using conditionally conjugate complete conditionals and coordinate ascent. Auxiliary variables make the count updates multinomial, while Gamma-Poisson conjugacy yields updates for latent weights and activity parameters.
- Variational inference minimizes KL divergence to the posterior by iteratively optimizing each parameter while holding the others fixed.
- HPF is conditionally conjugate with auxiliary variables, so each complete conditional belongs to an exponential family.
- Gamma-Poisson conjugacy produces complete conditionals for user and item weights, user activity, and item popularity.
- Auxiliary Poisson counts have multinomial complete conditionals because their values are conditioned on their observed sum.
- The variational factors for user and item weights use Gamma distributions with shape and rate parameters.
- The update equations use expected auxiliary counts and the Gamma expectation that equals shape divided by rate.