Source-linked AI summary
Item2Vec: Neural Item Embedding for Collaborative Filtering
Oren Barkan, Noam Koenigstein
TL;DR
Item-based collaborative filtering needs effective item-item similarities, including when user information is unavailable. The paper introduces item2vec by applying SGNS to item sets and finds it effective against an SVD-based item-similarity model, with a larger advantage for unpopular items.
Problem
Item-based collaborative filtering requires learning item-item similarities, including from item sets that lack user information.
Method
Item2vec applies SGNS to item sets or baskets, learns item embeddings, and computes affinity using cosine similarity.
Results
Item2vec consistently outperforms SVD in genre-consistency evaluation, achieving 68% versus 58.4% accuracy for unpopular items.
Takeaways & Limitations
Item2vec provides better item representations than the SVD baseline, with the gap becoming more significant for unpopular items.
Abstract
from arXiv · showhide
Many Collaborative Filtering (CF) algorithms are item-based in the sense that they analyze item-item relations in order to produce item similarities. Recently, several works in the field of Natural Language Processing (NLP) suggested to learn a latent representation of words using neural embedding algorithms. Among them, the Skip-gram with Negative Sampling (SGNS), also known as word2vec, was shown to provide state-of-the-art results on various linguistics tasks. In this paper, we show that item-based CF can be cast in the same framework of neural word embedding. Inspired by SGNS, we describe a method we name item2vec for item-based CF that produces embedding for items in a latent space. The method is capable of inferring item-item relations even when user information is not available. We present experimental results that demonstrate the effectiveness of the item2vec method and show it is competitive with SVD.
1. INTRODUCTION AND RELATED WORK
Item similarities are a central component of recommendation systems, especially for single-item recommendations and related shopping tasks. The paper focuses on learning these similarities directly in a low-dimensional item space.
- Recommendation motivation: Item similarities support single-item recommendations across online stores, including “People also like” lists for apps and games.Examples include Amazon, Netflix, Google Play, and iTunes.
- Research focus: The paper addresses the overlooked task of embedding items in a low-dimensional space to learn item-item similarities.It contrasts directly learning item relations with approaches that learn user-item connections implicitly.
- Recommendation motivation: Similarity-based recommendations also support lower-priced checkout suggestions, product bundles, and item exploration.These uses are presented as ways to improve discovery and overall user experience.
2. SKIP-GRAM WITH NEGATIVE SAMPLING
SGNS learns word representations from target-context relations by maximizing a Skip-gram objective. Negative sampling replaces the costly softmax computation, while subsampling addresses frequency imbalance.
- SGNS objective: SGNS learns target-word representations that capture relations between words and their surrounding context.The method operates on a finite word sequence and vocabulary.
- SGNS objective: The Skip-gram objective maximizes the log probability of context words within a specified context window.Target and context words use latent vectors in the model.
- Negative sampling: Negative sampling replaces the softmax computation because its gradient scales linearly with vocabulary size W.It draws N negative examples per positive example from a unigram distribution raised to the 3/4 power.
- Subsampling: Subsampling discards words with a frequency-dependent probability to reduce imbalance between rare and frequent words.The procedure was reported to accelerate learning and improve rare-word representations.
- Optimization: The model estimates target and context representations using stochastic gradient ascent on the objective.The learned matrices are updated with respect to the Skip-gram objective.
3. ITEM2VEC – SGNS FOR ITEM SIMILARITY
Item2vec applies SGNS to item sets or baskets by treating items like words in a sequence. It produces item embeddings whose cosine similarities represent item affinity, with alternative representations also possible.
- Item2vec formulation: Item2vec applies SGNS to user-generated item sets, treating a sequence of words as equivalent to a set or basket of items.The method can operate when multiple item sets cannot be linked to the same user.
- Item2vec formulation: The method can infer item-item relations from baskets even when information connecting items to users is unavailable.This setting includes order data containing baskets without the identity of the purchasing user.
- Item2vec formulation: Item2vec may preserve the original objective while shuffling each item set during runtime; the experiments found both options performed the same.The remaining process follows the SGNS procedure.
- Item representations: The final representation uses the target vector u_i, and item affinity is computed with cosine similarity.Alternative choices include the context vector, additive composition u_i + v_i, and concatenation; the latter options sometimes produce superior representations.
4. EXPERIMENTAL SETUP AND RESULTS
The evaluation compares item2vec with item-item SVD on music and store datasets using quantitative genre-consistency and qualitative nearest-neighbor analyses. item2vec performs better in both settings, including the weaker user-free store data, with larger gains for unpopular items.
- Datasets and setup: The evaluation uses private Xbox Music data with 9M user-artist events and Microsoft Store basket data lacking user information.The datasets contain 732K users and 49K artists in the Music set; Store orders are baskets without the purchasing user's identity.
- Datasets and setup: The baseline applies SVD to an item-by-item co-occurrence matrix and computes item affinity with cosine similarity between latent representations.The matrix entries count positive item pairs, are normalized by row and column sums, and use the top singular components.
- Evaluation: Genre consistency evaluates whether each item's nearest neighbors share its genre, using majority voting across neighborhood sizes k = 6, 8, 10, 12 and 16.Results reported in Table 2 use k = 8, with no significant change across the tested neighborhood sizes.
- Evaluation: 68% accuracy for item2vec versus 58.4% for SVD was obtained on 10K unpopular items with fewer than 15 users.The reported gap supports better representation quality for less popular items in this evaluation.
- Results: Qualitative comparisons show item2vec produces nearest-neighbor lists better related to seed items than SVD on both datasets.This advantage also appears for the Store dataset, despite its weaker information because user identities are unavailable.
5. CONCLUSION
The paper proposes item2vec, an SGNS-based neural embedding method for item-based collaborative filtering. Quantitative and qualitative evaluations find better item representations than SVD, with a more significant gap for unpopular items.
- item2vec is a neural embedding algorithm for item-based collaborative filtering based on SGNS with minor modifications.
- Quantitative and qualitative evaluations demonstrate item2vec's effectiveness compared with an SVD-based item similarity model.
- The performance gap over SVD becomes more significant for unpopular items, which the paper attributes to negative sampling and subsampling popular items.