Source-linked AI summary
Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation
Flavian Vasile, Elena Smirnova, Alexis Conneau
TL;DR
Recommendation systems need item representations that use available metadata while remaining practical for real-time deployment. Meta-Prod2Vec injects categorical metadata as training-time side information into Prod2Vec embeddings, and the paper reports consistent gains over Prod2Vec, especially in cold-start settings and when combined with collaborative filtering. The method is evaluated on next-song prediction using the 30Music dataset.
Problem
Prod2Vec uses local item co-occurrence information but does not incorporate available item metadata, limiting the information used to represent items.
Method
Meta-Prod2Vec extends Prod2Vec by incorporating categorical item side information in both neural-network input and output spaces, with metadata used during training.
Results
Meta-Prod2Vec consistently outperforms Prod2Vec globally and in cold-start recommendation, while its combination with collaborative filtering outperforms all other tested methods.
Takeaways & Limitations
The resulting item embeddings improve recommendation performance while retaining low implementation cost and supporting a constant real-time scoring memory footprint.
Takeaways & Limitations
The evaluation excludes methods that use metadata directly at prediction time because the study targets metadata use only during training.
Abstract
from arXiv · showhide
We propose Meta-Prod2vec, a novel method to compute item similarities for recommendation that leverages existing item metadata. Such scenarios are frequently encountered in applications such as content recommendation, ad targeting and web search. Our method leverages past user interactions with items and their attributes to compute low-dimensional embeddings of items. Specifically, the item metadata is in- jected into the model as side information to regularize the item embeddings. We show that the new item representa- tions lead to better performance on recommendation tasks on an open music dataset.
1. INTRODUCTION
Recommendation has become commercially important as online commerce expands, while real-world systems must scale and address cold-start constraints. Meta-Prod2Vec extends Prod2Vec with item metadata and reports improved recommendation performance at low implementation and integration cost.
- Motivation: Online commerce grew 15% in 2015 and represented $1.5 trillion in 2014 spending, increasing the importance of recommendation systems.Amazon attributed roughly 35% of overall sales to product recommendations.
- System constraints: Recommender systems must handle large volumes of interaction data, support real-time changes, and address cold-start cases.These constraints inform system architecture in real-world deployments.
- Prior progress: Neural probabilistic models provide scalable user and product embeddings and have shown improvements on cold-start problems.They have been applied to advertising, restaurant, and challenge recommendation tasks.
- Contribution: Meta-Prod2Vec extends Prod2Vec by adding categorical item side information in a simple and efficient way.Metadata can be used only during training, keeping the real-time scoring memory footprint constant.
- Reported result: The approach significantly improves recommendation performance on a subset of the 30Music dataset with low implementation and integration cost.The paper evaluates the method on an open music dataset.
2. RELATED WORK
Recommendation methods span collaborative filtering, content-based, hybrid, matrix-factorization, neural, and unified latent-representation approaches. These methods incorporate interactions and content in different ways, including shared factors, regression, factorization, graph models, and Boltzmann machines.
- Classical recommendation methods: Collaborative filtering uses user-item interactions without requiring domain knowledge, whereas content-based methods use user or product profiles.Collaborative filtering is more popular in practice but suffers from cold-start problems for niche or new items.
- Matrix factorization: Matrix factorization learns low-rank user and item representations from sparse interaction matrices for recommendation.Variants modify losses or ranking objectives, including Bayesian Personalized Ranking and Logistic MF.
- Neural embeddings: Prod2Vec applies Word2Vec-style learning to purchase sequences, generating product embeddings used to recommend similar products.The method learns from local product co-occurrence information.
- Content-aware latent models: Latent factors can incorporate content through regression, Factorization Machines, tensor factorization, and co-factorization.These approaches create unified representations from interaction and content information.
- Joint interaction models: Graph-based and Boltzmann-machine models jointly represent user-item interactions and side information.They model shared latent factors or interaction weights involving metadata and user actions.
3. PROPOSED APPROACH
Meta-Prod2Vec extends Prod2Vec by incorporating item metadata into both neural-network inputs and outputs, adding separately weighted side-information interactions to regularize item embeddings. The approach addresses information absent from sequence-only product embeddings by modeling relationships among items and their metadata.
- Prod2Vec: Prod2Vec learns product embeddings from local co-occurrence patterns in purchase sequences, using center products to predict surrounding products.Its architecture uses a single hidden layer and softmax output layer.
- Prod2Vec: Prod2Vec does not use available item metadata, limiting the interactions it can model between products, categories, and metadata sequences.Examples include category-conditioned product transitions and product-conditioned category transitions.
- Meta-Prod2Vec: Meta-Prod2Vec incorporates side information into both neural-network input and output spaces and parametrizes item–metadata interactions separately.This extends the architecture beyond sequence-only product information.
- Meta-Prod2Vec: The Meta-Prod2Vec loss adds four weighted interaction terms to the Prod2Vec loss: LM|I, LJ|M, LM|M, and LI|M.The regularization parameter λ controls the combined side-information contribution.
- Meta-Prod2Vec: LI|M models item identities conditioned on their own metadata, while LJ|M, LM|I, and LM|M model cross-item or metadata-sequence relationships.LM|M also represents a Word2Vec-like embedding of the metadata.
- Meta-Prod2Vec: The method assigns separate regularization parameters to the four side-information types and to each additional metadata source.The item and metadata losses encode separate sequence likelihoods, conditional likelihoods, and cross-item interactions.
4. EXPERIMENTS
The experiments evaluate next-item recommendation on the 30Music dataset using embedding, collaborative-filtering, popularity, and ensemble methods. Meta-Prod2Vec performs best overall and especially improves cold-start recommendations, while its side-information components each contribute meaningful lift.
- 4.1 Setup: The evaluation predicts each user's next item from time-ordered interaction sequences, using HR@K and NDCG@K averaged across users.Models are trained and validated on earlier sequence elements before final next-item prediction.
- 4.1 Setup: The comparison includes popularity, co-occurrence collaborative filtering, Prod2Vec, Meta-Prod2Vec, and ensembles that linearly blend embeddings with CoCounts.The ensemble blending factor is selected by benchmarking embedding-based similarity against CoCounts-based cosine similarity.
- 4.3 Results: Meta-Prod2Vec outperforms Prod2Vec both standalone and in the ensemble model, with results computed at 90% confidence levels.The experiments use 100k user sessions, 433k songs, and 67k artists from 30Music, with artist metadata used as side information.
- 4.3.1 Improvements on Cold-Start: Standalone Meta-Prod2Vec outperforms all other methods when query-next-item pairs have zero training co-occurrences, while the Meta-Prod2Vec–CoCounts mix performs best for low observed counts.CoCounts performs well on frequent pairs, whereas Meta-Prod2Vec helps generalize to unseen pairs.
- 4.3.2 Relative Importance of each type of Side Information: Each side-information type contributes relevant information, while the mm type contributes 39% on HR and 35% on NDCG when estimated from degraded performance.Individually, each side-information type accounts for at most 50% of the full Meta-Prod2Vec performance lift over BestOf.
5. CONCLUSIONS
The paper introduces Meta-Prod2Vec, which augments Prod2Vec with item metadata during training. It consistently outperforms Prod2Vec globally and in cold-start recommendation, while retaining low implementation cost and no online architectural or memory-footprint changes.
- Meta-Prod2Vec enhances Prod2Vec by incorporating item metadata at training time and connects embedding-based methods with matrix factorization using side information.The paper separately analyzes the relative value of four side-information types and finds each informative.
- Meta-Prod2Vec consistently outperforms Prod2Vec globally and in the cold-start regime, and its combination with collaborative filtering outperforms the other tested methods.The approach is presented as attractive when item embeddings are already used because it has reduced implementation cost and does not alter online recommendation.
- Future work will extend item-metadata usage to non-categorical information such as images and continuous variables.