Source-linked AI summary
Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks
Yongchun Zhu, Ruobing Xie, Fuzhen Zhuang, Kaikai Ge, Ying Sun, Xu Zhang, Leyu Lin, Juan Cao
TL;DR
Cold-start items have ID embeddings that fit deep recommendation models poorly and are distorted by noisy limited interactions. The paper proposes MWUF, which uses meta-generated scaling and shifting functions to warm and stabilize these embeddings. Across three datasets and multiple deep models, MWUF generally outperforms comparison methods, with some dataset-specific exceptions.
Problem
Limited interactions make cold item ID embeddings difficult to train, creating a model-fitting gap and sensitivity to noisy interactions.
Method
MWUF uses Meta Scaling and Meta Shifting Networks to transform cold embeddings into warm feature space and produce more stable embeddings.
Results
MWUF outperforms most compared methods on most tasks and is compatible with six deep recommendation models across three real-world datasets.
Takeaways & Limitations
MWUF provides a general framework for warming cold ID embeddings without affecting hot items.
Takeaways & Limitations
The study focuses mainly on the warm-up phase, while cold-start performance relies on common initialization.
Abstract
from arXiv · showhide
Recently, embedding techniques have achieved impressive success in recommender systems. However, the embedding techniques are data demanding and suffer from the cold-start problem. Especially, for the cold-start item which only has limited interactions, it is hard to train a reasonable item ID embedding, called cold ID embedding, which is a major challenge for the embedding techniques. The cold item ID embedding has two main problems: (1) A gap is existing between the cold ID embedding and the deep model. (2) Cold ID embedding would be seriously affected by noisy interaction. However, most existing methods do not consider both two issues in the cold-start problem, simultaneously. To address these problems, we adopt two key ideas: (1) Speed up the model fitting for the cold item ID embedding (fast adaptation). (2) Alleviate the influence of noise. Along this line, we propose Meta Scaling and Shifting Networks to generate scaling and shifting functions for each item, respectively. The scaling function can directly transform cold item ID embeddings into warm feature space which can fit the model better, and the shifting function is able to produce stable embeddings from the noisy embeddings. With the two meta networks, we propose Meta Warm Up Framework (MWUF) which learns to warm up cold ID embeddings. Moreover, MWUF is a general framework that can be applied upon various existing deep recommendation models. The proposed model is evaluated on three popular benchmarks, including both recommendation and advertising datasets. The evaluation results demonstrate its superior performance and compatibility.
1 INTRODUCTION
Deep recommendation models face cold-start difficulties because limited item interactions produce ID embeddings that fit the model poorly and are vulnerable to noise. MWUF addresses both issues by transforming and stabilizing cold item embeddings with two meta networks, and is evaluated across datasets and models.
- Limited interactions make it difficult to train reasonable cold item ID embeddings for data-demanding deep recommendation models.
- Noise in limited interactions can seriously affect cold ID embeddings and degrade recommendations for cold items.
- Cold ID embeddings poorly fit deep models because those models learn mainly from hot-item data, making fast adaptation important.
- MWUF transforms cold ID embeddings toward warm feature space and uses global interaction information to reduce noise.Its two ideas are direct transformation for fast adaptation and global interacted-user information for stabilization.
- MWUF is a general, online-compatible framework evaluated on three real-world datasets and applied to six deep recommendation models.
2 RELATED WORK
Related work divides cold-start recommendation into cold-start and warm-up phases and groups existing warm-up methods by how they adapt models or embeddings. MWUF instead predicts item-specific scaling and shifting functions for cold embeddings.
- Cold-start recommendation covers new users or items with limited interactions and includes zero-sample cold-start and few-sample warm-up phases.
- Cold-start methods use auxiliary information such as user or item attributes, knowledge graphs, and auxiliary-domain samples during the zero-sample phase.
- Existing warm-up methods personalize deep-model parameters, exploit pretrained embeddings, or learn more robust item embeddings.
- MWUF differs from these groups by using meta networks to predict scaling and shifting functions that transform cold ID embeddings.
- Meta-learning methods aim to transfer general knowledge across tasks so models can rapidly adapt from few examples.
3 MODEL
The model defines recommendation prediction from user and item embeddings, then warms cold item embeddings with meta-generated scaling and shifting functions. MWUF trains these components in a model-agnostic procedure while focusing on the warm-up phase.
- 3.1 Problem Definition: Each binary recommendation sample contains user, item, feature, and label information, and a deep model predicts the label from their embeddings.
- 3.1 Problem Definition: The paper focuses on the warm-up phase, where cold items have more than zero but limited interactions.
- 3.2 Framework: MWUF uses a common initial item embedding and two meta networks to transform cold embeddings into warmer, more stable representations.
- 3.3 Two Meta Networks: The Meta Scaling Network uses item features to generate an item-specific scaling function that maps cold embeddings toward warm feature space.
- 3.3 Two Meta Networks: The Meta Shifting Network aggregates interacted-user embeddings and generates a shifting function that stabilizes noisy item embeddings.
- 3.4 Overall Procedure: MWUF is model-agnostic: the base model and meta networks are updated separately, allowing application to various base models.
4 EXPERIMENTS
The experiments evaluate cold-start methods across three datasets and phases, test MWUF with multiple recommendation models, and examine its components and initialization. Results generally favor MWUF, while performance varies by model, dataset, and available interactions.
- Baseline Comparisons: AFN outperforms popular deep collaborative-filtering models in the cold-start setting and surpasses cold-start methods on some tasks.The comparison includes Wide & Deep, PNN, DCN, DropoutNet, MetaEmb, and MeLU.
- MWUF Results: MWUF outperforms most compared methods on most tasks and significantly surpasses the best baseline in most scenarios according to paired t-tests.The reported improvement mainly comes from warm ID embeddings of items.
- Generalization Experiments: MWUF and MetaEmb improve new-item performance across multiple base models, while generalized MWUF consistently achieves the best results and adapts faster than MetaEmb.The compatibility evaluation covers six base models and three datasets; averaged scores and standard deviations are reported over ten runs.
- Ablation and Initialization: The full MWUF performs best in ablation experiments, with both Meta Scaling and Meta Shifting improving over common initialization alone.MWUF(scale) outperforms MWUF(shift), which the authors associate with feature transformation producing larger gains than representation combination.
5 CONCLUSION
The paper addresses cold-start recommendation by warming up cold item ID embeddings with Meta Scaling and Meta Shifting Networks. The resulting Meta Warm Up Framework (MWUF) is designed for compatibility with existing embedding-based recommendation models and is evaluated on three real-world datasets.
- MWUF uses Meta Scaling and Meta Shifting Networks to warm up cold item ID embeddings.The scaling network transforms cold embeddings into warm feature space, while the shifting network produces stable embeddings from noisy embeddings.
- MWUF is a general framework that can be applied upon various existing models using embedding techniques.
- Experiments on three real-world datasets validate the effectiveness and compatibility of the proposed models.