PaperlayerView exact revision on arXiv

Source-linked AI summary

Zero-Shot Recommender Systems

Hao Ding, Yifei Ma, Anoop Deoras, Yuyang Wang, Hao Wang

arXiv:2105.08318v2cs.LGcs.AIcs.IR

TL;DR

Recommender systems rely on training data, leaving early-stage products in a chicken-and-egg bind. ZESREC combines sequential user histories with natural-language item identifiers to transfer recommendations across domains without overlapping users or items. The paper reports successful recommendations in this zero-shot setting across three dataset pairs.

  • Problem

    Recommender-system performance depends heavily on training data, creating a chicken-and-egg problem for early-stage products whose data depends on system performance.

    Source: §1 · p. 1

  • Method

    ZESREC combines sequential representations of users from their interaction histories with continuous item identifiers derived from natural-language descriptions.

    Source: §1 · p. 2

  • Results

    Across three pairs of real-world datasets, the authors report that ZESREC successfully recommends in the zero-shot setting where target users and items are unseen.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The approach supports recommendation in an extreme cold-start setting with no target-domain user or item seen during training.

    Source: §1 · p. 2

    In the zero-shot setting, the model cannot fine-tune or retrain on target-domain data.

    Source: §3 · p. 3

Abstract

from arXiv · show

Performance of recommender systems (RS) relies heavily on the amount of training data available. This poses a chicken-and-egg problem for early-stage products, whose amount of data, in turn, relies on the performance of their RS. On the other hand, zero-shot learning promises some degree of generalization from an old dataset to an entirely new dataset. In this paper, we explore the possibility of zero-shot learning in RS. We develop an algorithm, dubbed ZEro-Shot Recommenders (ZESRec), that is trained on an old dataset and generalize to a new one where there are neither overlapping users nor overlapping items, a setting that contrasts typical cross-domain RS that has either overlapping users or items. Different from categorical item indices, i.e., item ID, in previous methods, ZESRec uses items' natural-language descriptions (or description embeddings) as their continuous indices, and therefore naturally generalize to any unseen items. In terms of users, ZESRec builds upon recent advances on sequential RS to represent users using their interactions with items, thereby generalizing to unseen users as well. We study three pairs of real-world RS datasets and demonstrate that ZESRec can successfully enable recommendations in such a zero-shot setting, opening up new opportunities for resolving the chicken-and-egg problem for data-scarce startups or early-stage products.

1 INTRODUCTION

Recommender systems depend on training data, creating a chicken-and-egg problem for startups. ZESREC combines sequential user representations with continuous item identifiers to recommend across domains with no shared users or items.

1 INTRODUCTION

  • The approach addresses the startup data chicken-and-egg problem: recommendation performance depends on available training data, which itself depends on system performance.Modern systems learn user-to-item propensities from large collections of interaction tuples, sometimes using item content or user metadata.Source: §1 · p. 1
  • Sequential interaction histories represent users, while natural-language descriptions or other generic features provide continuous item identifiers that generalize to unseen items.ZESREC uses pretrained language embeddings to represent descriptions, enabling transfer when source and target descriptions share a similar vocabulary.Source: §1 · p. 2
  • Combining sequential recommendation with continuous identifiers, ZESREC learns transferable behavioral patterns and reports successful recommendations when target users and items are unseen.The paper also reports case studies of interpretable behavioral patterns generalizing across datasets.Source: §1 · p. 2
  • ZESREC is presented as a hierarchical Bayesian model for recommending in a target domain with neither users nor items shared with the source.The paper frames this as a zero-shot setting, unlike cross-domain recommendation with overlapping users or items.Source: §1 · p. 2

2 RELATED WORK

Prior work covers deep-learning and sequential recommendation, as well as cross-domain and cold-start settings. The paper distinguishes its setting by requiring transfer without shared users or items and across datasets.

Deep Learning for RecSys

  • Deep-learning recommender systems include Bayesian hierarchical models integrating content and rating information, alongside work on sequential recommendation.The related work also notes deep learning's scalability and reported superior performance [7].Source: §2 · pp. 2–3
  • Sequential systems use cross-session information, causal convolutions, or metadata and control signals; ZESREC is described as compatible with any sequential recommender.Examples include hierarchical GRU, TCN, and HRNN.Source: §2 · p. 3

Cross-Domain and Cold-Start RecSys

  • Cross-domain methods generally require common users or items, while cited cold-start methods either address only few-interaction cases or cannot generalize across domains.The authors argue these prior systems do not address zero-shot transfer with neither common users nor common items, or transferable behavior across datasets.Source: §2 · p. 3

3 ZERO-SHOT RECOMMENDER SYSTEMS

ZESREC addresses recommendation across source and target domains with no shared users or items, using content-based item embeddings and sequential user representations. Its Bayesian encoder-decoder combines universal embeddings with domain-specific offsets, which are removed for target-domain inference.

3 ZERO-SHOT RECOMMENDER SYSTEMS

  • The model trains only on source interaction sequences and serves a disjoint target domain without fine-tuning; a dummy starting item supports users with empty histories.Source: §3 · p. 3

3.1 FROM CATEGORICAL DOMAIN-SPECIFIC ITEM ID TO CONTINUOUS UNIVERSAL ITEM ID

  • Item content such as natural-language descriptions supplies continuous universal identifiers, while sequential models aggregate item embeddings to represent users across domains.The approach is compatible with sequential models; this paper focuses on descriptions and leaves images and videos for future exploration.Source: §3 · p. 3, §3.1 · p. 4

3.2 MODEL OVERVIEW

  • Source-domain item and user vectors add learned offsets to universal embeddings; target inference removes those offsets and scores recommendations without target interactions.The paper describes offset removal as an attempt to remove source-domain bias; recommendation scores use inner products.Source: §3.2 · pp. 4–5
  • The item UEN maps descriptions through pretrained BERT and a single-layer neural network to produce universal item embeddings, jointly trained with the sequential model.The embedding for BERT’s CLS token is used as the BERT output.Source: §3.2 · p. 5
  • The user UEN feeds embeddings of a user’s prior interactions into a sequential model to produce a user universal embedding for both training and inference.Source: §3.2 · p. 5

4 EXPERIMENTS

The experiments evaluate ZESREC on three source–target dataset pairs against zero-shot and target-trained baselines, then examine incremental training and cross-domain behavioral patterns. ZESREC performs competitively in zero-shot recommendation, while case studies illustrate patterns learned across domains.

4.5 EXPERIMENTAL RESULTS

  • Across three dataset pairs, ZESREC beats zero-shot Embedding-KNN and Random by a large margin in most cases and achieves performance comparable to in-domain baselines.Table 1 covers Amazon Grocery and Gourmet Food to Amazon Prime Pantry, Others to NFL, and Others to NCAA; its oracle methods are trained directly on target-domain data.Source: Table 1 · p. 6, §4.5 · p. 8

4.1 DATASETS

  • The evaluation uses Amazon Prime Pantry and Grocery and Gourmet Food plus two MIND sports pairs, Others-to-NFL and Others-to-NCAA.The MIND pairs are evaluated in first-day and whole-week settings; dataset sizes and splits differ across pairs.Source: §4.1 · p. 6

4.2 BASELINES AND ZESREC VARIANTS

  • Baselines include target-trained GRU4Rec, TCN, HRNN, their metadata variants, POP, and zero-shot EmbeddingKNN and Random; ZESREC uses GRU4Rec, TCN, or HRNN base models.The in-domain baselines are treated as oracle methods, while EmbeddingKNN uses user and item embeddings and Random samples from the item catalogue.Source: §4.2 · pp. 6–7

4.3 IMPLEMENTATION DETAILS

  • ZESREC processes item descriptions with pretrained BERT embeddings and feeds them to a single-layer neural network.The BERT embedding dimension is 768, and the network output dimension is D.Source: §4.3 · p. 7

4.4 EXPERIMENT SETUP AND EVALUATION METRICS

  • All models are tested on target-domain test sets; evaluation uses Recall (R@20) and NDCG (N@20), removing repeated consecutive interactions to focus on transitions from user history to the next item.Case-study users are randomly selected from the target-domain test set and retained only if ZESREC correctly predicts the sixth item given the first five as context.Source: §4.4 · pp. 7–8

4.5 EXPERIMENTAL RESULTS

  • On the MIND pairs, ZESREC consistently outperforms HRNN-Meta and HRNN-Interactions in the whole-week setting, while remaining reasonable against in-domain models in the first-day setting.The authors attribute in-domain models’ whole-week weakness to overfitting history amid strong recency bias in news recommendation.Source: §4.5 · p. 8
  • Nearly all in-domain baselines fail to surpass ZESREC within 10K target-domain interactions; Figure 2 compares incremental training on MIND-NCAA and Amazon Prime Pantry.The plotted comparisons show TCN- and HRNN-based models against ZESREC without target-domain data.Source: Figure 2 · p. 7, §4.5 · p. 8

4.6 CASE STUDIES

  • Case studies show ZESREC transferring snack-and-tea-to-herbal-tea and preferred-flavor patterns from source users to unseen target users and items.Examples pair Tension Tamer Tea with Lemon Zinger Herbal Tea, and multiple Vita Coconut Water flavors with a V8 Splash flavor recommendation.Source: §4.6 · p. 9

5 CONCLUSION

The conclusion presents ZESREC as a framework for recommendation when target-domain users and items are unseen during training. It reports quantitative and qualitative evidence of effectiveness and identifies additional modalities and information as future directions.

5 CONCLUSION

  • ZESREC addresses recommendation across domains with no overlapping users or items, using universal continuous identifiers grounded in natural-language descriptions.The paper presents it as the first general framework for this zero-shot setting.Source: §5 · p. 9
  • The reported quantitative and qualitative results demonstrate effectiveness and support that ZESREC learns user behavioral patterns that generalize across domains.Source: §5 · p. 9
  • Future work includes exploring images and videos as universal identifiers and investigating pretrained BERT interpretability and auxiliary information.Source: §5 · p. 9

Related papers

Loading 2105.08318v2…