PaperlayerView exact revision on arXiv

Source-linked AI summary

IDGenRec: LLM-RecSys Alignment with Textual ID Learning

Juntao Tan, Shuyuan Xu, Wenyue Hua, Yingqiang Ge, Zelong Li, Yongfeng Zhang

arXiv:2403.19021v2cs.IRcs.AIcs.CLcs.LG

TL;DR

Generative recommendation represents user histories and recommendations as text, but item IDs remain difficult to encode with concise, meaningful language tokens. IDGenRec learns textual IDs and aligns an ID generator with a recommender; it reports strong standard-setting results and promising zero-shot performance on unseen datasets. The findings suggest potential for a foundational generative recommendation model.

  • Problem

    Generative recommendation needs item IDs that fit the text-to-text framework while preserving meaningful item information.

    Source: §1 · p. 1

  • Method

    IDGenRec trains a generator to produce unique, concise, semantically rich textual item IDs from human-language tokens alongside an LLM-based recommender.

    Source: §1 · p. 2

  • Results

    The method outperforms baselines in standard sequential recommendation and shows promising zero-shot performance on six unseen datasets, comparable to or better than some supervised traditional models.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    The results suggest that textual ID learning has potential as a foundation-model approach for generative recommendation.

    Source: §1 · p. 3, §5 · p. 10

Abstract

from arXiv · show

Generative recommendation based on Large Language Models (LLMs) have transformed the traditional ranking-based recommendation style into a text-to-text generation paradigm. However, in contrast to standard NLP tasks that inherently operate on human vocabulary, current research in generative recommendations struggles to effectively encode recommendation items within the text-to-text framework using concise yet meaningful ID representations. To better align LLMs with recommendation needs, we propose IDGen, representing each item as a unique, concise, semantically rich, platform-agnostic textual ID using human language tokens. This is achieved by training a textual ID generator alongside the LLM-based recommender, enabling seamless integration of personalized recommendations into natural language generation. Notably, as user history is expressed in natural language and decoupled from the original dataset, our approach suggests the potential for a foundational generative recommendation model. Experiments show that our framework consistently surpasses existing models in sequential recommendation under standard experimental setting. Then, we explore the possibility of training a foundation recommendation model with the proposed method on data collected from 19 different datasets and tested its recommendation performance on 6 unseen datasets across different platforms under a completely zero-shot setting. The results show that the zero-shot performance of the pre-trained foundation model is comparable to or even better than some traditional recommendation models based on supervised training, showing the potential of the IDGen paradigm serving as the foundation model for generative recommendation. Code and data are open-sourced at https://github.com/agiresearch/IDGenRec.

1 INTRODUCTION

Generative recommendation casts recommendation as text-to-text generation, but item IDs that lack human-language meaning limit semantic use and transfer. IDGenRec learns concise, unique textual IDs and aligns an ID generator with a base recommender; experiments report gains in standard and zero-shot settings.

1 INTRODUCTION

  • In generative recommendation, user history is a textual prompt and the target item is generated as text, making item encoding in language tokens a central problem..Source: §1 · p. 1
  • Earlier numerical IDs lack contextual meaning and encode dataset-specific co-occurrence, limiting transfer to unseen datasets; prior methods also overlook item-description semantics.The paper argues that human-vocabulary item representations could help LLMs learn recommendation-related knowledge and improve accuracy and generalizability.Source: §1 · pp. 1–2
  • IDGenRec represents items with concise, unique, semantically rich textual IDs made from human-language tokens, and trains an ID generator alongside the base recommender.The generator uses item metadata; the same generator can optionally produce a user ID from the text of items in the user's history.Source: §1 · p. 2
  • The approach combines a diverse ID-generation algorithm for uniqueness with asynchronous alternating training to align the ID generator and recommender.The ID generator is fine-tuned for recommendation objectives, and generated IDs are intended to be short yet unique.Source: §1 · p. 2
  • Experiments report improvements over traditional and generative baselines on four public datasets, and promising zero-shot results on six unseen datasets after training on 19 datasets.The reported zero-shot performance even surpasses many traditional recommendation models trained with supervision.Source: §1 · p. 3

2 APPROACH

The approach builds a generative recommendation pipeline by converting recommendation inputs into text-to-text prompts, generating item IDs from item metadata, and training the ID generator and recommender alternately.

2 APPROACH

  • The method first constructs prompts and integrates generated IDs into a text-to-text format, then generates IDs from item metadata.The base recommender follows; the ID generator and recommender are trained alternately against the recommendation objective.Source: §2 · p. 3
  • A diverse ID-generation algorithm is used to ensure each item receives a unique ID.These IDs are used by the base recommender.Source: §2 · p. 3
  • Alternating training is intended to make the ID generator and base recommender collaborate effectively.The section order proceeds from the generation process to ID generation, the recommender, and training.Source: §2 · p. 3

2.1 Generative Process

The generative process places a user's ordered item history into a prompt, then has the decoder generate the target item's ID token by token.

2.1 Generative Process

  • A prompt template inserts item IDs from the user's history in their original sequential order and asks for the next possible item.The paper uses 10 prompt templates with minor variations, randomly selecting one for each training instance.Source: §2.1 · p. 3
  • A user ID can optionally be generated from metadata for all items in the user's history to represent a high-level preference profile.The paper reports that adding a user ID benefited recommendation performance in experiments.Source: §2.1 · p. 3
  • The decoder generates the target item ID token by token, with the generated ID uniquely identifying the recommendation.The completed prompt is illustrated in Figure 2.Source: §2.1 · p. 3

2.2 ID Generator

The ID generator converts an item's textual metadata into a concise ID, generating each ID token while attending to the full description and previously generated ID tokens.

2.2 ID Generator

  • The ID generator consumes plain-text item metadata and learns which information to prioritize when producing an item ID.Metadata can include titles, categories, prices, descriptions, creation time, popularity, and location, depending on the platform and dataset.Source: §2.2 · p. 3
  • The output is a concise sequence of ID tokens, with length n much smaller than the description length m.The description is represented as a sequence of tokens w, and the generated ID as tokens d.Source: §2.2 · p. 3
  • Each generated ID token is conditioned on the entire item description and the previously generated ID tokens.The formulation uses the ID generator's parameters to define the probability of the generated ID.Source: §2.2 · p. 3

2.3 Diverse ID Generation

The diverse ID generation algorithm aims to balance ID length with uniqueness. It uses diverse beam search and adjusts diversity or permitted length when duplicates occur.

2.3 Diverse ID Generation

  • Diverse beam search partitions beams into groups and applies a diversity penalty, balancing output diversity against the model’s probabilities.A higher λ encourages diversity; a lower λ gives more weight to model probabilities.Source: §2.3 · p. 3
  • The algorithm generates k groups of IDs and checks each against existing IDs, increasing the penalty when it detects a duplicate.If the penalty reaches its preset maximum without producing a unique ID, the algorithm extends the permitted ID length and retries with the initial penalty.Source: §2.3 · pp. 3–4
  • Generated item IDs are inserted into the prompt, which the base recommender uses to generate a recommendation ID through constrained decoding.Figure 2 also shows a user ID generated from the text of items in the user’s history.Source: Figure 2 · p. 4

2.4 Base Recommender

The base recommender receives a tokenized prompt containing generated IDs and produces a recommendation ID autoregressively. Constrained decoding restricts generation to IDs for existing items.

2.4 Base Recommender

  • The system tokenizes the completed prompt and feeds it to the recommender, which generates the target item ID autoregressively.Source: §2.4 · p. 4
  • A prefix tree constrains each newly generated token to continuations that can form an existing candidate ID.The input prompt is denoted x, and the generated sequence y is an ID for a real dataset item.Source: §2.4 · p. 4
  • The paper expresses a target item’s recommendation probability in terms of the generated ID tokens.Source: §2.4 · p. 4

2.5 Alternate Training

The ID generator and base recommender are trained in alternating, interdependent phases. The process uses precomputed item IDs for recommender training and a differentiable embedding representation to train the ID generator from recommendation loss.

2.5 Alternate Training

  • Alternating training separates the interdependent ID-generator and recommender updates to address instability from training both simultaneously.Both components minimize negative log-likelihood of the pipeline’s prediction against the ground-truth target item ID.Source: §2.5 · pp. 4–5

2.5.1 Training Base Recommender.

  • At each recommender-training round, the system precomputes item IDs and uses them in user-history prompts, training with teacher forcing.Source: §2.5.1 · p. 5
  • In the described update, the base recommender’s parameters are fixed while the ID generator is updated to produce IDs it can interpret.Source: §2.5.2 · p. 5

2.5.2 Training ID Generator.

  • Because generated IDs are discrete, the method interpolates differentiable ID embeddings into the prompt so recommendation loss can guide ID-generator updates.The base recommender is held fixed, and the loss is backpropagated only to the ID generator.Source: §2.5.2 · p. 5

2.6 Model Initialization

The model uses T5 for both the ID generator and base recommender to keep the study focused on ID generation and enable comparison with prior T5-based work. Their starting checkpoints differ to suit their roles.

2.6 Model Initialization

  • Both components use T5 to keep the model simple and support fair comparison with previous T5-based generative recommendation work.The paper attributes improvement to ID selection rather than a different model structure.Source: §2.6 · p. 5
  • The base recommender starts from pretrained T5, while the ID generator starts from T5 small fine-tuned for article tag generation.The tag-generation checkpoint was trained on 190k Medium articles and was selected for its similarity to summarizing items in a few words.Source: §2.6 · p. 5

3 EXPERIMENT

The experiments test single-dataset sequential recommendation and zero-shot transfer from a 19-dataset Amazon corpus to unseen datasets. They report gains over standard baselines and performance comparable to some supervised traditional recommenders.

3.3 Evaluation Metrics

  • Standard tests use Sports, Beauty, Toys, and Yelp, filtering users and items below five interactions; evaluation ranks all items under leave-one-out testing with HR and NDCG at 5 and 10.The same processing steps are used for comparison with baselines.Source: §3.1 · p. 5, §3.3 · p. 6

3.1 Datasets

  • The zero-shot corpus uses 19 Amazon datasets for training and six tests spanning five Amazon domains and Yelp, covering all Amazon density categories.The Amazon test domains are Sports, Beauty, Toys, Music, and Instruments; Yelp is the inter-platform test.Source: §3.1 · p. 5

3.2 Implementation Details

  • Foundation-model training performs best after one epoch; further training negatively affects zero-shot performance, while oversized Amazon datasets are downsampled to 30,000 users.The 30,000-user cap is around the median dataset user count.Source: §3.2 · p. 6, §3.1 · p. 6

3.4 Exp1: Standard Evaluation

  • 39.44% on Sports, 23.55% on Beauty, 42.37% on Toys, and 36.76% on Yelp: the method exceeds each dataset’s second-best baseline, averaging gains across four metrics.All improvements are significant at p<0.05 versus the best baseline under a student’s t-test.Source: §3.4.2 · p. 6, Table 4 · p. 7

3.4.3 Ablation Studies.

  • Alternate training significantly boosts performance; adding user IDs helps beyond item IDs, while user IDs alone are insufficient.The strategy alternates training the ID generator and base recommender for three iterations.Source: Table 5 · p. 7, §3.4.3 · p. 7, Table 6 · p. 9

3.5.1 Baselines.

  • For zero-shot comparison, UniSRec learns item representations from metadata and receives preselected Amazon categories, while Yelp uses all plain text information.The authors describe the comparison as fair because both models encounter unseen patterns.Source: §3.5.1 · p. 7

3.4.4 Case Studies.

  • After alternate training, generated IDs select more representative item words and generally contain less extraneous information than the initial IDs.The quality study compares two examples per dataset, one with longer and one with shorter plain-text item data.Source: §3.4.4 · p. 7, Figure 3 · p. 8

3.5.2 Results.

  • Zero-shot performance generally beats UniSRec (except Music HR@10) and matches or exceeds some supervised traditional models; on Yelp it beats all traditional methods, trailing only P5.Across the shared datasets, the comparison uses average HR and NDCG scores.Source: §3.5.2 · pp. 8–9, Table 7 · p. 9

4 RELATED WORKS

Related work distinguishes discriminative LLM recommenders, which use language-model representations in ranking, from generative systems that produce recommendations as text. Prior generative approaches use numerical or semantic IDs, while another line studies prompt-based recommendation without model fine-tuning.

4 RELATED WORKS

  • Discriminative LLM recommenders combine contextual text with collaborative filtering for ranking, whereas generative systems directly produce target items as text [3] [9].The paper places its work in the generative category and contrasts it with ranking-based recommendation.Source: §4 · p. 9
  • Earlier T5-based generative systems used numerical item IDs [9], while titles or categories as semantic IDs can duplicate or overlap as catalogs grow.The paper differentiates its contextual ID generator as a way to use rich item information for cross-platform, zero-shot recommendation.Source: §4 · p. 9
  • A separate research line probes recommendation without training or fine-tuning LLMs, focusing mainly on prompt design [4] [7] [13] [21] [22] [30].This line was inspired by ChatGPT’s zero-shot capabilities.Source: §4 · p. 9
  • ChatGPT’s recommendation accuracy was reported as not competitive with traditional recommendation methods [21].Source: §4 · p. 10

5 CONCLUSION

The paper introduces textual ID learning to encode recommendation items as concise, semantically rich, platform-agnostic IDs and align an ID generator with a recommender. It reports gains over existing baselines in standard sequential recommendation and strong zero-shot performance on unseen datasets, while presenting foundation-model use as a future possibility.

5 CONCLUSION

  • The framework learns unique, concise, semantically rich, platform-agnostic textual IDs and uses diverse ID generation and alternate training to align the ID generator with the recommender.Source: §5 · p. 10
  • The model outperforms existing recommendation baselines in standard sequential recommendation and shows strong performance when tested zero-shot on unseen datasets.Source: §5 · p. 10
  • The authors present learned textual IDs as a possible basis for future foundational recommendation models.Source: §5 · p. 10
Loading 2403.19021v2…