Source-linked AI summary
How to Index Item IDs for Recommendation Foundation Models
Wenyue Hua, Shuyuan Xu, Yingqiang Ge, Yongfeng Zhang
TL;DR
Recommendation foundation models need compact, unique item IDs to avoid long-text generation and hallucinated recommendations. This paper systematically compares indexing strategies in P5 and proposes sequential, collaborative, semantic, and hybrid alternatives with constrained decoding. Experiments across real-world datasets show that indexing method strongly affects performance, while the proposed methods satisfy suitable ID-length and uniqueness criteria.
Problem
LLM recommenders need compact, unique, natural-language-compatible item IDs because long descriptions are difficult to generate exactly and can cause hallucinated items.
Method
The paper evaluates three trivial indexing methods and four alternatives in P5, using constrained decoding over a trie of valid item IDs.
Results
Nearly all Collaborative Indexing results outperform baselines across tested configurations, while advanced indexing methods generally outperform trivial methods.
Takeaways & Limitations
Item indexing substantially influences LLM-based recommendation performance, and collaborative information can provide more beneficial item relationships than metadata alone.
Takeaways & Limitations
Title indexing can encode misleading semantics, while independent indexing ignores item relationships and may require prohibitively long training for many new tokens.
Abstract
from arXiv · showhide
Recommendation foundation model utilizes large language models (LLM) for recommendation by converting recommendation tasks into natural language tasks. It enables generative recommendation which directly generates the item(s) to recommend rather than calculating a ranking score for each and every candidate item as in traditional recommendation models, simplifying the recommendation pipeline from multi-stage filtering to single-stage filtering. To avoid generating excessively long text and hallucinated recommendations when deciding which item(s) to recommend, creating LLM-compatible item IDs to uniquely identify each item is essential for recommendation foundation models. In this study, we systematically examine the item ID creation and indexing problem for recommendation foundation models, using P5 as an example of the backbone LLM. To emphasize the importance of item indexing, we first discuss the issues of several trivial item indexing methods, such as random indexing, title indexing, and independent indexing. We then propose four simple yet effective solutions, including sequential indexing, collaborative indexing, semantic (content-based) indexing, and hybrid indexing. Our study highlights the significant influence of item indexing methods on the performance of LLM-based recommendation, and our results on real-world datasets validate the effectiveness of our proposed solutions. The research also demonstrates how recent advances on language modeling and traditional IR principles such as indexing can help each other for better learning and inference. Source code and data are available at https://github.com/Wenyueh/LLM-RecSys-ID.
1 INTRODUCTION
Recommendation foundation models must assign compact, unique, language-compatible item IDs because generating long item descriptions can produce hallucinated recommendations. This paper examines trivial indexing methods and proposes constrained decoding alongside four alternatives.
- Motivation: Long item descriptions are impractical for exact generation and may not correspond to real database items, causing hallucinations.Compact unique IDs represent items with a small number of characteristic tokens.
- Problem: LLM-compatible IDs must remain unique, integrate with natural-language prompts, and avoid misleading relationships between unrelated items.Random indexing can create arbitrary associations through shared sub-tokens.
- Solution: Constrained decoding can enforce valid item generation, but applying it to free-form long text is impractical because rigid patterns add memorization complexity.The method uses a prefix tree of valid IDs and assigns zero probability to nonexistent IDs during decoding.
- Study scope: The study evaluates Random, Title, and Independent Indexing, then explores Sequential, Collaborative, Semantic, and Hybrid Indexing using P5.P5 converts recommendation tasks into natural-language sentences based on personalized prompts.
2 RELATED WORK
Generative recommendation replaces one-by-one candidate scoring with natural-language generation, requiring users and items to be represented as token sequences or embeddings. This paper compares item-indexing strategies within P5 to identify suitable approaches for recommendation foundation models.
- Traditional recommendation: Traditional recommenders score and sort every candidate item, which becomes time consuming when the item pool is large.Matching-based models project users and items into a shared embedding space before calculating preference scores.
- Generative recommendation: Generative recommendation models instead formulate recommendation as language understanding or generation and directly generate items to recommend.P5 converts diverse recommendation tasks and interaction data into natural-language sequences using personalized prompts.
- Item indexing: P5 uses number-token IDs, M6Rec uses metadata-based embeddings, and LMRecSys uses item titles.The differing representations motivate a systematic comparison of indexing methods under P5.
- Research focus: The paper studies how item indexing affects LLM-based recommendation and which indexing methods best suit recommendation foundation-model pre-training.The comparison also examines the relationship between item indexing and foundation-model pre-training.
3 PRELIMINARIES AND PRECEDING STUDY
The study uses P5’s text-to-text recommendation paradigm and evaluates datasets, splits, token notation, and baseline indexing methods. Its motivating analysis identifies structural problems in random and title IDs, while independent IDs trade simplicity for cost and limited performance.
- 3.1 Introduction of P5 Paradigm: P5 represents recommendation inputs and outputs as text, with item interactions encoded by item IDs; this study focuses on sequential recommendation.The task predicts the next item from a user’s interaction sequence and is therefore sensitive to indexing choices.
- Datasets: Experiments use Amazon Sports & Outdoors, Amazon Beauty, and Yelp interaction data from January 1 through December 31, 2019.The Amazon datasets support product recommendation, while Yelp supports business recommendation.
- Data split: The leave-one-out split assigns each user’s second-to-last interaction to validation, the last to testing, and earlier interactions to training.Validation and test predictions use progressively longer observed interaction sequences.
- Trivial indexing methods: Random Indexing assigns tokenized random numbers, Title Indexing uses tokenized titles, and Independent Indexing allocates one extra token per item.IID tokens are specifically allocated and conventionally begin with “IID”.
- Random Indexing limitations: Random numerical IDs can share sub-tokens across unrelated items, creating arbitrary relationships and unwanted training bias.The overlap is imposed by index structure and cannot be eliminated through learning.
- Title and Independent Indexing limitations: Title IDs require memorizing lengthy text and can encode misleading semantics because similar titles may describe different items.Independent IDs are easier to learn but ignore item relationships, offer limited performance, and may require prohibitively long training when many new tokens are added.
4 NONTRIVIAL INDEXING METHODS
The paper develops nontrivial item-indexing methods that preserve short IDs while incorporating collaborative or semantic similarity. Sequential indexing uses co-occurrence order, whereas collaborative indexing uses spectral clustering to assign shared tokens along a hierarchical tree.
- An effective item index should remain short for generation while encoding prior information so similar items share tokens and dissimilar items share few.
- The paper explores Sequential, Collaborative, Semantic, and Hybrid Indexing to combine compact IDs with collaborative or metadata-based similarity.
- 4.1 Sequential Indexing: Sequential indexing assigns consecutive numerical IDs to consecutively interacted items, reusing established IDs and allowing token overlap after SentencePiece tokenization.
- 4.1 Sequential Indexing: Sequential indexing can share tokens between non-co-occurring adjacent items, misses co-occurrence-frequency differences, and changes when training-user order changes.
- 4.2 Collaborative Indexing: Collaborative indexing builds a weighted item co-appearance graph and applies spectral clustering so frequently co-occurring items are grouped into similar clusters.
- 4.2.2 Item Indexing based on the Spectral Clustering Tree: The clustering hierarchy assigns tokens to internal nodes and leaves, concatenating an item's ancestor and leaf tokens so items in the same final cluster share a prefix.
5 EXPERIMENTS
Experiments under P5 compare indexing methods across recommendation datasets and show that advanced collaborative, semantic, sequential, and hybrid designs generally outperform trivial indexing and baselines. Performance depends on ordering, hyperparameters, category structure, and how indexing signals are combined.
- 5.3 Overall Results: RID and TID consistently underperform baselines, while IID generally performs better but treats items independently and requires additional learnable tokens.The overall comparison includes baseline methods and multiple indexing strategies under P5.
- 5.3 Overall Results: CID generally outperforms SemID across datasets, suggesting collaborative information captures item relationships more effectively than metadata alone.SID performs worse than CID and SemID on Amazon datasets but better on Yelp.
- 5.4 Different Settings of Sequential Indexing: Time-Sensitive ordering generally performs best for SID, followed by Long-to-Short or Short-to-Long ordering, with Random ordering worst.The results support using temporal information when ordering users for sequential indexing.
- 5.5 Different Settings of Collaborative Indexing: CID performance is below 4.5% for hit@10 when k=50 but improves considerably for k>100, and nearly all tested CID configurations outperform baselines.The experiments vary k and N across the Beauty, Amazon, and Yelp datasets.
- 5.5 Different Settings of Collaborative Indexing: Optimal performance usually occurs when the average ID length is between 3 and 4, although the best length can vary slightly by dataset.Very small k values provide insufficient expressiveness, while different k and N combinations produce different ID lengths.
- 5.6 When will Semantic Indexing Work: Hierarchical category organization improves SemID, while CID+IID and SemID+IID outperform their component methods; SID+IID and SemID+CID are suboptimal.The hybrid improvements retain collaborative or metadata structure while adding an independent token, whereas other combinations interfere with temporal or hierarchical information.
6 CONCLUSION
The paper emphasizes that item-indexing choices substantially affect recommendation foundation model performance. It evaluates trivial indexing methods and examines four simple yet effective alternatives.
- Item-indexing method selection greatly impacts foundation recommendation model performance.
- The study examines Random, Title, and Independent Indexing and emphasizes their limitations.
- The paper then examines four simple yet effective indexing methods for recommendation foundation models.