PaperlayerView exact revision on arXiv

Source-linked AI summary

A Simple Contrastive Framework Of Item Tokenization For Generative Recommendation

Penglong Zhai, Yifang Yuan, Fanyi Di, Jie Li, Yue Liu, Chen Li, Jie Huang, Sicong Wang, Yao Xu, Xin Li

arXiv:2506.16683v1cs.IRcs.AI

TL;DR

For POI recommendation, spatial relationships and geographical knowledge are critical to capturing user-mobility patterns, yet prior item-tokenization methods often underutilize or oversimplify these dependencies. SimCIT uses contrastive learning over item modalities with learnable residual quantization to align multimodal signals and form semantic identifiers. SimCIT’s experiments demonstrate effectiveness in enhancing generative recommendation performance.

  • Problem

    For POI recommendation, spatial relationships and geographical knowledge are critical to capturing user-mobility patterns, yet prior item-tokenization methods often underutilize or oversimplify these dependencies.

    Source: §I · p. 2

  • Method

    SimCIT uses contrastive learning over item modalities with learnable residual quantization to align multimodal signals and form semantic identifiers.

    Source: §I · p. 2, C. Contrastive Item Tokenization · p. 5

  • Results

    SimCIT’s experiments demonstrate effectiveness in enhancing generative recommendation performance.

    Source: §V · p. 10

  • Takeaways & Limitations

    The paper concludes that contrastive quantization can capture item-similarity distributions for recommendation more effectively than reconstruction quantization.

    Source: §V · p. 10

    Future work could extend this to the generative recommendation framework and explore aligning the natural language token space.

    Source: §V · p. 11

Abstract

from arXiv · show

Generative retrieval-based recommendation has emerged as a promising paradigm aiming at directly generating the identifiers of the target candidates. However, in large-scale recommendation systems, this approach becomes increasingly cumbersome due to the redundancy and sheer scale of the token space. To overcome these limitations, recent research has explored the use of semantic tokens as an alternative to ID tokens, which typically leveraged reconstruction-based strategies, like RQ-VAE, to quantize content embeddings and significantly reduce the embedding size. However, reconstructive quantization aims for the precise reconstruction of each item embedding independently, which conflicts with the goal of generative retrieval tasks focusing more on differentiating among items. Moreover, multi-modal side information of items, such as descriptive text and images, geographical knowledge in location-based recommendation services, has been shown to be effective in improving recommendations by providing richer contexts for interactions. Nevertheless, effectively integrating such complementary knowledge into existing generative recommendation frameworks remains challenging. To overcome these challenges, we propose a novel unsupervised deep quantization exclusively based on contrastive learning, named SimCIT (a Simple Contrastive Item Tokenization framework). Specifically, different from existing reconstruction-based strategies, SimCIT propose to use a learnable residual quantization module to align with the signals from different modalities of the items, which combines multi-modal knowledge alignment and semantic tokenization in a mutually beneficial contrastive learning framework. Extensive experiments across public datasets and a large-scale industrial dataset from various domains demonstrate SimCIT's effectiveness in LLM-based generative recommendation.

I. INTRODUCTION

Generative recommendation replaces retrieval with autoregressive generation of item identifiers, but reconstruction-based tokenization can conflict with the need to distinguish items. SimCIT uses contrastive learning to tokenize items while integrating heterogeneous modalities.

I. INTRODUCTION

  • Generative recommendation tokenizes items into discrete semantic IDs, then autoregressively generates target-item tokens and maps them to candidate items.The introduction describes beam search during inference and contrasts this with conventional ANN retrieval.Source: §I · p. 1
  • Reconstruction-based quantization can conflict with generative retrieval’s need to distinguish items, contributing to ambiguous identifiers in high-frequency clusters.The introduction frames this as a problem for preserving recommendation systems’ differentiation efficacy.Source: §I · p. 2
  • Multimodal side information can provide richer interaction contexts and improve recommendations, particularly for cold-start items.The introduction also identifies spatial relationships and geographical knowledge as important in POI recommendation.Source: §I · p. 2
  • SimCIT replaces reconstruction loss with contrastive learning over multimodal views and soft residual quantization to improve token discrimination and reduce collisions.The item identifier acts as a bridge that implicitly aligns features across modalities.Source: §I · p. 2
  • The framework’s hierarchical identifier learning integrates text and image descriptions, collaborative-filtering signals, and spatial graphs for POI recommendation.The contribution describes this as extending feature-integration strategies to the tokenization architecture.Source: §I · p. 2
  • Experiments across e-commerce and location-based datasets report that SimCIT outperforms existing item-tokenization methods for generative recommendation.The study also includes experiments on a large-scale industrial dataset.Source: §I · p. 2

A. Generative Recommendation

This section reviews generative recommendation, semantic tokenization, multimodal recommendation, and contrastive learning before introducing the paper’s proposed method. It situates SimCIT among work on semantic and multimodal item representations.

A. Generative Recommendation

  • Generative recommendation directly predicts item identifiers, with TIGER decoding semantic code tuples and later work combining or extending generative retrieval frameworks.DSI maps queries to document IDs but can struggle to distinguish relevant documents from random ones.Source: A. Generative Recommendation · pp. 2–3

B. Semantic Indexing and Tokenization

  • Semantic indexing uses item content to create discrete tokens through vector quantization or hierarchical clustering, while SimCIT focuses on contrastive quantization for more discriminative tokens.TIGER and LC-Rec [24] use RQ-VAE; Recforest and EAGER use hierarchical k-means.Source: B. Semantic Indexing and Tokenization · p. 3

C. Multi-modal Recommendation

  • Multimodal recommendation uses item text, images, and graph information, while recent generative-retrieval work explores multimodal content and collaborative signals for tokenization.Prior approaches include quantitative translators [32] and Graph RQ-VAE in MMGRec [33].Source: C. Multi-modal Recommendation · p. 3

D. Contrastive Representation Learning

  • Contrastive learning aligns similar representations and separates dissimilar ones, but can face feature collapse, require many negatives, or underweight weaker modalities.Prior work examines the roles of views and negative samples in contrastive objectives [43].Source: D. Contrastive Representation Learning · pp. 3–4

III. PROPOSED METHOD

  • The paper introduces its proposed approach as an item-tokenization method for generative retrieval tasks.The section frames item tokenization as a critical step in the approach.Source: §III · p. 4

A. Problem Formulation

The problem formulation recasts sequential next-item prediction as generation over item-code sequences. The proposed tokenizer builds on item representations from multiple modalities and learns their fusion and quantization.

A. Problem Formulation

  • Sequential recommendation predicts the next item from a chronologically ordered history of interacted item IDs.The item set is denoted I in the problem formulation.Source: A. Problem Formulation · p. 4
  • In the generative setting, each item is represented by an L-code identifier derived from side features, and next-item prediction becomes autoregressive generation of that identifier.This recasts the task as a sequence-to-sequence problem.Source: A. Problem Formulation · p. 4
  • The paper contrasts discriminative next-token prediction with prior identifier-learning methods that reconstruct item embeddings using an MSE loss.It proposes a contrastive item tokenizer to address this mismatch.Source: A. Problem Formulation · p. 4

B. Multi-modal Semantic Fusion

  • Text and image representations come from frozen modality encoders followed by an MLP.The paper names LLaMa and ViT [44] as encoder examples.Source: B. Multi-modal Semantic Fusion · p. 4
  • Graph encoders produce item representations for collaborative or spatial signals, using distance and check-in graphs with edges weighted by distance and check-in connectivity.The graphs are used to learn embeddings over their respective relationships.Source: B. Multi-modal Semantic Fusion · p. 4
  • Because modality representations contain shared and complementary information, the method computes modality importance instead of relying only on summation or concatenation.The importance scores use parameters shared across modalities.Source: B. Multi-modal Semantic Fusion · p. 4
  • Residual quantization supplies the codebook used to align multimodal semantics.The method introduces a codebook with residual quantization for this alignment.Source: B. Multi-modal Semantic Fusion · p. 4

C. Contrastive Item Tokenization

SimCIT replaces VAE-based reconstruction and commitment losses with contrastive learning for residual quantization. It aligns item modalities through learned codebooks while learning discriminative item identifiers.

C. Contrastive Item Tokenization

  • Residual quantization iteratively quantizes each residual through L codebooks, producing a tuple of L semantic tokens.Each codebook has size K.Source: C. Contrastive Item Tokenization · pp. 4–5
  • A learnable attention module integrates modalities, while codebooks act as a bridge by aligning each modality's features and indirectly aligning them with one another.The integrated representation enters soft residual quantization, whose reconstruction serves as the query for contrastive alignment with all modalities.Source: Figure 2 · p. 5, C. Contrastive Item Tokenization · pp. 5–6
  • Gumbel-softmax relaxes non-differentiable nearest-neighbor assignments into trainable soft assignments, with temperature α annealed toward zero for hard assignments.The passage describes this schedule as beginning with codebook exploration.Source: C. Contrastive Item Tokenization · p. 5
  • Unlike conventional hard vector quantization, the differentiable strategy backpropagates through the entire codebooks using learnable soft assignment weights.This circumvents the gradient-blocking issue described for hard quantization.Source: C. Contrastive Item Tokenization · p. 5
  • NT-Xent aligns reconstructed embeddings with all modality representations, and its sole objective trains the network and codebooks end to end without other regularization terms.The authors say this encourages shared multimodal information and pushes different items apart.Source: C. Contrastive Item Tokenization · p. 5

1) Implicit Regularization to Promote Diversity:

The authors interpret SimCIT's contrastive objective as implicitly promoting identifier diversity through its treatment of negative items.

1) Implicit Regularization to Promote Diversity:

2) Minimal Sufficient Identifier:

The section frames item identification as finding a minimal identifier for the target, assuming each modality view can identify it. It states that adding sufficient modalities moves the representation toward that identifier.

2) Minimal Sufficient Identifier:

D. Autoregressive Generation

The generative recommender predicts an item’s semantic tokens autoregressively from prior item tokens, then maps generated tokens back to items. Experiments span public and industrial datasets, using chronological splits and a minimum-history filter.

D. Autoregressive Generation

  • The encoder-decoder predicts an item’s token tuple autoregressively from the preceding item tokens, then uses beam search and a token-item lookup to recommend items.The behavior sequence is trained as a seq2seq task.Source: D. Autoregressive Generation · p. 6

IV. EXPERIMENTS

  • Experiments address performance, token-generation and collision strategies, alignment intensity, and identifier quality across four public datasets and one industrial dataset.The paper notes that public datasets have short interaction histories and too few item tokens to demonstrate generative recommendation effectively.Source: §IV · pp. 6–7, 1) Dataset · p. 7

1) Dataset:

  • Check-in data exclude items and users with fewer than 10 historical visit records, then sort records chronologically and assign the first 80%, middle 10%, and last 10% to training, validation, and test sets, respectively.The validation and test sets have to contain all users and items appearing in the training set. Table I presents preprocessed-dataset statistics, with Avg.Len denoting average item-sequence length.Source: 1) Dataset · p. 7, Table I · p. 7

2) Baselines Methods:

The evaluation compares SimCIT with traditional sequential and item-tokenization recommenders, using full-item ranking and shared beam settings. Implementation varies side-information embeddings by recommendation domain.

2) Baselines Methods:

  • Baselines include traditional sequential models and tokenization-based generative recommenders, including TIGER and LETTER.The latter use RQ-VAE; their token sequences also initialize item identifiers in SimCIT.Source: 2) Baselines Methods · p. 7

3) Evaluating Settings:

  • Performance is measured with full-ranking Recall@5 and Recall@10 over the entire item set, with beam size 50 for generative models.Source: 3) Evaluating Settings · p. 7

B. Overall Performance

SimCIT outperforms the reported traditional and generative baselines across public benchmarks and shows a 15% advantage over the second-best model on AMap. Ablations report declines when components are removed.

B. Overall Performance

  • SimCIT maintains the best performance in all reported cases and surpasses TIGER on e-commerce benchmarks, with collaborative signals credited for the latter advantage.Generative recommenders also outperform traditional sequential models in these comparisons.Source: B. Overall Performance · p. 8
  • On Foursquare POI tasks, SimCIT improves more over baselines than on e-commerce tasks; the paper identifies spatial graph relationships as critical for POI recommendation.Source: B. Overall Performance · p. 8
  • 15% improvement over the second-best performer across Recall@K metrics at K=10, 100, and 1000 is reported on AMap.The comparison is among generative recommendation models on the industrial dataset.Source: Table III · p. 8, B. Overall Performance · p. 8

C. Ablation Study

  • Fig. 4 describes three identifier-training phases: decreasing loss with high collision, increasing loss during codebook exploration and diversity growth, then decreasing loss with hierarchical clustering.The visualizations use randomly sampled instances because of AMap’s size.Source: Figure 4 · p. 9
  • The AMap modality comparison reports Recall@10 for individual and composed text, image, spatial, and collaborative inputs.Source: Figure 5 · p. 9
  • Removing the projection head, Gumbel search, annealing schedule, or multimodal fusion lowers overall performance in the AMap ablation.The study compares SimCIT with variants lacking each of these components.Source: C. Ablation Study · pp. 8–9
  • Replacing the annealing schedule with a constant temperature is linked to codebook collapse, while hard search disrupts gradient flow and degrades performance.Removing multimodal information also removes collaborative and spatial knowledge from the tokenizer and degrades performance.Source: C. Ablation Study · p. 9

D. Further Analysis

The further analysis examines modality contributions and identifier-learning dynamics. It reports gains with progressive modality integration and a training trajectory from early collisions to a more diverse, hierarchically clustered code distribution.

1) Multi-modal Synergy Analysis:

  • Progressive modality integration is reported to improve generative recommendation monotonically, with spatial features providing the largest individual gain and the full four-modality setup achieving state-of-the-art results.The evaluated modalities are text, image, spatial information, and collaborative filtering signals.Source: 1) Multi-modal Synergy Analysis · p. 9

2) Training Dynamics Analysis:

  • Early training shows steep loss decay with stagnant identifier perplexity and high collision; later, increasing perplexity accompanies cluster dispersion and weaker hard-negative discrimination.The analysis describes a final phase of joint optimization of multimodal fusion and collision-aware diversity, with more diverse code embeddings and less biased code assignment; Fig. 6 depicts hierarchical codebook clustering.Source: 2) Training Dynamics Analysis · pp. 9–10, Figure 6 · p. 10

3) Code Assignment Distribution:

SimCIT uses contrastive quantization to model item similarities, with experiments reporting hierarchical code differentiation and improved recommendation metrics under several hyperparameter settings. The conclusion notes that its current focus is top-one neighborhood alignment.

3) Code Assignment Distribution:

  • Three codebook layers maintain hierarchical differentiation, with each visualized layer separating items into 24 categories or sub-clusters.The authors interpret this as evidence that SimCIT learns a multi-granular item taxonomy.Source: 3) Code Assignment Distribution · p. 10

4) Sensitivity Analysis of Temperature τ, Batch Size and Training Epochs:

5) Sensitivity Analysis of Codebooks:

  • NDCG trends upward as codebook size and codebook count increase; embedding dimension increases from 128 to 512 improve metrics, while dimension 256 slightly degrades performance due to overfitting.These sensitivity results are reported for codebook size K, number of codebooks M, and embedding dimension d.Source: 5) Sensitivity Analysis of Codebooks · p. 10, Figure 7 · p. 11

V. CONCLUSION

  • SimCIT’s contrastive loss maximizes top-one probability within a batch to capture item similarities; the experiments demonstrate its effectiveness for generative recommendation.The method currently focuses on top-one neighborhood alignment, which the authors identify as a direction for future extension.Source: §V · p. 10

Related papers

Loading 2506.16683v1…