PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiable Semantic ID for Generative Recommendation

Junchen Fu, Xuri Ge, Alexandros Karatzoglou, Ioannis Arapakis, Suzan Verberne, Joemon M. Jose, Zhaochun Ren

arXiv:2601.19711v3cs.IR

TL;DR

Generative recommendation often freezes semantic IDs learned for reconstruction, leaving them mismatched with the recommendation objective. DIGER jointly learns IDs with a recommender using exploratory code selection and uncertainty decay, and reports improved codebook utilization and competitive public-benchmark results.

  • Problem

    Semantic IDs are typically learned for reconstruction and then frozen, leaving semantic indexing misaligned with the recommendation objective.

    Source: §1 · p. 2, §9 · p. 8

  • Method

    DIGER jointly optimizes semantic IDs with a generative recommender through exploratory learning and two uncertainty-decay strategies.

    Source: §1 · p. 2

  • Results

    DIGER stabilizes training, prevents semantic ID collapse, improves codebook utilization, and achieves competitive results on public benchmarks.

    Source: §9 · p. 9

  • Takeaways & Limitations

    The findings support extending differentiable semantic IDs to user-side or interaction-level structures and richer recommendation signals as future directions.

    Source: §9 · p. 9

    Integration of external collaborative embeddings is outside DIGER’s focus and left for future work.

    Source: §7 · p. 7

Abstract

from arXiv · show

Generative recommendation provides a novel paradigm in which each item is represented by a discrete semantic ID (SID) learned from rich content. Most existing methods treat SIDs as predefined and train recommenders under static indexing. In practice, SIDs are typically optimized only for content reconstruction rather than recommendation accuracy. This leads to an objective mismatch: the system optimizes an indexing loss to learn the SID and a recommendation loss for interaction prediction, but because the tokenizer is trained independently, the recommendation loss cannot update it. A natural approach is to make semantic indexing differentiable so that recommendation gradients can directly influence SID learning, but this often causes codebook collapse, where only a few codes are used. We attribute this issue to early deterministic assignments that limit codebook exploration, resulting in imbalance and unstable optimization. In this paper, we propose DIGER (Differentiable Semantic ID for Generative Recommendation), a first step toward effective differentiable semantic IDs for generative recommendation. DIGER introduces Gumbel noise to explicitly encourage early-stage exploration over codes, mitigating codebook collapse and improving code utilization. To balance exploration and convergence, we further design two uncertainty decay strategies that gradually reduce the Gumbel noise, enabling a smooth transition from early exploration to exploitation of learned SIDs. Extensive experiments on multiple public datasets demonstrate consistent improvements from differentiable semantic IDs. These results confirm the effectiveness of aligning indexing and recommendation objectives through differentiable SIDs and highlight differentiable semantic indexing as a promising research direction. Our code is released under https://github.com/junchen-fu/DIGER.

1 Introduction

Generative recommendation uses content-derived semantic IDs for next-ID generation, but frozen, reconstruction-trained IDs are misaligned with recommendation objectives. DIGER enables joint optimization and uses Gumbel-driven exploration with uncertainty decay to mitigate codebook collapse.

1 Introduction

  • Generative recommendation represents content-rich items with discrete semantic IDs and predicts the next ID, offering compact indexing, semantic interpretability, and a sequence-modeling interface.The paper notes that recent methods commonly use textual descriptions to construct these IDs.Source: §1 · p. 1
  • Conventional pipelines train fixed semantic IDs for content reconstruction and independently train recommenders for next-ID prediction, preventing recommendation gradients from reaching the tokenizer.This leaves indexing misaligned with recommendation utility, limiting personalization and preference-aware representation learning.Source: §1 · pp. 1–2
  • Naive straight-through differentiable optimization can cause early codebook collapse, volatile code usage, and degraded recommendation quality, including NDCG@10.Early overconfidence lets a small subset of codes dominate while others are rarely selected.Source: §1 · p. 2
  • DIGER jointly learns semantic IDs and a generative recommender through DRIL, using Gumbel noise and two uncertainty-decay strategies to balance early exploration with later convergence.The noise encourages early exploration across codes, while uncertainty decay gradually reduces stochasticity to align training with its objective.Source: §1 · p. 2
  • Experiments on three public datasets demonstrate the effectiveness of jointly training recommenders with differentiable semantic IDs.The paper presents DIGER as a pioneering framework for direct joint optimization of semantic IDs and generative recommenders.Source: §1 · p. 2

2 Related Work

Prior work jointly optimizes recommendation representations and increasingly uses content-informed semantic IDs in generative recommenders, but direct gradient alignment through discrete semantic indexing remains unresolved.

Joint Optimization for Recommendation

  • Modality-based sequential recommenders jointly train encoders to address representation–task mismatch, including visual backbones [3], language encoders, and LLM-based recommenders [5].Joint optimization lets recommendation gradients update representation modules, though optimization can be challenging for large encoders or discrete components.Source: §2 · pp. 2–3

Generative Recommendation

  • Generative recommenders generate discrete item IDs [15] [24] [28] [31] [39] [40] [42] [46] [49] [51]; semantic-ID examples include P5 [8], TIGER [34], and LETTER [41].Relatively few studies investigate the objective mismatch between semantic-ID tokenizers and downstream generative recommenders.Source: §2 · p. 3
  • ETEGRec [29] alternates distillation to handle discrete IDs, leaving indexing without direct recommendation gradients; this work instead makes semantic IDs differentiable for joint optimization.Because recommendation loss cannot backpropagate through discrete indexing, ETEGRec optimizes the indexing module through indirect surrogate objectives.Source: §2 · p. 3

3 Preliminaries

Generative recommendation predicts the next item’s Semantic ID (SID) autoregressively from a user’s interaction history. The preliminaries define how item SIDs are encoded, predicted, and used for next-item recommendation.

3 Preliminaries

  • Following [34], generative recommendation predicts the next item’s SID autoregressively from a sequence formed by concatenating previously interacted items’ SIDs in chronological order.A sequence model, such as a Transformer decoder, learns the conditional distribution of the next-item SID.Source: §3 · p. 3
  • An RQ-VAE tokenizer maps each item to a length-m SID of discrete codes from a K-token vocabulary, with each code corresponding to an entry in a shared embedding/codebook.The resulting discrete codes are fed as tokens into the generative model.Source: §3 · p. 3
  • Training minimizes the negative log-likelihood of the ground-truth next-item SID, and inference maps the generated SID back to an item via the codebook or retrieval.Retrieval can search items sharing the generated SID, enabling next-item recommendation.Source: §3 · p. 3

4 Our Method: DIGER

DIGER replaces separate indexing and recommendation training with joint optimization, using Gumbel-noise exploration followed by uncertainty decay to learn differentiable semantic IDs and align training with inference.

4.1 DRIL: Differentiable Semantic ID with Exploratory Learning

  • Gumbel-noise DRIL counters premature codebook collapse by exploring discrete codes while preserving differentiable updates to the codebook.It uses hard assignments for SID indexing and soft Gumbel-Softmax probabilities for gradient updates; the noise supports stochastic code selection.Source: §4.1 · pp. 3–4, Figure 3 · p. 4

4.2 Two Uncertainty Decay Strategies

  • SDUD reduces noise as the generative loss falls, while FrqUD adds noise to overused codes and keeps underused codes deterministic.SDUD’s λ controls when the noise scale reaches zero; FrqUD uses smoothed code frequencies relative to a uniform-use threshold. The method keeps temperature fixed and instead reduces uncertainty toward zero.Source: §4.2 · p. 5

4.3 Model Training

  • Joint training optimizes next-SID generation with RQ-VAE quantization and reconstruction losses, while omitting the decoder and any code-diversity loss.The RQ-VAE losses stabilize optimization, while the generative loss primarily drives parameter updates.Source: §4.3 · p. 5

5 Experimental Setup

The experimental setup evaluates DIGER on three public recommendation datasets using leave-one-out testing, established baselines, and a T5-style generative recommender. It specifies dataset processing, evaluation metrics, and implementation and tuning choices.

Datasets

  • Experiments use B-Shop, I-Shop, and Yelp, encoding product text or restaurant names and categories while reserving review texts for interaction records.Item content is generated from original titles and descriptions with LLaMA-7B following [41]; users and items with fewer than five interactions are filtered [20] [17].Source: §5 · pp. 5–6

Evaluation

  • Evaluation uses Recall@10 and NDCG@10 under leave-one-out testing, ranking each predicted item against the full item set.All reported results are computed on the test set; the protocol follows [13] [29] [41].Source: §5 · p. 6

Baselines

  • Comparisons include STE [2], collaborative and sequential recommenders, and generative methods, with TIGER [34] as the main two-stage reference.Collaborative and sequential baselines are MF, LightGCN [12], Caser [37], HGN [33], SASRec [20], and BERT4Rec [35]; generative baselines also include BIGRec, P5 [8], LETTER [41], and ETEGRec [29]. Methods are tuned per dataset on validation data, with most results taken from [41] under matching configurations.Source: §5 · p. 6

Implementation Details

  • Implementation initializes the tokenizer with pretrained RQ-VAE, uses codebook size K=256 and sequence length m=3 plus a conflict code, and trains a 6-layer-per-side T5-style model.Training uses AdamW with weight decay 0.05 and validation-based early stopping; Gumbel temperature τ is tuned over {0.5, 1.0, 2.0, 4.0} and set to 2.0. The setup fixes β=0.25, tunes recommender and tokenizer learning rates and r, and searches λ over {1.0, 1.2, 1.4, 1.8, 2.0}; experiments use two H100 GPUs.Source: §5 · p. 6

6 Experimental results

Across three benchmarks, DIGER improves on two-stage training and performs at or near the best listed recommenders, while the naive STE approach performs poorly. The results support differentiable semantic IDs with stable joint optimization.

6.1 Comparison with differentiable SID and conventional (RQ1)

  • On B-Shop, DIGER R@10 is 0.0657–0.0696 versus Two-Stage 0.0610, and N@10 is 0.0361–0.0376 versus 0.0331.The passage reports similar gains on I-Shop and Yelp.Source: §6.1 · p. 6, Table 2 · p. 7
  • STE [2] performs poorly on every dataset (B-Shop R@10 0.0134), consistent with codebook collapse; DIGER's joint optimization enables measurable gains over Two-Stage [34].The results indicate that differentiability alone is insufficient for stable joint training.Source: §6.1 · p. 6

6.2 Comparison with SOTA baselines (RQ2)

  • DIGER surpasses TIGER across all datasets; ETEGRec slightly beats TIGER but remains behind DIGER despite requiring roughly twice the training time.The passage attributes DIGER's remaining advantage to direct gradient-based joint optimization.Source: §6.2 · pp. 6–7
  • DIGER leads all listed baselines on B-Shop and I-Shop, wins Yelp R@10 (0.0432), and is comparable to LETTER on Yelp N@10 (0.0227 vs 0.0231).LETTER uses additional collaborative signals, whereas DIGER is text-only.Source: §6.2 · pp. 6–7, Table 3 · p. 7

7 Ablation Study (RQ3)

The ablations attribute DIGER’s effectiveness to controlled stochastic code selection, soft codebook updates, and uncertainty decay. They also show robustness to reasonable hyperparameter choices and a favorable balance at K=256 and m=3.

Effect of core components

  • External collaborative embeddings were left for future work because incorporating them was considered orthogonal to this study’s focus.The paper frames their inclusion as model ensembling or auxiliary information injection.Source: §7 · p. 7
  • Removing uncertainty decay or Gumbel noise reduced performance; soft updates outperformed STE, and Gaussian noise degraded more than Gumbel noise.Further removing the soft codebook update caused slight additional degradation, while temperature annealing produced similar but slightly inferior results. Overall, DIGER’s effectiveness stems from the joint design of controlled stochastic code selection, soft updates, and uncertainty decay.Source: §7 · pp. 7–8, Table 4 · p. 7

Sensitivity to Uncertainty Decay hyperparameters

  • DIGER remained stable across broad ranges of λ and r, while extreme settings weakened exploration or convergence.Very small λ maintained excessive uncertainty late in training, whereas overly large λ suppressed early exploration; moderate r balanced the two effects. Performance varied little within reasonable ranges.Source: §7 · p. 7, Figure 5 · p. 8

Codebook capacity and semantic ID length

  • Performance peaked at K=256; reducing m to 2 caused a substantial drop, while increasing m to 4 yielded only marginal gains.Both smaller and larger codebooks performed worse, reflecting a tradeoff between representational capacity and optimization stability; K=256 and m=3 provided a favorable balance.Source: §7 · pp. 7–8, Table 5 · p. 8

8 Semantic ID Dynamics and Stability (RQ4)

During joint training, uncertainty decay makes semantic ID evolution more stable and effective. It aligns training and inference assignments, limits abrupt drift and code collapse, and promotes balanced code usage.

SID drift during joint training

  • Naive STE shows larger, more abrupt incremental SID drift than DIGER, while uncertainty-decay variants keep cumulative drift below 40%.DIGER without uncertainty decay has the smallest cumulative drift, below 15%, suggesting limited adaptation; uncertainty-decay variants perform better.Source: §8 · p. 8

Training–inference agreement

  • Uncertainty decay rapidly raises training–inference SID agreement after an early phase; FrqUD maintains high agreement throughout, while SDUD increases it as Gumbel noise falls.FrqUD applies stochastic sampling only to overused codes from the beginning; SDUD progressively reduces the Gumbel noise standard deviation σ.Source: §8 · p. 8

Code usage distribution

  • At higher quantization layers, uncertainty-decay DIGER uses codes most evenly, whereas STE has severe code collapse; usage is relatively uniform at the first layer across methods.DIGER without uncertainty decay shows moderate imbalance, with collapse in STE especially pronounced in deeper layers.Source: §8 · p. 8

Overall finding

  • Uncertainty decay stabilizes SID refinement, aligns training and inference assignments, prevents abrupt drift and code collapse, and supports reliable inference.Overall, it enables controlled refinement of semantic IDs during joint optimization.Source: §8 · p. 8

9 Conclusions

DIGER addresses the mismatch between semantic indexing and recommendation by making semantic IDs differentiable and guiding their exploration-to-exploitation transition. The conclusion reports improved training stability and codebook utilization, competitive public-benchmark results, and directions for future work.

9 Conclusions

  • DIGER’s DRIL supports effective differentiable semantic ID exploration, while two uncertainty decay strategies guide a smooth transition from exploration to exploitation.This addresses the objective mismatch caused when semantic IDs are learned for reconstruction and then frozen; the naive straight-through estimator approach suffers from severe code collapse.Source: §9 · pp. 8–9
  • DIGER stabilizes training, prevents semantic ID collapse, improves codebook utilization, and achieves competitive results on public benchmarks.Source: §9 · p. 9
  • Future work includes differentiable user-side or interaction-level structures, alternative discrete-latent estimators and optimization, and integration with richer collaborative signals or LLM-based recommendation.These directions aim to improve stability and expressivity and advance generative recommendation through joint SID and recommender training.Source: §9 · p. 9

A Theoretical Analysis

The analysis formalizes how freezing semantic indexing can restrict recommendation optimization and cause objective-mismatch gaps, while entropy maximization favors broader codebook utilization.

Theoretical justification

  • End-to-end optimization is no worse than two-stage freezing, and is strictly better when the restricted indexing set is compact and excludes every global minimizer.The two-stage value minimizes the recommendation value function over the restricted set A, whereas end-to-end training minimizes it over all Φ.Source: A Theoretical Analysis · p. 9
  • Objective mismatch can make the optimally trained two-stage solution arbitrarily worse than joint optimization, even under exact optimization.A continuous construction gives joint optimum value 0 and frozen-indexing value 1; scaling the recommendation loss by M makes the gap M.Source: A Theoretical Analysis · p. 9

Noise/entropy and codebook utilization

  • Effective code usage exp(H(q)) is uniquely maximized at uniform assignments, where it equals K; an entropy bonus therefore promotes better utilization independent of the optimizer.Equivalently, H(q) ≤ log K, with equality only when each code has marginal usage 1/K.Source: A Theoretical Analysis · pp. 9–10

B Derivation of the optimal 𝜎.

The appendix derives the optimal σ by differentiating Lσ while treating Lgen and λ as constants. Setting the derivative to zero yields a positive-root solution, with σ≥0 imposed as a practical constraint.

B Derivation of the optimal 𝜎.

  • Treating Lgen and λ as constants, differentiation and the zero-derivative condition give (σ + λ)^2 = Lgen.The derivation multiplies the stationary equation by (σ + λ)^3.Source: B Derivation of the optimal 𝜎 · p. 10
  • Because log(σ + λ) requires σ + λ > 0, the derivation takes the positive root and then imposes the practical constraint σ ≥ 0.Under that constraint, the appendix states that the closed-form solution is obtained.Source: B Derivation of the optimal 𝜎 · p. 10

Related papers

Loading 2601.19711v3…