Generative Recommender with End-to-End Learnable Item Tokenization

1 Introduction

• Existing approaches typically preprocess item tokenization separately, leaving the tokenizer unaware of recommendation objectives and limiting the recommender’s ability to fuse tokenizer knowledge.These concerns motivate a unified end-to-end framework.Source: §1 · p. 2
• ETEGRec jointly optimizes an RQ-VAE item tokenizer and a T5-like Transformer recommender in a dual encoder-decoder architecture.Unlike heuristic or pre-learned tokenizers, its tokenizer is optimized with the recommender.Source: §1 · p. 2
• Sequence-item alignment matches token distributions from sequential encoder states with the target item’s collaborative embedding.Preference-semantic alignment uses contrastive learning to align Transformer-decoder user preference with target-item semantics reconstructed by RQ-VAE.Source: §1 · p. 2
• Alternating optimization is introduced to support stable and effective end-to-end joint training.The framework’s stated aim is mutual enhancement of the tokenizer and recommender.Source: §1 · p. 2
• Experiments on three benchmarks report ETEGRec’s superiority over traditional sequential recommendation models and generative recommendation baselines.The supplied introduction gives no metric values.Source: §1 · p. 2

Sequential Recommendation

• Sequential recommendation predicts the next item from a user’s historical behavior, using architectures including RNNs, CNNs, GNNs, and Transformers.SASRec uses unidirectional attention, BERT4Rec bidirectional masked prediction, and S3Rec intrinsic data correlation for pretraining.Source: §2 · p. 2

Generative Recommendation

• Generative recommendation tokenizes item sequences and autoregressively predicts target-item tokens, with tokenization methods ranging from parameter-free approaches to multi-level vector quantization.Parameter-free methods include co-occurrence factorization, hierarchical clustering, and textual metadata identifiers; deep-learning methods include RQ-VAE-based approaches.Source: §2 · pp. 2–3
• Parameter-free tokenizers are described as efficient but may suffer length bias and miss deeper collaborative relationships, whereas multi-level VQ can produce expressive, equal-length identifiers.LETTER aligns RQ-VAE quantized embeddings with collaborative embeddings to use collaborative and semantic information.Source: §2 · pp. 2–3
• The paper identifies the separation of item tokenization and generative recommendation into independent stages as potentially suboptimal and integrates them through recommendation-oriented alignment.The stated aim is superior recommendation performance.Source: §2 · p. 3

3 Methodology

• The framework jointly optimizes item tokenization and generative recommendation.The methodology section presents this as its central framework design.Source: §3 · p. 3
• A dual encoder-decoder architecture comprises an item tokenizer and a generative recommender.The framework is illustrated in Fig. 1.Source: §3 · p. 3
• Recommendation-oriented alignment uses sequence-item and preference-semantic objectives to align the two components from different perspectives.The section then describes alternating optimization for joint component learning.Source: §3 · p. 3

3.1 Problem Formulation

• Given a user’s interaction sequence, sequential recommendation predicts the next item they are likely to interact with.The generative formulation represents that target item by an identifier to be generated.Source: §3.1 · p. 3
• Representing items by multiple tokens forms item tokenization; this paper follows RQ-VAE in using equal-length identifiers to reduce potential length bias.The sequence of interacted items is converted into a token sequence, and the target item is represented by its identifier.Source: §3.1 · p. 3
• The paper considers end-to-end learning of both the item tokenizer and recommender, rather than relying on heuristic or pre-learned tokenizers.The task is formulated as sequence-to-sequence learning.Source: §3.1 · p. 3

3.2 Dual Encoder-Decoder Architecture

• The item tokenizer maps item-level interaction sequences to tokens, which the generative recommender models before autoregressively generating recommendation tokens.The recommender uses a Transformer encoder-decoder similar to T5.Source: §3.2 · p. 3, §3.2.2 · p. 4

3.2.1 Item Tokenizer.

• The tokenizer uses hierarchical residual quantization to assign each item multiple tokens, organizing items as a tree whose shared prefixes can represent shared collaborative semantics.Its semantic quantization loss combines reconstruction and residual-quantization losses.Source: §3.2.1 · pp. 3–4

3.2.2 Generative Recommender.

• For recommendation, the encoder-decoder processes tokenized interaction sequences and predicts each target-item token autoregressively.The target-item tokens are trained with a sequence-to-sequence negative log-likelihood objective.Source: §3.2.2 · p. 4

3.3 Recommendation-oriented Alignment

• Prior approaches often fixed the tokenizer after preprocessing; ETEGRec instead proposes two alignment strategies to jointly learn the tokenizer and recommender.The stated goal is mutual enhancement between the components.Source: §3.3 · pp. 4–5

3.3.1 Sequence-Item Alignment.

• Sequence-item alignment makes the tokenizer produce similar token distributions for the encoder’s past-sequence state and the target item’s collaborative embedding.The objective uses symmetric KL divergence; the alignment loss can also improve encoder representations when the decoder might bypass the encoder.Source: §3.3.1 · p. 5

3.3.2 Preference-Semantic Alignment.

• Preference-semantic alignment links the decoder’s first hidden state, which reflects sequential user preference, with the reconstructed target-item embedding, which encodes collaborative semantics.The loss uses InfoNCE with in-batch negatives, incorporating the tokenizer through the reconstructed embedding.Source: §3.3.2 · p. 5

3.4 Alternating Optimization

• Alternating optimization trains the tokenizer with quantization and both alignment losses while the recommender is fixed, then trains the recommender with recommendation and alignment losses while the tokenizer is fixed.Each cycle updates the tokenizer in its first epoch and keeps its tokens fixed for the remaining recommender-training epochs; once the tokenizer converges, it is frozen while the recommender is trained to convergence.Source: §3.4 · pp. 5–6

3.5.1 Comparison with Existing methods.

• ETEGRec integrates tokenization and recommendation end to end, whereas heuristic and pre-learned tokenizers are used separately from recommender optimization in existing methods.The paper contrasts heuristic tokenizers, which may miss semantic relevance, with pre-learned tokenizers that are decoupled from the recommender.Source: §3.5.1 · p. 6
• ETEGRec aligns past user interactions with target items to incorporate behavioral preference information into the item tokenizer.Joint optimization produces diverse token sequences and gradually refined semantics, and the paper reports that their continuous enhancement contributes to performance.Source: §3.5.1 · p. 6

3.5.2 Complexity Analysis.

• ETEGRec's training cost is O(NLKd + N^2d + Nd^2 + Md), on the same order of magnitude as TIGER and LETTER; its inference complexity is consistent with TIGER.Item-tokenization results can be cached in advance for inference.Source: §3.5.2 · p. 6

4.1.1 Dataset.

• Experiments use three Amazon 2023 review subsets—Musical Instruments, Video Games, and Industrial Scientific—with chronological sequences capped at 50 items after 5-core filtering.The data cover reviews from May 1996 to September 2023.Source: §4.1.1 · p. 6

4.1.2 Baseline Models.

• The comparison includes traditional sequential recommenders and generative models, including Caser, SASRec, TIGER, TIGER-SAS, and LETTER.The traditional baselines span convolutional, recurrent, Transformer, and MLP approaches; generative baselines include semantic-ID and collaborative-tokenization methods.Source: §4.1.2 · p. 7

4.1.3 Evaluation Settings.

• Evaluation uses top-5 and top-10 Recall and NDCG with leave-one-out splits and full ranking over the entire item set.The latest interaction is test data, the second latest is validation data, and generative models use beam size 20.Source: §4.1.3 · p. 7

4.1.4 Semantic ID Generation.

• Item identifiers use 256-dimensional collaborative semantic embeddings from trained SASRec and an RQ-VAE tokenizer with three codebooks of 256 embeddings each.An additional token is appended to semantic tokens to ensure unique item IDs.Source: §4.1.4 · p. 7

4.1.5 Implementation Details.

• The generative recommender uses a six-layer-per-side T5 backbone, and training alternates tokenizer and recommender updates until validation performance converges.The tokenizer is initialized from a pretrained RQ-VAE; the tokenizer is trained for one epoch before recommender training for C−1 epochs per cycle.Source: §4.1.5 · p. 7

4.2 Overall Performance

• Among generative baselines, TIGER and TIGER-SAS outperform CID and SID across all three datasets, while LETTER performs best in most cases.ETEGRec consistently achieves the best results on all datasets among the compared methods.Source: §4.2 · p. 8
• ETEGRec consistently achieves the best results on all three datasets compared with baseline methods.The authors attribute the improvements to mutual enhancement between the item tokenizer and generative recommender through recommendation-oriented alignment.Source: Table 3 · p. 7, §4.2 · p. 8

4.3 Ablation Study

• Removing both alignment losses performs worse than removing either one alone, supporting positive contributions from sequence-item and preference-semantic alignment together.Removing sequence-item alignment worsens performance across datasets, while removing preference-semantic alignment also degrades performance.Source: §4.3 · p. 8
• Omitting alternating training causes a significant performance decline; the authors report that frequent tokenizer updates adversely affect recommender training.Alternating training is reported to stabilize training for both components while maintaining collaborative alignment.Source: §4.3 · p. 8
• Bypassing end-to-end optimization and retraining a recommender on ETEGRec's final item tokens shows that gains are attributed to more than the item identifiers alone.The authors also attribute gains to integrating tokenizer-encoded prior knowledge with the generative recommender.Source: §4.3 · p. 8

4.4.1 Generalizability Evaluation.

• ETEGRec outperforms LETTER and TIGER on both seen and unseen users in the evaluation using users with little interaction history as new users.The study selects 5% of users with the least history on the Instrument and Scientific datasets.Source: Figure 2 · p. 8, §4.4.1 · p. 9

4.4.2 Preference-Semantic Representation Visualization.

• In the representation visualization, preference points cluster near their corresponding target-item semantic points and remain separated from other semantic points.The analysis uses 10 items and 80 corresponding interaction histories from Instrument and Scientific datasets.Source: §4.4.2 · p. 9, Figure 3 · p. 9

4.4.3 Hyper-Parameter Analysis.

• For sequence-item alignment, the best μ is 3e-4 on Instrument and Scientific and 1e-3 on Game; the best λ is 1e-4 on all three datasets.Values of μ or λ that are too large can interfere with learning and yield suboptimal performance.Source: §4.4.3 · p. 9

• ETEGRec integrates item tokenization and generative recommendation through joint optimization.Source: §5 · p. 9
• ETEGRec outperforms traditional sequential recommendation models and generative recommendation baselines on three benchmarks.Source: §5 · p. 9
• Recommendation-oriented alignment combines sequence-item and preference-semantic objectives, while alternating optimization supports joint component learning.Source: §5 · p. 9