Source-linked AI summary
From a Static Multi-Level Small Semantic Codebook to a Dynamic Single-Level Large Semantic Codebook for Generative Recommendation
Tianlu Xie, Xin Ku, Mingjie Sun, Yunhao Sha, Lixiang Wang, Peng Wang, Yiyu Wang, Wenjin Wu, Zhaojie Liu, Peng Jiang, Wenwu Ou
TL;DR
Generative recommenders face costly sequential decoding and potentially sparse, drifting multi-level Semantic ID codebooks. The paper replaces residual semantic codes with one large semantic token plus a collaborative disambiguation token, adds exposure-aware dynamic updates and offline screening, and reports improved recommendation quality, serving efficiency, and online consumption.
Problem
Multi-level residual SIDs increase autoregressive decoding cost and can create conditionally sparse spaces, while static codebooks become misaligned with changing items and exposure distributions.
Method
The paper uses a single-level large semantic codebook with a separate collaborative disambiguation code, exposure-aware dynamic updates, and multidimensional offline codebook evaluation.
Results
Across two public datasets, the proposed architecture improves mean Recall@10 by 5.0%–8.8% and mean NDCG@10 by 4.1%–5.1% for OneRec-V1 and by 7.1%–8.7% and 3.8%–8.5% for OneRec-V2.
Takeaways & Limitations
The shorter SID reduces estimated autoregressive-decoding FLOPs by 47.93%–48.70%, increases single-card QPS by 28.57%–47.0%, and improves the primary consumption metric by 0.792% in a five-day online A/B test.
Takeaways & Limitations
Offline codebook measurements screen candidates, but downstream recommendation experiments remain necessary to establish task-level effectiveness.
Abstract
from arXiv · showhide
Generative recommendation represents each item with a sequence of discrete Semantic IDs (SIDs) and predicts the sequence to retrieve the next item. Typical systems use multi-level residual quantization, which increases autoregressive decoding cost and creates a large hierarchical space that may be sparsely occupied. Static codebooks also become misaligned with current traffic as new items arrive and exposure distributions change. We propose a single-level large semantic codebook that replaces multiple residual semantic codes with one semantic token while retaining a separate collaborative disambiguation token to reduce item collisions. We further introduce an exposure-aware dynamic update mechanism based on temporal weight decay, exponential moving-average center updates, and an exposure-weighted penalty on SID changes. We also develop an offline evaluation framework covering representation quality, code utilization, cluster load, full-SID collision, and temporal stability. On two public datasets, the two-level SID improves mean Recall@10 by 5.0%-8.8% and mean NDCG@10 by 4.1%-5.1% for OneRec-V1, and by 7.1%-8.7% and 3.8%-8.5%, respectively, for OneRec-V2. Dynamic updating provides further gains on KuaiRec. Across three serving architectures, the shorter SID reduces estimated autoregressive-decoding FLOPs by 47.93%-48.70% and increases single-card QPS by 28.57%-47.0%. A five-day online A/B test serving 2.5% of production traffic improves the primary consumption metric by 0.792%.
1 Introduction
The paper identifies inefficiencies in multi-level semantic IDs: conditional sparsity expands the hierarchical search space while adding sequential decoding steps. It proposes a shorter semantic-plus-disambiguation SID, exposure-aware dynamic updates, and offline codebook screening, with gains in recommendation quality and serving efficiency.
- Motivation: Multi-level residual quantization adds sequential prediction steps and enlarges beam-search processes during autoregressive SID generation.Each later code refines the residual representation, but every additional level increases decoding cost.
- Motivation: 93.31% of the SID2 vocabulary appears globally, but each active SID1 uses only 2.48% on average, creating a conditionally sparse hierarchical space.The analysis covers approximately 1.50 billion industrial samples.
- Method: The proposed SID replaces multiple residual semantic codes with one large semantic code and retains a collaborative disambiguation code to reduce collisions.The sequence is shortened from three codes to two, reducing autoregressive decoding steps.
- Method: The exposure-aware dynamic codebook uses temporal weight decay, exponential moving-average center updates, and an exposure-weighted penalty on SID changes.These mechanisms balance adaptation to changing traffic with stability for high-exposure items.
- Evaluation: The offline framework evaluates representation quality, code utilization, cluster load, full-SID collision, and temporal stability before costly retokenization and downstream retraining.It screens codebook candidates without requiring a full recommender training run for every candidate.
- Results: The two-level SID improves mean Recall@10 by 5.0%–8.8% and mean NDCG@10 by 4.1%–5.1% for OneRec-V1 across two public datasets.The reported result is an average across the two public datasets.
2 Related Work
Related work frames Semantic IDs as a tokenization problem involving semantics, collaboration, code capacity, and valid generation. It also connects SID evaluation with codebook diagnostics and streaming or adaptive clustering for non-stationary recommendation.
- Semantic ID construction: Codebook-based Semantic IDs offer compact, compositional item identifiers between atomic identifiers and textual identifiers.The passage contrasts distinctiveness without relations against semantics that may not map uniquely to catalog items.
- Recommendation-aligned tokenization: Recommendation-aligned tokenization adds semantic, collaborative, and assignment-diversity objectives beyond reconstruction fidelity.The cited methods integrate these signals into tokenizer or recommender training.
- Codebook capacity: Finite scalar quantization and lookup-free quantization increase discrete token capacity without relying exclusively on deeper learned code stacks.These approaches address large-vocabulary construction through implicit or lookup-free discrete representations.
- SID generation: Hierarchical SIDs are usually generated left to right with beam search and catalog constraints because only a small subset of code sequences identifies catalog items.Parallel alternatives still use graph-guided decoding or adapted beam search to recover valid identifiers.
- Streaming clustering: Streaming clustering updates assignments and centers incrementally, while forgetting factors reduce the influence of older observations.These mechanisms address continual item arrival and shifting cluster statistics.
- Adaptive recommendation indexing: Adaptive recommendation indexing explicitly models temporal change through dynamic identifiers or adaptive clusters for skewed, non-stationary item distributions.The related systems monitor changing content or cluster occupancy while constructing recommendation indices.
- Evaluation: Existing task-level evaluation reports Recall and NDCG but provides limited evidence about which SID property explains performance differences.Codebook-quality work separately examines reconstruction, usage, collapse, inactive codes, and assignment imbalance.
3 Method
The method replaces two semantic residual-quantization levels with one large semantic code while retaining a deterministic disambiguation token. Exposure-aware dynamic updates adapt the semantic codebook to changing traffic, and offline metrics screen candidates before downstream training.
- Single-level semantic codebook: The framework represents each item with a two-token SID comprising one semantic code and one disambiguation code.The semantic code is selected from a large codebook, while the stable item key is used only for disambiguation.
- Single-level semantic codebook: The single large semantic codebook replaces the two semantic residual-quantization levels while retaining the existing disambiguation token.The larger vocabulary concentrates semantic capacity in one code without adding another autoregressive position.
- Disambiguation: The deterministic disambiguation rule separates items sharing a semantic code and remains fixed across codebook updates.It does not participate in semantic reconstruction, and unchanged items retain the same disambiguation token when their semantic assignment changes.
- Autoregressive generation: The design reduces autoregressive predictions from three to two by removing the second semantic-token factor while preserving disambiguation.For user history H, the target factorizes into semantic-code prediction followed by conditional disambiguation-code prediction.
- Dynamic updating: Exposure-aware updates maintain decayed item weights, update active centers, and penalize exposure-weighted SID1 switches.The penalty discourages changes for highly exposed items while allowing reassignment when quantization distance improves sufficiently; SID2 remains unchanged.
- Offline evaluation: The offline framework evaluates reconstruction quality, code utilization, cluster load, full-SID collisions, and temporal stability before downstream recommendation experiments.Representation quality uses reconstruction cosine similarity, while cluster-load statistics include the 95th percentile, maximum, and coefficient of variation.
4 Experiments
Experiments compare static two-level and three-level SIDs across public datasets and model architectures, then evaluate dynamic updating, codebook quality, and serving efficiency. The two-level design generally improves recommendation metrics while shortening decoding and increasing throughput.
- Experimental setup: The experiments use Amazon Beauty and KuaiRec, with chronological preprocessing, paired codebook comparisons, and three-seed reporting.S3 uses [256, 256, 256], while S2, PV-S2, and Dynamic PV-S2 use [1024, 512].
- Offline codebook quality: PV-S2 reduces maximum group load from 376 to 248 and load CV from 1.9938 to 1.6390 among single-level variants.S2 has the highest mean reconstruction cosine similarity, and every semantic level retains at least 94.92% utilization.
- Static comparison on KuaiRec: On KuaiRec, S2 improves OneRec-V1 Recall@10 and NDCG@10 over S3 by 8.8% and 5.1%, while OneRec-V2 gains 7.1% and 3.8%.PV-S2 also improves both OneRec variants, but neither two-level variant dominates uniformly across models and ranking metrics.
- Dynamic comparison: Dynamic C2 improves OneRec-V1 Recall@10 by 1.4% and NDCG@10 by 7.0% relative to Static C1, with corresponding OneRec-V2 gains of 2.7% and 2.7%.The update changes SID1 for 46 of 10,728 items, a 0.4288% item-level change rate, while keeping assignments largely stable.
- Industrial evaluation: Across decoder, LazyAR, and MTP architectures, the shorter SID decreases estimated autoregressive-decoding FLOPs by 47.93%–48.70% and raises single-card QPS by 28.57%–47.0%.A five-day online A/B test serving 2.5% of production traffic increased the primary consumption metric by 0.792%.
5 Conclusion
The paper replaces multi-level residual semantic representations with a single large semantic codebook plus collaborative disambiguation, adds exposure-aware dynamic updates, and introduces offline codebook screening. It reports gains in recommendation quality, serving efficiency, and online consumption.
- The proposed SID uses one large semantic code and a separate collaborative disambiguation code to reduce full-SID collisions.This replaces the multi-level residual semantic representation while retaining disambiguation separately from semantic reconstruction.
- The exposure-aware dynamic codebook combines temporally decayed exposure weights, stability-aware assignment, and moving-average center updates.
- The offline evaluation framework measures reconstruction quality, code utilization and load, full-SID collisions, and temporal stability before downstream training.
- 5.0%–8.8% mean Recall@10 and 4.1%–5.1% mean NDCG@10 gains are reported for OneRec-V1 across two public datasets.
- 47.93%–48.70% lower estimated autoregressive-decoding FLOPs and 28.57%–47.0% higher single-card QPS are reported across three serving architectures.
- A five-day online A/B test serving 2.5% of production traffic improves the primary consumption metric by 0.792%.
A Inference-Cost Derivation
The derivation estimates decoder-core FLOPs for standard and shortened SIDs by summing per-step projection, attention, feed-forward, and vocabulary costs. It separately accounts for KV-cache movement because memory access is not a floating-point operation.
- The FLOP convention counts one multiplication and one addition as two floating-point operations.
- At decoding step j, the cost depends on decoder layers, model widths, context length, beam count, and output vocabulary size.
- Per-step decoder cost combines cross-attention, self-attention with KV caching, two feed-forward blocks, and vocabulary projection.
- For q generated tokens, the decoder-core cost is compared with the shortened schedule after removing the final autoregressive level.
- The reported decoder-core reduction is 1 − Fq−1/Fq, excluding computation shared by both SID structures.
- KV-cache occupancy and logical beam-search reads are reported separately from FLOPs because memory access is hardware dependent.
B OneRec Public-Data Model Configurations
The appendix describes the public-data configurations used for OneRec-V1 and OneRec-V2. These configurations are public-data adaptations rather than parameter-exact reproductions of production systems.
- The reported OneRec configurations are public-data adaptations of the corresponding architectures, not parameter-exact reproductions of production systems.
C OneRec Results on Amazon Beauty
The Amazon Beauty results are documented through configuration tables and repeated-run result tables. The reported entries provide means and sample standard deviations over three seeds.
- The OneRec-V1 Amazon Beauty results are provided as exact values underlying Figure 3.
- Each OneRec-V1 result entry reports the mean ± sample standard deviation over seeds {7, 42, 2024}.
- OneRec-V1 and OneRec-V2 Amazon Beauty configurations are provided in Tables 7 and 8.
- The OneRec-V2 Amazon Beauty results are reported in Table 10.
D OneRec Results on KuaiRec
This section reports the exact numerical values underlying Figure 4 for the OneRec-V1/V2 sweeps on KuaiRec. Each entry is summarized as a three-seed mean with a shaded sample standard deviation.
- Figure 4 presents the complete OneRec-V1/V2 sweeps over three static SID variants.The corresponding numerical values are provided in Appendix D.
- The reported curves use three-seed means, with shaded regions indicating sample standard deviation.
- Tables 11 and 12 provide the exact values underlying Figure 4.
E OneRec Fixed-Date Static–Dynamic Results on KuaiRec
This section provides fixed-date KuaiRec comparisons for OneRec-V1 and OneRec-V2. Tables 13 and 14 contain the exact values underlying Figure 6, reported as mean ± sample standard deviation across three seeds.
- Figure 6: Tables 13 and 14 provide the exact values underlying Figure 6.
- Evaluation protocol: Each entry is reported as the mean ± sample standard deviation over seeds {7, 42, 2024}.
- OneRec-V1: Table 13 reports OneRec-V1 results in the fixed-date KuaiRec comparison.
- OneRec-V2: Table 14 reports OneRec-V2 results in the fixed-date KuaiRec comparison.