Source-linked AI summary
OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment
Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, Guorui Zhou
TL;DR
Existing recommender systems commonly separate retrieval and ranking, while generative retrieval often serves only as a retrieval-stage selector. OneRec unifies recommendation in an end-to-end generative model using encoder-decoder modeling, session-wise generation, and iterative preference alignment; online evaluation reports a 1.68% improvement in total watch time.
Problem
Existing recommender systems commonly use cascaded retrieve-and-rank pipelines, and generative retrieval models typically function only during candidate selection.
Method
OneRec uses an encoder-decoder generative recommender with sparse Mixture-of-Experts, session-wise list generation, and iterative Direct Preference Optimization using reward-model-based self-hard negatives.
Results
1.68% improvement in total watch time was achieved by OneRec in online evaluation against a current multi-stage recommender system.
Takeaways & Limitations
Session-wise modeling captures dependencies among recommended videos, while iterative preference alignment improves generalization across diverse user preference patterns.
Takeaways & Limitations
The session-quality training data requires users to watch at least five videos and exceed a total-watch-duration threshold, limiting the defined session scope.
Abstract
from arXiv · showhide
Recently, generative retrieval-based recommendation systems have emerged as a promising paradigm. However, most modern recommender systems adopt a retrieve-and-rank strategy, where the generative model functions only as a selector during the retrieval stage. In this paper, we propose OneRec, which replaces the cascaded learning framework with a unified generative model. To the best of our knowledge, this is the first end-to-end generative model that significantly surpasses current complex and well-designed recommender systems in real-world scenarios. Specifically, OneRec includes: 1) an encoder-decoder structure, which encodes the user's historical behavior sequences and gradually decodes the videos that the user may be interested in. We adopt sparse Mixture-of-Experts (MoE) to scale model capacity without proportionally increasing computational FLOPs. 2) a session-wise generation approach. In contrast to traditional next-item prediction, we propose a session-wise generation, which is more elegant and contextually coherent than point-by-point generation that relies on hand-crafted rules to properly combine the generated results. 3) an Iterative Preference Alignment module combined with Direct Preference Optimization (DPO) to enhance the quality of the generated results. Unlike DPO in NLP, a recommendation system typically has only one opportunity to display results for each user's browsing request, making it impossible to obtain positive and negative samples simultaneously. To address this limitation, We design a reward model to simulate user generation and customize the sampling strategy. Extensive experiments have demonstrated that a limited number of DPO samples can align user interest preferences and significantly improve the quality of generated results. We deployed OneRec in the main scene of Kuaishou, achieving a 1.6\% increase in watch-time, which is a substantial improvement.
1 Introduction
OneRec replaces the conventional cascade ranking pipeline with a unified, single-stage generative recommender. It combines scalable model capacity, session-wise generation, and preference optimization to improve recommendation quality.
- Cascade ranking separates recall, pre-ranking, and ranking into stages that each select top-k items, balancing response time and sorting accuracy.
- Existing independently trained ranking stages limit overall performance because each stage bounds the effectiveness of the subsequent stage.
- OneRec introduces a unified end-to-end generative framework that replaces the traditional multi-stage ranking pipeline with single-stage recommendation.
- Its encoder-decoder architecture uses sparse Mixture-of-Experts to scale recommendation-model capacity without proportionally increasing computation.
- Session-wise generation models the relative content and order of recommended items, avoiding hand-crafted strategies needed to combine point-by-point predictions coherently and diversely.
- Iterative preference optimization with personalized self-hard negative sampling improves generalization across a broader range of user preferences.
2 Related Work
Related work covers generative retrieval, semantic item indexing, and preference alignment methods developed for language models. OneRec’s framework connects these research directions through session generation and iterative preference optimization.
- Generative retrieval formulates retrieval as sequential generation of relevant document tokens, contrasting with embedding-based retrieval using two-tower scoring and MIPS or ANN search.
- Semantic indexing methods tokenize items using content-derived representations, including residual quantization and hierarchical clustering of textual embeddings.
- RLHF aligns language models with human values through reward-model-guided reinforcement learning, while DPO directly optimizes preference data in closed form.
- OneRec’s framework is organized into session training and iterative preference optimization with self-hard negatives.
3 Methods
OneRec is an end-to-end single-stage generative recommendation framework that encodes user histories and generates coherent video sessions. It combines balanced semantic indexing, an encoder-decoder with sparse MoE, and iterative preference alignment using a reward model and DPO.
- Framework: OneRec constructs a single-stage generative recommendation pipeline from user behavior features, semantic item indexing, session-wise generation, and iterative preference alignment.The framework is organized around feature engineering, session-wise generation, and self-hard negative sampling with direct preference optimization.
- Feature Engineering: User histories contain videos watched or interacted with through likes, follows, or shares, and the model outputs a session list of m videos.The behavior sequence has length n, while the generated session contains m videos.
- Feature Engineering: Balanced residual K-Means transforms multimodal item embeddings into semantic tokens to address the unbalanced code distribution known as the hourglass phenomenon.Balanced K-means partitions videos into equal-sized clusters by assigning nearest unallocated videos and recalibrating centroids until convergence.
- Session-wise List Generation: Session-wise generation predicts a list of typically 5 to 10 videos, allowing the model to capture dependencies, user interest, coherence, and diversity within recommendations.High-quality sessions require at least five watched videos, sufficient total watch duration, and interactions such as liking, collecting, or sharing.
- Session-wise List Generation: A Transformer encoder models historical interactions while its decoder autoregressively generates target-session semantic IDs, using sparse MoE experts to scale capacity efficiently.Only K_MoE of N_MoE experts process each token, preserving computational efficiency within an MoE layer.
- Iterative Preference Alignment: Iterative Preference Alignment trains successive models by generating beam-search responses, scoring them with a personalized reward model, selecting best and worst responses, and applying DPO with NTP loss.The reward model uses target-aware item representations, self-attention across session items, multitarget prediction towers, and binary cross-entropy training; preference alignment samples 1% of data for efficiency.
4 System Deployment
OneRec was deployed for online services using an architecture with training, serving, and DPO sample-server components.
- OneRec-1B was deployed for online services to balance stability and performance.The deployment processes interaction logs, trains a seed model with next-token prediction, and adds DPO loss for preference alignment.
5 Experiment
The experiments compare OneRec with point-wise methods and DPO variants, evaluate its modules through ablations, and validate online performance with a Kuaishou A/B test.
- The evaluation includes offline comparisons, module ablations, and an online Kuaishou A/B test.
5.1 Experimental Settings
The experimental setup specifies optimization, hardware, generation, semantic-identifier, MoE, baseline, metric, and DPO-ratio configurations. Figure 4 examines how changing the DPO sample ratio affects performance.
- Implementation: OneRec is optimized with Adam at an initial learning rate of 2×10^-4 on NVIDIA A800 GPUs.
- Implementation: The setup generates N=128 responses per user, uses K=8192 clusters, L=3 codebook layers, and fixes the DPO sample ratio at 1%.
- Baselines: The evaluation compares OneRec with SASRec, BERT4Rec, FDSA, TIGER, DPO, and several DPO variants.
- Evaluation: A pretrained reward model estimates recommended-session value across target interaction metrics, including session watch time and like probability.
- DPO Ablation: A 1% DPO training ratio produces significant gains, while further sample-ratio increases yield limited improvements.
5.2 Offline Performance
Offline results show that session-wise generation outperforms point-wise approaches, limited DPO training substantially improves the base model, and IPA exceeds alternative DPO implementations.
- Session-wise Generation: OneRec-1B achieves 1.78% higher maximum swt and 3.36% higher maximum ltr than TIGER-1B.The result is attributed to session-wise modeling, which maintains contextual coherence across recommendations while balancing coherence and diversity.
- Session-wise Generation: Session-wise generation significantly outperforms traditional dot-product and point-wise methods such as TIGER.
- Preference Alignment: With a 1% DPO training ratio, OneRec-1B+IPA surpasses OneRec-1B by 4.04% in maximum swt and 5.43% in maximum ltr.
- Preference Alignment: IPA achieves superior performance compared with alternative DPO implementations.Some DPO baselines underperform the non-aligned OneRec-1B model, while iterative self-generated-output mining is reported as more effective for preference selection.
5.3 Ablation Study
The ablations examine DPO sample-ratio efficiency and model scaling. OneRec retains most observed performance with a small DPO sample ratio, while larger models yield consistent accuracy gains.
- DPO Sample Ratio Ablation: 1% DPO sampling achieves average 95% of maximum observed performance while using only 20% of the computational resources required by higher ratios.The 5% ratio requires 5× more GPU resources than the 1% baseline, while gains beyond 1% remain insignificant.
- Model Scaling: Scaling OneRec from 0.05B to 1B produces consistent accuracy gains.Relative to OneRec-0.05B, the 0.1B model gains 14.45% maximum accuracy, followed by additional gains of 5.09%, 5.70%, and 5.69% at 0.2B, 0.5B, and 1B.
5.4 Prediction Dynamics of OneRec
OneRec’s prediction distributions shift toward preferred generation patterns after preference alignment. Across semantic-ID layers, uncertainty decreases as autoregressive decoding accumulates context.
- Prediction Distributions: Figure 5 visualizes softmax probability distributions for 8192 semantic-ID codes across layers, marking the highest-reward item’s code with a red star.The red star identifies the semantic ID of the item with the highest reward value.
- Model Scalability: Figure 6 reports that OneRec benefits from performance improvement as model parameters are scaled up.The figure summarizes the model-scaling behavior described in the scalability evaluation.
- Preference Alignment: OneRec+IPA shifts prediction confidence toward preferred generation patterns relative to the OneRec baseline.The reported shift indicates that preference alignment changes the base model’s generation distributions.
- Hierarchical Uncertainty Reduction: Entropy decreases across layers, from 6.00 in the first layer to 3.71 on average in the second and 0.048 in the third.The passage attributes this progressively concentrated distribution to autoregressive decoding and accumulated context.
5.5 Online A/B Test
The online A/B test compares OneRec with the current multi-stage recommender on Kuaishou’s main-page video recommendation traffic. OneRec improves both total watch time and average view duration.
- Online Results: 1.68% improvement in total watch time and 6.56% improvement in average view duration were observed for OneRec online.Total Watch Time measures aggregate user watching time, while Average View Duration measures average watch time per exposed video.
- Experimental Setting: The experiment compared OneRec against the current multi-stage recommender system using 1% of main traffic.The evaluation was conducted in Kuaishou’s main-page video recommendation scenario.
- Comparison: Table 2 reports absolute improvements of OneRec relative to the current multi-stage system in the online A/B-testing setting.The supplied table caption identifies the comparison and evaluation setting but provides no additional table values.
6 Conclusion
The conclusion presents OneRec as an industrial single-stage generative recommendation solution built around efficient scaling, session-wise generation, and iterative preference alignment. Offline experiments and online A/B testing verify its effectiveness and efficiency, while multi-objective modeling remains a future direction.
- Model Scaling: OneRec applies sparse MoE to scale model parameters with high computational efficiency for large-scale industrial recommendation.The conclusion frames this architecture as a scalable blueprint for industrial recommendation.
- Session-wise Generation: Session-wise generation models target-item context and captures user preference dynamics better than isolated point-wise generation.The conclusion links this design to the contextual information and sequence modeling of recommendation sessions.
- Preference Alignment: Iterative Preference Alignment is proposed to improve OneRec’s generalization across diverse user preference patterns.The conclusion lists IPA as one of OneRec’s three key contributions.
- Validation: Extensive offline experiments and online A/B testing verify OneRec’s effectiveness and efficiency.The conclusion states this validation without specifying an additional metric in the supplied passage.
- Limitations and Future Work: OneRec has limitations in interactive indicators such as likes, motivating future work on multi-objective modeling.The stated goal is to provide a better user experience through broader objective modeling.