Unifying Generative Recall and Multi-Objective Ranking in a Single Decoder-Only Sequence

1 Introduction

• Cascaded recall and ranking use different objectives and feature organizations, compress recall information into a candidate list, and repeatedly encode user context.These issues are described as objective inconsistency, representation loss, and redundant computation in training and serving.Source: §1 · p. 2
• Transformer-based generative recall and ranking share tokenized inputs, user-context modeling, attention-based feature fusion, and deep Transformer stacks, motivating a unified sequence.Generative recall predicts hierarchical SIDs, while ranking uses attention to model heterogeneous features.Source: §1 · p. 2
• Directly sharing parameters is challenging because SID generation requires autoregressive visibility while ranking uses simultaneously available item features and dense and sparse objectives.The paper warns that sharing all parameters can create a seesaw effect between retrieval quality and ranking accuracy.Source: §1 · p. 2
• Prior collaboration methods still separate recall trajectories from fine-grained ranking features, leaving ranking with compressed representations or extra fusion networks and potentially duplicated context processing.The stated challenge is to couple representations in the forward pass while keeping optimization isolated.Source: §1 · p. 2
• The proposed decoder-only Transformer places user context, SID trajectory, and item features in one sequence, using task-specific attention and ranking-side LoRA to couple representations while isolating optimization.The generated trajectory lets ranking consume the recall decision process rather than only the discrete candidate output.Source: §1 · p. 2
• UniR² consistently improves generative recall and multi-objective ranking over deployed baselines and achieves significant gains in online A/B tests on Kuaishou services.The authors report these findings from extensive experiments on large-scale industrial data.Source: §1 · p. 2

2.1 Generative Recommendation

• Generative recommendation uses autoregressive SID generation for large-scale retrieval, while existing approaches still do not explicitly model the multi-objective scores needed in ranking.Methods discussed include path evaluation and candidate scoring, but retain a single preference signal in the cited examples.Source: §2.1 · pp. 2–3

2.2 Transformer-Based Ranking

• Transformer ranking models tokenize heterogeneous features and use attention for feature interaction, but the surveyed approaches operate only at the ranking stage.Examples include OneTrans, MixFormer, SORT, and TokenFormer.Source: §2.2 · p. 3

2.3 Unified Recall and Ranking

• Existing recall-ranking collaboration methods use scoring modules, representation transfer, or shared Transformers, but do not place recall trajectories and fine-grained ranking features in one sequence for layer-wise interaction.UniR² is described as combining them at both the token-sequence and layer-wise computation levels.Source: §2.3 · p. 3

3 Preliminary

• Items have raw features and a three-layer SID from Res-Kmeans with codebook size 8129, while observed user-item interactions carry binary labels such as click, long-view, and gift.Users are associated with personalized profiles and multiple behavior histories.Source: §3 · p. 3
• The recall task learns an autoregressive SID distribution conditioned on user context, trained with next-token prediction and served with beam search.Ranking estimates per-objective probabilities for a user-item pair and is trained with multi-objective binary cross-entropy.Source: §3 · p. 3
• Unlike cascades with separate recall and ranking parameters, the alternative learns one model that produces both the generative distribution and ranking scores in a single service pass.Source: §3 · p. 3

4 Methodology

• UniR2 unifies generative recall and multi-objective ranking in one decoder-only Transformer sequence containing user context, SID trajectory, and item features.The sequence lets ranking consume the recall decision process rather than only its discrete candidate output.Source: §4 · p. 3

4.1 Unified Sequence Composition

• The input concatenates user, generative, and ranking segments, which share Transformer parameters and contextual memory.The user segment acts as a shared memory read by downstream target tokens; ranking feature tokens summarize complementary item and user-item signals for the ranking tower.Source: §4.1 · pp. 3–4

4.2 Dual-Query Prefix-Causal Attention

• Dual-Query Prefix-Causal Attention gives SID queries access to user context and preceding SID positions, while ranking queries fuse item features with the generative trajectory.Both queries read personalized identity information, and ranking outputs incorporate the full generative segment.Source: §4.2 · pp. 4–5

4.3 Generative Recall

• The recall head predicts next-level SID codes from generative hidden states using level-specific output projections and a weighted cross-entropy objective.The generative loss reads only generative hidden states, so its gradients flow through generative attention paths and not item-side components.Source: §4.3 · p. 5

4.4 Multi-Objective Ranking

• The ranking tower combines contextualized ranking states, original user and author features, and the final SID-position state to estimate multiple objectives.The combined representation is fed to an MMoE model whose task-specific towers produce objective predictions.Source: §4.4 · p. 5
• Ranking-side LoRA adapts ranking attention while stop-gradient and separate parameter groups limit ranking-gradient effects on the generative backbone.The base attention projections remain shared, while ranking LoRA is active only in ranking-view attention; a simple stop-gradient boundary would otherwise freeze ranking-side feature fusion.Source: §4.4 · pp. 5–6

5.1 Training Strategy

• Ranker objectives use independent masks over all incoming user-item samples, while the generative branch downsamples and retains clicked and randomly exposed samples as positives.The separate masks provide ranking contrast and preserve recall’s online-exposure distribution-fitting target in the same forward pass.Source: §5.1 · p. 6
• Training first enables only the GRM loss, then warm-starts joint training with the ranker loss after the SID representations have stabilized.The staged schedule is intended to reduce noise-driven negative transfer from introducing ranking while SID semantics are still drifting.Source: §5.1 · p. 6

5.2 Inference Pipeline

• During inference, ranking reuses cached user-context and SID-trajectory states, avoiding recomputation while strategy filtering runs in parallel with ranking.The ranking pass computes only newly appended item-feature tokens and ranking-specific LoRA paths.Source: §5.2 · pp. 6–7

• RQ1 asks how the unified model performs against recall and ranking baselines.Source: §6 · p. 7
• RQ2 asks how the proposed key structural components contribute to overall model performance.Source: §6 · p. 7
• RQ3 asks how the unified model performs in online services.Source: §6 · p. 7

6.1.1 Dataset.

• The dataset contains logs from 400 million users and 3 million authors, with billions of interactions; recall tests use exposure and click records.Ranking reports click, long-view, and gift objectives as CTR, LVTR, and GTR.Source: §6.1.1 · p. 7

6.1.2 Evaluation Metrics.

• Recall is evaluated with ACC, HR, and MRR, while ranking is evaluated with AUC and UAUC.Source: §6.1.2 · p. 7

6.1.3 Baselines.

• Recall baselines include traditional and generative methods, while ranking is compared with the deployed HoME MMoE-style model using conventional feature fusion.Recall comparisons include KuaiFormer, GNN, OneRec, OneLive, NEZHA, and PROMISE.Source: §6.1.3 · p. 7

6.2.1 Recall Performance. As the results shown in Table 1, UniR2

• For recall, prefix-cross attention exposes historical user-interest representations to SID generation, while decoupled gradients mitigate ranking-objective conflict.The authors also identify shared embeddings and positive–negative sample contrast as a possible contributor to recall gains.Source: §6.2.1 · p. 7

6.2.2 Rank Performance.

• For ranking, LoRA preserves task-specific feature interactions, and the generated SID trajectory supplies a target-aware semantic view that complements original ranking features.The authors report that this operates while preserving the discriminative contrast required by ranking.Source: §6.2.2 · pp. 7–8

6.2.1 Recall Performance. As the results shown in Table 1, UniR2

• UniR2 consistently outperforms conventional and generative baselines in recall and improves XTR prediction on both sparse and dense ranking targets.The reported overall offline results indicate stronger candidate retrieval quality alongside improved XTR prediction.Source: §6.2.2 · pp. 7–8, §6.2.1 · p. 7

6.3.1 Components Ablation.

• Removing DQ-PCA and gradient truncation improves ranking metrics but significantly degrades generative performance during long joint training.The analysis attributes this imbalance to ranking's more frequent dense-objective gradients versus SID generation's need to preserve token-level semantic consistency.Source: §6.3.1 · p. 8
• Discarding the long history sequence for the ranking query greatly reduces FLOPs with little ranking degradation, while ranking-side LoRA enables feature fusion without changing base generative parameters.The LoRA adds a small parameter and computation cost, with particularly significant impact on sparse targets such as gift.Source: §6.3.1 · p. 8

6.3.2 Attention Analysis.

• Recall SID queries attend broadly to the behavioral prefix, while ranking-specific LoRA sharpens attention around the user profile and generation trajectory.The analysis says the trajectory carries target-aware matching signals distilled by recall; LoRA restores ranking-specific fusion without perturbing the generative backbone.Source: Figure 4 · p. 8, §6.3.2 · pp. 8–9

6.3.3 Scaling Unified Architecture.

• Scaling depth and width substantially improves generative and ranking capabilities without an imbalanced trade-off, though gains diminish at larger sizes.The online configuration uses 3 layers and hidden dimension 640 to balance performance and resource consumption.Source: Figure 5 · p. 9, §6.3.3 · p. 9

6.3.4 Inference Latency.

• Reusing cached user-context and decoder representations, alongside parallelization and ranking scheduling, reduces end-to-end pipeline latency.The baseline runs separate recall and ranking models and incurs repeated initialization, feature preparation, and information transfer.Source: Figure 6 · p. 9, §6.3.4 · p. 9

6.4 Online A/B Test (RQ3)

• The two-week A/B test replaced the OneLive-based recall model and production pre-ranking model on Kuaishou and Kuaishou Lite App.It launched in June 2026 and covered 5% of online traffic.Source: §6.4 · p. 9
• Online results show UniR2 improved Kuaishou App play volume by +1.177%, follow rate by +0.655%, and like rate by +2.560%, while Kuaishou Lite gifting users, intention, and amount improved by +0.717%, +1.567%, and +2.569%, respectively.Source: §6.4 · p. 9

7 Conclusion.

• UniR2 organizes user context, SID trajectory, and item features in one heterogeneous decoder-only sequence.Dual-Query Prefix-Causal Attention supports causal SID generation and ranking feature fusion under task-specific visibility.Source: §7 · p. 9
• Offline experiments show consistent improvements in generative recall and multi-objective ranking, with the SID trajectory serving as a representation bridge between tasks.The conclusion also describes task-specific forward paths and ranking-side LoRA as balancing representation sharing with optimization isolation.Source: §7 · p. 9
• Reusing cached user-context and SID states reduces end-to-end inference time, and long-term online A/B tests show consistent positive gains.The paper presents these findings as evidence of UniR2's effectiveness and practicality in large-scale industrial recommendation systems.Source: §7 · p. 9