Uncertainty-aware Generative Recommendation

1 Introduction

• Generative recommendation formulates recommendation as end-to-end conditional sequence generation rather than multi-stage retrieval and ranking.Items are serialized as discrete tokens and generated autoregressively.Source: §1 · pp. 1–2
• Existing preference optimization overlooks generation confidence and sample difficulty, treating errors uniformly and leaving decision risk unquantified.The paper terms this limitation uncertainty blindness.Source: §1 · p. 2
• UGR combines uncertainty-weighted rewards, difficulty-aware optimization, and explicit confidence alignment to use uncertainty as an optimization signal.The mechanisms penalize confident hallucinations, sustain learning on difficult samples, and express confidence as quantifiable risk signals.Source: §1 · p. 2
• UGR's learned confidence provides reliable signals for downstream risk-aware applications such as list truncation and reranking.The paper argues uncertainty modeling is an indispensable part of preference optimization, not merely a calibration task.Source: §1 · p. 2
• UGR yields superior recommendation performance and stabilizes training, preventing the degradation often observed with standard methods.Its aligned confidence also supports downstream risk-aware applications.Source: §1 · p. 2

2.1 Generative Recommendation Formulation

• A user's interaction history is the context for generating a sequence representing a recommended item; this work represents items with Semantic IDs encoding categorical hierarchy.The model generates the sequence autoregressively.Source: §2.1 · p. 2
• Binary rewards penalize any mismatch, while rank-based rewards penalize highly ranked incorrect items; both remain static signals that omit generation uncertainty and instance difficulty.The rank-based penalty uses the generated item's position in the model's probability distribution.Source: §2.1 · pp. 2–3

2.2 Confidence and Uncertainty in Recommendation

• Recommendation uncertainty is intrinsic to the subjectivity and stochasticity of user interactions, not solely a reflection of model capability limits..Source: §2.2 · p. 3

2.2.1 Implicit Generation Uncertainty.

• Implicit uncertainty is characterized by sequence negative log-likelihood, with higher values indicating lower prediction probability; it reflects token-level generation dynamics but is not designed to express decision-level confidence.Its normalization also limits absolute comparability across samples.Source: §2.2.1 · p. 3

2.2.2 Explicit Confidence.

• Explicit confidence directly assesses a recommendation's overall quality or plausibility; relative confidence ordering suits recommendation better than absolute calibration in sparse settings.It complements implicit uncertainty with a decision-level signal for preference modeling and risk-aware applications.Source: §2.2.2 · p. 3

3.1 Overview

• UGR combines uncertainty-aware rewards, difficulty-aware optimization, and explicit confidence alignment in a closed-loop framework for preference alignment.The framework builds on Semantic IDs and constrained beam search to make uncertainty signals measurable and comparable.Source: §3.1 · pp. 3–4

3.2 Uncertainty-aware Rollout Reward

• Continuous rollout rewards use sequence logit intensity to penalize incorrect generations more when the model is more certain about them.The reward addresses the inability of binary or rank-based penalties to distinguish confident hallucinations from low-confidence hesitations.Source: §3.2 · p. 4

3.3 Difficulty-aware Optimization Dynamics

• Difficulty-aware weighting shifts gradient focus toward hard samples, using the ground truth’s beam-search rank as a proxy for instance difficulty.A missed ground truth receives the maximum difficulty rank, while the weighting is intended to balance learning easy patterns with exploration of hard samples.Source: §3.3 · pp. 4–5

3.4 Explicit Confidence Modeling and Alignment

• Explicit confidence alignment uses SID hierarchy to distinguish fully correct, partially correct, and completely wrong generations in confidence space.The model’s confidence is made interpretable for risk-aware decisions; stronger penalties for complete errors induce the hierarchy c(y*) > c(ypartial) > c(ywrong).Source: §3.4 · p. 5

3.5 Overall Objective and Training

• End-to-end training sums difficulty-aware GRPO and confidence-ranking losses, using shared rollout samples to update recommendation and confidence outputs.The auxiliary loss is described as scale-compatible with the main loss, so the framework introduces no additional global hyperparameter.Source: §3.5 · p. 5

4.2 Overall Performance Comparison (RQ1)

• UGR achieves the best performance across all datasets and metrics, outperforming the strongest baselines and establishing new state-of-the-art results.The comparison includes traditional, LLM-based, SID-based generative, and reasoning-enhanced models.Source: Table 1 · p. 6, §4.2 · pp. 6–7

4.1 Experimental Setup

• The evaluation uses Office, Industrial, and Yelp benchmarks, with HR@K and NG@K metrics and beam search for inference.The datasets are chronologically split 8:1:1; the implementation uses Qwen3-8B and four-layer SIDs.Source: §4.1.1 · p. 6, §4.1.2 · p. 6, §4.1.4 · p. 6

4.3 Component Analysis and Ablation (RQ2)

• The full UGR ablation performs best; difficulty-aware optimization adds consistent gains, while explicit confidence alignment enables confidence expression and reinforces SID structure.The uncertainty-weighted reward recovers performance to match SFT, but reward-level gains alone do not significantly surpass SFT.Source: Table 2 · p. 6, §4.3.3 · p. 7, §4.3.2 · p. 7

4.4 Training Dynamics Analysis (RQ3)

• Under uniform weighting (α=1.0), HR@1 and HR@16 regress after an initial peak; UGR (α=0) maintains an upward training trend.The reported analysis links uniform-weighting instability to overfitting on trivial samples and UGR’s sustained coverage to continued refinement of hard instances.Source: Figure 3 · p. 7, §4.4.1 · p. 7, §4.4.2 · p. 7

4.4.3 Analysis of Candidate Collapse.

• With α=1.0, unique valid candidates per rollout steadily erode; UGR (α=0) preserves a healthy, diverse search space.The analysis attributes candidate collapse to overconfidence on easy samples and describes UGR as dynamically down-weighting their gradients.Source: §4.4.3 · pp. 7–8, Figure 4 · p. 8

4.5.1 Confidence-based Ranking Enhancement.

• Confidence-based reranking consistently improves all reported metrics over logits alone on the Office dataset.The authors describe confidence as a complementary assessment of logical correctness that can correct high-confidence but factually incorrect outputs.Source: Table 3 · p. 8, §4.5.1 · p. 8

4.5.2 Risk-aware Recommendation.

• On Office, filtering low-confidence user requests steadily raises NDCG@10, while truncating low-confidence tail items steadily increases Precision.The reported applications use confidence thresholds for user-level rejection and dynamic item-list truncation.Source: Figure 5 · p. 8, §4.5.2 · p. 8

4.6 Hyperparameter Sensitivity

• Across datasets, margin configurations yield comparable performance; a small positive γ1 with larger γ2 performs best or near-best, while bringing the margins close slightly degrades results.The authors attribute the degradation to reduced differentiation between partially correct and completely wrong outputs.Source: §4.6 · p. 8

5.1 Generative Recommendation

• Generative recommenders have moved from text generation and grounding toward SID-based generation, while current approaches still lack explicit confidence alongside recommendations.Grounding mechanisms [15] and candidate-list restrictions [32] are described as constraining end-to-end generation; SID approaches include TIGER [39] and LC-Rec [58].Source: §5.1 · p. 9

5.2 Preference Alignment in Recommendation

• Preference alignment methods including DPO [38] and GRPO [41] remain focused on outcome correctness, treating confident errors and tentative exploration alike while ignoring sample difficulty.The passage also discusses offline and group-wise optimization, including ReRe [43] and MiniOneRec [28].Source: §5.2 · p. 9

5.3 Confidence and Uncertainty in LLMs and Recommendation

• LLM uncertainty research includes calibration rewards [12], but recommendation’s subjective, sparse interactions make absolute numerical calibration less meaningful and uncertainty’s use in generative recommendation remains unexplored.The passage positions UGR as applying uncertainty to preference alignment while adding risk-awareness capabilities.Source: §5.3 · p. 9

6 Conclusion

• UGR combines uncertainty-weighted rewards, difficulty-aware optimization, and explicit confidence alignment, with reported state-of-the-art performance, greater training stability, and confidence for risk-aware decisions.The paper says UGR was validated on three moderate-scale benchmarks and expects its optimization and reward-shaping principles to generalize to larger settings.Source: §6 · p. 9
• Future work will test uncertainty-aware optimization beyond GRPO, develop finer-grained uncertainty decomposition, and extend UGR to online interactive and dynamic domains.The stated directions include adapting the approach to DPO to explore sample efficiency.Source: §6 · p. 9

A Datasets

• The three datasets are 5-core filtered, time-windowed, truncated to user histories of at most 10 interactions, and split chronologically into training, validation, and test sets at 8:1:1.Dataset statistics are summarized in Table 5.Source: A Datasets · p. 11

• Baselines are grouped into traditional recommendation, LLM-based generative models, and reasoning-enhanced models.LLM-based methods are subdivided into text-based and SID-based generative approaches.Source: B Baselines · p. 11

1. Traditional Recommendation Models

• SASRec uses self-attention to capture long-term semantic dependencies in sequential recommendation.Source: §1 · p. 11

2. LLM-based Generative Models

• BIGRec grounds generated semantic tokens to items, while SPRec uses DPO with self-generated negatives to mitigate filter bubbles.Source: §1 · p. 11
• TIGER and MiniOneRec use RQ-VAE item SIDs; MiniOneRec also optimizes with reinforcement learning and hybrid rewards.Source: §1 · p. 11

3. Reasoning-enhanced Models

• ReaRec enhances user representations through inference-time multi-step reasoning, while R2ec jointly generates reasoning chains and predicts items.Source: §3 · p. 12

• UGR uses Qwen3-8B with four-layer SIDs, then LoRA SFT warm-started into preference alignment on eight NVIDIA A100 GPUs.Preference alignment uses learning rate 2e-6, KL penalty β=0, and 16 candidates per sample; SFT runs for 10 epochs.Source: C Implementation Details · p. 12
• At inference, beam size 10 generates recommendation lists, and explicit confidence is computed after generation via Eq. (10).Source: C Implementation Details · p. 12
• Risk-aware ranking combines exp(sequence log-likelihood) with explicit confidence at λ=0.5; user rejection uses the maximum hybrid score among generated candidates.Source: C Implementation Details · p. 12
• Compared with GRPO, UGR adds 10G memory and 15 minutes per epoch, plus 4.1 ms inference latency.The added inference latency enables downstream risk-aware applications that standard methods cannot support.Source: C Implementation Details · p. 12

• Only complete U+D+E consistently achieves the best performance across all metrics; D+E is the strongest two-component variant but trails it.E alone falls below the SFT baseline, indicating confidence alignment requires the reward structure from U or D.Source: D Additional Ablation Studies · p. 12, Table 7 · p. 12
• On Office, binary confidence tokens outperform multi-level discrete tokens and a regression head.The authors attribute this to sparse recommendation supervision and binary tokens’ fit with GRPO pairwise ranking.Source: D Additional Ablation Studies · p. 12, Table 8 · p. 12