PaperlayerView exact revision on arXiv

Source-linked AI summary

Reasoning over Semantic IDs Enhances Generative Recommendation

Yingzhi He, Yan Sun, Junfei Tan, Yuxin Chen, Xiaoyu Kong, Chunxu Shen, Xiang Wang, An Zhang, Tat-Seng Chua

arXiv:2603.23183v2cs.IRcs.AI

TL;DR

Generative recommendation needs effective reasoning over Semantic IDs, but itemic tokens lack native meaning and recommendation reasoning is difficult to supervise and evaluate. SIDReasoner strengthens SID–language alignment with multi-task training and teacher-enriched data, then applies outcome-driven reinforcement; experiments show effective reasoning with cross-domain generalization and improved interpretability.

  • Problem

    Reasoning over Semantic IDs is underexplored because itemic tokens are not natively meaningful to LLMs and recommendation reasoning is difficult to supervise and evaluate.

  • Method

    SIDReasoner uses multi-task and teacher-assisted SID–language alignment, followed by outcome-driven reinforcement to guide effective reasoning trajectories.

  • Results

    Experiments on three real-world datasets demonstrate effective SID reasoning and recommendation, with strong cross-domain generalization and improved interpretability.

  • Takeaways & Limitations

    Reasoning-enhanced SID-based generation can extend recommendation benefits beyond accuracy to cross-domain transfer and interpretable reasoning processes.

  • Takeaways & Limitations

    Scaling to larger backbone models and substantially larger training datasets remains future work because of limited time and computational resources.

Abstract

from arXiv · show

Recent advances in generative recommendation have leveraged pretrained LLMs by formulating sequential recommendation as autoregressive generation over a unified token space comprising language tokens and itemic identifiers, where each item is represented by a compact sequence of discrete tokens, namely Semantic IDs (SIDs). This SID-based formulation enables efficient decoding over large-scale item corpora and provides a natural interface for LLM-based recommenders to leverage rich world knowledge. Meanwhile, breakthroughs in LLM reasoning motivate reasoning-enhanced recommendation, yet effective reasoning over SIDs remains underexplored and challenging. Itemic tokens are not natively meaningful to LLMs; moreover, recommendation-oriented SID reasoning is hard to evaluate, making high-quality supervision scarce. To address these challenges, we propose SIDReasoner, a two-stage framework that elicits reasoning over SIDs by strengthening SID--language alignment to unlock transferable LLM reasoning, rather than relying on large amounts of recommendation-specific reasoning traces. Concretely, SIDReasoner first enhances SID-language alignment via multi-task training on an enriched SID-centered corpus synthesized by a stronger teacher model, grounding itemic tokens in diverse semantic and behavioral contexts. Building on this enhanced alignment, SIDReasoner further improves recommendation reasoning through outcome-driven reinforced optimization, which guides the model toward effective reasoning trajectories without requiring explicit reasoning annotations. Extensive experiments on three real-world datasets demonstrate the effectiveness of our reasoning-augmented SID-based generative recommendation. Beyond accuracy, the results highlight the broader potential of large reasoning models for generative recommendation, including improved interpretability and cross-domain generalization.

1 Introduction

SID-based generative recommendation connects compact item identifiers with pretrained LLMs, but effective reasoning over SIDs remains difficult because itemic tokens lack native meaning and recommendation reasoning is hard to supervise and evaluate. SIDReasoner addresses these challenges through stronger SID–language alignment and outcome-driven optimization, with experiments showing improved recommendation reasoning and broader benefits.

  • Motivation: Semantic IDs represent items as compact discrete sequences, enabling autoregressive recommendation within a language-model token space.This formulation supports efficient decoding and integration with pretrained LLMs.
  • Challenges: Reasoning over SIDs is challenging because itemic tokens are not inherently meaningful, while recommendation preferences are implicit and difficult to evaluate.These conditions make high-quality reasoning supervision scarce and expensive.
  • Approach: SIDReasoner strengthens SID–language alignment so general LLM knowledge and reasoning can transfer to recommendation without relying only on recommendation-specific reasoning traces.The framework uses multi-task alignment over enriched SID-centered data.
  • Approach: Outcome-based reinforcement with GRPO steers the model toward effective reasoning trajectories despite the absence of a direct reasoning-quality criterion.The optimization provides feedback from recommendation outcomes rather than explicit reasoning annotations.
  • Findings: Experiments on three real-world datasets show effective SID reasoning, strong cross-domain generalization, and improved interpretability through explicit reasoning processes.The authors also frame the contribution as extending recommendation reasoning beyond accuracy improvements.

2 Related Work

Related work positions SID-based generation as a compact semantic alternative for recommendation and reviews reasoning approaches in large language models and recommender systems. The literature distinguishes explicit natural-language reasoning from latent reasoning and motivates the paper’s focus on reasoning over itemic tokens.

  • Generative Recommendation: Generative recommendation methods include sparse ID-based, text-based, and semantic ID-based approaches.Semantic IDs compactly encode item semantics while supporting efficient decoding.
  • Generative Recommendation: Semantic IDs are obtained by quantizing continuous item embeddings into short sequences of discrete codes.This creates a trade-off between semantic expressiveness and decoding efficiency.
  • Large Reasoning Models: LLM reasoning commonly uses explicit intermediate steps, with additional inference-time computation or reinforcement learning refining reasoning processes.The related work discusses parallel sampling, iterative refinement, and verifiable rewards.
  • Reasoning in Recommendation: Recommendation reasoning is broadly divided into explicit reasoning with natural-language rationales and latent reasoning that avoids explicit rationales.These paradigms differ in how they expose intermediate inference to the user.

3 Methodoly

The method section presents SIDReasoner as a reasoning-enhanced generative recommendation framework with three stages: task formalization, SID quantization and language alignment, and reinforced optimization.

  • Method Overview: SIDReasoner formalizes recommendation, aligns itemic tokens with natural language through multi-task training and enriched semantic data, then enhances reasoning through reinforced optimization.The framework is presented as a two-stage reasoning approach built on this overall pipeline.

3.1 Task Formulation

The task formulation represents each item with a compact Semantic ID and generates the next item through autoregressive prediction over itemic tokens. Reasoning is modeled as an explicit intermediate sequence conditioned on the user’s interaction history.

  • 3.1 Task Formulation: Generative recommendation predicts the next item a user is likely to interact with from chronological interaction history.The history is represented as H_u = (i_1, i_2, ..., i_T), and the target is i_T+1.
  • 3.1.1 Tokenization with Semantic IDs: Each item is represented by a fixed-length Semantic ID sequence from a recommendation-specific itemic vocabulary.The itemic vocabulary S is distinct from the LLM language-token vocabulary V_LM.
  • 3.1.1 Tokenization with Semantic IDs: An item’s metadata is encoded into a continuous semantic embedding and quantized into L discrete semantic tokens.This tokenization preserves item semantics in a compact representation.
  • 3.1.1 Tokenization with Semantic IDs: Generating one next item requires L consecutive next-token predictions over the itemic vocabulary.The model’s unified token space combines natural-language and itemic tokens.
  • 3.1.1 Tokenization with Semantic IDs: The unified vocabulary V = V_LM ∪ S allows textual reasoning and item generation within one autoregressive model.This connects language-token processing with Semantic ID decoding.
  • 3.1.2 Generative Recommendation with Reasoning: The reasoning sequence τ functions as an explicit intermediate inference step guiding next-item generation from historical behavior.The generative policy is denoted by π_θ.

3.2 Enriched SID-Language Alignment

SIDReasoner strengthens the connection between Semantic IDs and language so LLMs can interpret and reason over itemic tokens. It combines residual-quantized item representations with multi-task and teacher-enriched alignment data while preserving general reasoning ability.

  • 3.2.1 Item Quantization: Residual quantization represents each item as a compact sequence of discrete SID tokens while preserving semantic fidelity.Items are encoded into continuous representations and quantized across multiple codebooks; the selected indices form the item’s SID sequence.
  • 3.2.2 Multi-task SID-Language Alignment: SID tokens begin as semantically meaningless vocabulary additions, so multi-task fine-tuning grounds them in recommendation semantics and behavior.The tasks jointly expose SID and language tokens through item prediction and SID translation under next-token prediction.
  • 3.2.3 Enriched Corpus Pre-training: Teacher-generated item-centric and user-centric enrichment provides more diverse SID-language associations than limited task formats alone.The teacher expands item metadata into structured semantic descriptions and interleaves SID tokens with natural language, while also enriching interaction sequences with inferred user information.
  • 3.2.3 Enriched Corpus Pre-training: General-domain reasoning data is mixed into the alignment corpus to prevent recommendation-focused training from degrading general language and reasoning abilities.The auxiliary data helps preserve the model’s original general reasoning ability while learning enriched SID-language associations.

3.3 Reinforced Reasoning Enhancement

SIDReasoner activates explicit reasoning and then refines reasoning–prediction trajectories through outcome-driven reinforcement learning. Its rewards combine prefix-sensitive item accuracy, format validity, and group-wise policy optimization.

  • Reasoning Activation: A lightweight supervised activation stage teaches the model to generate reasoning before itemic predictions, mainly improving response-format reliability.The stage uses teacher-generated reasoning and requires only a single epoch of fine-tuning because prior SID-language alignment already provides reasoning-based recommendation capability.
  • Group-wise Reinforcement Learning: The reinforcement-learning stage directly optimizes the recommendation policy using rewards from recommendation accuracy.The policy is refined after reasoning activation on recommendation tasks.
  • Reward Design: The reward combines a stepwise rule-based score for matching the ground-truth next item with a format reward for valid catalog items.The rule-based reward increases as more predicted SID tokens match the correct prefix, while the format reward checks structural validity and catalog existence.
  • Group-wise Reinforcement Learning: GRPO samples groups of reasoning–prediction trajectories, assigns outcome-based rewards, and updates the policy using normalized clipped objectives.The trajectory-level importance ratio compares the current policy with the previous policy, while clipping and KL regularization constrain updates.
  • Group-wise Reinforcement Learning: The reinforced optimization progressively improves both generated reasoning quality and item-prediction accuracy.It increases the likelihood of higher-reward reasoning–prediction trajectories while suppressing less effective paths.

4 Experiments

Experiments evaluate SIDReasoner across three real-world datasets using full-item ranking and comparisons with discriminative, generative, and reasoning-based recommenders. Results show strong recommendation performance, domain-transferable reasoning, and clear benefits from SID–language alignment while preserving general abilities through mixed training data.

  • Experimental Setup: Experiments use Amazon Games, Office, and Industrial datasets with 5-core filtering, Recall@K and NDCG@K at K∈{5,10}, and full-item ranking.The evaluation compares performance across the entire item set rather than sampled negatives.
  • Experimental Setup: SIDReasoner is compared with traditional discriminative, generative, and reasoning-based recommendation methods.Baselines include Caser, GRU4Rec, SASRec, TIGER, HSTU, LETTER, LCRec, ReaRec, and R2ec.
  • Main Results: After reinforcement learning, SIDReasoner achieves the strongest SID prediction performance across three datasets, surpassing conventional generative and reasoning-enhanced recommenders.The results indicate that effective reasoning over SIDs can be learned and used to improve recommendation quality.
  • Main Results: Reasoning gains vary across domains: they are substantial on semantically rich Games items but more limited on Industrial items with less domain-relevant LLM knowledge.A consistent trend also appears in R2ec, linking reasoning effectiveness to the availability and quality of usable semantic knowledge.
  • Cross-Domain Generalization: Single-domain reinforcement learning improves reasoning effectiveness on both in-domain and out-of-domain datasets after shared alignment across Games, Office, and Industrial.The result suggests that learned recommendation reasoning can transfer across domains rather than remaining tied to domain-specific item distributions.
  • Alignment and Ablation: Best-of-N reasoning performance tracks post-reinforcement-learning convergence, making it an effective indicator of reasoning capacity and optimization potential.Figure 4 evaluates Recall@10 by selecting the best of N sampled reasoning trajectories according to the groundtruth item.
  • Alignment and Ablation: Explicit SID–language alignment is necessary for effective reasoning-based recommendation, while enriched alignment expands reasoning capacity and mixed general reasoning data mitigates catastrophic forgetting.Recommendation-only training degrades general language abilities, whereas incorporating general reasoning data preserves abilities important for recommendation reasoning.

4.4 Model Study (RQ3)

The study examines how corpus enrichment and reinforcement learning shape SID reasoning, finding that stronger teacher models improve alignment while RL produces shorter, more targeted reasoning that accompanies better recommendation performance.

  • Effect of Teacher Model on Corpus Enrichment: Stronger teacher models produce more informative enriched corpora and improve SID-language alignment and recommendation reasoning.Rule-based metadata performs weakest, while GPT-4o-mini achieves the best performance among evaluated corpus-construction variants.
  • Figures: Figure 5 illustrates an explicit-reasoning case study, while Figure 6 tracks reasoning length and Recall@10 across RL training steps.The figures provide complementary views of interpretability and training dynamics.
  • Evolution of Reasoning during RL Training: During RL training, reasoning length decreases while Recall@10 steadily improves.The authors attribute this pattern to RL removing redundant or uninformative reasoning components learned from teacher-generated samples.
  • Case Study: Explicit SID-language alignment enables reasoning to reveal user interests and directly guide SID decoding toward relevant recommendations.In the case study, reasoning identifies strategic role-playing games and Nintendo amiibo interests before prioritizing Nintendo amiibo SIDs.

5 Conclusion

SIDReasoner enables LLMs to reason over itemic tokens by combining SID-language alignment with outcome-driven reinforcement learning. Experiments report competitive performance alongside improved cross-domain generalization and interpretability, while scaling to larger models and datasets remains future work.

  • Conclusion: SIDReasoner combines multi-task alignment, teacher-assisted semantic expansion, and outcome-driven reinforcement to support reasoning over Semantic IDs.The framework strengthens semantic grounding before using reinforcement learning to guide effective reasoning patterns.
  • Conclusion: Experiments on real-world datasets report competitive performance with improved cross-domain generalization and interpretability.These results support the broader use of reasoning in SID-based generative recommendation.
  • Future Work: The study leaves evaluation with larger backbone models and substantially larger training datasets for future work.The authors cite limited time and computational resources as the reason for this scope boundary.

A Baseline Methods

The baseline suite spans convolutional, recurrent, transformer-based, and Semantic ID-based generative recommenders for sequential next-item prediction.

  • Caser: Caser applies convolutional neural networks to interaction sequences to capture local and global sequential patterns.It predicts the next item using binary cross-entropy loss.
  • GRU4Rec: GRU4Rec uses GRU modules to model sequential dependencies and predict the next purchased item.Binary cross-entropy is used as the optimization objective.
  • SASRec: SASRec uses self-attention to capture long-range dependencies in user interaction sequences.It predicts future interactions with binary cross-entropy optimization.
  • TIGER: TIGER represents items as Semantic IDs and autoregressively predicts the next item’s SID sequence with a Transformer.It serves as a representative generative recommender baseline.

B Alignment Task Formats

The alignment tasks bridge item titles and Semantic IDs through bidirectional translation and next-interaction prediction across title and SID representations.

  • SID Translation: SID translation trains the model to emit a SID from a title and recover a title from a SID.This bidirectional mapping grounds the SID vocabulary in natural language.
  • Generative Next-Item Prediction: Generative next-item prediction alternates titles and SIDs across four directional variants: title→title, title→SID, SID→title, and SID→SID.The templates combine recommendation prediction with alignment between textual and itemic representations.

C Prompts for Corpus Enrichment

The paper uses a two-stage prompting framework to construct item-centric and user-centric enrichment corpora, beginning with structured analysis of raw item metadata.

  • A two-stage prompting framework constructs both item-centric and user-centric enrichment corpora.
  • Item-centric Semantic Enrichment: Its first item-centric stage prompts a model to reason over titles, brands, categories, descriptions, and features to produce structured insights.
  • Prompt templates, output-format constraints, and additional implementation details are released in the open-source repository.
Loading 2603.23183v2…