PaperlayerView exact revision on arXiv

Source-linked AI summary

MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation

Xiaoyu Kong, Leheng Sheng, Junfei Tan, Yuxin Chen, Jiancan Wu, An Zhang, Xiang Wang, Xiangnan He

arXiv:2510.24431v1cs.IRcs.AI

TL;DR

Embedding-heavy recommenders have struggled to scale, and proprietary generative deployments leave open whether their scaling benefits hold on public data and what post-training recipe works. MiniOneRec provides an open-source SID-based framework with SFT and recommendation-oriented RL. Larger models show lower training and evaluation losses, while the post-training approach improves reported ranking and diversity outcomes.

  • Problem

    Embedding-heavy recommenders plateau as they grow, while proprietary generative deployments leave public-data scaling and minimal post-training questions unresolved.

    Source: §1 · p. 3

  • Method

    MiniOneRec builds SIDs with RQ-VAE and post-trains Qwen models using SFT followed by recommendation-oriented RL with alignment and constrained generation.

    Source: §1 · pp. 3–4

  • Results

    Training and evaluation losses decrease with model size, and MiniOneRec outperforms previous generative solutions across most reported metrics.

    Source: §5.2 · p. 9, §5.3 · p. 10

  • Takeaways & Limitations

    On the reported public benchmarks, the SID-based generative approach exhibits parameter efficiency and the post-training pipeline improves ranking accuracy and candidate diversity.

    Source: §1 · p. 4, §6 · p. 12

Abstract

from arXiv · show

The recent success of large language models (LLMs) has renewed interest in whether recommender systems can achieve similar scaling benefits. Conventional recommenders, dominated by massive embedding tables, tend to plateau as embedding dimensions grow. In contrast, the emerging generative paradigm replaces embeddings with compact Semantic ID (SID) sequences produced by autoregressive Transformers. Yet most industrial deployments remain proprietary, leaving two fundamental questions open: (1) Do the expected scaling laws hold on public benchmarks? (2) What is the minimal post-training recipe that enables competitive performance? We present MiniOneRec, to the best of our knowledge, the first fully open-source generative recommendation framework, which provides an end-to-end workflow spanning SID construction, supervised fine-tuning, and recommendation-oriented reinforcement learning. We generate SIDs via a Residual Quantized VAE and post-train Qwen backbones ranging from 0.5B to 7B parameters on the Amazon Review dataset. Our experiments reveal a consistent downward trend in both training and evaluation losses with increasing model size, validating the parameter efficiency of the generative approach. To further enhance performance, we propose a lightweight yet effective post-training pipeline that (1) enforces full-process SID alignment and (2) applies reinforcement learning with constrained decoding and hybrid rewards. Together, these techniques yield significant improvements in both ranking accuracy and candidate diversity.

1 INTRODUCTION

Traditional embedding-heavy recommenders have struggled to scale, while generative recommendation raises questions about whether its scaling advantages hold on public data and what post-training is needed. MiniOneRec addresses these questions with an open-source framework and a pipeline combining SID alignment with reinforcement learning.

1 INTRODUCTION

  • Embedding-heavy recommenders plateau as embedding dimensions or table sizes grow, motivating investigation of scaling in generative recommendation.Generative models instead use compact SID sequences and autoregressive Transformers.Source: §1 · p. 3
  • Proprietary industrial deployments leave open whether generative recommendation scales on public datasets and what minimal post-training recipe achieves strong performance.The paper frames these as two research questions for the community.Source: §1 · p. 3
  • MiniOneRec provides an open-source workflow for SID generation, supervised fine-tuning, and recommendation-driven reinforcement learning.It uses RQ-VAE for SID construction and post-trains Qwen-based models from 0.5B to 7B parameters on Amazon Review data.Source: §1 · p. 3
  • Across four model sizes trained on public Amazon Review data, training and held-out evaluation losses decrease as model size grows.The variants use identical data splits and hyperparameter settings.Source: §1 · p. 3
  • The post-training recipe maintains SID alignment through SFT and RL, while constrained generation and ranking-aware rewards improve candidate diversity and ranking fidelity.Invalid tokens are masked, beam search explores diverse candidates, and the reward combines rule-based accuracy with a penalty for hard negatives.Source: §1 · pp. 3–4

2 BACKGROUND AND RELATED WORK

Generative recommendation represents items as discrete codes for autoregressive generation, with prior work exploring code construction, LLM alignment, and industrial architectures. LLM reinforcement learning includes preference-based and online methods, and MiniOneRec’s framework combines SFT with RL while maintaining SID alignment.

2.1 GENERATIVE RECOMMENDATION

  • Generative recommendation trains a Transformer to predict next-item tokens, replacing the traditional multi-stage matching–ranking pipeline.Earlier approaches use discrete item codes, including RQ-VAE-derived SIDs, hierarchical clusters, or fused collaborative and semantic evidence.Source: §2.1 · p. 4
  • Industrial generative recommenders explore compressed features, lazy decoder layouts, and latent-space inference to address serving cost or performance.The cited examples include MTGR, OneRec, and OnePiece.Source: §2.1 · p. 4

2.2 LLM AND RL

  • PPO is memory-intensive at billion-scale, while DPO removes a separate value network but preference-based objectives can be off-policy and converge prematurely.S-DPO adapts preference optimization to recommendation through implicit pairwise preferences from softmax-based negative sampling.Source: §2.2 · p. 4
  • GRPO normalizes rewards across small batches of roll-outs and uses rule-based signals instead of a learned reward model.The cited work reports gains in mathematics and code generation.Source: §2.2 · p. 4
  • MiniOneRec uses SFT followed by RL, with constrained beam search producing valid, distinct SID recommendations and alignment enforced through both stages.Its framework uses GRPO to update the policy while preserving the SID alignment objective.Source: Figure 2 · p. 5

3 MODELING

MiniOneRec frames recommendation as autoregressive generation over Semantic IDs, with item tokenization and language-model alignment preceding supervised and reinforcement learning. Its RL design addresses limited candidate diversity and sparse ranking feedback through constrained sampling and a combined reward.

3 MODELING

  • MiniOneRec converts item text into SIDs, aligns language models with those codes, then trains through next-token prediction followed by reinforced preference optimization.The modeling pipeline uses an RQ-VAE tokenizer and adds alignment to the original OneRec architecture.Source: §3 · p. 5

3.1 TASK FORMULATION

  • A generative policy reads each user's chronological interaction history and predicts the next item, returning recommendations from its top-k beam-search candidates.Each item is represented by a three-level structural ID.Source: §3.1 · p. 5

3.2 ITEM TOKENIZATION

  • RQ-VAE maps title-description semantic vectors into residual code indices that form the discrete item tokens consumed by the recommender.The text encoder remains frozen while the codebooks, encoder, and decoder are trained jointly.Source: §3.2 · pp. 5–6

3.3 ALIGNMENT WITH LLMS

  • Recommendation tasks predict the next-item SID, while alignment tasks connect natural language and SID space through a two-way mapping.Both task groups are optimized during SFT and RL, with constrained decoding during RL limiting outputs to valid item SIDs and canonical titles.Source: §3.3 · p. 6

3.4 REINFORCED PREFERENCE OPTIMIZATION

  • After SFT, GRPO samples multiple candidates per prompt, scores them, standardizes rewards within each group, and keeps the updated policy near a reference model through a KL term.The method addresses limited sampling diversity and sparse ranking supervision with diverse valid candidates and an auxiliary ranking-shaped reward.Source: §3.4 · pp. 6–7

3.4.1 SAMPLING STRATEGY

  • Because repeated sampling can produce duplicate items, MiniOneRec uses beam search to provide distinct candidates with a better diversity-efficiency trade-off.Its default constrained beam search keeps generated items valid while providing diverse candidate trajectories.Source: §3.4.1 · p. 7

3.4.2 REWARD DESIGN

  • The rank-aware reward penalizes negatives more when the model ranks them prominently, and the default reward combines this ranking signal with rule-based correctness.The paper also explores collaborative scores from a pretrained collaborative-filtering model as an alternative source of reward.Source: §3.4.2 · pp. 7–8

4 TRAINING

Training builds item SIDs with RQ-VAE, fine-tunes a Qwen backbone, and then applies GRPO. The comparison protocol uses shared Qwen2.5-Instruct backbones for LLM-powered systems and specifies separate settings for conventional recommenders.

4 TRAINING

  • The RQ-VAE tokenizer is trained on item text, and its resulting SID vocabulary is plugged into a Qwen2.5-Instruct backbone.Tokenizer training uses one GPU, batch size 20,480, learning rate 1 × 10^-3, and 10,000 epochs.Source: §4 · p. 8
  • SFT runs on eight NVIDIA H100 GPUs for up to ten epochs with early stopping, followed by two GRPO epochs from the SFT checkpoint.SFT uses a 3 × 10^-4 initial learning rate with cosine decay; RL rollouts use beam search of width 16.Source: §4 · p. 8
  • For comparisons, conventional recommenders use binary cross-entropy and Adam, while LLM-powered systems share the Qwen2.5-Instruct backbone and use AdamW.The comparison passage also specifies batch sizes and learning-rate settings for SFT, preference alignment, and RL.Source: §4 · p. 8

5 EVALUATION

The evaluation section outlines studies of scaling, baseline performance, unseen-domain generalization, ablations, and pretrained LLM weights. It uses Office and Industrial slices of Amazon Reviews and reports top-K accuracy measures.

5 EVALUATION

  • The evaluation plan covers scaling, comparisons with three baseline groups, unseen-domain recommendation, ablations, and the influence of pretrained LLM weights.The section frames unseen-domain SID next-item prediction as recommendation rule discovery.Source: §5 · p. 8
  • Table 1 is scoped as a performance comparison of MiniOneRec with traditional, generative, and LLM-based methods.The supplied caption identifies the comparison groups but does not provide outcome values.Source: Table 1 · p. 9

5.1 EVALUATION SETUP

  • Experiments use the Office and Industrial slices of the Amazon Review dataset and measure top-K accuracy with HR@K and NDCG@K.The section identifies these as real-world dataset slices.Source: §5.1 · p. 9

5.2 SCALING

MiniOneRec’s losses decrease as model size grows, and it outperforms previous generative solutions across most reported metrics.

5.2 SCALING

  • Evaluation losses remain lower for larger models through nearly the entire training process, while convergence is faster.Convergence and evaluation losses decrease as model size increases.Source: §5.2 · p. 9

5.3 PERFORMANCE COMPARISON

  • LLM-based recommenders BIGRec and D3 surpass traditional GRU4Rec and Caser in recommendation accuracy.Source: §5.3 · p. 10
  • MiniOneRec outperforms previous generative solutions across most reported metrics through full-process alignment and reinforced preference optimization.Its compact SID space also requires fewer context tokens, with faster inference, lower latency, and smaller serving memory footprints.Source: §5.3 · p. 10

5.4 TRANSFERABILITY

The unseen-domain test finds competitive transfer from RL-only training, while component ablations favor full-process alignment, beam search, and the full reward design.

5.4 TRANSFERABILITY

  • RL-only MiniOneRec transfers competitively to the unseen Office catalog after training on Industrial, though it falls short of full MiniOneRec on in-domain scores.The comparison also finds Qwen-SID performs noticeably better than Qwen-Text.Source: §5.4 · p. 10, Table 2 · p. 11

5.5.1 ALIGNING STRATEGY

  • Full-process alignment earns the highest scores on every metric, while removing alignment performs worst.The compared variants remove alignment, restrict it to SFT, or introduce it only during RL.Source: §5.5.1 · pp. 10–11, Figure 4 · p. 11

5.5.2 SAMPLING STRATEGY

  • Beam search delivers the highest accuracy while using roughly two-thirds of the samples required by dynamic sampling.The full model uses beam search with width 16.Source: §5.5.2 · p. 11, Figure 4 · p. 11

5.5.3 REWARD DESIGN

  • The full reward design achieves the best overall performance, whereas injecting collaborative reward information degrades performance.The authors hypothesize reward hacking because the collaborative reward rises even as recommendation accuracy declines.Source: §5.5.3 · p. 11, Figure 4 · p. 11

5.6 PRE-TRAINED LLM IMPACT

Across two benchmark datasets, MiniOneRec initialized with pre-trained weights significantly outperforms its randomly initialized counterpart.

5.6 PRE-TRAINED LLM IMPACT

  • Pre-trained MiniOneRec significantly outperforms random initialization across two benchmark datasets.The authors conjecture that language-pretraining reasoning and encoded factual knowledge help with SID pattern discovery and item semantics.Source: §5.6 · p. 12

6 CONCLUSION

MiniOneRec is an open-source end-to-end generative recommendation framework whose experiments report lower losses with larger models and strong baseline comparisons. The authors plan to extend its codebase with new datasets, tokenization schemes, backbones, and training pipelines.

6 CONCLUSION

  • MiniOneRec provides an open-source workflow spanning SID construction, SFT, and recommendation-oriented RL.The conclusion reports lower training and evaluation losses with larger models and says MiniOneRec surpasses strong sequential, generative, and LLM-based baselines on Amazon Review.Source: §6 · p. 12
  • Planned codebase updates include new datasets, advanced tokenization schemes, larger backbones, and enhanced training pipelines.The authors also welcome community contributions.Source: §6 · p. 12

A ALIGNMENT PROMPTS

The framework aligns pretrained LLM knowledge with the SID space through added SID tokens and auxiliary alignment tasks, alongside tasks that train recommendation behavior. Its prompts cover next-item generation, cross-modal SID prediction, description reconstruction, and user-preference summarization.

A.1 OVERVIEW

  • A three-layer codebook of 256 SIDs per layer is appended as indivisible tokenizer tokens, then auxiliary alignment and recommendation tasks bridge text and SID representations.The overview frames these steps as aligning LLM world knowledge with the SID space for generative recommendation.Source: §A.1 · p. 18

A.2 RECOMMENDATION TASKS

  • Chronological user-interaction SIDs plus a “Recommend the next item” instruction form the input for predicting the next-item SID.Figure 5 presents a sample generative-retrieval prompt.Source: §A.2 · p. 18, Figure 5 · p. 18
  • Asymmetric item prediction maps textual user histories to next-item SIDs and SID-only histories to next-item titles.Sample prompts appear in Figures 6 and 7.Source: §A.2 · p. 18, Figure 6 · p. 18, Figure 7 · p. 18

A.3 ALIGNMENT TASKS

  • Item description reconstruction generates a description from one SID and infers the SID from a description, only during SFT because descriptions are large and diverse.Figure 10 shows a sample prompt.Source: §A.3 · p. 19, Figure 10 · p. 19
  • User preference summarization turns SID sequences into short natural-language profiles, using DeepSeek-generated metadata and review summaries as pseudo-labels because explicit preference labels are absent.This open-ended task is also restricted to SFT; Figure 11 shows a sample prompt.Source: §A.3 · p. 19, Figure 11 · p. 19

B DATASETS

The evaluation uses two Amazon Review subsets and applies filtering, history truncation, and chronological train-validation-test splits. Dataset statistics are reported for the resulting training splits.

B DATASETS

  • The evaluation uses the Industrial_and_Scientific and Office_Products Amazon Review subsets.The passage also specifies trimming steps for Toys_and_Games and Industrial_and_Scientific.Source: B DATASETS · p. 19
  • Users and items with fewer than five interactions are removed, histories are capped at ten items, and datasets are split chronologically at an 8:1:1 ratio.The passage gives date windows for Toys_and_Games and Industrial_and_Scientific.Source: B DATASETS · p. 19
  • Table 4 reports statistics for the resulting training splits.The supplied caption identifies it as a dataset-statistics table.Source: Table 4 · p. 19, B DATASETS · p. 19

Related papers

Loading 2510.24431v1…