PaperlayerView exact revision on arXiv

Source-linked AI summary

KUAISHOU Explorer LLM-Rec Challenge 2026: Reasoning Generative Recommendation

Jiangxia Cao, Hao Peng, Wenlong Xu, Jiaxin Deng, Zhixin Ling, Xingmei Wang, Kun Shang, Can Tang, Zhihuai Cai, Jun Du, Fang Su, Xiaojuan Liu, Yiling Li, Chenglong Yu, Chongling Rao, Haixuan Gao, Haitao Xu, Jian Liang, Ruiming Tang, Chenglong Chu, Guohong Mu, Honghui Bao, Hui Wang, Jialong Chen, Jiao Ou, Muhao Wei, Peng Zhang, Renpu Liu, Ruochen Yang, Shugui Liu, Xinqi Jin, Yan Sun, Yifan Wang, Yingzhi He, Yufei Ye, Yusen Huo, Tingkuo Wang, Jihong Zhang, Lanxi Zhu, Pengyuan Liu, Zhipeng Yi, Luankang Zhang, Hang Lv, Xuyang Zhi, Tianyu Li, Bintao Wu, Chuang Ou, Siyue Su, Ziyuan Wang, Yuliang Sun, Baiyan Che, Feiyang Xu, Shiwen Zhang, Shiteng Cao, Chongcong Jiang, Yuan Fang, Xiangwu Yang, Hao Deng, Zijian Du, Pengxun Wang, Xiaoming Wang, Shun Qin, Yingqi Song, Tianyi Li, Naixiao Peng, Chenyu Zhou, Qiliang Jiang, Quan Zheng, Cheng Jin, Siying Zeng, Hongjia Xu, Junwu Hu, Teng Fu, Zhengkang Mei, Haijun Yu, Kai Li, Shengyang Zhou, Zhijia Wei, Siyi Xiong, Bo Liu, Zichun Guo, Zhubin Han, Jinpeng Fu, Bingqian Liu, Yuyi Wang, Yu Liu, Qinghai Tan, Ruijie Zhou, Zhuohang Li, Zhijia Zhong, Xiangnan He, Jirong Wen, Min Zhang, Wenwu Ou, Peng Jiang, Han Li, Kaiqiao Zhan, Yanan Niu, Lantao Hu, Kun Gai

arXiv:2609.39828v1cs.IR

TL;DR

Reasoning CoT does not always improve recommendation, and OneReason addresses this by strengthening Semantic ID–language alignment and structuring interest-reasoning supervision. The challenge supplies OneReason-based models and data for studying reasoning recommendation; the authors report that recommendation-oriented CoT can enhance pretrained semantic-alignment LLM performance.

  • Problem

    Reasoning CoT can be generated yet still underperform direct generation, leaving its benefit to recommendation uncertain.

    Source: §1 · p. 2

  • Method

    The challenge provides OneReason-based pretrained models and SFT data, with SIDs combined with natural language to train the model on item content for user-understanding and recommendation tasks.

    Source: §1 · p. 2, §3.1 · p. 5

  • Results

    The authors report that recommendation-oriented CoT can enhance pretrained semantic-alignment LLM performance and support beneficial reasoning trajectories for RL/RFT.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The report opens the challenge data and benchmark, including multi-domain interactions, item semantics, task-formatted SFT data, and evaluation benchmarks, for reproducible comparisons and method exploration.

    Source: §8 · p. 19

    In the Bridge analysis, removing the transition phrase yields more candidates outside history but does not consistently increase true-target hits, and lower reasoning-text loss does not correspond to better SID prediction.

    Source: §7.2 · p. 18

Abstract

from arXiv · show

Generative recommendation, has been attracted a surge of attentions in industrial and academic research community, towards to build more smart system to build next-generation recommender. Under the significant developing wave of large language model, our team have been developed Semantic ID based OneRec/OneRec-V2. These models have been widely deployed in production and demonstrate the scaling potential of the autoregressive next-item prediction paradigm for industrial recommender systems. Building on the success of OneRec, we further explored a series of models, including OneRec-Think, OpenOneRec, and OneReason, that connect item Semantic IDs with natural language in a unified representation space and seek to unlock the potential of natural-language chain-of-thought (CoT) reasoning for recommendation. However, our preliminary works found that introducing reasoning CoT does not always improve the recommendation performance. To address this issue, OneReason strengthens the semantic alignment between items and language, introduces structured template-based supervision for interest reasoning, and applies advanced reinforcement learning techniques to make reasoning more beneficial to recommendation. As a frontier topic to building recommendation foundation models, we believe this topic has significant research value and hope to encourage more researchers to explore it together. To this end, together with the SIGIR 2026 community, we organized the KUAISHOU Explorer LLM-Rec Challenge 2026: Reasoning Generative Recommendation.

1. Introduction

Generative recommendation frames next-interaction prediction as sequence generation, and OneReason connects item Semantic IDs with natural language. Because reasoning CoT does not always improve recommendation, OneReason strengthens semantic alignment, structures interest-reasoning supervision, and applies reinforcement techniques.

1. Introduction

  • Generative recommendation formulates prediction of users’ subsequent interactions as sequence generation, a direction for scaling recommendation models in the large-model era.The section situates this paradigm alongside the OneRec family’s Semantic IDs and OneReason’s connection of Semantic IDs with natural language.Source: §1 · p. 2
  • OneReason-related models use natural language to understand item content, analyze user interests, and reason before recommendations, combining content semantics with behavioral evidence.The authors describe this as a way to understand user interests and how they evolve.Source: §1 · p. 2
  • Reasoning CoT can underperform direct generation, motivating OneReason’s stronger Semantic ID–language alignment, structured interest-reasoning supervision, and reinforcement techniques.The authors report that recommendation-oriented CoT can enhance pretrained semantic-alignment LLM performance and support beneficial reasoning trajectories for RL/RFT.Source: §1 · p. 2
  • The challenge differs from competitions focused on iterating recommendation models by exploring how RecSys can connect with LLMs to understand content and users and improve recommendation performance.The comparison is with the TAAC 2025 advertising-sequence challenge and TAAC × KDD Cup 2026 conversion-ranking challenge.Source: §1 · p. 2
  • The challenge provides OneReason-based pretrained models, 792,797 instruction SFT examples (412,438 with CoT), and 500,000 anonymous user records across four domains.The user records span short video, e-commerce, advertising, and live streaming, with SIDs, content descriptions, and category information linked through item identifiers.Source: §1 · p. 2

2. Challenge Setting

The challenge evaluates a single OneReason-based model on item, user, recommendation, and general-knowledge tasks. It proceeds through preliminary, semi-final, and onsite final rounds, with changes to the base model and item-understanding task in the semi-final.

2.1. Problem Definition Overview

  • Item understanding uses Semantic IDs across four domains; the preliminary task generates captions from SIDs, and the semi-final task also generates captions from SIDs.The passage states the same SID-to-caption direction for both rounds.Source: §2.1 · p. 3
  • User understanding selects history interactions relevant to a specified topic and generates an interest-evolution chain describing how needs change around it.Source: §2.1 · p. 3
  • Recommendation evaluation predicts target-domain item SIDs from multi-domain histories, then merges separate thinking and non-thinking Pass@32 candidate sets for final scoring.The metric is designed to assess whether thinking produces diverse user-interested items that non-thinking does not predict.Source: §2.1 · p. 3
  • General-knowledge evaluation asks the model to select the correct option for a question, assessing knowledge retained after recommendation-task training.Source: §2.1 · p. 3

2.2. Competition Stages Overview

  • The preliminary round uses OneReason-0.8B-pretrain, 500,000 user sequences, partial seed SFT data, and evaluates four task types; the top 50 teams advance.Item understanding in this round generates Semantic IDs from content descriptions.Source: §2.2 · pp. 3–4
  • The semi-final upgrades to OneReason-8B-pretrain, full seed SFT data, and supplied compute; the onsite final reviews reproducibility before teams present their methods and findings.The semi-final changes item understanding to SID-to-caption generation with an LLM judge; the onsite final selects 20 teams after reviewing code and training data.Source: §2.2 · p. 4

3. OneReason-Pretrain Baseline

OneReason represents items with Semantic IDs and trains the language model to connect those codes with content semantics and user behavior. Its pretraining spans token, item, relation, and user levels, followed by staged training that extends support for longer histories.

3.1. Item Representation and Vocabulary Expansion

  • A multimodal encoder and language decoder produce semantic item vectors, which RQ-KMeans quantizes into three residual-code levels with 8,192 codewords each.Each SID is a domain marker followed by three discrete codes in quantization order.Source: §3.1 · p. 4
  • Semantic-alignment training teaches the model the content associated with SIDs, enabling their use in user-understanding and recommendation tasks.SIDs are added as special vocabulary tokens and combined with natural language in training sequences.Source: §3.1 · pp. 4–5

3.2. Multi-Granularity Semantic Alignment Pretraining

  • OneReason constructs pretraining data at token, item, relation, and user levels to incorporate item-content semantics into behavioral modeling.Source: §3.2 · p. 5
  • Token-level tasks align codes and code combinations with language through code explanations, bidirectional prefix-description mappings, and full-description composition.Source: §3.2 · p. 5
  • Item-level tasks map complete SIDs and descriptions in both directions and answer content questions, while controlling description granularity to avoid details codes cannot recover.The retained description information includes distinguishing topics, categories, and brands.Source: §3.2 · p. 5
  • Relation-level data pairs item codes with language explanations of behavioral links, while user-level data models cross-domain conditioning or chronologically ordered histories.Relation signals include searches following video views, item transitions, and cross-domain co-occurrence; user-level histories can replace some item codes with descriptions.Source: §3.2 · p. 5
  • Pretraining has three stages—new-token parameter training, full-parameter training, and context-length extension—using 110B, 449B, and 19B tokens, respectively.Source: §3.2 · p. 5

3. OneReason-Pretrain Baseline

  • The challenge leaves supervised fine-tuning and reinforcement-learning design to participants, who can construct reasoning supervision and rewards on the pretrained base.Source: §3.2 · p. 6

4. Data Resources

The challenge releases SFT examples, multi-domain user histories, and item-semantic mappings. These resources support four task types, with think and nothink response formats in SFT and item information linked to interactions by domain and anonymized item identifier.

4. Data Resources

  • The challenge provides 944,802 SFT seed examples, 500,000 anonymized multi-domain user histories, and SID-caption alignment data for 35,914,095 items.Source: §4 · p. 6

4.1. Supervised Fine-Tuning Data

  • SFT conversations cover item understanding, user understanding, recommendation, and general-purpose tasks, using think or nothink response formats.Table 1 reports example and token counts and average lengths by task; recommendation-related token volumes and example-length distributions are shown in Figure 2.Source: §4.1 · p. 6, Figure 2 · p. 7

4.2. User Interactions

  • User histories span short video, e-commerce, advertising, and live streaming, with domain-specific feedback fields and cross-domain interactions stored in each user record.Feedback includes watch duration and engagement, shopping impressions through purchases, ad clicks and conversions, and live-streaming engagement counts.Source: §4.2 · p. 7
  • Table 2 summarizes user coverage, historical interaction counts, and sequence lengths by domain; Figure 3 shows per-domain and total-history length distributions.Source: §4.2 · p. 7, Table 2 · p. 7, Figure 3 · p. 8

4.3. Item Semantics Alignment

  • Item semantics are supplied as SIDs, natural-language captions, and three-level category labels, mapped to interactions through domain and anonymized item identifiers.Each SID contains a domain marker and three discrete codes; multiple items can share an SID, and Table 3 summarizes item, code, and description scale by domain.Source: §4.3 · p. 8, Table 3 · p. 8

5. Tasks and Evaluation

The challenge evaluates one model across item, user, recommendation, and general-knowledge tasks, using task-specific test data and metrics. Its evaluation covers item-content recovery, user-interest reasoning, target-domain recommendation, and knowledge-question answering.

5.1. Overview of Evaluation Tasks

  • The four capability dimensions are item understanding, user understanding, recommendation, and general knowledge, each assessed through a distinct task.Tasks require recovering item content from SIDs, finding relevant historical interactions and interest evolution, predicting items in a target domain, or answering multiple-choice questions.Source: §5.1 · p. 9, Table 4 · p. 9

5.2. Test Data Construction

  • Test examples draw on item content, user interaction logs, and knowledge questions, with recommendation targets drawn from subsequent interactions in a specified domain.Each knowledge example has one correct answer; recommendation examples may contain multiple ground-truth targets within the single specified prediction domain.Source: §5.2 · pp. 9, 11, Table 5 · p. 10
  • Item-understanding examples pair an SID with a reference description, whose weighted factual points are used to assess generated descriptions.Descriptions are generated from original item content and checked by model review and manual quality checks; evaluation matches generated descriptions against weighted reference information points.Source: §5.2 · pp. 9–10
  • User-understanding examples use chronological, multi-domain histories to select topic-relevant interactions and generate interest-evolution chains.Candidate chains are filtered for chronological order, behavioral support, and clear progression or shifts in interest; chains are limited to five steps.Source: §5.2 · p. 10
  • Recommendation examples use histories to predict subsequent interactions in a target domain, with domain-specific high-value targets and filtering, downsampling, and category balancing.Targets include product clicks, video interactions above the duration-bucket 75th percentile, advertising conversions, and first-time live-stream tipping.Source: §5.2 · pp. 10–11

5.3. Evaluation Methods and Metrics

  • Subtask scores use separate metrics and task weights to form the overall ranking score; item descriptions are scored for matched, missing, and additional information.Item scoring uses weighted information points and BERTScore F1 for matched pairs; both importance and match quality contribute.Source: §5.3 · pp. 11–12
  • Relevant-behavior selection uses mean F1 across examples, while interest-evolution chains are scored by event alignment and explanation similarity.Event matching preserves order in both chains and maximizes total similarity; explanation similarity equally weights Token F1 and ROUGE-L F1, and unparseable responses receive zero.Source: §5.3 · pp. 12–13
  • Recommendation merges and deduplicates think-mode and nothink-mode candidates, then scores whether any predicted item matches a subsequent target.General-knowledge responses are scored by whether the extracted option letter matches the reference answer.Source: §5.3 · p. 13

6. Competition Platform and Fairness

The competition used Kuaishou’s Vanchin platform for model training and consistent evaluation, with different compute provisions by round. Reproducibility and fairness checks covered pipeline reproduction, duplicate data, and potential test leakage.

6.1. Platform and Computational Resources

  • Preliminary teams received a weekly 0.3B-token training quota, while semi-final teams received a development server with 4 × 80GB GPUs and an unlimited token quota.Teams could also use their own compute; Vanchin provided inference resources in both rounds.Source: §6.1 · pp. 13–14
  • Vanchin applied a common inference and scoring procedure to submissions, reporting sub-task and overall scores and updating the leaderboard daily.This kept evaluation conditions consistent across submissions while participants varied their data and training methods.Source: §6.1 · p. 14

6.2. Reproducibility and Fairness

  • Each semi-final training pipeline was reproduced from OneReason-8B-pretrain within 72 hours, then evaluated five times and compared with its leaderboard score.The review preserved data and teacher-model snapshots and also assessed generalization on additional same-distribution test data.Source: §6.2 · p. 14
  • Fairness checks detected exact cross-team duplicates and screened for test leakage across user histories, item SIDs, and SID–description pairs.Flagged records were manually reviewed alongside their data sources and construction procedures.Source: §6.2 · p. 14

7. Participation and Leading Approaches

The challenge drew 2,031 registered participants across 1,206 teams, and leading approaches combined data refinement, reasoning training, and preference optimization. Team solutions also explored hierarchical SID feedback and natural-language transitions in recommendation generation.

7.1. Participation and Competition Results

  • 2,031 participants formed 1,206 teams; 812 teams completed preliminary evaluations, and 50 advanced to the semi-final, where rankings covered four capability areas.The preliminary round received 33,059 valid submissions and the semi-final received 4,861.Source: §7.1 · p. 14

7.2. Leading Participant Approaches

  • Target-SID-masked training on added self-sampled reasoning trajectories improved candidate complementarity and recommendation performance over full-trajectory supervision or merely downweighting repeated answers.Limited numbers of distinct trajectories helped, while adding more reduced gains; self-sampled trajectories also outperformed larger-teacher-generated data under target masking.Source: §7.2 · p. 15
  • The runner-up refined item descriptions and used domain-specific recommendation supervision, with a mixed think/nothink configuration outperforming either format across all four domains.Constrained rewriting improved item understanding in short video and e-commerce; the final approach also retained original user-understanding data, while changes to user-intent cues did not improve both user subtasks simultaneously.Source: §7.2 · p. 15
  • Graded rewards raised first-level SID accuracy without improving two-level or complete-SID accuracy; prefix guidance supplied correct prefixes when free sampling lacked a sufficiently correct candidate.Hierarchical credit assignment compared code choices sharing a parent prefix and excluded supplied prefixes and preceding segments from policy-gradient computation.Source: Figure 4 · p. 16, §7.2 · p. 17, Figure 5 · p. 17
  • The third-place approach staged SFT, task-specific reinforcement learning, and recommendation preference optimization, using pack-level cross-entropy to balance packed sequences.For general knowledge, single-answer constraints, a progressive length penalty, and groups with differing rewards reduced repetition and improved performance; user-understanding RL used behavior-selection F1 feedback.Source: §7.2 · p. 16
  • Recommendation preference pairs held reasoning text fixed while varying the final SID, and their training stage improved short-video, e-commerce, and live-streaming performance while advertising remained largely stable.The stage combined a label-smoothed preference loss with a positive-example generation loss.Source: §7.2 · p. 16
  • Removing the Bridge improved local validation but reduced online test performance, while Bridge-trained models reused more historical items without consistently increasing true-target hits.The team hypothesized that the transition phrase affects competition between historical items and targets outside the history, and that output-format alignment alone is insufficient.Source: Figure 6 · p. 18, §7.2 · p. 18
  • Bridge-GRPO removed the fixed domain-prefix constraint, letting the model generate the natural-language transition and subsequent SID autonomously under recommendation rewards.It retained a two-stage process of reasoning followed by answer generation.Source: §7.2 · p. 18

8. Conclusion

The report reviews the challenge, participant solutions, and lessons, and releases its data and benchmark as shared research resources. It frames the challenge as a small step toward connecting large language models with recommender systems.

8. Conclusion

  • Participants explored semantic alignment, reasoning supervision, multi-task learning, reinforcement learning, preference optimization, and hierarchical recommendation, illustrating both promise and remaining challenges.The report reviews leading solutions and summarizes insights from the competition.Source: §8 · p. 18
  • The report open-sources challenge data and benchmarks, including multi-domain interactions, item semantics, task-formatted SFT data, and evaluation benchmarks.The authors hope these resources will support reproducible comparisons and exploration of new methods.Source: §8 · p. 19
  • The authors describe the challenge as a small step in the broader effort to connect large language models with recommender systems.They invite further research and contributions to the field.Source: §8 · p. 19

Related papers

Loading 2609.39828v1…