PaperlayerView exact revision on arXiv

Source-linked AI summary

RecGPT-V3 Technical Report

Bowen Zheng, Chao Yi, Dian Chen, Gaoyang Guo, Han Zhu, Jiakai Tang, Jian Wu, Mao Zhang, Wen Chen, Yifan Lu, Yujie Luo, Yuning Jiang, Zhujin Gao, Bo Zheng, Chenchi Zhang, Dixuan Wang, Hao Fang, Jiancai Liu, Jing Yu, Junjun Zheng, Ke Chen, Kewei Zhu, Mengyan Li, Mingke Xu, Wenjun Yang, Xiangheng Kong, Xinming Zhang, Xunke Xi, Zile Zhou

arXiv:2607.15591v2cs.IR

TL;DR

Industrial-scale LLM recommenders still face stateless user modeling, a tag-to-item bottleneck, and costly explicit reasoning. RecGPT-V3 addresses these challenges with persistent memory, joint text–Semantic ID reasoning, and latent intent tokens, improving online recommendation and business metrics while reducing serving compute by 52.4%.

  • Problem

    Industrial-scale LLM recommenders face stateless user modeling, lossy tag-to-item grounding, and inefficient explicit reasoning that limit further progress.

  • Method

    RecGPT-V3 combines a structured evolving Memory Hub, joint natural-language and Semantic ID reasoning, and compact decodable latent intent tokens.

  • Results

    +1.28% IPV, +1.00% CTR, +1.97% TC, and +3.97% GMV were achieved in large-scale online A/B tests, while end-to-end serving compute fell 52.4%.

  • Takeaways & Limitations

    RecGPT-V3 shows that LLMs can support real-world industrial recommendation pipelines while improving engagement and business metrics at lower serving cost.

  • Takeaways & Limitations

    Continual pre-training on domain-specific patterns can cause overfitting and loss of general capabilities needed for real-world deployment.

Abstract

from arXiv · show

Large language models (LLMs) are transforming recommender systems from matching co-occurrence patterns in historical behavior toward reasoning about the intent that drives it. RecGPT-V1 pioneered this paradigm on Taobao by centering user understanding, and RecGPT-V2 scaled it via coordinated multi-agent reasoning; both are deployed in production with consistent gains in user experience and commercial outcomes. However, operating RecGPT at scale reveals three challenges: (1) stateless behavior modeling, where each request reprocesses full user history, wasting computation and discarding prior analysis; (2) a tag-to-item information bottleneck, where natural-language tags form a lossy channel between user understanding and item grounding; and (3) inefficient explicit reasoning, whose lengthy chain-of-thought incurs untenable latency and compute overhead. We present RecGPT-V3, a stateful, hybrid-modal recommender that reasons over natural language for open-world knowledge and Semantic IDs (SIDs) for concrete item grounding. A Memory Hub maintains structured, continually evolving user memory that distills long-horizon behavior into condensed units, cutting user-modeling computation by 55.8%. A Hybrid-modal Foundation Model allows the LLM jointly reason over text tags and SIDs, opening a high-bandwidth channel into the item space. Latent Intent Reasoning internalizes verbose rationales into compact learnable latent tokens that remain decodable into readable explanations, lowering output token cost by 200x. Deployed in Taobao's "Guess What You Like" feed, RecGPT-V3 achieves consistent gains in large-scale online A/B tests: IPV +1.28%, CTR +1.00%, TC +1.97%, GMV +3.97%, while cutting end-to-end serving resource consumption by 52.4%.

1. Introduction

RecGPT-V3 addresses the limitations of interaction-prediction recommenders by reasoning about user intent with a stateful memory, hybrid natural-language/SID item representation, and compact latent reasoning. Deployed on Taobao, it improves engagement and business metrics while reducing computation and serving cost.

  • Motivation: LLMs recast recommendation as reasoning about the motivation and potential interest behind user behavior, beyond statistical co-occurrence prediction.Traditional recommenders primarily predict the next interaction from historical behavioral regularities, whereas LLMs use contextual interpretation and broad world knowledge.
  • Challenges: RecGPT-V3 targets stateless history reprocessing, lossy tag-to-item grounding, and costly explicit chain-of-thought generation.The explicit rationales average approximately 3,000 tokens in the task, creating prohibitive latency at billion-user scale and high QPS.
  • S1 Memory Hub: 55.8% lower user-modeling computation comes from the Memory Hub’s continually curated memory units, which replace repeated full-history processing with consolidated memory and recent behavior.The memory compresses tokens by 94.5% in condensed form, preserves provenance, and evolves as new interactions arrive.
  • S2 Hybrid-modal Recommendation Foundation Model: Hybrid-modal reasoning combines natural language for open-ended intent and world knowledge with Semantic IDs for concrete item representations enriched by collaborative signals.Using SIDs as a second modality addresses the information bottleneck between natural-language tags and concrete item retrieval.
  • S3 Latent Intent Reasoning: 200× lower reasoning token cost is achieved by replacing verbose chain-of-thought with learnable latent tokens that can decode into readable rationales on demand.Dedicated latent <CoT> slots encode the reasoning trace while retaining explainability for production recommendation use.
  • Online deployment and results: +1.28% IPV, +1.00% CTR, +1.97% TC, and +3.97% GMV were achieved against live RecGPT-V2 A/B testing, alongside 52.4% lower end-to-end serving compute.RecGPT-V3 was deployed in Taobao’s “Guess What You Like” homepage scenario.

2. Memory Hub

The Memory Hub replaces repeated full-history analysis with a compact, structured memory that is initialized once and continually updated as new behaviors arrive. This stateful design preserves traceability and recommendation quality while reducing Global Planner computation by 55.8%.

  • Structured Behavior Compression: 80% token reduction comes from consolidating full behavioral histories into compact, schema-defined memory units containing coherent, high-confidence patterns.Memory units retain provenance links to source behaviors, supporting interpretability and downstream recommendation quality.
  • Memory Hub: 55.8% lower Global Planner compute results from combining initial behavior compression with periodically curated memory updates.The two mechanisms preserve recommendation quality while maintaining a continually evolving user representation.
  • Structured Behavior Compression: The initial compression pass produces persistent memory units labeled by a taxonomy of hundreds of behavioral patterns, with controlled extension for uncovered high-confidence clusters.The memory is constructed once and serves as the foundation for later incremental updates.
  • Evolving Memory Curation: Evolving curation updates relevant units, retains unaffected units, and extracts new units from unmatched behaviors using only compressed memory and the new behavioral delta.A single model forward pass jointly performs selective updates and new-pattern extraction, while semantic continuity resynthesizes affected units into current snapshots.
  • Evolving Memory Curation: Incremental curation lets the system reason over compact memory plus recent behavior instead of re-deriving user understanding from the full interaction history at every request.This allows user understanding to accrue and refine over time; production curation runs every two months.

3. Hybrid-Modal Recommendation Foundation Model

The Hybrid-Modal Recommendation Foundation Model augments Qwen3-14B with Semantic IDs (SIDs) alongside natural language, jointly supporting open-ended intent reasoning and concrete item grounding. It builds SIDs from multimodal item representations and trains the unified vocabulary through continual pre-training and instruction tuning.

  • Motivation and model design: Natural-language tags create a lossy bottleneck, whereas SIDs provide retrieval-compatible, fine-grained item identifiers alongside language.Natural language expresses rich intent and open-world knowledge; SIDs ground that intent in item representations enriched by collaborative signals.
  • Hybrid tokenization: 65,536 SID tokens extend the Qwen3-14B vocabulary, produced by CN-CLIP and a two-level RQ-VAE quantization pipeline.Each of the two codebook levels has 32,768 entries, with coarse-to-fine codes preserving distinctions among related items.
  • Hybrid tokenization: Multimodal item representations fuse text, images, and side information, while behavior-mined co-occurrences provide positive pairs for contrastive learning.The contrastive objective stabilizes subsequent quantization by producing a well-separated embedding space that avoids codebook collapse.
  • Continual pre-training: Continual pre-training grounds SID tokens in item semantics while preserving general capabilities through SID-grounding data and approximately 10% general-domain text.Grounding samples pair each SID with textual attributes so the model can recover item content from identifiers it already understands.
  • Instruction tuning: Instruction tuning teaches bidirectional SID–text translation, sequential recommendation directly in SID space, and general-domain capability preservation.SID-only operation provides direct evidence that collaborative-filtering signals are internalized through the tokens without textual item descriptions.

4. Latent Intent Reasoning

Latent Intent Reasoning replaces costly explicit chain-of-thought with compact learnable latent tokens that support efficient inference while preserving readable explanations. A two-stage post-training pipeline distills and aligns these tokens, then optimizes them with ranking feedback.

  • Latent representation: Latent Intent Reasoning internalizes explicit traces into short learnable latent tokens that encode contiguous reasoning segments.The trace is partitioned into K segments, with each latent token representing one segment and K much smaller than the original number of steps.
  • Motivation and overview: 200× lower reasoning-token cost comes from compressing thousands of autoregressively decoded rationale steps into a handful of latent tokens.The approach shifts computation from slow autoregressive decoding to parallelizable prefill, reducing inference latency.
  • Explainability and alignment: Human-readable rationales remain decodable from the latent tokens, preserving interpretability at substantially lower token cost.Multi-granularity reconstruction trains the latent sequence as a sufficient encoding of the replaced reasoning trace.
  • Post-training pipeline: The two-stage post-training pipeline first distills teacher reasoning and compresses it through multi-task alignment, then optimizes the latent model against online business objectives.The second stage uses Reinforcement Learning from Ranking Feedback to obtain rewards directly from the production ranking model.

5. Experiments

Experiments on Taobao show that RecGPT-V3 improves recommendation quality across online scenarios while reducing computation through memory compression and latent reasoning. Ablations further show that hybrid text–SID retrieval, general-domain data mixing, and latent reasoning are important for preserving capabilities and balancing effectiveness with efficiency.

  • Online A/B Tests: +1.28% IPV, +1.00% CTR, +3.97% GMV, and +1.97% TC were achieved in the feed scenario, while item-level gains reached +7.51% GMV and +3.10% TC.RecGPT-V3 consistently outperformed RecGPT-V2 across both online evaluation scenarios.
  • Memory Efficiency: 55.80% lower Global Planner compute cost was obtained versus the RecGPT-V2 baseline by conditioning on compressed memory and recent behavioral changes.Inference cost fell to 33.43% per pass, while incremental memory curation added 10.77% overhead.
  • General-Domain Data: 4.70% GSM8K, 0.12% MMLU, 0.01% CMMLU, and 23.29% IFEval resulted without general-domain data, indicating catastrophic capability collapse.Mixing general-domain data preserved most backbone capabilities and maintained SID–language alignment.
  • Latent Intent Reasoning: HR@30 improved from 0.3050 to 0.3508 with explicit CoT, while latent reasoning reached 0.3462 HR@30 and 0.0649 CTR using 10 latent tokens.RL further improved performance to 0.3693 on HR@30 and 0.0679 on CTR, surpassing explicit CoT on both metrics.
  • Latent Intent Reasoning: 95.7% fewer output tokens and a 3.46× end-to-end speedup were achieved by reducing reasoning output from 2,840 to 122 tokens.End-to-end time decreased from 1,020s to 295s, with input throughput increasing from 166K to 498K tokens per minute.
  • Hybrid Retrieval: 0.1571 HR@500 was achieved by hybrid retrieval, exceeding 0.1539 for SID retrieval and 0.1503 for text-tag retrieval.The hybrid model combines the broad coverage of text tags with the collaborative precision of SIDs.

6. Conclusion

RecGPT-V3 is an LLM-based recommender designed for industrial-scale recommendation. It addresses stateless behavior modeling and the tag-to-item information bottleneck through structured user memory and joint reasoning over natural language and Semantic IDs.

  • Conclusion: RecGPT-V3 addresses three industrial-scale challenges: stateless behavior modeling, the tag-to-item information bottleneck, and inefficient explicit reasoning.The conclusion frames these as challenges facing LLM-based recommendation at industrial scale.
  • Conclusion: The Memory Hub maintains structured, continually evolving user memory instead of repeatedly encoding the full history in a one-shot process.This replaces one-shot full-history encoding with persistent user memory.
  • Conclusion: The Hybrid-modal Foundation Model jointly reasons over natural language and Semantic IDs to address the tag-to-item information bottleneck.The passage identifies joint reasoning over these two modalities as RecGPT-V3’s grounding mechanism.

Appendix · A. Contributors · Core Contributors

The appendix’s Core Contributors section lists the paper’s authors in alphabetical order by first name. The contributors are presented across two author-list passages.

  • Core Contributors: Bowen Zheng, Chao Yi, Dian Chen, Gaoyang Guo, Han Zhu, and Jiakai Tang are listed as core contributors.These names appear in the first contributor listing.
  • Core Contributors: Jian Wu, Mao Zhang, Wen Chen, Yifan Lu, Yujie Luo, and Yuning Jiang are listed as core contributors.These names continue the first contributor listing.
  • Core Contributors: Zhujin Gao, Bo Zheng, Chenchi Zhang, Dixuan Wang, Hao Fang, and Jiancai Liu are listed as core contributors.Zhujin Gao concludes the first listing, followed by names from the second listing.
  • Core Contributors: Jing Yu, Junjun Zheng, Ke Chen, Kewei Zhu, Mengyan Li, and Mingke Xu are listed as core contributors.These names appear in the second contributor listing.
  • Core Contributors: Wenjun Yang, Xiangheng Kong, Xinming Zhang, Xunke Xi, and Zile Zhou are listed as core contributors.These names conclude the second contributor listing.
  • Core Contributors: The authors are listed in alphabetical order based on their first names.The appendix explicitly states the ordering rule for the author listing.

B. Intent-to-Item Retrieval

Intent-to-item retrieval uses a hybrid model that combines natural-language tags with Semantic IDs, preserving both open-ended intent understanding and concrete item grounding. The model learns to rank candidate items by likely click preference while maintaining alignment with upstream intent predictions.

  • Hybrid Intent Representation: Tags provide generalizable open-world knowledge, while SIDs ground user intent in concrete item representations enriched by collaborative signals.The two channels serve distinct roles in expressing user intent.
  • Hybrid Retrieval Model: The hybrid retrieval model consumes both text tags and SIDs to learn a preference ordering over candidate items.Its purpose is accurate downstream item retrieval from the hybrid signals.
  • Hybrid Retrieval Model: The model favors items a user is likely to click while preserving semantic alignment with upstream intent predictions.The section develops its hybrid input architecture and joint training objective.

B.1. Architecture

The retriever jointly models natural-language tags for semantic intent and Semantic IDs (SIDs) for concrete item grounding. A dual-embedding architecture fuses both channels into unified intent queries for target-attention retrieval, combining language semantics with collaborative item-level signals.

  • Dual-channel inputs: Text tags express user intent, while paired SIDs identify which items satisfy that intent in the discrete item space.The two channels capture complementary sides of the same intent: “what the user wants” and “which items satisfy that want.”
  • Dual-embedding fusion: Each tag and SID is independently embedded, concatenated, and projected through a learnable linear transformation into a unified intent representation.The resulting intent vectors serve as queries for target attention over candidate item representations.
  • Target-attention retrieval: The unified intent vectors attend over candidate item representations to compute retrieval scores while preserving channel-specific information and enabling cross-channel interaction.Target attention performs the shared computation after the channels remain distinct in earlier layers.
  • Collaborative evidence: Contrastive pre-training gives the SID embedding space item-level signals unavailable to text alone, including co-purchase patterns, popularity dynamics, and item-to-item affinity.The dual channels also cover different regions of the item space, pairing language’s semantic breadth with collaborative precision.

B.2. Training Recipe

The retriever is trained with a joint utility–relevance objective because click-only supervision favors popular items and can misalign retrieval with predicted tags and SIDs. Hierarchical preference ordering prioritizes click utility while using semantic relevance to discriminate among similarly useful candidates.

  • Motivation: Click-only supervision favors globally popular items and can retrieve clickable products inconsistent with upstream tags and SIDs.Click propensity is confounded with item popularity, weakening intent grounding.
  • Preference Ordering: The hierarchical preference treats click likelihood as primary and semantic relevance as the discriminator among candidates with comparable utility.This ordering integrates utility and relevance signals.
  • Relevance Objective: The relevance loss uses multi-level contrastive learning to order candidates sharing the same click status.It constructs relevance-positive sets at progressively coarser matching criteria and supplies discrimination beyond the utility loss.
  • Joint Learning Objective: The final objective is a weighted sum of utility and relevance losses, with α = 1 and β = 0.5 in production experiments.Priority ordering is clicked > exposed > non-exposed, and negatives are restricted to candidates no higher priority than the positive.

C. Latent Reasoning Reconstruction

Latent Reasoning Reconstruction uses prompts tailored to reconstruct either individual reasoning segments or an entire chain of thought from latent <cot> tokens. The reconstructed explanations preserve recommendation context and recover coherent natural-language rationales for representative production cases.

  • Reconstruction prompts: Two prompt variants reconstruct either single or multiple masked segments, or the full chain of thought when all K segments are masked.The single-/multi-segment prompt asks what each <cot> token represents, while full-trace reconstruction recovers the entire chain of thought.
  • Reconstruction inputs: Each reconstruction sample combines the recommendation context, latent-token <think> block, and model output before decoding masked tokens into natural-language explanations.The context includes an expert persona, user profile, and click history.
  • Representative cases: The bag-recommendation case reconstructs sustained interest in backpacks and crossbody bags, including neo-Chinese, vintage, rhinestone, embroidery, waterproof, and large-capacity preferences.These preferences are inferred from the user profile and click history represented in the reconstructed reasoning.
  • Representative cases: The full-trace car-accessory case produces recommendations such as a cute cartoon car ornament and sun-protective cartoon sunshade from purchase records and practical-aesthetic demand dimensions.The reconstructed rationale covers interior ambiance, practical accessories, and exterior decoration while connecting them to the user’s preferences.
Loading 2607.15591v2…