Source-linked AI summary
Generative Large Recommendation Models: Emerging Trends in LLMs for Recommendation
Hao Wang, Wei Guo, Luankang Zhang, Jin Yao Chin, Yufei Ye, Huifeng Guo, Yong Liu, Defu Lian, Ruiming Tang, Enhong Chen
TL;DR
Existing surveys and tutorials have mainly covered LLM-enhanced recommendation, leaving generative large recommendation models less systematically explored. This tutorial reviews both approaches, with particular attention to generative models, their challenges, and future directions. It provides a systematic introduction to scaling-law generative recommendation model design.
Problem
Prior surveys and tutorials mainly focused on LLM-enhanced recommendations, leaving generative large recommendation models without systematic exploration.
Source: §3.1 · p. 2Method
The tutorial surveys LLMs for recommendation and examines generative models through data quality, scaling, behavior mining, efficiency, and future directions.
Source: §3.1 · p. 2, §3.2 · p. 3, §3.4 · p. 3Results
The tutorial systematically introduces generative large recommendation model design alongside a summary of advances in LLM-enhanced recommendations.
Source: §3.3 · p. 3Takeaways & Limitations
Participants can gain insights into current developments, challenges, and future research directions for applying or studying generative large recommendation models.
Source: §3.1 · p. 2, §3.2 · p. 3
Abstract
from arXiv · showhide
In the era of information overload, recommendation systems play a pivotal role in filtering data and delivering personalized content. Recent advancements in feature interaction and user behavior modeling have significantly enhanced the recall and ranking processes of these systems. With the rise of large language models (LLMs), new opportunities have emerged to further improve recommendation systems. This tutorial explores two primary approaches for integrating LLMs: LLMs-enhanced recommendations, which leverage the reasoning capabilities of general LLMs, and generative large recommendation models, which focus on scaling and sophistication. While the former has been extensively covered in existing literature, the latter remains underexplored. This tutorial aims to fill this gap by providing a comprehensive overview of generative large recommendation models, including their recent advancements, challenges, and potential research directions. Key topics include data quality, scaling laws, user behavior mining, and efficiency in training and inference. By engaging with this tutorial, participants will gain insights into the latest developments and future opportunities in the field, aiding both academic research and practical applications. The timely nature of this exploration supports the rapid evolution of recommendation systems, offering valuable guidance for researchers and practitioners alike.
1 FORMAT AND INTENDED AUDIENCE
The tutorial is a three-hour, on-site session combining lectures and interactive discussion for people interested in generative recommendation models.
1 FORMAT AND INTENDED AUDIENCE
- The three-hour tutorial is an on-site, lecture-style session with in-person presenters and questions and feedback throughout.It mixes presentations with interactive discussions.Source: §1 · p. 1
- It is designed for researchers and academics interested in generative recommendation, with basic familiarity with LLMs and traditional recommendation systems.Source: §1 · p. 1
- Information retrieval professionals can deepen their expertise, while professionals in other fields can gain insights for designing domain-specific generative models.Source: §1 · pp. 1–2
2 PRESENTERS
The presenters bring experience from USTC and Huawei Noah’s Ark Lab across recommender systems, data mining, information retrieval, and large language models.
2 PRESENTERS
- Hao Wang specializes in representation learning, graph mining, sequential modeling, and LLMs; Luankang Zhang focuses on recommender systems and data mining.Source: §2 · p. 2
- Wei Guo researches recommender systems, deep learning, and data mining, while Jin Yao Chin focuses on recommender systems and deep learning at Huawei Noah’s Ark Lab.Source: §2 · p. 2
- Yufei Ye is an undergraduate at USTC researching recommendation systems under Dr. Hao Wang and Prof. Defu Lian.Source: §2 · p. 2
- Huifeng Guo specializes in recommendation, information retrieval, and advertising, while Yong Liu researches LLMs, search, and recommendation systems.Source: §2 · p. 2
- Defu Lian is a USTC professor focused on data mining, deep learning, and recommender systems; Enhong Chen is a USTC professor and executive dean.Source: §2 · p. 2
- Ruiming Tang directs Huawei Noah’s Ark Lab’s Recommendation & Search Lab and researches recommender systems and deep learning.Source: §2 · p. 2
3 TOPIC AND RELEVANCE
The tutorial addresses the gap in systematic coverage of generative large recommendation models, pairing an overview of LLM-enhanced recommendations with a focused treatment of scaling-oriented generative models and their challenges.
3.1 Motivation
- Earlier surveys and tutorials mainly covered LLM-enhanced recommendation, while generative large recommendation models follow scaling laws and have recently reached trillion-parameter online deployment.The tutorial presents both approaches and focuses on the less systematically explored generative-model path.Source: §3.1 · pp. 2–3
- The tutorial examines data quality and regeneration, data and model scaling, complex user-behavior mining, and training and inference efficiency.Efficiency is identified as important for enabling model-size scalability.Source: §3.1 · p. 2
- Future directions include data distillation and selection, tokenizer technology and side information, and incremental updates for practical applications.The tutorial aims to help participants understand developments, challenges, and research directions.Source: §3.1 · p. 2
3.2 Objective
- The tutorial aims to provide an in-depth account of LLMs for recommendation, especially the underexplored systematic study of generative large recommendation models.Source: §3.2 · p. 3
3.3 Relevance
- The tutorial aligns with Web Conference themes and distinguishes itself from prior tutorials by systematically introducing scaling-law generative recommendation model design.Source: §3.3 · p. 3
3.4 Outline
- The outline spans LLM-based recommendation, data-centric methods and scaling laws, complex user behavior, efficiency, representation enhancement, and incremental learning.Source: §3.4 · p. 3
4 TUTORIAL DETAILS
The tutorial is planned as a three-hour, in-person event, with materials posted online and contingency options for remote access.
4 TUTORIAL DETAILS
- The tutorial is planned for three hours, with all presenters attending in person.Source: §4 · p. 3
- Slides will be posted on the tutorial website, and organizers can obtain the necessary copyright permissions.Source: §4 · p. 3
- If unforeseen circumstances arise, organizers can use prerecorded lectures; live streaming is also possible pending approval.Source: §4 · p. 3