Source-linked AI summary
MTGR: Industrial-Scale Generative Recommendation Framework in Meituan
Ruidong Han, Bin Yin, Shangyu Chen, He Jiang, Fei Jiang, Xiang Li, Chi Ma, Mincong Huang, Xiaoguang Li, Chunzhen Jing, Yueming Han, Menglei Zhou, Lei Yu, Chuan Liu, Wei Lin
TL;DR
Generative recommendation improves scalability but traditionally discards DLRM cross features, whose removal severely harms performance. MTGR retains those features in an HSTU-based framework with user-level compression, normalization, masking, and optimized training, achieving 65x forward FLOPs with higher conversion and CTR, unchanged training cost, and lower inference cost.
Problem
Generative recommendation requires removing DLRM cross features, and the resulting performance degradation cannot be compensated by scaling up.
Method
MTGR combines DLRM features, including cross features, with HSTU-based tokenized modeling, Group-Layer Normalization, dynamic masking, and optimized training.
Results
65x FLOPs per sample raised conversion volume by 1.22% and CTR by 1.31%, while training cost remained unchanged and inference cost decreased by 12%.
Takeaways & Limitations
MTGR was deployed in Meituan’s take-away recommendation system, serving hundreds of millions of users.
Abstract
from arXiv · showhide
Scaling law has been extensively validated in many domains such as natural language processing and computer vision. In the recommendation system, recent work has adopted generative recommendations to achieve scalability, but their generative approaches require abandoning the carefully constructed cross features of traditional recommendation models. We found that this approach significantly degrades model performance, and scaling up cannot compensate for it at all. In this paper, we propose MTGR (Meituan Generative Recommendation) to address this issue. MTGR is modeling based on the HSTU architecture and can retain the original deep learning recommendation model (DLRM) features, including cross features. Additionally, MTGR achieves training and inference acceleration through user-level compression to ensure efficient scaling. We also propose Group-Layer Normalization (GLN) to enhance the performance of encoding within different semantic spaces and the dynamic masking strategy to avoid information leakage. We further optimize the training frameworks, enabling support for our models with 10 to 100 times computational complexity compared to the DLRM, without significant cost increases. MTGR achieved 65x FLOPs for single-sample forward inference compared to the DLRM model, resulting in the largest gain in nearly two years both offline and online. This breakthrough was successfully deployed on Meituan, the world's largest food delivery platform, where it has been handling the main traffic.
1 Introduction
MTGR addresses the tension between DLRM’s cross-feature effectiveness and GRM’s scalability by reorganizing features within an HSTU-based framework. It combines normalization, masking, and training-system optimizations, achieving strong offline and online gains while reducing inference cost.
- Motivation: DLRM scaling is limited because long user behavior is difficult to process and candidate-dependent computation raises training and inference costs.These constraints motivate a ranking model that scales efficiently under high-QPS and low-latency requirements.
- Motivation: GRM compresses user exposures and uses efficient transformer attention, but removing candidate cross features severely damages performance.The reported degradation cannot be compensated by scaling up.
- MTGR Framework: MTGR retains DLRM inputs, including cross features, while converting user and candidate features into tokens and optimizing them with a discriminative loss.This feature arrangement enables the framework to combine DLRM inputs with GRM-style scaling.
- MTGR Framework: MTGR adds Group-Layer Normalization for heterogeneous token types and dynamic masking with full-attention, autoregressive, and self-only visibility patterns.The masking strategy is designed to avoid information leakage.
- Results: 65x FLOPs per sample increased conversion volume by 1.22% and CTR by 1.31%, while training cost stayed unchanged and inference cost fell 12%.MTGR-large was deployed in Meituan’s take-away recommendation system and serves hundreds of millions of users.
- Results: The authors report a power-law relationship between MTGR performance and computational complexity, with superiority to DLRM shown through offline and online experiments.The contribution summary describes systematic TorchRec-based training optimizations supporting substantially larger computational complexity.
2 Related Works
Related work scales recommendation models through DLRM user or cross modules, while GRM uses HSTU and generative modeling for broader sequence processing and scalability.
- DLRM Scaling: DLRM scaling either enlarges the cross module for feature interactions or enlarges the user module for more inference-friendly user representations.User representations can be cached or broadcast across candidate items to reduce online inference costs.
- DLRM Scaling: Cross-module scaling improves user-candidate interactions but increases computation linearly with candidate count, whereas user-module scaling avoids that burden by sharing user representations.The two approaches differ mainly in which portion of DLRM receives additional capacity.
- Generative Recommendation: GRM validates scaling through HSTU and can replace traditional ID representations with semantic coding in a unified generative model.The cited work also combines semantic coding with DPO optimization and transformer-based modeling.
3 Preliminary
Traditional ranking independently embeds user, sequence, real-time, cross, and candidate features, applies target attention to behavior sequences, and predicts a candidate logit with an MLP. Its cross-module scaling creates latency that motivates MTGR’s rearranged, sub-linear approach.
- Data Arrangement: A traditional sample combines user profile U, historical sequence −→S, real-time sequence −→R, cross features C, and candidate features I.Historical and recent interaction sequences contain item features such as IDs, tags, and item-level statistics.
- Data Arrangement: Figure 1 arranges simplified user, sequence, real-time, cross, and candidate features before embedding and downstream ranking.The example uses age and gender for U, two items in each sequence, and CTR and exposure count in C.
- Ranking Workflow: The system embeds feature groups separately, processes behavior sequences with target attention, concatenates the resulting representations, and feeds them to an MLP.Target attention uses the candidate as query and historical sequence features as keys and values.
- Ranking Workflow: The MLP outputs one logit per candidate for training and inference-time ranking.The concatenated representation includes user, processed historical and real-time sequences, cross features, and candidate embeddings.
- Scaling Dilemma: Scaling the cross module increases computation linearly with candidate count and can produce unacceptable system latency.This creates a need for efficient feature interaction with sub-linear inference cost as candidates increase.
4 Data Rearrangement and Architecture of MTGR
MTGR rearranges user and candidate features into shared token sequences, aggregates candidates per user, and processes them with an HSTU-style encoder. Its masking and normalization choices support efficient scaling while preventing temporal information leakage.
- User Sample Aggregation: MTGR aggregates candidate features with one user representation, producing scores for multiple candidates through a shared computation.Candidates are grouped by user for training and by request for inference, reducing repeated user-side computation.
- Dynamic Masking: The customized mask prevents future interactions from leaking into earlier candidate-ranking decisions within an aggregated sample.Interactions are ordered from recent to past, with visibility restricted according to temporal position.
- Feature Tokenization: User, sequence, real-time, cross, and item features are converted into unified-dimension tokens before self-attention.Scalar features become individual tokens, while sequence and candidate features are embedded, concatenated, and projected by MLPs.
- Unified HSTU Encoder: MTGR uses stacked self-attention layers with Group-Layer Normalization, four projections, value updates, and residual MLP blocks.Group-Layer Normalization separately aligns tokens from different feature domains before attention and after value updating.
- Dynamic Masking: Dynamic masking separates static, dynamic, and candidate visibility to preserve causality across real-time interactions and candidate predictions.Static user and historical-sequence tokens are visible to all; dynamic tokens follow temporal causality; candidate tokens see themselves only.
5 Training System
MTGR rebuilds training in the PyTorch ecosystem by extending TorchRec for industrial streaming data and distributed computation. The framework targets embedding management, load balancing, communication, and pipeline efficiency.
- Framework Reconstruction: MTGR reconstructs its training framework around PyTorch and optimizes TorchRec for the model’s characteristics and efficient training.The redesign replaces the previous TensorFlow-based framework.
- Embedding Management: Dynamic hash tables support real-time sparse-embedding growth and compact key scanning for large-scale streaming training.Keys map to embedding vectors and metadata, enabling capacity expansion by replicating key storage rather than the full embeddings.
- Embedding Lookup: Embedding ID de-duplication reduces duplicate transfers during cross-device all-to-all embedding exchange.The process ensures IDs remain unique before and after communication.
- Load Balance: Dynamic sequence balancing addresses GPU load imbalance caused by long-tail user-behavior sequence lengths.The system targets uneven computation arising when fixed-size batches contain users with substantially different sequence lengths.
- Other Optimizations: Three separate streams overlap data copying, embedding dispatch, and computation to improve training throughput.Copy, dispatch, and compute streams handle CPU-to-GPU transfer, table lookup, and forward/backward computation concurrently.
6 Experiments
Experiments evaluate MTGR on large-scale Meituan data across baselines, model scales, ablations, scalability tests, and online deployment. MTGR consistently improves over DLRM while scaling smoothly and reducing inference cost.
- 6.1 Experiment setup: MTGR is evaluated on 10-day offline data and more than six months of online-training data, with DLRM baselines including SIM, E2E, DNN, MoE, Wukong, MultiEmbed, and UserTower.Online comparison uses a DLRM baseline trained for over two years.
- 6.1 Experiment setup: Offline evaluation measures CTR and CTCVR with AUC and GAUC, while online evaluation uses PV_CTR and UV_CTCVR, with UV_CTCVR identified as the most crucial business metric.GAUC emphasizes ranking ability within the same user.
- 6.2 Overall Performance Comparison: The smallest MTGR model exceeds the strongest DLRM model, and performance increases smoothly across three MTGR sizes as computational complexity grows.The experiments use three model scales to verify scalability.
- 6.3 Ablation Study: Removing either Dynamic Masking or GLN causes a significant performance decline, while removing cross features erases MTGR-large’s gain over DLRM.The component ablation decline is comparable to the improvement from MTGR-small to MTGR-medium.
- 6.4 Scalability: MTGR performance improves smoothly with more HSTU blocks, larger d_model, and longer training sequences, while performance follows a power-law relationship with computational complexity.Figure 3 measures gain in CTCVR GAUC relative to UserTower-SIM against logarithmic computational complexity.
7 Conclusion
MTGR is a ranking framework based on HSTU that combines DLRM’s cross-features with GRM’s scalability. It has been deployed in Meituan and yielded significant benefits.
- MTGR explores scaling laws in recommendation systems through a new HSTU-based ranking framework.
- MTGR combines DLRM and GRM advantages by retaining cross-features while providing GRM-like scalability.
- MTGR has been deployed in Meituan and has yielded significant benefits in its operating scenario.