Source-linked AI summary
Monolith: Real Time Recommendation System With Collisionless Embedding Table
Zhuoran Liu, Leqi Zou, Xuan Zou, Caihua Wang, Biao Zhang, Da Tang, Bolin Zhu, Yijie Zhu, Peng Wu, Ke Wang, Youlong Cheng
TL;DR
Recommendation systems need to adapt to sparse, changing features and recent user feedback, while general-purpose frameworks separate training from serving. Monolith combines collisionless embeddings with online training and parameter synchronization; the authors report improved performance over colliding hash-based systems at roughly similar memory usage and state-of-the-art online-serving AUC.
Problem
Recommendation workloads have sparse, dynamic features and non-stationary user data, while general-purpose frameworks separate batch training from serving.
Source: §1 · p. 1Method
Monolith combines a collisionless embedding table with online training and parameter synchronization, using frequency filtering to reduce embedding memory.
Source: §2.1 · p. 3, §5 · p. 10Results
Monolith outperforms systems using colliding hash tricks at roughly similar memory usage and achieves state-of-the-art online-serving AUC.
Source: §1.2 · p. 2Takeaways & Limitations
The authors conclude that parameter synchronization should be as short as possible for ultimate model performance, and that trading off parameter-server reliability for real-time learning can be robust.
Source: §5 · p. 10The synchronization design assumes sparse parameters dominate model size and that only a small subset changes within a short time window.
Source: §2.2.3 · p. 5
Abstract
from arXiv · showhide
Building a scalable and real-time recommendation system is vital for many businesses driven by time-sensitive customer feedback, such as short-videos ranking or online ads. Despite the ubiquitous adoption of production-scale deep learning frameworks like TensorFlow or PyTorch, these general-purpose frameworks fall short of business demands in recommendation scenarios for various reasons: on one hand, tweaking systems based on static parameters and dense computations for recommendation with dynamic and sparse features is detrimental to model quality; on the other hand, such frameworks are designed with batch-training stage and serving stage completely separated, preventing the model from interacting with customer feedback in real-time. These issues led us to reexamine traditional approaches and explore radically different design choices. In this paper, we present Monolith, a system tailored for online training. Our design has been driven by observations of our application workloads and production environment that reflects a marked departure from other recommendations systems. Our contributions are manifold: first, we crafted a collisionless embedding table with optimizations such as expirable embeddings and frequency filtering to reduce its memory footprint; second, we provide an production-ready online training architecture with high fault-tolerance; finally, we proved that system reliability could be traded-off for real-time learning. Monolith has successfully landed in the BytePlus Recommend product.
1 INTRODUCTION
Monolith addresses recommendation workloads shaped by sparse, dynamic features and changing user interests with collisionless embeddings and online training. The authors report better performance than colliding hash-based systems at roughly similar memory usage, alongside state-of-the-art online-serving AUC.
1 INTRODUCTION
- Recommendation models use recent user interactions, but real-world recommendation data include sparse, categorical, dynamically changing features and non-stationary distributions.Source: §1 · p. 1
1.1 Sparsity and Dynamism
- Embedding tables must capture many features and grow as users and items are added, while low-collision hashing can degrade model quality under skewed ID frequencies.Source: §1.1 · p. 2
1.2 Non-stationary Distribution
- Recent user history can help predict changing behavior, motivating serving models updated from new feedback as close to real time as possible.Source: §1.2 · p. 2
- Monolith uses a collisionless hash table and dynamic feature eviction to represent sparse features, and feeds serving feedback into training in real time.Source: §1.2 · p. 2
- Monolith outperforms systems using colliding hash tricks at roughly similar memory usage and achieves state-of-the-art online-serving AUC.Source: §1.2 · pp. 2–3
2 DESIGN
Monolith uses a distributed Worker–ParameterServer architecture with dense and sparse parameters stored on parameter servers, plus native HashTable operations for sparse features. Its online-training design combines collisionless, memory-conscious embeddings with real-time data processing, parameter synchronization, and fault tolerance.
2 DESIGN
- Monolith places dense and sparse parameters on parameter servers and adds native HashTable operations for sparse parameters, online synchronization, and fault-tolerance mechanisms.Workers compute model updates; parameter servers store and update parameters.Source: §2 · p. 3
2.1 Hash Table
- Cuckoo HashMap provides collisionless key insertion; Monolith also expires inactive IDs and filters admissions by occurrence threshold and probabilistic filtering.The occurrence threshold is tunable by model; embedding expiration periods can also be tuned by table.Source: §2.1 · pp. 3–4
2.2 Online Training
- Online training consumes real-time data and periodically synchronizes training parameters to serving, allowing the model to adapt to user feedback in real time.Batch training reads stored mini-batches and trains the dataset for one pass; online training follows deployment.Source: §2.2 · p. 4
2.2.1 Streaming Engine.
- A streaming engine lets Monolith switch between batch and online training, with joined training examples routed to both workflows.Online workers read from Kafka; batch workers receive data dumped to HDFS.Source: §2.2.1 · p. 4
2.2.2 Online Joiner.
- The Online Joiner pairs unordered user actions and features by request key, and uses on-disk storage when delayed actions outlast the in-memory cache.Serving applies log-odds correction after negative sampling to estimate the original distribution without bias.Source: §2.2.2 · pp. 4–5
2.2.3 Parameter Synchronization.
- Incremental synchronization sends recently touched sparse embeddings more frequently than dense parameters, reducing transfer and memory demands while tolerating relatively stale dense parameters.The design addresses the need to update multi-terabyte serving models on the fly without stopping service or transferring the full model.Source: §2.2.3 · p. 5
2.3 Fault Tolerance
- Monolith snapshots training parameter servers daily; experiments found the resulting loss of one day’s updates after failure caused tolerable performance degradation.Snapshot frequency trades model quality against the memory-copy and disk-I/O overhead of snapshotting.Source: §2.3 · pp. 5–6
3 EVALUATION
The evaluation tests collisionless embeddings, online updates, and synchronization robustness through public-dataset, production-dataset, and live-traffic experiments. It reports quality gains without collisions and with more frequent synchronization, alongside tolerance to infrequent snapshots.
3.1.1 Embedding Table.
- MovieLens ratings of at least 3.5 become positive labels, and standard DeepFM is evaluated by AUC against MD5-hashed IDs that can share embeddings.The internal production model uses two smaller embedding tables to generate each ID's embedding by vector combination; performance is measured by online AUC.Source: §3.1.1 · p. 6
3.1.2 Online Training.
- The Criteo simulation uses five days for batch training and two chronologically sharded days for online training, testing sync intervals of 5 hr, 1 hr, and 30 min.The interval is determined by the number of data shards.Source: §3.1.2 · pp. 6–7
3.2.1 The Effect of Embedding Collision.
- Collisionless embeddings outperform colliding embeddings in MovieLens and internal experiments, and remain robust as user and item contexts change.The authors also report no overfitting after convergence for the collisionless model.Source: Figure 7 · p. 7, Figure 8 · p. 7, §3.2.1 · p. 7
3.2.2 Online Training: Trading-off Reliability For Realtime.
- Online-trained Criteo models outperform batch-trained models, and shorter sync intervals of 5 hr, 1 hr, and 30 min yield better online-serving AUC.The comparisons are shown for streaming evaluation and online-serving AUC.Source: Figure 9 · p. 8, Table 2 · p. 8, §3.2.2 · p. 8
- The live A/B experiment on a production Ads model found a significant bump in online-serving AUC with online training.Table 3 reports the improvement over batch training.Source: Table 3 · p. 9, §3.2.2 · p. 9
- With a PS-machine failure rate of 0.01% per day, a previous-day model showed nearly no quality loss, prompting lower snapshot frequency and computation overhead.The paper estimates that one of 1,000 PS machines fails every 10 days, losing one day of updates for that shard.Source: §3.2.2 · p. 9
4 RELATED WORK
Related work addresses sparse features through hashing or dynamic key-value tables, and addresses online updates through feature eviction and parameter synchronization. Monolith is presented as a native TensorFlow hash-table approach with lightweight synchronization.
4 RELATED WORK
- Earlier industrial recommendation work applied deep learning and varied techniques to address challenges in production-scale systems.The passage points to [10] as early successful applications.Source: §4 · p. 9
- Hash-trick methods reduce memory and accommodate growing IDs, while native key-value tables allow dynamic growth; Monolith's TensorFlow operation is described as developer-friendly and cross-platform.The cited alternatives use specialized software mechanisms or hardware to access and manage hash tables.Source: §4 · p. 9
- Prior online-training systems use feature eviction and short-interval parameter synchronization with fault tolerance; Monolith takes a similar approach with lightweight synchronization.The cited systems include, [14], and.Source: §4 · p. 9
5 CONCLUSION
The conclusion presents Monolith as a production-scale recommendation system combining collisionless embeddings with real-time serving. It highlights model-quality benefits and synchronization and fault-tolerance trade-offs.
5 CONCLUSION
- The authors report that their Cuckoo HashMap-based collisionless embedding table is memory-efficient and improves online-serving metrics.They describe collisionless embeddings as essential for model quality.Source: §5 · p. 10
- Monolith's online-serving design trades parameter-version consistency for lower network bandwidth and PS reliability for real-time updates.The conclusion describes the fault-tolerance strategy as robust.Source: §5 · p. 10
- The authors conclude that Monolith provides a general solution for production-scale recommendation systems.They also report best performance compared with existing solutions.Source: §5 · p. 10