Source-linked AI summary
One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction
Xiang-Rong Sheng, Liqin Zhao, Guorui Zhou, Xinyao Ding, Binding Dai, Qiang Luo, Siran Yang, Jingshan Lv, Chi Zhang, Hongbo Deng, Xiaoqiang Zhu
TL;DR
Multi-domain CTR prediction must exploit shared behavior while accounting for domain-specific distributions. STAR combines shared centered parameters with domain-specific parameters, and production results report 8.0% higher CTR and 6.0% higher RPM after Alibaba deployment since 2020.
Problem
Multi-domain CTR prediction must model commonalities across domains while capturing their differing data distributions for accurate, resource-efficient predictions.
Method
STAR combines shared centered parameters, which learn transferable common behavior, with domain-specific parameters, which capture distinctions for refined CTR prediction.
Results
8.0% improvement on CTR and 6.0% on RPM were obtained after STAR was deployed in Alibaba’s display advertising system since 2020.
Takeaways & Limitations
STAR’s superior performance across all domains supports combining information sharing with domain-specific modeling in multi-domain CTR prediction.
Takeaways & Limitations
STAR shares the embedding layer across domains, while samples are assumed locally i.i.d. only within each domain and normalization moments differ across domains.
Abstract
from arXiv · showhide
Traditional industrial recommenders are usually trained on a single business domain and then serve for this domain. However, in large commercial platforms, it is often the case that the recommenders need to make click-through rate (CTR) predictions for multiple business domains. Different domains have overlapping user groups and items. Thus, there exist commonalities. Since the specific user groups have disparity and the user behaviors may change in various business domains, there also have distinctions. The distinctions result in domain-specific data distributions, making it hard for a single shared model to work well on all domains. To learn an effective and efficient CTR model to handle multiple domains simultaneously, we present Star Topology Adaptive Recommender (STAR). Concretely, STAR has the star topology, which consists of the shared centered parameters and domain-specific parameters. The shared parameters are applied to learn commonalities of all domains, and the domain-specific parameters capture domain distinction for more refined prediction. Given requests from different business domains, STAR can adapt its parameters conditioned on the domain characteristics. The experimental result from production data validates the superiority of the proposed STAR model. Since 2020, STAR has been deployed in the display advertising system of Alibaba, obtaining averaging 8.0% improvement on CTR and 6.0% on RPM (Revenue Per Mille).
1 INTRODUCTION
Multi-domain CTR prediction must balance shared cross-domain information with distinct domain distributions and behaviors. STAR addresses this through shared centered parameters combined with domain-specific parameters, while targeting efficient multi-domain serving.
- Motivation: Large commercial recommenders often need CTR predictions across multiple business domains with overlapping users and items but differing behaviors.These overlaps create commonalities, while domain-specific behavior changes create distinctions.
- Motivation: Using one separate model per domain can waste common information, underuse data in smaller domains, and impose substantial maintenance and resource costs.The burden grows when platforms operate hundreds of business domains.
- Problem Setting: Multi-domain CTR prediction covers the same task across domains, but each domain has its own data distribution and requires both commonality learning and distinction capture.This differs from multi-task learning, which commonly addresses different tasks within one domain.
- STAR: STAR combines shared centered parameters, which learn transferable general behaviors, with domain-specific parameters, which capture domain-specific behaviors for refined prediction.The final model for each domain is formed by combining the shared and domain-specific parameters.
- STAR: STAR adds little computational and memory cost because its extra domain-specific parameters are small relative to industrial recommender embedding layers.The implementation combines parameters through element-wise products of weights in each layer.
- Evaluation: The paper evaluates STAR on industrial production data and reports deployment in Alibaba’s display advertising system in 2020.The contribution summary describes consistent superiority in the production evaluation.
2 RELATED WORK
Related work spans single-domain CTR prediction, multi-task learning, and multi-domain learning. The paper distinguishes multi-domain CTR prediction by its same-task, cross-domain setting and direct use of domain information.
- Single-Domain CTR Prediction: Traditional CTR models are generally trained on data from one business domain and then serve that domain.This single-domain formulation motivates extending CTR prediction to multiple business domains.
- Single-Domain CTR Prediction: Deep CTR models commonly follow an embedding-and-MLP paradigm, with prior work adding feature interactions, recurrent attention, or multiple user-item representations.Examples include Wide & Deep, DeepFM, PNN, DIEN, MIND, and DMIN.
- Multi-Task and Multi-Domain Learning: Multi-task learning shares knowledge across related tasks, whereas multi-domain learning transfers knowledge across domains that may collect data from different distributions.The related-work discussion contrasts multi-domain learning with one-way domain adaptation.
- Multi-Task and Multi-Domain Learning: Multi-domain CTR prediction applies the same CTR task across business domains, using a domain indicator and semantic embeddings to represent domain distinctions.The paper identifies direct domain-indicator use as neglected by previous multi-domain learning literature.
- Proposed Approach: STAR’s architecture combines partitioned normalization, a star-topology fully connected network, and an auxiliary network that models domain information.The figure description specifies shared centered and domain-specific FCNs with element-wise weight products.
3 THE PROPOSED APPROACH
STAR is a single multi-domain CTR model combining shared and domain-specific components to learn commonalities while adapting to domain differences. Its architecture uses shared embeddings, partitioned normalization, a star-topology FCN, and an auxiliary network.
- Multi-Domain CTR Prediction: Multi-domain CTR prediction seeks one accurate model for M domains with low resource and human cost despite domain-specific distributions.Ignoring domain differences harms performance, while separate models reduce per-domain data and increase production costs.
- STAR Architecture: STAR combines shared centered parameters with domain-specific parameters to transfer common knowledge while capturing domain-specific behavior.The model comprises partitioned normalization, a star-topology fully-connected network, and an auxiliary network.
- STAR Architecture: All domains share the embedding layer because embedding parameters greatly outnumber fully-connected parameters, making domain-specific embeddings difficult to learn with limited data.The paper reports that embedding parameters can be 10,000 times more numerous than fully-connected-layer parameters.
- Partitioned Normalization: Partitioned normalization uses domain-specific statistics and parameters so intermediate representations reflect each domain’s distinct data characteristics.During training it uses current-domain batch moments; during testing it accumulates domain-specific moving averages and adapts scale and bias by domain.
- Star Topology FCN and Auxiliary Network: The star-topology FCN combines shared and independent domain-specific FCNs, with shared parameters learning general behavior and specific parameters refining domain predictions.The auxiliary network also uses domain information and other domain features to capture domain distinction directly.
4 EXPERIMENTS
The experiments evaluate STAR through production-data setup, comparative results, ablations, and online-environment evidence.
- The evaluation uses a production dataset, compared methods, and implementation details.
- Results and discussion assess STAR's performance against alternative approaches.
- Ablation studies examine the contributions of STAR's components.
4.1 Experimental Settings
The study evaluates multi-domain CTR prediction on Alibaba production data using several model comparisons, shared training settings, and domain-level AUC metrics.
- Dataset: Alibaba production data covers user click behavior across 19 business domains, with one day for training and the following day for testing.The training set contains billions of examples.
- Compared models: The compared models include Base, Shared Bottom, and MulANN.
- Compared models: All compared methods and STAR use the proposed auxiliary network for fair comparison.The auxiliary network is evaluated separately in an ablation study.
- Implementation details: All models use Adam with learning rate 0.001, batch size 2000, and cross-entropy loss over samples from all domains.
- Metrics: Evaluation uses user-weighted AUC for each domain and overall AUC across mixed-domain samples.The weighted AUC averages user-level AUC according to impression counts.
4.2 Results
On the offline Alibaba production dataset, STAR consistently improves performance across domains while modeling domain relationships and distinctions more explicitly than alternatives.
- Shared Bottom, MMoE, Cross-Stitch, and STAR achieve better overall performance than Base, whereas MulANN performs worse than Base.
- The results indicate that exploiting domain relationships while capturing domain distinctions enhances prediction performance.
- STAR exhibits superior performance across all domains compared with the Base model.
- Shared Bottom, MMoE, and Cross-Stitch underperform Base in some domains, including domains #5, #6, and #16.The authors hypothesize that learning conflicts across domains explain these declines.
- STAR consistently outperforms Shared Bottom, MMoE, and Cross-Stitch in the reported comparisons.The results support explicit modeling of domain relationships over implicit gate or cross-stitch mechanisms.
4.3 Ablation Study
Ablations evaluate STAR FCN, Partitioned Normalization, the auxiliary network, normalization choices, and calibration across domains.
- 4.3 Ablation Study: STAR FCN and Partitioned Normalization each outperform the Base model, and combining them further boosts performance.
- 4.3.2 Normalization: Partitioned Normalization and Layer Normalization outperform Batch Normalization for multi-domain CTR prediction.The result supports using domain-specific or instance-specific normalization instead of global statistics.
- 4.3.3 Auxiliary network: The auxiliary network consistently improves all evaluated methods.Its improvement is slightly weaker for MulANN, possibly because adversarial domain obscuring conflicts with domain-feature modeling.
- 4.3.4 Ability to Capture Domain Distinction: CTR calibration matters in CPC advertising because impression allocation uses bid multiplied by CTR.
- 4.3.4 Ability to Capture Domain Distinction: STAR's PCOCs are more compact and concentrated around 1.0 across domains than Base's PCOCs.PCOC closer to 1.0 indicates more accurate CTR prediction.
4.4 Production
STAR is deployed across Alibaba’s display advertising system to serve multiple business domains, with domain-specific weights pre-computed for faster inference. Since 2020, it has served more than 60 domains and improved overall CTR and RPM in an online A/B test.
- Pre-computed domain-specific FCN weights make STAR’s serving computational time equal to the Shared Bottom model.This optimization supports faster inference.
- Since 2020, STAR has served more than 60 business domains in Alibaba’s display advertising system.
- +8.0% overall CTR lift and +6.0% overall RPM lift were achieved over the previous production Base model in an online A/B test.
5 CONCLUSION
The paper proposes STAR for multi-domain CTR prediction, combining shared and domain-specific parameters to learn commonalities and domain distinctions. STAR adapts its parameters by domain and achieved reported improvements in Alibaba’s advertising system.
- STAR combines shared centered parameters with domain-specific parameters for multi-domain CTR prediction.Shared parameters learn from all examples, while domain-specific parameters learn from each domain’s examples.
- Shared parameters learn commonalities across domains, while domain-specific parameters capture distinctions for more refined prediction.
- STAR adaptively modulates its parameters conditioned on the domain.
- STAR obtained 8.0% improvement on CTR and 6.0% on RPM after deployment in Alibaba’s advertising system.