PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information

Jianxin Chang; Chenbin Zhang; Yiqun Hui; Dewei Leng; Yanan Niu; Yang Song; Kun Gai

Source-linked AI summary

PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information

Jianxin Chang, Chenbin Zhang, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, Kun Gai

arXiv:2302.01115v3Feb 2, 2023cs.IR

TL;DR

Industrial recommender systems need to model user interests across multiple domains and tasks without losing domain or task differences. PEPNet injects personalized priors through gated embedding selection and DNN-parameter modification. It reports consistent offline gains and online improvements exceeding 1% across multiple task metrics, while serving over 300 million daily users in Kuaishou apps.

Problem
Recommender systems must capture shared information across multiple domains and tasks while handling differences in feature semantics, user preferences, and target sparsity.
Method
PEPNet uses personalized gates to select embedding features across domains and modify DNN parameters across tasks.
Results
PEPNet consistently outperforms baselines across six task metrics and three domains, with reported GAUC gains of around 0.01–0.02 by domain and online improvements exceeding 1% across multiple task metrics.
Takeaways & Limitations
PEPNet provides a plug-and-play personalization approach deployed in Kuaishou apps at a scale exceeding 300 million daily users.

Abstract

from arXiv · show

With the increase of content pages and interactive buttons in online services such as online-shopping and video-watching websites, industrial-scale recommender systems face challenges in multi-domain and multi-task recommendations. The core of multi-task and multi-domain recommendation is to accurately capture user interests in multiple scenarios given multiple user behaviors. In this paper, we propose a plug-and-play \textit{\textbf{P}arameter and \textbf{E}mbedding \textbf{P}ersonalized \textbf{Net}work (\textbf{PEPNet})} for multi-domain and multi-task recommendation. PEPNet takes personalized prior information as input and dynamically scales the bottom-level Embedding and top-level DNN hidden units through gate mechanisms. \textit{Embedding Personalized Network (EPNet)} performs personalized selection on Embedding to fuse features with different importance for different users in multiple domains. \textit{Parameter Personalized Network (PPNet)} executes personalized modification on DNN parameters to balance targets with different sparsity for different users in multiple tasks. We have made a series of special engineering optimizations combining the Kuaishou training framework and the online deployment environment. By infusing personalized selection of Embedding and personalized modification of DNN parameters, PEPNet tailored to the interests of each individual obtains significant performance gains, with online improvements exceeding 1\% in multiple task metrics across multiple domains. We have deployed PEPNet in Kuaishou apps, serving over 300 million users every day.

1 INTRODUCTION

Industrial recommenders must jointly handle fragmented domains and multiple related tasks while preserving differences in feature semantics, target sparsity, and user preferences. PEPNet addresses this with personalized gating of embeddings and DNN parameters, achieving reported gains in large-scale deployment.

Motivation: Real-world recommendation fragments data across scenarios such as Taobao purchase stages and Kuaishou video tabs, creating multi-domain and multi-task requirements.Kuaishou domains include multiple tasks such as like, follow, forward, collect, and comment.
Motivation: Separate models waste deployment resources and shared data, whereas one unified model can ignore differences between domains and tasks.The resulting tension motivates methods that capture commonalities without erasing domain- and task-specific differences.
Motivation: The imperfectly double seesaw combines gaps in feature semantics with differences in target sparsity and interdependence across domains and tasks.The paper identifies this problem as increasingly severe as the number of domains and tasks grows.
Approach: PEPNet dynamically scales bottom-level embeddings and top-level DNN hidden units using personalized prior information and gate mechanisms.EPNet performs personalized embedding selection across domains, while PPNet modifies DNN parameters across tasks.
Results: More than 300 million daily active users are served by the deployed system, with over 1% higher watch time and around 2% improvement on multiple interactive targets.The reported deployment results support the method’s practical impact in Kuaishou recommendation.

2 METHODOLOGY

PEPNet addresses multi-domain, multi-task recommendation by injecting personalized prior information into shared embeddings and DNN hidden units. Its Gate NU generates personalized gates, while EPNet adapts embeddings across domains and PPNet adapts DNN parameters across tasks.

Network Structure: PEPNet uses Gate NU, EPNet, and PPNet to inject personalized prior information into a multi-domain, multi-task recommendation network.Gate NU generates personalized gates; EPNet operates on embeddings, and PPNet operates on DNN parameters.
Gate Neural Unit: Gate NU crosses prior-information features and generates gate vectors that control signal importance, with outputs limited to [0,2].The first layer uses nonlinear feature crossing, while the second applies a sigmoid with scaling factor γ=2.
Embedding Personalized Network: EPNet uses domain-specific features to personalize a shared embedding, selecting feature contributions for different users across domains.It applies an element-wise product between the domain gate and the embedding, while optionally omitting vector-wise products for large inputs.
Parameter Personalized Network: PEPNet targets the limitations of shared DNN parameters, which make balancing multiple tasks difficult because users differ in preferences for behaviors.The method builds user-tailored DNN behavior by applying personalized modifications to the shared tower parameters.
Parameter Personalized Network: PPNet uses user, item, and author features to generate task gates that modify DNN hidden units for users with different task preferences and target sparsity.The gate output is split across task towers and applied to hidden units in each DNN layer.

3 EXPERIMENT

The experiment section evaluates PEPNet through comparisons with state-of-the-art recommenders, analyses of its components, and tests in multi-task, multi-domain, and online settings.

Evaluation Questions: The experiments compare PEPNet with state-of-the-art recommenders in multi-task and multi-domain scenarios.The evaluation also examines whether PPNet and EPNet address the imperfectly double seesaw problems.
Evaluation Questions: The evaluation studies component and implementation effects, including PEPNet’s performance in real online recommendation scenarios.These experiments address the effects of different components and implementations as well as online performance.

3.1 Experimental Settings

The experiments use an industrial Kuaishou dataset spanning three domains and six binary interaction tasks, evaluate AUC and GAUC, and compare PEPNet with general, multi-task, and domain-specific baselines.

Dataset: The dataset covers three domains and six binary interaction targets: Like, Follow, Forward, Hate, Click, and EffView.The domains are Double-Columned Discovery, Featured-Video, and Single-Columned Slide tabs.
Dataset: EffView denotes an effective view when watch time reaches at least the 50th percentile of all samples.
Evaluation: Models are evaluated with AUC and GAUC after filtering users and items with fewer than 10 interactions.Training uses the first 10 days, validation the 11th day, and testing the last day.
Baselines: Baselines include general recommenders, multi-task recommenders, and multi-task multi-domain recommenders.General recommenders train each task and domain separately, while multi-task models share components across tasks.
Baselines: SpecificAll distinguishes domain-specific top DNN towers and bottom Embedding layers.
Implementation: Offline implementations use TensorFlow, Adam with learning rate 0.001, batch size 1024, embedding size 40, and two-layer networks with hidden sizes [100, 64].

3.2 Overall Performance (RQ1)

Across six tasks and three domains, PEPNet achieves the best reported performance and improves GAUC over baselines, especially for sparse domains and tasks. Its shared gated architecture captures personalized cross-domain and cross-task effects with few additional parameters.

Overall results: PEPNet significantly outperforms all baselines across six task metrics and three domains, with GAUC gains of about 0.01–0.02 by domain.The reported average GAUC improvements are around 0.01 on Domain A, 0.02 on Domain B, and 0.02 on Domain C, with p-value < 0.05.
Overall results: Task-average improvements are 0.01 for Like, 0.02 for Follow, 0.02 for Forward, 0.03 for Hate, 0.002 for Click, and 0.005 for EffView.
Overall results: The gains are more apparent on sparse domains and tasks, reducing the difficulty of cross-domain and cross-task modeling.
Baseline analysis: SpecificTop improves some domains but increases parameters several times, whereas SpecificAll can deteriorate performance by discarding shared knowledge between domains.
Model interpretation: Gated networks over shared bottom Embeddings and shared top DNN task towers capture personalized bias with a small number of parameters.

3.3 Ablation Study (RQ2)

Ablation results show that both EPNet and PPNet contribute to PEPNet’s performance across their intended settings. Parameter personalization also benefits conventional single-task single-domain recommendation.

Module ablations: Removing either PPNet or EPNet, or both, is compared against the complete PEPNet model to assess each module’s contribution.
Generalization settings: PPNet is evaluated for multi-task single-domain recommendation, while EPNet is evaluated for single-task multi-domain recommendation.
Module ablations: EPNet and PPNet each improve performance by capturing cross-domain and cross-task information through embedding and parameter personalization.
Generalization settings: Adding pure parameter personalization to single-task single-domain models also improves general recommendation performance.

3.4 Hyper-parameter Study (RQ3)

The hyper-parameter study finds EPNet robust to embedding dimension, PPNet sensitive to DNN depth, and Gate NU strongest with coefficient 2. Input and backpropagation settings also affect personalization.

Embedding dimension: EPNet remains effective across embedding dimensions, including dimension 16.
DNN depth: PPNet performance improves with more DNN layers until excessive depth causes overfitting.
Gate coefficient: The Sigmoid coefficient in Gate NU performs best at 2 because its output range is (0, 2) centered at 1.
Input and backpropagation: Adding general input while removing backpropagation performs better than the other tested input settings.This setting is reported to use input information without affecting the backbone network.

3.5 Online A/B Testing (RQ4)

Online A/B testing evaluates PEPNet across three representative short-video domains using Like, Follow, Forward, and Watch Time metrics. The reported results show significant improvement over the previous state-of-the-art method, with Watch Time gains meeting Kuaishou’s business threshold.

Online evaluation: PEPNet is evaluated online in three representative domains: Double-Columned Discovery Tab, Featured-Video Tab, and Single-Columned Slide Tab.The evaluation uses rigorous online A/B testing.
Metrics: Short-video evaluation tracks Like, Follow, Forward, and Watch Time rather than e-commerce metrics such as CTR and GMV.Watch Time measures each user’s average video-watching time.
Results: All reported metrics improve significantly compared with the previous SOTA method.The passage reports this outcome across the short-video metrics used in the online evaluation.
Business impact: 0.1% increase in Watch Time is considered effective in Kuaishou, and PEPNet therefore achieves significant business benefits.PEPNet is deployed in the online service serving over 300 million users every day.

4 RELATE WORK

Related work spans CTR prediction, multi-domain learning, multi-task learning, and gating mechanisms. Existing recommendation approaches address feature interactions, transfer across domains, task sharing, and adaptive information weighting, while PEPNet’s context is joint recommendation across multiple domains and tasks.

Click-Through Rate Prediction: CTR prediction evolved from lightweight shallow models such as LR, FM, and GBDT toward deep models that capture high-order feature interactions.Examples include FNN, PNN, WDL, and DeepFM.
Multi-Domain Learning: Multi-domain recommendation extends beyond single-domain CTR prediction by jointly training on different domains and emphasizing simultaneous improvement across domains.Recommendation scenarios weaken the traditional source-target distinction used in domain adaptation.
Multi-Task Learning: Multi-task learning shares information across related tasks, but hard parameter sharing can cause negative transfer when task differences are substantial.Soft parameter sharing methods learn combinations of task-specific hidden layers to address task differences.
Multi-Task Learning: Recommendation systems use shared-bottom and mixture-of-experts architectures to share information across tasks while allowing task-specific processing.MMoE uses different gates for each task while sharing experts.
Gating Mechanisms: Gating mechanisms adaptively strengthen important information and weaken irrelevant information in recommendation models.Prior work includes feature-level and instance-level gates and squeeze-excitation-based dynamic weighting.

5 CONCLUSION

The conclusion frames PEPNet as a response to uneven domain data and sparse task labels in multi-domain, multi-task recommendation. It reports personalized embedding and parameter modeling, engineering optimization, deployment at Kuaishou, and significant online and offline improvements.

Problem: PEPNet addresses the imperfectly double seesaw problem caused by data imbalance across domains and sparse labels across tasks.The conclusion identifies both uneven domain data and task-label sparsity as the motivating conditions.
Approach: PEPNet learns heterogeneous relationships across domains and tasks through embedding personalization and parameter personalization.The conclusion states that both forms of personalization are considered in Kuaishou’s recommendation setting.
Deployment and results: PEPNet’s engineering strategies support training and online inference for short-video recommendation, with deployment in Kuaishou Apps.The conclusion reports significant improvements in App usage and engagement across online and offline multi-task, multi-domain experiments.