Source-linked AI summary
Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks
Yabo Ni, Dan Ou, Shichen Liu, Xiang Li, Wenwu Ou, Anxiang Zeng, Luo Si
TL;DR
E-commerce portals need personalization across multiple search and recommendation tasks, but conventional task-specific models do not share user information. DUPN learns universal representations from behavior sequences using LSTM, attention, and multi-task learning; across five tasks and online Taobao deployment, it reports stronger personalization and transfer results.
Problem
Separate models for different personalization tasks fail to share valuable user information across tasks, limiting universal user modeling.
Method
DUPN uses LSTM and attention to model user behavior sequences while sharing representations end-to-end across multiple tasks.
Results
Across five tasks, DUPN consistently achieves better results, transfers to a new task, and improves Taobao online search outcomes in A/B testing.
Takeaways & Limitations
Universal user representations can support more effective personalization across learned and related new e-commerce tasks.
Abstract
from arXiv · showhide
Tasks such as search and recommendation have become increas- ingly important for E-commerce to deal with the information over- load problem. To meet the diverse needs of di erent users, person- alization plays an important role. In many large portals such as Taobao and Amazon, there are a bunch of di erent types of search and recommendation tasks operating simultaneously for person- alization. However, most of current techniques address each task separately. This is suboptimal as no information about users shared across di erent tasks. In this work, we propose to learn universal user representations across multiple tasks for more e ective personalization. In partic- ular, user behavior sequences (e.g., click, bookmark or purchase of products) are modeled by LSTM and attention mechanism by integrating all the corresponding content, behavior and temporal information. User representations are shared and learned in an end-to-end setting across multiple tasks. Bene ting from better information utilization of multiple tasks, the user representations are more e ective to re ect their interests and are more general to be transferred to new tasks. We refer this work as Deep User Perception Network (DUPN) and conduct an extensive set of o ine and online experiments. Across all tested ve di erent tasks, our DUPN consistently achieves better results by giving more e ective user representations. Moreover, we deploy DUPN in large scale operational tasks in Taobao. Detailed implementations, e.g., incre- mental model updating, are also provided to address the practical issues for the real world applications.
1 INTRODUCTION
E-commerce personalization must address information overload across many search and recommendation tasks, but separate user models fail to share valuable cross-task information. DUPN learns universal user representations from behavior sequences with multi-task learning and improves offline, transfer, and online personalization results.
- Separate personalization models for individual tasks do not share valuable user information across tasks.
- DUPN combines RNNs, attention, and multi-task learning to build universal user representations from complex behavior sequences.The model integrates content, behavior, and temporal information while sharing representations across related tasks.
- Four Taobao tasks show that multi-task universal representations outperform single-task learning, while a fifth task demonstrates transferability to a new task.
- Online A/B testing in Taobao search indicates that DUPN better reflects user preferences, generates more business value, and improves efficiency.The deployed model avoids rebuilding complex models from scratch for related tasks.
2 RELATED WORK
Prior personalization research uses deep and recurrent models for recommendation and search, while multi-task learning supports shared representations. This paper positions DUPN as an early deep multi-task approach for general and transferable user representations in operational e-commerce.
- Research on recommendation and search personalization has expanded from matrix-factorization and neighborhood methods to DNNs and RNNs.
- RNN-based methods model sequential user behavior to infer future intent and improve session-based recommendation or search advertising.
- Single-task representations capture task-specific information, whereas multi-task learning enables statistical strength sharing and knowledge transfer.
- The paper claims to be the first study using deep multi-task representation learning for general and transferable user representations in operational e-commerce portals.
3 SYSTEM OVERVIEW
Taobao’s retrieval system recalls query-matching items, ranks them using relevance, user, item, and personalization features, and presents a final ranked list. User behavior supports hierarchical personalization features such as price preference and personalized CTR prediction.
- Given a user query, Taobao retrieves items containing query words, ranks them, and presents the top-ranked list.
- Ranking uses item-query relevance together with user profiles and item quality.
- Figure 2 identifies items, behavior properties, and user representations as distinct components of the general model architecture.
- User behavior can infer hidden preferences that feed personalized ranking features such as price-matching scores and CTR predictions.
4 MODEL ARCHITECTURE
DUPN encodes time-ordered user behaviors with item and behavior-property features, models them using Property Gated LSTM and query-aware attention, and shares the resulting user representation across multiple tasks.
- 4.1 The Input & Behavior Embedding: DUPN represents each time-ordered behavior as an item paired with properties such as behavior type, scenario, and time gap.Item features include shop, brand, category, tags, and item ID; behavior features include clicks, bookmarks, add-to-cart events, and purchases.
- 4.1 The Input & Behavior Embedding: Separate feature embeddings are concatenated into a dense behavior vector, reducing high-dimensional item and property inputs before sequence modeling.The resulting behavior representation is 172-dimensional.
- 4.2 Property Gated LSTM & Attention Net: Property Gated LSTM uses behavior properties to control what the recurrent network extracts, remembers, and forwards, while item features provide the LSTM input.This separates behavior importance signals from item features that reflect user interests.
- 4.2 Property Gated LSTM & Attention Net: Attention forms a weighted sum of hidden states using query, user profile, hidden-state, and behavior-property information, then concatenates it with the profile embedding.The resulting user representation is 256-dimensional.
- 4.3 Multi-tasks: The shared user representation is learned across related tasks, including CTR, price-preference prediction, ranking, and transfer to shop-preference prediction.CTR uses the representation with the current item to estimate click probability, while price-preference prediction classifies seven price levels; SPP is treated as a transfer task.
5 EXPERIMENTAL METHODOLOGY
The evaluation uses a large-scale Taobao benchmark spanning five tasks and an online search deployment, with simultaneous real-time prediction and distributed training under specified experimental settings.
- Offline Dataset: The offline benchmark contains about 6×10^9 instances from five Taobao task-specific subsets.Samples are drawn from daily logs describing historical user behaviors and task labels.
- Offline Dataset: Training uses ten days of data and evaluation uses the following day, with shuffled batches of 1024 instances.
- Online Environment: The online Taobao search system predicts CTR, L2R, and PPP simultaneously in real time.CTR supplies a personalized ranking feature, L2R produces the final ranking score, and PPP supports user analysis.
- Online Environment: More than 100 million users search Taobao daily, with 10% randomly assigned to the A/B-test experiment group.
- Experimental Configuration: DUPN is trained with dropout, L2 regularization, SGD using AdaGrad, and a learning rate of 10^-3.The reported behavior-sequence length is 100 and the dropout rate is 0.8.
- Training Infrastructure: Distributed training uses 96 parameter servers and 2000 workers, processing 400 batches per second over more than four days.Each worker runs with 15 CPU cores.
6 EXPERIMENT AND ANALYSIS
DUPN is evaluated against independent-task baselines, model variants, multi-task training, representation transfer, attention behavior, and online deployment. Across these settings, the experiments report stronger task performance, faster or effective transfer, and practical operational gains, while also identifying efficiency trade-offs.
- 6.1 Offline Comparison of Different Networks: DUPN-all outperforms all other compared models across the evaluated tasks.Behavior properties improve performance when used in both LSTM gates and the attention mechanism.
- 6.2 Single Task VS Multi-task: Multi-task learning outperforms single-task learning on all four tasks, converges faster, and gains about 1% finally.Price-preference precision increases from 40% to 44%, and the reported comparisons associate the gains with shared general representations.
- 6.2 Single Task VS Multi-task: One multi-task model reduces CPU cost, memory use, and response latency relative to several independent models, even when some task improvements are modest.The paper emphasizes this advantage for real-time ranking systems that infer multiple tasks simultaneously.
- 6.3 Representation Transferability: Fine-tuning and representation adaptation achieve the best transfer results, with AUC 2.5% above RS and 3% above RT; fine-tuning converges faster.The paper also reports a new big model as a cost and latency burden, while RT offers higher inference efficiency despite slightly lower final AUC.
- 6.5 Online A/B Testing: Online deployment increases CTR by 2.23% on average over 7 days, sales volume by 3.17%, and price precision from 33.2% to 44.2%.Across price classes, DUPN precision exceeds MaxEnt, while recall is reported as 2% to 10% higher for seven classes.
7 PRACTICAL GUIDELINES ON MODEL IMPLEMENTATION
The paper addresses large-scale deployment constraints through incremental updating and network disassembly. These practices reduce training time and avoid repeatedly inferring the full network for every candidate item.
- Daily incremental updating: Frequent updating is necessary because user preferences, items, brands, and styles change over time.
- Daily incremental updating: Incremental learning reduces daily model-update time after initially training on 10 days of data.The initial distributed training takes three or four days, while later fine-tuning with new data takes less than 10 hours.
- Network disassembly: Network disassembly addresses the cost of scoring about ten thousand items per query in real time.The full large-scale network would otherwise need to be inferred approximately ten thousand times for each user query.
- Network disassembly: DUPN can be divided into a user-representation component and a task component because shallow layers do not cross user and item features.The first part performs most calculations and supports tasks based only on user representations, while the second uses both user and item representations.
8 CONCLUSION
DUPN learns user representations from heterogeneous behavior sequences using LSTM and attention within a multi-task framework. Experiments and Taobao deployment show competitive, transferable representations and improved shopping efficiency.
- DUPN uses LSTM and attention to model heterogeneous behavior sequences while incorporating behavior properties.
- Multi-task information makes the learned user representations more effective at reflecting user interests.
- Extensive experiments show DUPN has competitive performance and that its user representations are general and transferable.
- Online A/B testing in Taobao shows that DUPN meets user preferences better and improves shopping efficiency.