Source-linked AI summary
POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion
Wen Chen, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, Huan Zhao, Binqiang Zhao
TL;DR
Fashion outfit recommendation must jointly address item compatibility and user personalization, while existing work often studies them separately or relies on explicit queries. POG bridges these requirements with a Transformer-based model that connects historical item behavior to outfit generation. It outperforms alternatives across compatibility and recommendation evaluations, is deployed in iFashion, and is accompanied by a large-scale fashion behavior dataset.
Problem
Fashion outfit recommendation needs compatible generated outfits and personalized recommendations, but prior work commonly addresses these requirements separately or uses explicit user queries.
Method
POG uses a Transformer encoder-decoder to connect users’ historical item preferences with outfit compatibility and generate personalized outfits.
Results
POG outperforms alternative methods on compatibility, outfit generation, and recommendation, including a 70% CTR increase over CF in online experiments.
Takeaways & Limitations
POG provides a deployed bridge between personalized recommendation and outfit generation, supported by a dataset containing outfit, item, and user-behavior data.
Takeaways & Limitations
Future work targets the cold-start problem by building user profiles from similar user groups.
Abstract
from arXiv · showhide
Increasing demand for fashion recommendation raises a lot of challenges for online shopping platforms and fashion communities. In particular, there exist two requirements for fashion outfit recommendation: the Compatibility of the generated fashion outfits, and the Personalization in the recommendation process. In this paper, we demonstrate these two requirements can be satisfied via building a bridge between outfit generation and recommendation. Through large data analysis, we observe that people have similar tastes in individual items and outfits. Therefore, we propose a Personalized Outfit Generation (POG) model, which connects user preferences regarding individual items and outfits with Transformer architecture. Extensive offline and online experiments provide strong quantitative evidence that our method outperforms alternative methods regarding both compatibility and personalization metrics. Furthermore, we deploy POG on a platform named Dida in Alibaba to generate personalized outfits for the users of the online application iFashion. This work represents a first step towards an industrial-scale fashion outfit generation and recommendation solution, which goes beyond generating outfits based on explicit queries, or merely recommending from existing outfit pools. As part of this work, we release a large-scale dataset consisting of 1.01 million outfits with rich context information, and 0.28 billion user click actions from 3.57 million users. To the best of our knowledge, this dataset is the largest, publicly available, fashion related dataset, and the first to provide user behaviors relating to both outfits and fashion items.
1 INTRODUCTION
Fashion outfit recommendation must balance compatibility among generated items with personalization to users’ tastes, but prior approaches often treat these requirements separately. POG bridges them by using historical item interactions to generate personalized outfits, with strong offline and online results and deployment in iFashion.
- 1 INTRODUCTION: Fashion outfit recommendation requires both compatibility among generated items and personalization to individual users.Compatibility concerns harmony within an outfit, while personalization concerns alignment with users’ fashion tastes.
- 1 INTRODUCTION: Users tend to show similar tastes in individual fashion items and outfits, supporting a bridge between item behavior and outfit recommendation.Analysis of iFashion behavior found high overlap in categories, brands, styles, and patterns between recent item clicks and clicked outfits.
- 1 INTRODUCTION: POG connects user preferences from historical item interactions with outfit compatibility through a Transformer encoder-decoder architecture.Its Fashion Outfit Model learns interactions among items, while the encoder-decoder models user preference and outfit compatibility together.
- 1 INTRODUCTION: 68.79% on FITB and 86.32% on CP show POG’s compatibility gains over alternative methods.These correspond to 5.98% and 25.81% relative improvements, respectively.
- 1 INTRODUCTION: POG increased CTR by 70% over collaborative filtering in extensive online experiments on Dida, Alibaba’s outfit generation platform.Dida deployed POG to assist outfit generation and recommendation in iFashion.
- 1 INTRODUCTION: The work released data covering 1.01 million outfits, 583 thousand fashion items, and 0.28 billion clicks from 3.57 million users.The dataset includes rich context information and user behavior associated with both outfits and individual fashion items.
2 RELATED WORK
Prior work spans outfit compatibility modeling, outfit generation, and fashion recommendation, while Transformer self-attention is adopted here to model compatibility in outfit generation.
- Fashion research covers image retrieval, clothing recognition and parsing, attribute learning, outfit compatibility, recommendation, and automatic outfit generation.
- Outfit-generation methods include pairwise compatibility metrics and approaches that model outfits as item sequences.Pairwise methods use visual features or user-behavior co-occurrence to estimate compatibility.
- Recommendation research mainly models user preferences from behavior histories but has largely focused on individual items rather than complete outfits.Some outfit methods require image or text queries, while others target specific occasions.
- Self-attention relates positions within a sequence, and this paper applies it to model compatibility in fashion outfit generation.The Transformer is described as a transduction model relying on self-attention.
- Table 2 provides dataset statistics for the related fashion datasets discussed in the paper.
3 DATASET
The dataset combines expert-reviewed outfits with item metadata and user click histories, producing training samples that connect outfit interactions to preceding item behavior.
- Taobao fashion experts created and reviewed about 1.43 million outfits, from which the dataset retains items in 80 frequent leaf categories.Items from less frequent categories are removed from outfits.
- User data come from 3.57 million active users who viewed more than 40 outfits, with outfit clicks recorded after more than 10 prior item clicks.
- The training set contains 19.2 million samples pairing an outfit click with each user’s latest 50 preceding item clicks.
- The released datasets separately describe outfits, items, and user behaviors, and provide outfit data with associated behavior data for future fashion recommendation research.The paper characterizes the dataset as the largest publicly available fashion-item dataset with rich information.
4 METHODOLOGY
POG is built in three stages: multimodal item embedding, compatibility learning with FOM, and personalized outfit generation with a Transformer encoder-decoder. The resulting system combines user preference and outfit compatibility signals and is deployed in Dida for large-scale recommendation.
- 4.1 Multi-modal Embedding: POG first embeds fashion items using image, title, and collaborative-filtering signals, then projects their concatenation into a lower-dimensional representation.The item embedding model uses a CNN for white-background images, TextCNN for titles, and Alibaba’s Behemoth Graph Embedding for click-session co-occurrences.
- 4.1 Multi-modal Embedding: Triplet loss defines same-leaf-category items as positives and different-category items as negatives, encouraging category-aware embedding distances.The loss uses Euclidean distance and a margin α.
- 4.2 FOM: Fashion Outfit Model: FOM masks one outfit item at a time and predicts it from the remaining items, learning compatibility without position embeddings because outfits are treated as sets.A bidirectional Transformer encoder models interactions among the outfit items, and the masked-item probability is computed with a softmax layer.
- 4.2 FOM: Fashion Outfit Model: FOM samples three non-outfit transition embeddings alongside the masked-item embedding, providing a diverse candidate set for compatibility learning.The sampled representations are outputs of the transition layer rather than the original item embeddings.
- 4.3 POG: Personalized Outfit Generation Model: POG translates a user’s historical item-click sequence into an outfit by predicting each next item from previous outfit items and the user’s clicks.Its Per encoder supplies preference information, while the Gen decoder generates items autoregressively until [END].
5 EXPERIMENT
The experiment section reports that both offline and online experiments compare compatibility and recommendation performance.
- 5 EXPERIMENT: Offline and online experiments compare the methods’ compatibility and recommendation performance.
5.1 Fashion Outfit Compatibility
The compatibility evaluation compares multimodal representations and outfit models on FITB and CP tasks. FOM performs best across ordered and unordered inputs, while set-based modeling avoids the order sensitivity of sequence models.
- Task Settings & Evaluation Metrics: FITB and CP evaluate item compatibility and whole-outfit compatibility, respectively.FITB measures accuracy when selecting a compatible missing item; CP predicts whether a candidate outfit is compatible.
- Modality Comparison: Text performs best alone on both tasks, while image and CF embeddings provide complementary information.CF alone performs poorly because it lacks semantic visual and textual information important for fashion compatibility.
- Model Comparison: Sequence models are sensitive to input order, whereas SetNN and FOM produce similar results with ordered and unordered inputs.The comparison uses a fixed tops, bottoms, shoes, and accessories order for ordered inputs.
- Model Comparison: FOM performs best on FITB with ordered and unordered inputs, improving 18.04% over Bi-LSTM unordered and 5.98% ordered.The results are attributed to Transformer self-attention, which weights interactions among outfit items.
- Model Comparison: FOM improves CP over Bi-LSTM by 34.90% with unordered inputs and 25.81% with ordered inputs.Multi-head self-attention learns compatibilities between each item and all other items before combining them into an outfit score.
5.2 Fashion Outfit Generation and Recommendation
Online experiments compare generation models and recommendation methods using seven-day outfit CTR measurements. POG-based methods achieve the strongest performance, with pre-trained FOM further supporting personalized generation.
- Online Evaluation: CTR evaluates outfit compatibility and recommendation effectiveness in seven-day online experiments.Outfits are generated and recommended from a manually selected pool of 1.57 million items on Dida.
- Compared Methods: POG uses both Per and Gen networks to generate and recommend personalized outfits, unlike existing generation models paired with random or CF recommendation.Gen-only variants evaluate generation without user information, while POG incorporates personalization.
- Online Results: CF methods have higher CTR than RR methods, while POG-based methods achieve the best online performances.Figure 6 reports CTR variation over seven days and compares random recommendation with collaborative filtering.
- Online Results: POG with pre-trained FOM improves CTR by more than 70% over Bi-LSTM with CF recommendation.The authors attribute this to individual item preference not necessarily indicating a user’s preferred outfit.
- Online Results: The Gen network usually has the highest CTR among RR and CF methods, while pre-trained FOM provides only a small improvement over POG alone.The authors conclude that Gen is already sufficient to learn outfit compatibility in this setting.
- Online Cases: POG generates compatible outfits from recent clicked items, adapting either to varied preferences or to repeated similarities in users’ histories.Figure 7 shows four users’ recent clicked items and the outfit generated in response.
6 CONCLUSION AND FUTURE WORK
The paper presents POG as a Transformer-based bridge between outfit compatibility and personalization, and deploys it on Alibaba’s Dida platform. It also shares data for future research and identifies cold-start handling as future work.
- Conclusion: POG combines multimodal embedding, fashion outfit modeling, and personalized fashion outfit modeling with Transformer architectures.The model captures item interactions within outfits and connects user behaviors on items and outfits.
- Conclusion: The model outperforms alternative methods in outfit compatibility, generation, and recommendation by significant margins.The conclusion summarizes the reported offline and online advantages without specifying a single metric.
- Deployment: POG is deployed on Alibaba’s Dida platform for personalized outfit generation in the iFashion application.Online examples illustrate generated outfits conditioned on users’ clicked-item histories.
- Future Work: The authors share their data for future research and plan to address cold-start users through profiles built from similar user groups.Cold-start handling is explicitly identified as future work.