Source-linked AI summary
Aesthetic-based Clothing Recommendation
Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, Zheng Qin
TL;DR
Existing clothing recommenders often use image features that do not directly represent users’ aesthetic preferences, which vary across people and over time. The paper extracts aesthetic features with a brain-inspired network and incorporates them into a dynamic tensor-factorization recommender. Experiments on real-world datasets show that the proposed method captures consumers’ aesthetic preferences and dramatically outperforms state-of-the-art models.
Problem
Existing clothing recommendation methods seldom consider aesthetic features, although clothing decisions depend partly on whether products fit users’ aesthetics.
Method
The paper extracts aesthetic features with a pretrained brain-inspired network and incorporates them with CNN features into a dynamic tensor-factorization model across users, items, and time.
Results
The proposed method captures consumers’ aesthetic preferences and dramatically outperforms state-of-the-art models on challenging real-world datasets.
Takeaways & Limitations
Aesthetic features can represent consumers’ style preferences in personalized clothing recommendation.
Takeaways & Limitations
PITF can recommend an item that fits a user but not the current time because its additive scoring may produce a high total score.
Abstract
from arXiv · showhide
Recently, product images have gained increasing attention in clothing recommendation since the visual appearance of clothing products has a significant impact on consumers' decision. Most existing methods rely on conventional features to represent an image, such as the visual features extracted by convolutional neural networks (CNN features) and the scale-invariant feature transform algorithm (SIFT features), color histograms, and so on. Nevertheless, one important type of features, the \emph{aesthetic features}, is seldom considered. It plays a vital role in clothing recommendation since a users' decision depends largely on whether the clothing is in line with her aesthetics, however the conventional image features cannot portray this directly. To bridge this gap, we propose to introduce the aesthetic information, which is highly relevant with user preference, into clothing recommender systems. To achieve this, we first present the aesthetic features extracted by a pre-trained neural network, which is a brain-inspired deep structure trained for the aesthetic assessment task. Considering that the aesthetic preference varies significantly from user to user and by time, we then propose a new tensor factorization model to incorporate the aesthetic features in a personalized manner. We conduct extensive experiments on real-world datasets, which demonstrate that our approach can capture the aesthetic preference of users and significantly outperform several state-of-the-art recommendation methods.
1 INTRODUCTION
Clothing images strongly influence recommendation decisions, but conventional image features capture product semantics more readily than personalized aesthetic appeal. The paper introduces aesthetic features and a dynamic tensor-factorization approach to model aesthetic preferences across users and time.
- Product images expose clothing attributes such as design, color, patterns, texture, thickness, and quality that influence recommendation decisions.
- Existing systems use CNN, SIFT, and color-histogram features, but these methods have rarely modeled aesthetics directly.
- The paper uses a brain-inspired aesthetic network to extract high-level features representing clothing elements such as color, structure, proportion, and style.
- Aesthetic preferences differ across people and change with seasons and longer-term fashion trends.
- Experiments compare aesthetic features with conventional features and evaluate the effectiveness of the resulting DCFA method on real-world datasets.
- The proposed DCF model represents purchase events across users, items, and time, then incorporates aesthetic and CNN features while using coupled matrices to alleviate sparsity.
2 RELATED WORK
The paper situates aesthetic-aware clothing recommendation at the intersection of aesthetic networks, image-based recommendation, and tensor factorization. Prior work provides image and contextual modeling tools, while tensor methods face sparsity and scalability challenges.
- The paper reviews aesthetic networks, image-based recommendation, and tensor factorization as foundations for aesthetic-aware clothing recommendation.
- Aesthetic-network research progressed from handcrafted features toward deep networks intended to represent complex human aesthetic perception.
- Image-based recommendation extends conventional recommender systems by incorporating product images alongside consumer interaction data.
- Tensor factorization models time as contextual information in recommendation, including CP, Tucker, PITF, and CMTF frameworks.
- Tensor-based recommendation methods can suffer from poor convergence on sparse data and limited scalability to large datasets.
3 PRELIMINARIES
The paper uses a brain-inspired deep network to derive aesthetic representations from image-related visual attributes. These representations are intended to complement semantic CNN features and improve clothing recommendation.
- BDN is a deep CNN structure with parallel pathways and a high-level synthesis network for image aesthetic assessment.
- Its pathways learn abstract aesthetic features from individual style labels, including complementary colors, duotones, and rule of thirds.
- The aesthetic representation uses raw low-level visual attributes such as hue, saturation, and value as inputs to the network.
- CNN features mainly encode semantic information, whereas BDN captures high-level aesthetic information for beauty estimation.
- The paper characterizes BDN features as more abundant and data-driven than handcrafted color histograms for representing aesthetic information.
4 CLOTHING RECOMMENDATION WITH AESTHETIC FEATURES
This section develops DCF, a time-aware tensor factorization model for user–clothing–time purchase events, and extends it with image features in DCFA. The model uses coupled matrices and BPR optimization to address sparsity and improve ranking-oriented recommendation.
- Aesthetic feature integration: DCFA combines DCF with image features, including concatenated CNN and aesthetic features, so latent and visual information jointly contribute to predictions.The image feature matrix supplies clothing representations, while user and time preference matrices encode aesthetic preferences across users and time intervals.
- Dynamic tensor factorization: DCF represents clothing purchases across users, items, and time, using a P × Q × R tensor of purchase events.Missing tensor entries correspond to purchases the model seeks to predict.
- Existing methods and limitations: Tucker decomposition is expressive but time consuming and hard to converge, while CP decomposition shares the same latent features across users, clothing, and time.PITF has linear complexity and strong representation ability but combines pairwise scores additively, which can recommend items that do not fit the current time.
- Dynamic tensor factorization: The proposed DCF separates latent clothing features for user preference from those for temporal information, making the model more expressive than CP decomposition.The model couples its tensor factorization with user–clothing and time–clothing matrices to mitigate sparsity.
- Model learning: BPR optimizes pairwise ranking from implicit purchase feedback, using mini-batch gradient descent with positive and randomly sampled non-observed items.Each batch includes b positive samples and 5b negative samples to construct 5b preference pairs.
5 EXPERIMENT
Experiments on real-world Amazon clothing data evaluate DCFA against recommendation baselines, feature variants, training settings, and qualitative recommendations. DCFA achieves the strongest reported performance and combines semantic and aesthetic information to improve recommendation quality.
- 5.1.1 Datasets.: The experiments use Amazon clothing records for recommendation and AVA photographic images with aesthetic ratings, tags, and styles for aesthetic-network training.The Amazon clothing data contain 39,371 users, 23,022 items, and 278,677 records after filtering.
- 5.2 Performance of Our Model (RQ1): DCFA outperforms VBPR by 8.53% on Recall@50 and 8.73% on NDCG@5.Models are evaluated on the test set after training, using top-n recommendations.
- 5.2 Performance of Our Model (RQ1): Recall increases with n, whereas NDCG generally decreases for non-random methods as the recommendation list grows.The evaluation varies n on the test set; RAND maintains constant ordering quality because it ranks items randomly.
- 5.3 Necessity of the aesthetic features (RQ2): DCFA combines CNN and aesthetic features, while DCF, DCFH, DCFCo, and DCFAo isolate no-image, color, CNN-only, and aesthetic-only variants.This comparison tests the contribution of aesthetic features alongside conventional visual information.
- 5.3 Necessity of the aesthetic features (RQ2): DCFA performs best among the feature variants because semantic and aesthetic information mutually enhance each other.The paper reports that DCFA outperforms DCFCo and DCFAo by about 5.06% and 8.7%, respectively.
- 5.3 Necessity of the aesthetic features (RQ2): Qualitative examples show DCFA recommendations matching consumers’ purchased styles more closely than DCFCo recommendations.The comparison uses selected consumers with explicit style preferences and single-category purchases, displaying five items from each model’s 50 recommendations.
6 CONCLUSION
The paper proposes a tensor factorization model incorporating aesthetic features to capture consumers’ style preferences over time, and reports strong performance on real-world implicit-feedback datasets. Future work includes better aesthetic-assessment data and networks, explicit-feedback evaluation, and integration of aesthetic-assessment knowledge.
- The proposed model incorporates aesthetic features into tensor factorization to capture consumers’ aesthetic preferences at particular times.
- Experiments on challenging real-world datasets show that the method dramatically outperforms state-of-the-art models and recommends items fitting consumers’ styles.
- Future work targets larger product-aesthetic datasets, improved aesthetic feature extraction, explicit-feedback evaluation, and integration of aesthetic-assessment domain knowledge.