Source-linked AI summary
Neural Compatibility Modeling with Attentive Knowledge Distillation
Xuemeng Song, Fuli Feng, Xianjing Han, Xin Yang, Wei Liu, Liqiang Nie
TL;DR
Clothing-matching research has largely overlooked fuzzy fashion-domain rules despite the practical challenge of creating compatible outfits. This paper integrates deep neural networks with attentive knowledge distillation to model compatibility, outperforming state-of-the-art baselines on a real-world dataset.
Problem
Existing clothing-matching studies overlook valuable fashion-domain matching rules, while constructing structured rules is difficult because expert knowledge is implicit, unstructured, and fuzzy.
Method
The proposed attentive knowledge distillation scheme jointly learns from data samples and fashion knowledge rules, assigning rule confidence attentively for compatibility modeling.
Results
The proposed scheme outperforms several state-of-the-art baselines on extensive experiments using a real-world dataset.
Takeaways & Limitations
The comparisons indicate benefits from incorporating fashion-domain knowledge into compatibility modeling and reveal fashion insights relevant to matching study.
Takeaways & Limitations
Rule activation currently relies only on contextual metadata, whose incomplete and noisy descriptions constrain identification; visual signals remain future work.
Abstract
from arXiv · showhide
Recently, the booming fashion sector and its huge potential benefits have attracted tremendous attention from many research communities. In particular, increasing research efforts have been dedicated to the complementary clothing matching as matching clothes to make a suitable outfit has become a daily headache for many people, especially those who do not have the sense of aesthetics. Thanks to the remarkable success of neural networks in various applications such as image classification and speech recognition, the researchers are enabled to adopt the data-driven learning methods to analyze fashion items. Nevertheless, existing studies overlook the rich valuable knowledge (rules) accumulated in fashion domain, especially the rules regarding clothing matching. Towards this end, in this work, we shed light on complementary clothing matching by integrating the advanced deep neural networks and the rich fashion domain knowledge. Considering that the rules can be fuzzy and different rules may have different confidence levels to different samples, we present a neural compatibility modeling scheme with attentive knowledge distillation based on the teacher-student network scheme. Extensive experiments on the real-world dataset show the superiority of our model over several state-of-the-art baselines. Based upon the comparisons, we observe certain fashion insights that add value to the fashion matching study. As a byproduct, we released the codes, and involved parameters to benefit other researchers.
1 INTRODUCTION
The paper addresses complementary clothing matching by combining data-driven neural networks with fashion-domain knowledge. Its attentive knowledge-distillation scheme handles fuzzy rules and sample-dependent rule confidence.
- Motivation: Existing clothing-matching methods rely mainly on deep neural networks, overlooking valuable fashion knowledge and suffering from poor interpretability.The paper motivates integrating human knowledge with data-driven learning.
- Challenges: Clothing matching requires structured rules despite fashion knowledge being implicitly conveyed, unstructured, fuzzy, and variably confident across samples.The authors identify rule construction, rule integration, and confidence assignment as central challenges.
- Proposed approach: AKD-DBPR uses a teacher-student framework in which the student learns a latent compatibility space while the teacher encodes domain knowledge.The student and teacher networks jointly support compatibility modeling through knowledge distillation.
- Contributions: The proposed attentive knowledge-distillation scheme encodes fashion-domain knowledge into neural networks and assigns rule confidence flexibly.The contribution extends traditional neural networks with attention over rule confidence.
2 RELATED WORK
Prior fashion research covers several prediction and retrieval tasks, but compatibility modeling has largely remained data-driven. The paper positions its approach as an effort to incorporate domain knowledge and knowledge distillation into fashion analysis.
- Fashion analysis: Existing fashion research addresses clothing retrieval, fashion trending prediction, fashionability prediction, and compatibility modeling.These studies represent the broader fashion-analysis landscape reviewed by the paper.
- Compatibility modeling: Most compatibility studies use data-driven deep learning and overlook domain knowledge, motivating knowledge-guided compatibility modeling.The paper also connects this direction with improved interpretability as a side product.
- Knowledge distillation: Knowledge distillation transfers knowledge from a teacher to a student, and related work has combined neural networks with structured logic rules or linguistic statistics.Prior applications include visual relationship detection, sentiment analysis, and named entity recognition.
- Research gap: The paper identifies fashion as a domain with limited prior knowledge-distillation research despite broader applications of the technique.This establishes the research context for applying knowledge distillation to clothing compatibility.
3 NEURAL COMPATIBILITY MODELING
The model learns clothing compatibility from heterogeneous item modalities in a latent space, then incorporates fashion rules through attentive teacher-student distillation. Training combines ranking-based compatibility learning, rule regularization, and dynamically weighted imitation of the teacher.
- 3.2 Problem Formulation: Clothing compatibility is modeled between tops and bottoms using visual and contextual item representations, positive outfit pairs, and positive or negative domain rules.Compatibility scores generate ranked bottoms for each top, while positive pairs come from expert-composed Polyvore outfits.
- 3.3 Data-driven Compatibility Modeling: A nonlinear neural network maps tops and bottoms from heterogeneous spaces into a latent compatibility space where compatible items can share high similarity.The model uses an MLP to combine visual and contextual modalities before producing latent representations and compatibility scores.
- 3.3 Data-driven Compatibility Modeling: Bayesian Personalized Ranking trains the student to score observed compatible pairs above unobserved pairs while regularization controls overfitting.The training triplet (i,j,k) indicates that bottom b_j should be more compatible with top t_i than bottom b_k.
- 3.4 Attentive Knowledge Distillation: Knowledge distillation constructs a teacher that combines rule regularization with closeness to the student’s compatibility predictions, while the student learns from both data and teacher guidance.The imitation parameter ρ calibrates the relative importance of ground-truth learning and teacher imitation.
- 3.5 Teacher Network Construction: Attention assigns sample-specific confidence to rules because different rules may have different confidence levels for different clothing pairs.Rule functions reward compatibility predictions satisfying positive rules and penalize predictions violating the corresponding constraints; rule notation uses value1 + value2 for positive rules and no value1 + value2 for negative rules.
4 EXPERIMENT
Experiments on FashionVC evaluate convergence, model comparisons, rule guidance, attention, and retrieval. AKD-DBPR generally improves compatibility modeling and retrieval, while results show that rule usefulness varies by rule type and sample.
- Experimental Settings: FashionVC contains 20,726 expert-composed outfits, with 14,871 tops and 13,663 bottoms, each associated with visual, category, and title information.
- Experimental Settings: The model’s objective and training AUC change rapidly during early epochs before becoming steady, indicating convergence of the learning scheme.
- Model Comparison: AKD-DBPR-p and AKD-DBPR-q both surpass DBPR, while DBPR outperforms the other pure data-driven baselines.
- Model Comparison: Knowledge rules help especially when competing bottoms are visually compatible, but some rules produce failed triplets because they are not universally applicable.
- Rule Guidance: Negative rules and category rules provide stronger guidance than positive rules and rules based on attributes such as material and color.
- Attention Mechanism: AKD-DBPR consistently outperforms uniformly weighted UKD-DBPR across modality settings, with relative improvements of 6.97% using visual input and 2.69% using contextual input versus DBPR.
- Attention Mechanism: Attention assigns different confidence levels to the same rule across triplets and can reduce the influence of fuzzy rules when visual evidence suggests incompatibility.
- Fashion Item Retrieval: AKD-DBPR and DBPR outperform other baselines across candidate counts and scenarios, handle observed and unobserved tops, and AKD-DBPR exceeds DBPR especially for observed tops.
5 CONCLUSION AND FUTURE WORK
The paper presents attentive knowledge distillation for compatibility modeling by jointly learning from fashion samples and domain rules. Experiments support its effectiveness, while rule identification remains constrained by incomplete and noisy contextual metadata.
- The proposed scheme jointly learns compatibility from specific fashion samples and general knowledge rules.
- The experiments indicate that negative matching rules and category-related rules are more powerful than other rules.
- Rule identification currently relies on contextual metadata, whose incompleteness and noise constrain the method.The authors plan to explore visual signals for identifying activated rules.