PaperlayerView exact revision on arXiv

Source-linked AI summary

Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering

Ruining He, Julian McAuley

arXiv:1602.01585v1cs.AIcs.IR

TL;DR

The paper addresses the difficulty of modeling personalized visual preferences and their non-linear evolution in sparse, large-scale fashion recommendation. It combines deep CNN visual features, implicit user feedback, and community trends in One-Class Collaborative Filtering, outperforming state-of-the-art methods on two Amazon clothing datasets while visualizing fashion change over time.

  • Problem

    Fashion recommendation must model subtle visual preferences that are personal, temporally evolving, and mixed with non-visual factors, while also facing cold-start sparsity in long-tailed product collections.

  • Method

    The method combines deep visual features, implicit feedback, evolving community trends, non-visual dynamics, and automatically discovered fashion epochs within One-Class Collaborative Filtering.

  • Results

    The models significantly outperform state-of-the-art methods on two Amazon clothing datasets in both warm- and cold-start settings, while visualizations reveal non-linear fashion shifts.

  • Takeaways & Limitations

    Learned models can estimate personalized rankings while supporting visualizations of how fashion and visual dimensions change over time.

  • Takeaways & Limitations

    Visual dimensions may capture image artifacts such as model poses or photo setups in addition to clothing characteristics.

Abstract

from arXiv · show

Building a successful recommender system depends on understanding both the dimensions of people's preferences as well as their dynamics. In certain domains, such as fashion, modeling such preferences can be incredibly difficult, due to the need to simultaneously model the visual appearance of products as well as their evolution over time. The subtle semantics and non-linear dynamics of fashion evolution raise unique challenges especially considering the sparsity and large scale of the underlying datasets. In this paper we build novel models for the One-Class Collaborative Filtering setting, where our goal is to estimate users' fashion-aware personalized ranking functions based on their past feedback. To uncover the complex and evolving visual factors that people consider when evaluating products, our method combines high-level visual features extracted from a deep convolutional neural network, users' past feedback, as well as evolving trends within the community. Experimentally we evaluate our method on two large real-world datasets from Amazon.com, where we show it to outperform state-of-the-art personalized ranking measures, and also use it to visualize the high-level fashion trends across the 11-year span of our dataset.

1. INTRODUCTION

The paper targets visually aware recommendation by modeling personalized preferences alongside evolving fashion trends. It addresses visual complexity, personalization, temporal change, non-visual influences, and cold-start sparsity with scalable models evaluated on Amazon clothing data.

  • Motivation: Visual fashion preferences are difficult to model because they are subtle, personal, temporally evolving, and entangled with non-visual factors.Large product and purchase corpora are needed to extract meaningful visual signals, while fashion changes over time and product quality also affects decisions.
  • Goals: The paper aims to build recommender systems that are scalable, personalized, temporally evolving, and interpretable.The intended scope includes fashion domains with long-tailed, continually expanding product collections and cold-start items.
  • Approach: The proposed models capture temporal dynamics in One-Class Collaborative Filtering by discovering fashion epochs with distinct prevailing visual decision factors.They learn from implicit feedback such as purchase histories, bookmarks, browsing logs, and mouse activities.
  • Approach: The method combines non-visual dimensions and temporal dynamics with BPR-based training that jointly learns epoch segmentation and model parameters.This accounts for non-visual sources while producing a full recommendation system.
  • Results: The models significantly outperform state-of-the-art methods on Amazon Women’s and Men’s Clothing & Accessories data in both warm- and cold-start settings.The paper also visualizes learned fashion trends and reports complex, non-linear shifts over recent years.

2. MODELING THE TEMPORAL DYNAMICS OF VISUAL STYLES

The paper extends visual recommendation to model how fashion preferences evolve over time from implicit purchase feedback. It combines visual representations, temporal dynamics, and category information to predict personalized item rankings.

  • Purchase histories provide implicit feedback for learning time-dependent personalized rankings of items users have not interacted with.The model uses purchases rather than star ratings and aims to rank likely purchases above nonpurchases.
  • Deep CNN image features are linearly embedded into a low-dimensional visual style space whose dimensions are learned to predict users’ opinions.The embedding matrix E maps high-dimensional image features into K′ visual dimensions, while θu and θi model user-item visual compatibility.
  • The temporal model represents item visual factors as functions of time to capture fashion dynamics while keeping the parameterization tractable for sparse datasets.It considers temporal attractiveness drift, temporal weighting drift, and combined global temporal structures.
  • Global fashion dynamics use a stationary embedding plus time-dependent deviations, modeling community-wide evolution with expressive low-rank structures.The stationary component is E, while ΔE(t) captures drifting visual factors with a modest number of parameters.
  • A temporal weighting vector models changing emphasis on visual dimensions, while the Hadamard product combines dimension-specific weights with visual factors.This allows users’ attention to dimensions such as colorfulness to vary over time.
  • Temporal subcategory biases explicitly model changing opinions toward categories, discouraging visual factors from merely predicting product subcategories.The model adds β_Ci(t) for the item’s subcategory when category trees are available.
  • Global fashion trends were particularly useful for personalized ranking, whereas personal visual drift had little effect because the datasets were sparse and users were active briefly.The experiments therefore retained stationary user visual factors while allowing their preferences to be affected by global fashion trends.

3. LEARNING THE MODEL

The model learns personalized temporal rankings by combining BPR likelihood optimization with alternating updates to model parameters and fashion-epoch segmentation. Dynamic programming efficiently finds the segmentation that best predicts rankings within continuous timeline bins.

  • BPR objective: BPR directly optimizes personalized rankings by comparing observed items with non-observed alternatives for each user.Training uses user-item-item triples, with observed items preferred over unobserved items through a sigmoid preference model.
  • Temporal ranking: Temporal training tuples add each positive feedback timestamp, so observed items are ranked above non-observed items at the time of feedback.The temporal training set consists of quadruples (u, i, j, t_ui).
  • Joint learning: The objective jointly fits model parameters Θ and timeline segmentation Λ into N fashion epochs.Both components are optimized through the regularized log-likelihood.
  • Coordinate ascent: Coordinate ascent alternates between fitting Θ with fixed Λ and segmenting Λ with fixed Θ.The procedure repeats these two fitting steps until convergence or until validation performance stops improving.
  • Timeline segmentation: Dynamic programming partitions the timeline into N equal-sized initial bins and finds the segmentation maximizing ranking accuracy within bins.The segmentation step admits an O(|D+_S|×N) solution; sampling non-observed items approximates the full log-likelihood for scalability.

4. EXPERIMENTS

The experiments evaluate the method on two long-span Amazon fashion datasets using visual features, temporal held-out evaluation, and AUC-based test comparisons. The design targets fashion dynamics while preserving a user-level personalized ranking setting.

  • Datasets: Experiments use two Amazon.com datasets covering U.S. Women’s and Men’s Clothing & Accessories over roughly the past decade.The categories provide broad clothing vocabularies and long temporal spans for studying changing visual decision factors.
  • Preprocessing: Users’ review histories provide implicit feedback, while one image per item supplies visual features; users with fewer than five actions are discarded.Dataset statistics after processing are reported in Table 2.
  • Visual features: A pretrained Caffe convolutional neural network produces 4096-dimensional visual feature vectors for items.The network has five convolutional layers and three fully connected layers and was pretrained on 1.2 million ImageNet images.
  • Evaluation: Evaluation ranks held-out positive items for each user at their feedback timestamps, reflecting the model’s time-dependent recommendation setting.Data are split per user into training, validation, and test interactions, with hyperparameters selected on validation data.
  • Evaluation: All methods are evaluated on the test set with the widely used AUC measure.The test set is evaluated after selecting the best hyperparameters on the validation set.

4.4 Comparison Methods

The comparison includes popularity and matrix-factorization baselines spanning point-wise, pairwise, visual, temporal, and taxonomy-aware recommendation. These models isolate the value of personalization, visual signals, and temporal dynamics.

  • Baseline families: The study primarily compares against state-of-the-art matrix-factorization methods for implicit feedback.The baselines include both point-wise and pairwise matrix-factorization models.
  • Baseline families: Popularity ranks items by their observed popularity rather than personalized preference.POP provides a non-personalized reference point.
  • Matrix factorization: WR-MF is a point-wise model that assigns confidence levels to feedback before factorizing a weighted matrix.It represents a confidence-weighted implicit-feedback approach.
  • Matrix factorization: BPR-MF performs personalized ranking with standard matrix factorization, while BPR-TMF adds taxonomy and temporal category and item biases.These models distinguish pairwise ranking from added temporal and taxonomy effects.
  • Visual and temporal models: VBPR models raw visual signals without temporal dynamics, whereas TVBPR adds visual temporal dynamics without non-visual dynamics.TVBPR+ further includes non-visual temporal dynamics and uses all terms in the predictor.
  • Comparison dimensions: The comparison tracks whether each model is personalized, visually aware, temporally aware, and taxonomy aware.All time-aware methods use the proposed coordinate-ascent training procedure.

4.5 Performance

Performance is evaluated with AUC on all test items and on cold-start items, which comprise around 60% of the test set. Visual and temporal modeling improve ranking, with TVBPR+ producing the strongest reported gains over VBPR.

  • Evaluation settings: AUC measures overall ranking in “All Items” and cold-start ranking separately; cold-start items had fewer than five training positives and comprise around 60% of the test set.The evaluation targets both general accuracy and the sparse-item setting that is especially prevalent in these fashion datasets.
  • Baseline comparisons: BPR-MF outperforms WR-MF and POP, while POP is especially ineffective for cold-start items because they are inherently unpopular.BPR-MF is described as a state-of-the-art personalized-ranking method for implicit feedback.
  • Baseline comparisons: BPR-TMF improves over BPR-MF by 1.5% for all items and 4.3% for cold-start items on average.The improvement uses taxonomy information and temporal dynamics.
  • Visual modeling: VBPR improves over BPR-MF by up to 11.6% on Women’s Clothing and 10.4% on Men’s Clothing, reaching 29.0% and 25.1% in cold start.Visual signals are reported as especially helpful when BPR-MF lacks enough observations to learn reliable item factors.
  • Temporal modeling: TVBPR captures three types of fashion dynamics and yields significant improvements over VBPR, addressing VBPR’s inability to model system dynamics.The paper motivates finer-grained temporal modeling because the feedback spans more than a decade.
  • Temporal modeling: TVBPR+ outperforms VBPR by 4.8% and 3.1% for all items, and by 9.6% and 8.1% for cold start, on Women’s and Men’s Clothing respectively.TVBPR+ incorporates non-visual dynamics into TVBPR.
  • Observed patterns: Temporal models show comparably larger improvements on Women’s Clothing than Men’s Clothing, possibly because of dataset size or richer temporal dynamics.The passage presents these explanations as presumptive rather than established causes.

4.6 Visualization

The model visualizes interpretable visual dimensions and their evolving weights, then maps items into a time-dependent visual space to reveal broader fashion shifts.

  • The model discovers visual dimensions combining color, shape, texture, and human-relevant distinctions such as semi-formal versus casual styles.
  • Figure 3 ranks representative tees for each dimension and plots its weight across epochs, revealing generally continuous but occasionally abrupt changes.
  • Longer ranked lists indicate that the learned dimensions focus on clothing characteristics rather than incidental photo properties such as poses or setups.
  • The visual score averages each item's visual predictor component across users, providing an overall measure of visual popularity within an epoch.
  • A t-SNE map of 30,000 test images combines item proximity with normalized epoch scores to visualize styles and fashion popularity over time.
  • Heat maps show a shoe style gaining popularity and then losing appeal in recent years.

4.7 Case Study: Men’s Fashion in the 2000s

The men’s-fashion case study tests whether the model captures documented resurgences by examining visual neighbors and popularity across fashion epochs.

  • The case study uses TVBPR+ trained on Amazon Men’s Clothing to visualize interpretable dynamics in men’s fashion during the 2000s.
  • Four external query images are matched to nearest neighbors in the learned visual space, and their normalized visual scores are computed for each epoch.
  • The model predicts these styles gaining popularity especially since 2009, supporting its ability to capture real-world fashion dynamics.

5. RELATED WORK

Related work spans one-class collaborative filtering, temporal modeling, visual representations, and visually-aware recommendation; the paper combines these strands while retaining scalability and personalization.

  • One-Class Collaborative Filtering: One-Class Collaborative Filtering addresses recommendation when only positive feedback, such as purchases, is observed.
  • One-Class Collaborative Filtering: Bayesian Personalized Ranking is presented as the state-of-the-art OCCF framework, and the proposed model extends BPR-MF with fashion awareness.
  • Modeling Temporal Dynamics: Prior temporal approaches model concept drift and time-dependent collaborative filtering, but fashion dynamics can involve multiple simultaneous temporal sources.
  • Visual Models: Visual compatibility methods learn image relationships but do not use historical user feedback for personalized preferences or account for non-visual recommendation factors.
  • Visually-aware Collaborative Filtering: Earlier visually-aware collaborative filtering combines visual interactions but ignores fashion’s temporal dynamics, limiting questions about evolving styles.

6. CONCLUSION

The paper concludes that scalable models combining product images and user feedback can capture temporal drifts in fashion and personal tastes.

  • Deep CNN features model visual dimensions and their temporal dynamics, while low-rank structures efficiently capture fashion dynamics.
  • The models significantly outperform state-of-the-art approaches and visualize non-linear changes in visual dimensions and fashion over the past decade.

A. SCALABILITY ANALYSIS

The scalability analysis compares TVBPR+ with BPR-MF and VBPR on parameter-update complexity and training time. TVBPR+ limits visual updates to the relevant fashion epoch, while epoch segmentation is trained less frequently.

  • Parameter fitting: The method samples |P| training tuples per iteration and repeats parameter updates for 100 iterations.The sampling scheme follows BPR-MF as implemented in MyMediaLite.
  • Parameter fitting: TVBPR+ updates only the visual parameters associated with the sampled triple’s timestamp epoch, despite modeling multiple fashion epochs.This design avoids updating all visual parameters for every training triple.
  • Parameter fitting: BPR-MF requires O(K) per training triple, whereas VBPR requires O(K+K′) because it also updates visual parameters.
  • Epoch segmentation: Epoch segmentation is trained at much lower frequency than parameter fitting and consumes much less time.The segmentation procedure is described as the outer loop.
  • Training time: 20 hours of training on the Women’s Clothing dataset included approximately 45 minutes for epoch fitting on a commodity desktop machine.TVBPR+ generally requires more iterations to converge than VBPR because it learns temporal dynamics.
Loading 1602.01585v1…