PaperlayerView exact revision on arXiv

Source-linked AI summary

Fashion Outfit Generation via Unified Sequential Composition Models

Kaicheng Pang, Xingxing Zou, Ruohan Xu, Waikeung Wong

arXiv:2608.13888v1cs.LG

TL;DR

Generating stylistically coherent outfits from huge item libraries is difficult because aesthetic compatibility is implicit, non-monotonic, and combinatorial. The paper formalizes this as constrained ensemble generation and combines USCM with LE-MCTS, achieving state-of-the-art results across discriminative and generation tasks, including zero-shot settings.

  • Problem

    Fashion outfit generation must balance stylistic coherence within a large item library despite combinatorial search and implicit, non-monotonic compatibility.

  • Method

    The framework formalizes outfit generation as deterministic constrained ensemble generation and couples USCM’s compatibility and intent priors with LE-MCTS for candidate selection.

  • Results

    The framework achieves state-of-the-art performance across discriminative benchmarks and constrained outfit generation, including robust zero-shot transfer to unseen data.

  • Takeaways & Limitations

    The results support USCM with LE-MCTS as a framework for navigating complex aesthetic landscapes in constrained fashion outfit generation.

  • Takeaways & Limitations

    Evaluation may not capture the full spectrum of subjective fashion preferences, and the model lacks professional stylists’ creativity and contextual sensitivity.

Abstract

from arXiv · show

The task of synthesizing stylistically coherent fashion outfits from massive item libraries, known as fashion outfit generation, remains a non-trivial challenge, primarily due to the non-monotonic and implicit nature of aesthetic compatibility, coupled with the exponentially large combinatorial search space. In this paper, we formalize this task as Constrained Ensemble Generation (CEG) and model it as a finite-horizon deterministic Markov Decision Process. To address CEG in fashion, we propose the Unified Sequential Composition Model (USCM), which jointly models set-level compatibility and latent composition intents. Guided by USCM's learned priors, a Latent Expansion Monte Carlo Tree Search (LE-MCTS) mechanism is proposed to handle item retrieval during composition, balancing local aesthetic synergy with global structural balance. Extensive experiments on the Polyvore Outfits dataset, along with zero-shot evaluations on the iFashion and PolyvoreU datasets, demonstrate that our framework achieves state-of-the-art performance across independent human preference evaluations, automated aesthetic proxies, and structural validity metrics for constrained fashion outfit generation.

1 Introduction

The paper formulates fashion outfit generation as Constrained Ensemble Generation (CEG), a deterministic finite-horizon MDP addressing combinatorial search, non-monotonic objectives, and sparse feedback. It introduces USCM with LE-MCTS to jointly model compatibility and latent composition intent while guiding forward-looking item selection, achieving state-of-the-art results across datasets and evaluation settings.

  • Problem formulation: CEG models outfit synthesis as a deterministic MDP over permutation-invariant item collections, where actions add library items and terminal states maximize constraint-aware compatibility.The formulation targets combinatorial explosion, non-monotonic objectives, and sparse feedback in set synthesis.
  • Method: USCM jointly learns outfit compatibility evaluation and next-item latent intent prediction in a shared multimodal manifold for ensembles and items.This representation engine is coupled with LE-MCTS for lookahead candidate selection through dynamic latent branch expansion.
  • Method: LE-MCTS performs forward-looking candidate selection by dynamically expanding latent branches, balancing the search demands of constrained ensemble generation.The mechanism is guided by USCM’s learned compatibility and policy priors.
  • Empirical validation: Experiments on Polyvore Outfits, iFashion, and PolyvoreU show state-of-the-art performance on discriminative benchmarks and constrained fashion generation.The framework surpasses greedy and retrieval-based methods and transfers zero-shot to unseen data.
  • Empirical validation: Structural evaluations find that the approach best satisfies structural rules without explicit structural rules, while ablations support its architecture, search strategies, budgets, and zero-shot transferability.These findings accompany evaluations across structural metrics and unseen data.

2 Related Work

Related work has primarily used discriminative models to predict outfit compatibility, later incorporating large multimodal and graph attention models. Generative methods have emerged because passive evaluators cannot navigate the expensive search space of ensemble composition.

  • Discriminative and Generative Methods: Prior fashion outfit-generation studies used metric learning, sub-space projections, and graph neural networks to predict outfit compatibility.The field later incorporated large multimodal models and graph attention networks.
  • Discriminative and Generative Methods: These approaches remain passive evaluators and cannot navigate the expensive search space required for ensemble composition.Generative methods have recently emerged to address this limitation.

3 Methodology

The methodology formulates constrained fashion outfit generation as a finite-horizon deterministic MDP and solves its intractable search problem with USCM-guided LE-MCTS. USCM jointly estimates compatibility and latent composition intents, while LE-MCTS retrieves and expands promising feasible items under explicit constraints.

  • CEG Formulation: CEG represents outfit composition as a finite-horizon deterministic MDP over nonempty subsets of a fashion-item library, starting from a seed ensemble and adding one item per step.States preserve permutation invariance, with transitions defined by set union.
  • Unified Sequential Composition Model: USCM uses a shared, permutation-invariant manifold to jointly evaluate ensemble compatibility, encode next-item intents, and index library items for nearest-neighbor retrieval.Its Value Head predicts complete-ensemble aesthetic compatibility and serves as a heuristic for intermediate states, while the Policy Head generates latent intent priors.
  • CEG Formulation: The objective maximizes implicit compatibility within explicitly constrained reachable ensembles, but non-monotonic stylistic regularities make exhaustive search computationally intractable.The framework therefore uses learned value estimates and policy priors to guide near-optimal composition.
  • USCM Training: USCM minimizes L = Lv + λpLp, combining Focal Loss compatibility classification with robust 1:2:2 positive-to-negative sampling and InfoNCE latent-intent alignment.The policy objective aligns partial-ensemble queries with positive counterpart items and treats other batch items as negatives.
  • LE-MCTS Inference: LE-MCTS restricts search to feasible states, retrieves top-k latent-similar candidates, evaluates expanded states with USCM, and backpropagates values across M simulations.Progressive widening limits branching through |children(u)| < Cpw · (nu)^αpw, balancing exploration and exploitation.

4 Experiments

Experiments on Polyvore Outfits and zero-shot tests on iFashion and PolyvoreU show that the framework achieves strong aesthetic and structural performance for constrained fashion outfit generation. Ablations further attribute these gains to USCM’s unified representation and LE-MCTS’s look-ahead planning.

  • Metrics: Evaluation combines human preference, neural aesthetic, category-distribution, and rule-based structural-validity metrics to reduce evaluator and category-inflation bias.Category JSD measures alignment with empirical category proportions, while Svalid audits functional coverage and logical compositional consistency.
  • Constrained Ensemble Generation Results: 41.81% of human evaluators preferred the proposed method, versus 32.65% for VLLM and 25.54% for OutfitTransformer; Sneural was 13.43 versus 11.62 and 5.28.The human-preference difference was significant (χ2, p < 0.01).
  • Constrained Ensemble Generation Results: Category JSD reached 0.0059, while VLLM achieved the highest functional completeness with Svalid = 0.845; the proposed framework led across metrics except Svalid.Random performed worst, whereas Type-aware’s greedy step-wise ranking limited long-horizon structural coordination.
  • Model Architecture Ablation: USCM’s unified architecture improved structural outcomes over the separate model, achieving Category JSD 0.0059 versus 0.0134 and ensemble validity 0.815 versus 0.509.The unified model’s substantial FITB advantage provided more accurate guidance for LE-MCTS candidate identification.
  • Search Strategy Ablation: LE-MCTS’s look-ahead planning outperformed greedy search early, and M = 10 offered the best cost-quality trade-off with final score 5.146 versus 5.172 for LE-MCTS-100.Type-Aware reached only 1.79, while incremental gains peaked at Step 2 and declined thereafter.

5 Conclusion · A LE-MCTS Algorithmic Details

The paper concludes by framing fashion outfit generation as constrained ensemble generation and introducing USCM with LE-MCTS to balance structural constraints, aesthetic harmony, and large search spaces. The appendix specifies LE-MCTS’s progressive-widening search, candidate expansion, value backpropagation, and visit-based action selection.

  • 5 Conclusion: The framework formulates fashion outfit generation as Constrained Ensemble Generation within a deterministic Markov Decision Process.This addresses the tension between explicit structural constraints, implicit aesthetic harmony, and an exponentially large search space.
  • 5 Conclusion: USCM jointly aligns continuous policy priors and compatibility modeling to solve the constrained ensemble generation problem.
  • 5 Conclusion: Current evaluation protocols may not fully capture human preferences because fashion aesthetics are highly subjective.
  • 5 Conclusion: The USCM Value Head provides a zero-shot heuristic for intermediate states but is trained only on complete outfits, not partial ensembles.
  • A LE-MCTS Algorithmic Details: LE-MCTS initializes search at root state S0, uses USCM fθ and VectorDB D, and configures simulations with Cpuct, Cpw, αpw, τ, and k.
  • A LE-MCTS Algorithmic Details: During selection, progressive widening initializes node values and latent representations, retrieves k candidates, and assigns candidate priors with softmax(sim/τ).
  • A LE-MCTS Algorithmic Details: Expansion appends the best candidate to the current state, evaluates the new state with USCM, creates a child node, and backpropagates visit and value statistics.
  • A LE-MCTS Algorithmic Details: After all simulations, LE-MCTS returns the root action whose child has the most visits.

B Implementation Details

USCM is implemented as a Transformer-based model trained with FashionCLIP visual features on Polyvore Outfits, while LE-MCTS uses specified retrieval, widening, exploration, and termination settings. Experiments constrain outfit ensembles to six items and initialize them from randomly sampled core-category items.

  • USCM Details: USCM uses a 6-layer Transformer encoder with 16 attention heads, feed-forward dimension 2024, and 512-dimensional FashionCLIP visual features.The model is trained for 50,000 steps with batch size 1,000 using AdamW with peak learning rate 2 × 10−5.
  • Training Resources: USCM training runs on a single NVIDIA GeForce RTX 4090 D GPU with 24GB VRAM for approximately 8 hours and 20 minutes.Peak GPU memory consumption is approximately 12.5 GB during the 50,000-iteration schedule.
  • LE-MCTS Details: LE-MCTS uses M = 100 simulations, candidate retrieval breadth k = 10, Cpw = 1.0, αpw = 0.5, Cpuct = 40.0, and δ = 0.1.The prior-probability temperature is τ = 1.0, and search terminates when marginal value gain falls below δ.
  • CEG Experiment Details: All methods limit maximum ensemble length to six items, matching the Polyvore dataset’s average outfit size of 5.34.The initial ensemble randomly samples one or two items from core categories within ground-truth outfits.

C Detailed Definitions of Structural Metrics · C.1 Category Jensen-Shannon Divergence (Cate. JSD)

This section formally defines structural metrics for generated fashion ensembles, focusing on Category Jensen-Shannon Divergence (Cate. JSD) as a measure of similarity between generated and natural category distributions. Cate. JSD uses probability-vector distributions and penalizes over-generation of safe categories that can inflate compatibility scores.

  • C Detailed Definitions of Structural Metrics: The section introduces formal definitions for Category Jensen-Shannon Divergence (Cate. JSD) and the Ensemble Completeness Score Sfull.
  • C.1 Category Jensen-Shannon Divergence (Cate. JSD): Cate. JSD quantifies similarity between the generated category distribution and the dataset’s natural distribution.
  • C.1 Category Jensen-Shannon Divergence (Cate. JSD): The metric represents ground-truth P and generated Q as probability vectors over K unique fashion categories, with elements encoding category frequencies.
  • C.1 Category Jensen-Shannon Divergence (Cate. JSD): The Jensen-Shannon Divergence uses the average distribution 2(P + Q) and the Kullback-Leibler divergence DKL.
  • C.1 Category Jensen-Shannon Divergence (Cate. JSD): Using Shannon entropy H(P) = −PK i=1 pi log2 pi, the JSD can be computed from entropy terms.
  • C.1 Category Jensen-Shannon Divergence (Cate. JSD): A value of 0 indicates identical distributions, while the metric penalizes over-generation of safe categories such as accessories to inflate compatibility scores.

C.2 Ensemble Validity Score (Svalid)

Svalid is a heuristic structural-integrity metric that marks ensembles as valid when they meet functional and logical outfit requirements. Across generated outfits, it averages the validity successes, while 78.38% of human-authored ground-truth outfits satisfy the criteria.

  • Validity criteria: Svalid uses a binary validity function fvalid(S) ∈ {0, 1} to assess whether an ensemble is functionally complete and logically consistent.The rule is presented as an approximate but practical benchmark because perfect categorical grammar cannot be exhaustively hand-coded.
  • Score definition: Svalid is defined for a test set of N generated outfits as the average success rate.The score summarizes the proportion of generated ensembles passing the validity rule.
  • Empirical validation: 78.38% of human-authored ground-truth outfits naturally satisfy the validity criteria, supporting the heuristic’s practical relevance as an evaluative baseline.This dataset-level result is used to empirically validate the approximate rule.

D Qualitative Results · D.1 Extensive Qualitative Comparisons

The qualitative comparisons show that the proposed method generates aesthetically coordinated and structurally coherent outfits across four seed-item initialization scenarios. Despite noisy supervision and flawed human-composed references, it outperforms the compared baselines in aesthetic coordination and category coverage.

  • D.1 Extensive Qualitative Comparisons: The study presents an extensive qualitative comparison of generated ensembles against OutfitTransformer, Type-aware, and original Human Composed outfits.The comparison appears in Figure 4.
  • D.1 Extensive Qualitative Comparisons: Evaluations cover four initialization scenarios based on seed-item category: tops, bottoms, shoes, and all-body items.These scenarios test generation across different starting categories.
  • D.1 Extensive Qualitative Comparisons: The proposed method consistently produces superior aesthetic coordination and more reasonable category coverage than the baselines.This qualitative advantage is reported across the comparison settings.
  • D.1 Extensive Qualitative Comparisons: Human Composed outfits frequently contain uncoordinated stylistic choices and redundant category overlaps.These issues are visible in the dataset’s ground-truth outfit columns.
  • D. Qualitative Results: The dataset’s ground-truth outfits therefore provide noisy and imperfect supervision for learning outfit composition.The observation follows from their stylistic and categorical inconsistencies.
  • D.1 Extensive Qualitative Comparisons: Despite this supervision, the proposed method consistently synthesizes more aesthetically harmonious and structurally coherent outfits than the baseline methods.This finding summarizes the method’s qualitative robustness under imperfect training data.

D.2 Qualitative Analysis: Generating Variations from Same Seed

The qualitative analysis shows that replacing LE-MCTS’s deterministic final selection with temperature-controlled stochastic sampling enables diverse outfit variations from identical seed items while preserving global harmony.

  • Selection Strategy: The study relaxes LE-MCTS’s deterministic maximum-visit selection rule to examine the richness of the explored search space.Standard LE-MCTS selects the child node with maximum visit count n, whereas this qualitative analysis permits variation.
  • Selection Strategy: After the simulation budget M is exhausted, child nodes are sampled from a Softmax distribution over their visit counts ni.The temperature parameter τ controls the distribution, allowing different high-potential branches explored during MCTS to be followed.
  • Generated Variations: From identical seed items, the framework generates variations differing in silhouette, item category, and texture while maintaining consistent global harmony.With a brown knit dress seed, examples range from formal outfits with long coats and leather boots to casual outfits with lighter cardigans and suede accessories.

D.3 Qualitative Analysis: Generalization to Unseen Datasets

The framework demonstrates zero-shot transferability on unseen iFashion and PolyvoreU datasets across diverse initial seed categories. LE-MCTS adapts to dataset-specific distributions without fine-tuning, including PolyvoreU’s shorter, accessory-free outfits.

  • Zero-shot evaluation: Zero-shot generation is evaluated on unseen iFashion and PolyvoreU datasets, with each Figure 6 column fixing the initial seed-item category.This setup tests consistency across diverse starting points.
  • iFashion: On iFashion, the method effectively adapts to the dataset’s distinct style distribution.The result is shown in Figure 6(a).
  • PolyvoreU: For PolyvoreU, the maximum ensemble length is set to T = 4 to match its average outfit length of 3.35 and absence of accessory items.The constraint aligns generation with PolyvoreU’s underlying structural distribution.
  • PolyvoreU: Despite structural differences and no fine-tuning on these domains, the LE-MCTS planner successfully navigates PolyvoreU generation under the imposed constraint.The supplied passage states this conclusion but is truncated after “successfully navig”.
Loading 2608.13888v1…