RA-CoA: Training-free Fashion Image Captioning via Retrieval-Augmented Chain-of-Attributes

1 Introduction

• Fashion image captioning must capture fine-grained details such as stylistic patterns, closure mechanisms, and collar and sleeve silhouettes.The need is heightened by evolving inventories and vocabularies, which make repeated dataset curation and model retraining operationally impractical.Source: §1 · pp. 1–2
• General-purpose VLMs often hallucinate or misidentify fashion attributes, while in-context learning can copy exemplar attributes and chain-of-thought can still miss subtle details.The cited examples include difficulty distinguishing a mandarin collar from a band collar.Source: §1 · p. 2
• RA-CoA retrieves candidate attributes from a product knowledge base, then grounds each attribute value in the image before synthesizing a caption.This decomposition is designed to avoid copying exemplar attributes and to constrain reasoning to visually plausible candidates.Source: §1 · p. 3
• Across open- and closed-source VLMs, automatic metrics and user studies show RA-CoA consistently improves attribute precision and semantic coherence over prompting baselines.The comparisons include zero-shot, in-context learning, and implicit and explicit chain-of-thought prompting.Source: §1 · p. 3
• On held-out, web-curated data, RA-CoA outperforms a state-of-the-art supervised fashion captioning method without task-specific training or fine-tuning.The authors present this as evidence of generalization under distribution shift and suitability for dynamic e-commerce settings.Source: §1 · p. 3

2.1 Fashion image captioning

• Fashion captioning research spans attribute-aware models and vision-language pretraining, but many existing methods require substantial fashion-specific fine-tuning.RA-CoA is contrasted as a training-free, model-agnostic approach for attribute-grounded captions in a zero-shot setting.Source: §2.1 · pp. 3–4

2.2 Retrieval-augmented image captioning

• Earlier retrieval-augmented captioning methods condition generation on retrieved captions or train retrieval-conditioned models, whereas RA-CoA retrieves attribute keys and infers their values with a frozen VLM.The related-work discussion distinguishes this structured, training-free approach from methods focused on concept completion or retrieved-caption conditioning.Source: §2.2 · p. 4

2.3 Zero-shot prompting strategies for Vision Language Models

• Zero-shot prompting, in-context learning, and chain-of-thought can struggle with fashion attribute precision; ICL may reproduce exemplar content and CoT has limited effectiveness for attribute-centric captioning.The paper positions retrieval plus explicit attribute reasoning as its response to these limitations.Source: §2.3 · p. 4

3 Methodology

• RA-CoA identifies relevant attribute types, predicts their values, then synthesizes a caption from the attribute-value pairs.The task is framed around an image, product name, and attribute-value pairs.Source: §3 · p. 5

3.1 ProductKB Construction

• ProductKB stores cropped product images, names, captions, and structured attribute-value pairs extracted from FashionGen captions.Florence-2 crops the target product using its name to address images containing multiple products.Source: §3.1 · p. 5

3.2.1 Image embedding and retrieval

• The query product crop is embedded with CLIP, and the top-K visually similar ProductKB items are retrieved by cosine similarity.The retrieved items are later used as in-context exemplars.Source: §3.2.1 · p. 6

3.2.2 Attribute extraction

• RA-CoA aggregates unique attribute keys from retrieved products, excluding their values so that similar products do not transfer potentially incorrect specifics.The resulting keys indicate which values to predict from the query image.Source: §3.2.2 · p. 6

3.2.3 Chain-of-Attributes (CoA)

• CoA queries the VLM separately for each retrieved attribute, producing an attribute-value set with focused predictions.The authors state that this isolation reduces ambiguity and cognitive load, yielding more accurate and interpretable value predictions.Source: §3.2.3 · p. 7

3.2.4 Caption generation

• Caption generation conditions the VLM on the query image, product name, predicted attribute-value pairs, and retrieved examples.Retrieved entries guide the caption's structure and style, and the final synthesis integrates the analyzed attributes into a holistic description.Source: §3.2.4 · pp. 7–8

4.1 Dataset

• Evaluation uses 7K FashionGen test samples, with 60K training-split products forming a retrieval knowledge base that is not used to train or validate the VLMs.The dataset contains roughly 300K fashion-product images; product names guide attention to the intended item in images showing multiple products.Source: §4.1 · p. 8

4.2 VLMs used

• The evaluated VLMs include TinyLLaVA and Qwen-2.5VL at 3B, Qwen-2.5VL at 7B, InternVL2 at 8B, and GPT-4o.The study covers small and large open models as well as a closed-source model.Source: §4.2 · p. 8

4.2.1 VLM Paradigms

• RA-CoA is compared with direct zero-shot, few-shot ICL, implicit CoT, explicit CoT, and ICL combined with explicit CoT prompting.The ICL baselines retrieve random training-set samples as exemplars.Source: §4.2.1 · p. 8

4.3 Prompts used for different paradigms

• The prompting templates are kept consistent across all evaluated VLMs, including GPT-4o.The compared settings include zero-shot, ICL, implicit CoT, explicit CoT, and ICL+CoT-e.Source: §4.3 · p. 9
• Zero-shot requests a concise product caption, while implicit CoT asks the model to identify fine-grained attributes before writing the caption but output only the caption.Explicit CoT instead asks for both attributes and caption in JSON.Source: §4.3 · p. 9
• ICL prompts provide retrieved product examples for reference, and ICL+CoT-e combines examples with explicit attribute listing and a final caption.The combined prompt requests both attributes and caption in JSON.Source: §4.3 · p. 9
• All ICL baselines except TinyLLaVA-3B receive two randomly retrieved training samples; TinyLLaVA receives only their product names and captions because it cannot accept multiple images.Source: §4.3 · p. 10

4.4.1 Traditional metrics

• BLEU, ROUGE, and METEOR measure caption performance, with higher scores preferred.Source: §4.4.1 · p. 10

4.4.2 LLM-as-Judge

• Traditional metrics capture fluency and lexical overlap but may miss whether a caption accurately describes relevant visual attributes.The authors note that small fashion-attribute errors can mislead.Source: §4.4.2 · p. 10
• The LLM judge asks whether each oracle attribute-value pair is correctly captured in a caption and averages the Yes responses per sample.LLaMA-3.2-V-11B serves as the judgment engine.Source: §4.4.2 · p. 10

4.6 Implementation Details

• PyTorch and HuggingFace Transformers support most of the implementation.VLMs are run using official repositories or public HuggingFace implementations, prioritizing reproducibility.Source: §4.5 · p. 10
• FAISS retrieves ProductKB neighbors using precomputed CLIP visual embeddings.The embeddings are produced by a CLIP vision encoder.Source: §4.5 · p. 10
• All VLMs are frozen and inference-only, with K=1 for Chain-of-Attributes and Kicl=2 for ICL exemplars by default.Experiments run on one machine with three NVIDIA A6000 GPUs.Source: §4.5 · p. 10

4.6 Results and Discussion

• ICL improves METEOR by 1.5–18 points over zero-shot, but still misses complete attribute coverage; CoT with ICL underperforms ICL alone.CoT prompting alone shows limited or degraded performance, while adding ICL exemplars partially recovers the degradation of explicit CoT with ICL.Source: §4.6 · pp. 10–11
• 67.6 METEOR and 63.5 LLM Judge: GPT-4o with RA-CoA achieves the highest scores, while InternVL2-8B leads open-source models at 38.1 METEOR and 61.1 LLM Judge.RA-CoA has the highest performance across all evaluated models.Source: §4.6 · pp. 10–11, Table 1 · p. 11
• 12.4 and 24.3 METEOR points: RA-CoA improves over zero-shot for TinyLLaVA (3B) and InternVL2 (8B), respectively.For TinyLLaVA, RA-CoA doubles METEOR versus zero-shot (24.6 vs. 12.2) and improves 79% over ICL; gains scale with model size.Source: §4.6 · p. 11, Table 1 · p. 11
• Qualitative examples show RA-CoA captions consistently capturing fine-grained product attributes across diverse fashion categories.The authors describe the resulting attribute coverage as accurate.Source: §4.6 · p. 11

4.7.1 Optimal top-K; 4.7.2 Contribution of ICL exemplars in RA-CoA

• K=1 offers the best attribute-relevance and caption-accuracy balance; without retrieved exemplars, RA-CoA still beats zero-shot and CoT but InternVL2-8B falls from 38.1 to 19.8 METEOR.Higher K can add weakly aligned attributes, noise, and redundancy; retrieved exemplars guide caption structure and style.Source: Table 2 · p. 12, §4.7.1 · p. 12, Table 3 · p. 13

4.7.3 Comparison with Oracle Variants

• With oracle attribute-value pairs, InternVL2 (8B) reaches 97.1 METEOR versus 38.1 for RA-CoA; oracle attribute keys alone yield more modest gains, such as 42.5 versus 38.1.The larger value-pair gains point to attribute-value prediction as a key bottleneck.Source: Table 3 · p. 13, §4.7.3 · pp. 13–14

4.7.4 Effect of product-aware cropping on retrieval and captioning

• Florence–CLIP cropping improves retrieval alignment over global CLIP, while cropped caption-generation input adds only +0.8 METEOR over full-image input for InternVL2-8B.Full-image captioning already performs effectively; cropping is necessary for retrieval but optional at generation.Source: §4.7.4 · pp. 14–15, Table 4 · p. 14, Figure 4 · p. 14

4.7.5 Effect of ProductKB Quality and Scale

• With InternVL2-8B in RA-CoA, 60K ProductKB entries yield 38.1 METEOR; 75% value noise lowers it to 34.3, while dropping 75% of key–value pairs yields 25.5.Gains beyond 20K entries are incremental.Source: §4.7.5 · p. 15

4.7.6 Comparison with prior SOTA and Generalization of RA-CoA beyond FashionGen dataset

• On 500 held-out web images, training-free RA-CoA (InternVL2-8B) beat supervised UniFashion (7B) by 4.0 METEOR points and across all reported metrics.The comparison was designed to avoid evaluating UniFashion on FashionGen, where it was trained.Source: Table 8 · p. 16, §4.7.6 · p. 16

4.7.7 Generalization of RA-CoA beyond e-commerce Fashion domain; 4.7.8 User preference study

• On Furniture, RA-CoA outperforms all VLM baselines; in a 300-image user study, it is preferred in 68.6% of cases.The furniture evaluation uses structured attribute annotations and captions generated from them; the preference study evaluates InternVL2-8B captions.Source: §4.7.7 · pp. 16–17, Table 9 · p. 17, Table 10 · p. 17

4.7.9 Attribute-wise sensitivity analysis; 4.7.10 Error analysis

• Scores exceed 3.5 for prominent attributes but fall below 2.5 for occluded or viewpoint-sensitive ones; remaining errors include missing attributes and hallucinated values.Higher-scoring attributes include color, closure, sleeve type, front print, and hood; lower-scoring ones include hemline, fly type, and pockets.Source: Figure 5 · p. 18, §4.7.9 · p. 18, §4.7.10 · p. 18

4.7.11 Latency analysis

• Average inference takes approximately 3.5 seconds per sample on one A6000 GPU; VLM prompting takes 1.4s and 1.8s, while 60K-item Faiss retrieval takes 0.01s.ROI extraction takes 0.3s and CLIP embedding extraction takes 0.04s.Source: Table 11 · p. 18, §4.7.11 · pp. 18–19

• RA-CoA is a training-free, model-agnostic framework that uses a product knowledge base and frozen VLMs to infer relevant attributes and synthesize captions.Source: §5 · p. 19
• The framework disentangles captioning into chain-of-attribute reasoning followed by generation to enhance attribute-level precision and interpretability.Source: §5 · p. 19
• The authors report reduced hallucination, improved caption faithfulness, and support for scalable deployment in dynamic fashion environments.Source: §5 · p. 19

• RA-CoA depends on ProductKB quality and diversity, while the current knowledge base and evaluations focus on western fashion vocabularies.The authors warn that database biases or gaps may propagate to generated captions and propose validation on culturally diverse sets.Source: §6 · p. 19
• VLMs struggle to infer values for fine-grained product attributes, particularly subtle details such as button designs or small garment text.The authors identify improved semantic grounding through VLM pretraining as a possible direction for future work.Source: §6 · p. 19

• FashionGen contains human models wearing fashion products, raising privacy concerns and potential demographic biases.Source: §7 · p. 19
• The authors use the dataset under its original license and avoid identity analysis as mitigation measures.Source: §7 · p. 19
• Retrieval may propagate biases when ProductKB lacks diversity in represented styles or cultural demographics.The authors also identify possible misuse for misleading product descriptions, while noting safeguards from attribute grounding.Source: §7 · p. 19
• Future work should explore dataset de-identification, demographic fairness measures, and reducing computational requirements while improving attribute inference.Source: §7 · p. 19