MulVec: Fine-Grained Role-Aware Matching for Training-Free Zero-Shot Composed Image Retrieval

• Zero-shot composed image retrieval searches a gallery from a reference image and text edit without task-specific triplet training.Source: §I · p. 1
• Existing methods can entangle requested constraints because a fused target description and global representation dilute decisive local evidence.Source: §I · p. 1
• MULVEC maps each query to Global, Desired, Preserve, and Forbidden roles with distinct matching purposes.Source: §I · p. 1
• Frozen encoders produce target and role-specific probe vectors, while candidates use global and local visual vectors ranked in one fixed weighted retrieval pass.Source: §I · p. 1
• 23.0%: MULVEC improves CIRCO mAP@5 over the strongest compared method and achieves the best results on CIRR and FashionIQ.Source: §I · p. 2

• Supervised CIR methods learn composition from annotated reference–text–target triplets using shared embeddings and increasingly fine-grained interactions.Source: A. Learned CIR and Zero-Shot Transfer · p. 2
• Zero-shot CIR methods may still learn task-specific mappings, projections, fusion modules, or composition components despite removing annotated target-domain triplets.Source: A. Learned CIR and Zero-Shot Transfer · p. 2
• Training-free methods construct queries and retrieval scores at inference using rewritten captions, local-concept reranking, diverse edited captions, or direct reasoning.Source: B. Training-Free Zero-Shot CIR · p. 2
• Other training-free approaches combine textual and visual transformations, dual descriptions, proxy images, prompt directions, or separate reference and text similarities.Source: B. Training-Free Zero-Shot CIR · p. 2
• Recent methods structure retrieval with global and object-scale descriptions, semantic transitions, prescriptive and proscriptive constraints, or reference-aware bias correction.Source: B. Training-Free Zero-Shot CIR · p. 2

• Local or set-valued features expose small visual details through region–word interaction, reranking, object-scale scoring, and set transportation.Source: C. Role-Aware Matching with… · p. 2
• SoFT separates must-have and must-avoid constraints with global filters, while SDR-CIR uses reference-aware bias around one composed target description.Source: C. Role-Aware Matching with… · p. 2
• MULVEC separately compiles Global, Desired, Preserve, and Forbidden so each role reads the same candidate evidence bank differently.Source: C. Role-Aware Matching with… · pp. 2–3
• Desired rewards local support, Preserve checks local continuity against the reference, and Forbidden penalizes local support.Source: C. Role-Aware Matching with… · p. 3, Figure 2 · p. 3
• The system assigns each candidate a score from the reference and modification, then returns the gallery in descending score order without inference-time relevance labels.Source: A. Problem Formulation · p. 3

• MULVEC maps Desired, Preserve, and Forbidden content to different scoring behaviors: reward appearance, check reference continuity, and penalize states that should disappear.Source: B. Method Overview · p. 3, C. Role Compilation · p. 4
• A frozen compiler maps the reference and edit into Global, Desired, Preserve, and Forbidden fields without accessing targets, candidates, or relevance labels.Source: B. Method Overview · p. 3, C. Role Compilation · pp. 3–4
• A frozen vision–language encoder represents Global with one target vector and the other roles with role-specific probe-vector sets.Source: C. Role Compilation · p. 4, D. Role-Aware Evidence Matching · p. 4
• Each candidate is represented by one global visual vector and a bank of local visual vectors, while the local operators share a smooth late-interaction primitive.Source: D. Role-Aware Evidence Matching · p. 4
• Role-aware dispatch reads shared candidate evidence differently: Global uses global compatibility, Desired rewards required local content, Preserve uses reference-dependent continuity, and Forbidden measures removable source states.Source: D. Role-Aware Evidence Matching · p. 4
• A fixed weighted sum ranks the gallery once, with inactive Desired and Forbidden roles omitted and active weights renormalized.Source: D. Role-Aware Evidence Matching · p. 4

• MULVEC is evaluated on CIRCO, CIRR, and FashionIQ using their respective mAP and recall metrics.Source: A. Experimental Setup · p. 4
• The experiments first compare the complete framework, then use matched controls and targeted interventions to distinguish explicit role handling from simply adding candidate vectors.Source: A. Experimental Setup · p. 4
• All results use frozen OpenCLIP towers at ViT-B/32, ViT-L/14, or ViT-G/14 scale, with cached global and local image vectors.Source: A. Experimental Setup · p. 4
• The comparison is restricted to training-free methods that directly score the gallery in one pass; candidate-conditioned calls, iterative retrieval, verification, and rescue loops are excluded.Source: A. Experimental Setup · pp. 4–5
• Compared methods span caption rewriting, reflective reasoning, composed-caption generation, prompt-direction transfer, dual signals, proxy images, and structured retrieval evidence.Source: A. Experimental Setup · p. 5

• 23.0%: MULVEC improves CIRCO mAP@5 over the best available comparator with ViT-B/32, alongside 22.4% for ViT-L/14 and 19.7% for ViT-G/14.Source: B. Overall Retrieval Effectiveness · p. 5
• MULVEC leads all three CIRR full-gallery recalls and all three subset recalls at every evaluated backbone scale.Source: B. Overall Retrieval Effectiveness · p. 5
• MULVEC achieves the highest category-specific and averaged FashionIQ R@10 and R@50 at all three backbone scales, leading all 24 reported metrics.Source: B. Overall Retrieval Effectiveness · p. 5, Table II · p. 6
• The results are consistent across open-domain CIRCO and CIRR and category-specific FashionIQ, suggesting the benefit is not tied to one query style or visual domain.Source: B. Overall Retrieval Effectiveness · pp. 5–6
• The same role-aware scorer remains useful from ViT-B to ViT-G with frozen towers, while case analyses show global-description optimization can miss local constraints.Source: B. Overall Retrieval Effectiveness · p. 6

Matched control

• Matched controls keep compiler outputs and frozen weights fixed while comparing a shared read against role-specific dispatch.The shared read concatenates nonempty Desired, Preserve, and Forbidden probes; role-aware dispatch applies positive, reference-aware, and negative operators.Source: 1) Role-Aware Matching in the Global… · p. 6
• 7.08 and 5.58 points are the role-aware gains on CIRCO for Global only and Global + local evidence, respectively.The corresponding gains are 5.26 and 5.50 on CIRR, and 16.46 and 12.36 on FashionIQ.Source: 1) Role-Aware Matching in the Global… · p. 6

Cross-dataset evidence

• Role-aware dispatch improves the reported metric across both candidate-evidence settings on CIRCO, CIRR, and FashionIQ.This result is reported for both single-global and global-plus-local candidate representations.Source: 1) Role-Aware Matching in the Global… · p. 6

Qualitative evidence

• The qualitative examples involve multiple constraints, including age, added objects, style, and role binding.MULVEC retrieves labeled targets at rank 1 in the reported examples.Source: Figure 3 · p. 7

Evidence granularity

• Local visual vectors improve both shared and role-aware reads, but their gains overlap rather than add fully.The strongest configuration combines local vectors with role-aware dispatch, showing that roles determine how evidence affects ranking while vectors increase its resolution.Source: 1) Role-Aware Matching in the Global… · p. 6

Role components

• Global gives the largest contribution on all three datasets when removed from the G/D/P/F combination.Removing it lowers CIRCO mAP@5, CIRR Recall@1, and FashionIQ macro Recall@10 by 15.15, 13.64, and 14.34 points.Source: 2) Role Components and Role-Specific… · p. 7
• Desired is consistently useful, Preserve matters most on FashionIQ, and Forbidden has metric-dependent effects.Forbidden removal lowers CIRR full-gallery Recall@1 by 1.20 points and CIRR subset Recall@1 from 71.06 to 69.29.Source: 2) Role Components and Role-Specific… · p. 7

Role-specific operators

• Each isolated intervention—role reassignment, removing reference continuity, or rewarding Forbidden—lowers all three reported metrics.These interventions preserve other components sufficiently to distinguish role assignment and operator effects.Source: 2) Role Components and Role-Specific… · p. 7
• Role reassignment shows that extra probes are insufficient unless their meanings control the matching operators.Preserved content requires a source-relative test, while removed content acts as counterevidence.Source: 2) Role Components and Role-Specific… · p. 8

Design boundary

• CIRCO 60 improves from best-positive rank 5 to 2 with Global only evidence and from 4 to 1 with Global + local evidence.For CIRCO 22, no configuration retrieves a labeled positive within the Top-50.Source: 2) Role Components and Role-Specific… · p. 8, Figure 4 · p. 8

Temperature sweep

• τ = 0.02 gives the highest score on CIRCO, CIRR, and FashionIQ under the tested temperature grid.Only τ changes; compiler outputs, retrieval weights, and other settings remain frozen.Source: 3) Softmax Temperature Sensitivity · p. 8
• Adjacent temperatures 0.01 and 0.04 are lower by 0.70 and 0.52 points on CIRCO, 0.96 and 0.24 on CIRR, and 0.62 and 0.28 on FashionIQ.Small τ approaches hard local selection, whereas large τ spreads weight across weaker matches.Source: 3) Softmax Temperature Sensitivity · p. 8

Dataset sensitivity

• FashionIQ varies by only 0.65 points across the full grid, compared with 1.53 on CIRCO and 1.68 on CIRR.The two open-domain settings are more sensitive to local-evidence concentration in this experiment.Source: 3) Softmax Temperature Sensitivity · p. 8

Robustness

• The robustness conclusion does not rely on one isolated weight setting.The sweep scales one frozen coefficient at a time across five values with query-specific active-role normalization.Source: 4) Role-Weight Sensitivity · pp. 8–9

Weight sweep

• Across 16 non-anchor settings per dataset, the metric drops by at most 1.56 points on CIRCO, 0.45 on CIRR, and 0.55 on FashionIQ.The best sampled neighbor differs from the frozen point by −0.07, +0.19, and +0.02 points, respectively.Source: 4) Role-Weight Sensitivity · pp. 8–9

Dataset sensitivity

• Reducing Global by 25% causes the largest CIRCO drop, while increasing Preserve by 25% causes the largest FashionIQ drop.This identifies different local sensitivities despite the shared frozen weighting scheme.Source: 4) Role-Weight Sensitivity · p. 9
• The response is locally stable but not uniform across datasets and role coefficients.CIRCO is most sensitive to weakening the target description vector, whereas FashionIQ is most sensitive to overemphasizing preservation.Source: 4) Role-Weight Sensitivity · p. 9