PaperlayerView exact revision on arXiv

Source-linked AI summary

UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings

Zhibin Lan, Liqiang Niu, Fandong Meng, Jie Zhou, Jinsong Su

arXiv:2511.00405v2cs.LGcs.AI

TL;DR

Existing multimodal embedding models are mainly discriminative, limiting their use of reasoning-driven generation. UME-R1 introduces generative embeddings through reasoning-aware supervised fine-tuning and reinforcement learning, and outperforms conventional discriminative models across MMEB-V2 while supporting complementary embedding modes.

  • Problem

    Existing MLLM-based multimodal embeddings remain discriminative at inference and do not generate intermediate reasoning content.

  • Method

    UME-R1 uses cold-start supervised fine-tuning followed by reinforcement learning to generate reasoning-driven embeddings while retaining discriminative embeddings.

  • Results

    UME-R1 outperforms conventional discriminative embedding models across MMEB-V2’s 78 video, image, and visual-document tasks, with a 2.1-point overall improvement over VLM2Vec-V2.

  • Takeaways & Limitations

    Discriminative and reasoning-driven generative embeddings are complementary, with oracle mode selection improving overall scores by 4.3 points for UME-R1-2B and 3.6 points for UME-R1-7B.

  • Takeaways & Limitations

    Future work is needed for adaptive mode selection, more challenging RL data or strategies, and further inference-time scaling.

Abstract

from arXiv · show

The remarkable success of multimodal large language models (MLLMs) has driven advances in multimodal embeddings, yet existing models remain inherently discriminative, limiting their ability to benefit from reasoning-driven generation paradigm. In this work, we pioneer the exploration of generative embeddings, unifying embedding tasks within a generative paradigm. We propose UME-R1, a universal multimodal embedding framework consisting of a two-stage training strategy: a cold-start supervised fine-tuning equips the model with reasoning capabilities and enables it to generate both discriminative and generative embeddings; a subsequent reinforcement learning enhances reasoning and further optimizes generative embedding quality. This pioneering work reveals four key insights: 1) generative embeddings unlock substantial performance gains over conventional discriminative embeddings by leveraging the powerful generative reasoning capabilities of MLLMs; 2) discriminative and generative embeddings are complementary, whose combined oracle performance far exceeding that of either alone; 3) RL can effectively enhance generative embeddings, establishing a scalable optimization paradigm.; 4) repeated sampling at inference boosts downstream task coverage (pass@k), highlighting the inference-time scalability potential of generative embeddings. Evaluated on the MMEB-V2 benchmark across 78 tasks spanning video, image, and visual documents, UME-R1 significantly outperforms conventional discriminative embedding models and offers a foundation for more interpretable, reasoning-driven generative multimodal embeddings. Our code, models, and datasets will be publicly available at https://github.com/XMUDeepLIT/UME-R1.

1 INTRODUCTION

Existing MLLM-based multimodal embeddings remain discriminative at inference, motivating UME-R1’s reasoning-driven generative alternative. The framework combines supervised reasoning-aware training with reinforcement learning and supports both embedding modes.

  • UME-R1 addresses the limitation that existing MLLM-based embeddings encode inputs directly without generating intermediate reasoning content.
  • The framework trains models to generate reasoning and summaries before producing embeddings, while retaining discriminative embeddings.Cold-start SFT applies autoregressive loss to reasoning and summary tokens and contrastive loss to subsequent embedding tokens.
  • UME-R1 further applies a reward policy combining ranking and similarity gaps to avoid overly easy or difficult reinforcement-learning pairs.The design targets zero policy gradients caused by fixed-threshold rewards under varying pairwise similarity.
  • The framework is evaluated across 78 MMEB-V2 tasks spanning video, image, and visual-document modalities.
  • The paper reports that UME-R1 outperforms conventional discriminative embedding models while retaining potential for oracle mode selection and inference-time scaling.

2 DATASET CONSTRUCTION

The training corpus combines multimodal embedding data from diverse datasets with model-generated chain-of-thought rationales. Filtering produces a large cold-start SFT set and a smaller balanced RL set.

  • The corpus combines samples from 20 MMEB in-distribution datasets with LLaVA-Hound, ViDoRe, and VisRAG, yielding 1.76 million pairs.
  • Figure 1 depicts the data-construction pipeline used to assemble the multimodal training corpus.
  • Filtering removes repetitive, excessively long, or improperly formatted reasoning responses, leaving 1.46 million cold-start SFT pairs.
  • A balanced set of 11,136 pairs is sampled across image, video, and visual-document datasets for reinforcement-learning training.

3 UME-R1

UME-R1 trains a multimodal embedding model to produce both discriminative and reasoning-driven generative embeddings through supervised fine-tuning followed by reinforcement learning. Its architecture generates reasoning and summaries before forming generative representations, while RL rewards format adherence and embedding quality.

  • 3.1 PRELIMINARIES: The InfoNCE objective maximizes similarity for each query-positive target pair against other targets treated as negatives, using normalized final-token representations and a temperature parameter.Queries and targets may be text, images, or interleaved text-image inputs.
  • 3.2 ARCHITECTURE: The model generates reasoning and summaries for each query and target, then uses their concatenation with the original inputs to form generative representations.The final generative embedding comes from the model-generated <gen emb> token, while the <disc emb> token provides a discriminative embedding without additional computation.
  • 3.2 ARCHITECTURE: UME-R1 trains both discriminative and reasoning-driven generative embeddings through a two-stage supervised fine-tuning and RLVR framework.SFT introduces reasoning-aware objectives, while RLVR further refines the model using sampled responses and verifiable rewards.
  • 3.3 MODEL TRAINING: SFT combines discriminative contrastive learning, generative contrastive learning, and next-token prediction over reasoning trajectories and summaries.Reasoning and summaries provide more detailed information than the original input and often enhance embedding performance.
  • 3.3 MODEL TRAINING: RLVR samples response groups with GRPO and optimizes advantages computed from format and embedding rewards.Embedding rewards jointly evaluate positive-versus-negative ranking and similarity gaps, encouraging reasoning trajectories that produce higher-quality generative embeddings.

4 EXPERIMENTS

UME-R1 is evaluated on MMEB-V2 across 78 tasks and three visual modalities, with comparisons covering overall performance, RL components, discriminative embeddings, inference-time sampling, and external reasoning.

  • 4.2 MAIN RESULTS: 2.1 overall improvement over VLM2Vec-V2 and gains of 4.1, 9.0, and 11.1 points over DUME on images, videos, and visual documents, respectively.UME-R1 achieves the best performance in images and videos with the same backbone while using two-thirds of VLM2Vec-V2’s training data.
  • 4.2 MAIN RESULTS: 4.3 and 3.6 overall-score improvements are achieved by the 2B and 7B oracle settings, which select the better discriminative or generative embedding per instance.The oracle substantially outperforms using only reasoning-driven generative embeddings, supporting flexible mode selection.
  • 4.3 ABLATION STUDY: RL substantially improves performance despite using a small training dataset, while combining ranking and similarity gaps provides essential reward guidance.Ranking aligns with downstream tasks, whereas similarity differences help on relatively easy samples where ranking rewards saturate.
  • 4.3 ABLATION STUDY: 3 points improve discriminative embeddings across 78 tasks after SFT, including 7.5 points on visual documents, while RL adds 0.4 points without compromising them.The results indicate that generative objectives provide richer supervision and that discriminative and generative embeddings do not conflict during training.
  • 4.4 DEEP ANALYSIS: Repeated sampling improves embedding representations for both UME-R1-2B and UME-R1-7B across four image and video test sets under pass@k evaluation.Pass@k counts a problem as solved when any of k sampled outputs is correct.
  • 4.4 DEEP ANALYSIS: UME-R1 consistently outperforms DUME+Gen, whereas external reasoning improves K700 and MSVD by 19.7 and 3.9 but reduces CIRR by 12.3 points.DUME+Gen uses an external model to generate reasoning and summaries before DUME encodes them.

5 RELATED WORK

Prior work advances multimodal embeddings through MLLMs, contrastive learning, synthetic data, and methods that preserve generative capabilities, but existing models remain discriminative at inference.

  • 5.1 MULTIMODAL LARGE LANGUAGE MODEL: MLLMs have advanced multimodal understanding, while reinforcement learning with verifiable rewards has mainly targeted visual reasoning rather than embedding tasks.
  • 5.2 UNIVERSAL MULTIMODAL EMBEDDINGS: Universal multimodal embedding models encode varied modalities for tasks including retrieval, evaluation, and retrieval-augmented generation.
  • 5.2 UNIVERSAL MULTIMODAL EMBEDDINGS: VLM2Vec and MM-Embed convert MLLMs into embedding models through contrastive learning, while MegaPairs and GME synthesize larger training datasets.
  • 5.2 UNIVERSAL MULTIMODAL EMBEDDINGS: Although prior methods combine contrastive and autoregressive objectives, existing MLLM-based embedding models still produce only discriminative embeddings at inference.

6 CONCLUSION

UME-R1 pioneers reasoning-driven generative multimodal embeddings by unifying discriminative and generative embeddings, while identifying adaptive selection, stronger RL, and inference-time scaling as future directions.

  • 6 CONCLUSION: UME-R1 unifies discriminative and reasoning-driven generative embeddings through an SFT dataset augmented with reasoning and summaries, followed by reinforcement learning using ranking and similarity gaps.
  • 6 CONCLUSION: Experiments on MMEB-V2 across 78 video, image, and visual-document tasks show significant gains from reasoning-driven generative embeddings over discriminative ones.
  • 6 CONCLUSION: Future work includes adaptive embedding-type selection, more challenging RL data or strategies, and inference-time scaling to improve generative embedding quality.

ETHICS STATEMENT

The study reports compliance with the ICLR Code of Ethics and relies on public, non-sensitive data sources.

  • ETHICS STATEMENT: The work does not involve collecting new human-subject data or personally identifiable information.
  • ETHICS STATEMENT: All datasets used are publicly available and widely adopted in the research community.
  • ETHICS STATEMENT: Constructed experimental data derives from existing models and datasets without adding sensitive, private, or proprietary content.

REPRODUCIBILITY STATEMENT

The authors plan to support reproducibility by releasing the code, datasets, and trained models and by documenting the method in the paper and appendix.

  • REPRODUCIBILITY STATEMENT: Code, datasets, and trained models used in the study will be released to facilitate reproducibility.
  • REPRODUCIBILITY STATEMENT: The code is already included in the supplementary materials submitted with the paper.
  • REPRODUCIBILITY STATEMENT: Dataset construction, model architectures, and training procedures are described in the main text and appendix.

USE OF LARGE LANGUAGE MODELS

The paper reports that LLMs were used only to polish the manuscript, not to generate scientific content or conduct the research. Technical ideas, experiments, analyses, and conclusions are attributed entirely to the authors.

  • LLMs assisted only with writing, clarity, grammar, and readability.
  • LLMs were not used for scientific content, experimental design, analysis, or conclusions.
  • The authors state that all technical ideas, experiments, results, and conclusions are their own work.

B TRAINING AND INFERENCE COST

UME-R1 requires more training and inference resources than discriminative embeddings, while its inference-speed disadvantage narrows as input length increases. The section also presents the benchmark table and supporting implementation materials.

  • Training cost: 2336 H20 GPU-hours for SFT plus 1344 H20 GPU-hours for RL exceed DUME’s 1487 H20 GPU-hours for fine-tuning.
  • Inference cost: Reasoning-driven generative embeddings introduce noticeably higher inference overhead, especially for short inputs, but the speed gap narrows as input length increases.
  • RL data: RL instances are sampled across modalities and balanced across datasets, with image pairs drawn from five specified datasets.
  • Data construction: The cold-start dataset augments query–target pairs with prompts, reasoning, and summaries, with examples shown in Figures 5–7.
  • Benchmark reporting: Table 5 reports detailed baseline and UME-R1 results across the full MMEB-V2 benchmark, with task counts shown by category.

E MMEB-V1 BENCHMARK SCORES

On MMEB-V1, UME-R1 achieves the best overall score among same-size models. Comparative examples show that reasoning and summaries can improve retrieval, while repeated sampling and training dynamics reveal interpretability and optimization behavior.

  • MMEB-V1 results: UME-R1 achieves the best overall MMEB-V1 score among models of the same size.
  • Comparative examples: Reasoning-driven embeddings produce higher-quality retrieval cues by summarizing key content such as “hot dog,” clothing requirements, “Nail art tutorial demonstration,” and “WUFT 5 News.”
  • Repeated sampling: Correctly retrieved examples from repeated sampling tend to contain more effective reasoning and summaries, making embedding quality more interpretable.
  • Training dynamics: For both 2B and 7B models, the lowest reward increases during training, while reward remains non-monotonic because batch difficulty varies across modalities and datasets.
Loading 2511.00405v2…