MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control

Motivation

• Existing MLLM embedding methods mainly use models as static feature extractors, leaving their generative abstraction, reasoning, and structured-understanding capabilities underused.This motivates the question of how generative reasoning should be formulated for embedding learning.Source: §1 · pp. 1–2

Challenges

• Reasoning introduces structural misalignment because instance-level chains of thought are weakly coupled to pairwise contrastive objectives, encouraging shortcut behavior that mimics reasoning format.Prior embedding tokens attend heavily to original inputs but minimally to CoT tokens, indicating that reasoning can become a deterministic procedural prefix.Source: §1 · p. 2
• Reasoning is not universally beneficial: forced autoregressive reasoning can cause overthinking, add computation and latency, obscure salient semantics, and degrade performance on simple inputs.The supplied example contrasts reasoning benefits for a complex query with irrelevant noise for the simple target “cat”.Source: §1 · p. 2, Figure 2 · p. 2

Method

• MMEmb-R1 treats reasoning paths as latent variables and selects them with a pair-aware mechanism aligned to query-target contrastive supervision.Multiple heterogeneous worker MLLMs generate diverse candidates, while an evaluator measures each rationale’s marginal contribution by comparing matching confidence with and without it.Source: §1 · p. 2
• The framework adaptively invokes reasoning through reinforcement learning when its utility is substantial, balancing embedding effectiveness against computation and latency.The utility reward is the similarity gap between reasoning-enhanced and direct embeddings, optimized with GRPO.Source: §1 · p. 2

Results

• 71.2 overall with Qwen3-VL-4B and 68.3 with Qwen3-VL-2B establish state-of-the-art performance on MMEB-V2, surpassing Embed-RL (66.8) and RzenEmbed-v1 (68.9) with fewer parameters.Adaptive reasoning also increases inference throughput by 3.1× compared to UME-R1 while improving retrieval accuracy.Source: §1 · pp. 2–3

2.1 Multimodal Embedding Models

• Multimodal embedding progressed from CLIP’s dual-tower image-text alignment to broader modalities, improved contrastive objectives, and MLLM-based frameworks such as VLM2Vec, GME, and ColPali.Recent work further expands modality coverage, scales data quality, and develops specialized architectures or training strategies.Source: §2.1 · p. 3
• Recent reasoning-oriented embedding methods include UME-R1, TTE, and Embed-RL, but they largely overlook structural misalignment between instance-level reasoning and pair-level contrastive supervision.This limitation motivates MMEmb-R1’s design.Source: §2.1 · p. 3

2.2 Large Reasoning Models

• Chain-of-thought prompting and reinforcement learning have advanced reasoning in LLMs and MLLMs across visual, generative, mathematical, financial, and medical domains.These developments include GRPO-inspired optimization of reasoning trajectories following DeepSeek-R1.Source: §2.2 · p. 3
• Despite these advances, multimodal embedding and representation learning has yet to benefit substantially from large reasoning models, leaving a gap that MMEmb-R1 aims to bridge.Source: §2.2 · p. 3

3.1 Preliminaries and Framework Overview

• MMEmb-R1 uses a multimodal architecture with direct embeddings zd = E(x) and reasoning-enhanced embeddings zr = E(x ⊕ r), treating reasoning paths as latent variables sampled from P(R).Visual inputs are processed by a ViT and projected into the language-token space; latent reasoning is approximated through diverse generation, pair-aware selection, and adaptive sampling.Source: §3.1 · p. 4

3.2.1 Diverse Prior Simulation via

• Heterogeneous worker MLLMs generate diverse single-sided rationales whose candidates collectively approximate the latent prior P(R) while reducing single-teacher bias and avoiding information leakage.Workers provide complementary concise, exploratory, and knowledge-rich reasoning candidates for each query or target.Source: §3.2.1 · pp. 4–5

3.2.2 Posterior Selection

• Counterfactual posterior selection retains reasoning paths with positive gain Δr = cr − c0, then weights them by normalized gains so pair-aligned rationales contribute more strongly to training.The evaluator compares query–target matching confidence with and without each rationale, using c0 as a sample-wise baseline and retaining candidates with Δr > ϵ.Source: §3.2.2 · p. 5

3.3 Joint Reasoning and Embedding Training

• Joint training combines contrastive losses for reasoning-enhanced and direct embedding paths with next-token prediction over selected chain-of-thought trajectories.The reasoning path samples candidates according to posterior weights, while the direct path preserves embedding quality without reasoning overhead.Source: §3.3 · p. 5

3.4.2 Policy Optimization with GRPO

• GRPO trains a DIRECT-versus-REASON policy using rewards for adaptive benefit, format, and embedding quality while penalizing computational cost and excessively long rationales.The policy uses symmetric query-to-target and target-to-query rewards, encourages DIRECT exploration early, and applies an extra penalty beyond 512 reasoning tokens.Source: §3.4 · p. 5, §3.4.2 · p. 6

3.4.1 Reasoning Utility Estimation

• Reasoning utility is the continuous gap δi between reasoning-enhanced and direct query–target similarities: δi > 0 favors reasoning, whereas δi ≤ 0 indicates direct embedding suffices or is preferable.This intrinsic signal is derived from normalized embeddings after joint training and avoids coarse binary supervision.Source: §3.4.1 · pp. 5–6

4.1.1 Implementation Details

• The experiments train MMEmb-R1 on Qwen-VL using approximately 1.2M joint-training samples and 10K adaptive-reasoning reinforcement-learning samples, then evaluate 78 tasks in MMEB-V2.Evaluation reports Hit@1 for image/video tasks and NDCG@5 for visual document tasks; training uses MMEB-Train and reinforcement learning uses GRPO.Source: §4.1.2 · p. 6, §4.1.1 · p. 6

4.2 Main Results

• 71.2 overall with 4B parameters surpasses the strongest medium-size baseline RzenEmbed-v1-7B, while the 2B model reaches 68.3 and outperforms Embed-RL and RzenEmbed-v1 by 1.5 and 3.9 points.The 2B model achieves 55.6 on Video, outperforming Embed-RL by 3.5 points, with improvements consistent across modality groups.Source: §4.2 · pp. 6–7, Table 1 · p. 7

4.3.1 Further Analysis of Adaptive Reasoning

• MMEmb-R1 Adaptive achieves 10.02 queries per second, providing 1.7× and 3.1× speedups over always-reasoning and UME-R1 while achieving the highest accuracy.Pair-aware selection produces more concise rationales, and the adaptive policy skips unnecessary reasoning for simple queries; the direct variant is faster but has a severe performance drop.Source: §4.3.1 · pp. 7–8, Table 2 · p. 7
• Reasoning accuracy peaks at a 74.3% invocation ratio, rising from 57.2 to 62.7 before declining to 61.9 under near-universal reasoning.The results indicate that the policy prioritizes reasoning-critical instances first, with later reasoning choices yielding diminishing or negative gains.Source: §4.3.1 · p. 8, Figure 4 · p. 8

4.3.2 Ablation Studies

• Removing reasoning causes a 5.8-point drop, while always-reasoning reaches 63.6, 1.4 points below the full model, confirming that selective reasoning drives performance.Uniform sampling without pair-aware scoring loses 2.2 points, removing counterfactual baseline c0 loses 0.9 points, and the oracle reaches 66.2.Source: §4.3.2 · p. 8, Table 3 · p. 8

5 Conclusion

• MMEmb-R1 combines pair-aware selection to address instance–pair supervision misalignment with utility-aware reinforcement learning for selective reasoning invocation.The framework aims to avoid the overhead of indiscriminate reasoning while integrating generative reasoning into multimodal embedding learning.Source: §5 · p. 8
• MMEmb-R1 achieves state-of-the-art performance on MMEB-V2 with substantially reduced inference latency.Source: §5 · p. 8

Limitations

• The pipeline separately performs reasoning generation, pair-aware selection, and two-stage training, preventing their joint optimization.An end-to-end training loop could unify these components and improve overall performance.Source: Limitations · p. 9
• The adaptive policy makes only binary invoke-or-skip decisions, which may limit efficient reasoning allocation.Controlling reasoning depth or granularity, such as brief versus detailed chains, could enable finer-grained computational allocation.Source: Limitations · p. 9
• Reasoning-enhanced embeddings incur additional inference cost, and reducing latency remains an open challenge.Precomputing corpus-side embeddings partially alleviates this cost in retrieval scenarios but does not eliminate the underlying latency issue.Source: Limitations · p. 9

A.1 Qualitative Analysis

• MMEmb-R1 skips reasoning for an unambiguous penguin query but invokes it for temporal cooking inference, outperforming enforced- and non-reasoning baselines in the respective cases.The examples show adaptive reasoning avoids spurious alternatives on simple inputs and captures the logical next action in complex sequences.Source: §A.1 · p. 13, Figure 7 · p. 14
• Pair-aware selection assigns the highest worker weight to the Proprietary worker (w = 0.55) because it best bridges query–target semantics rather than merely producing elaborate reasoning.The Instruct worker extracts numbers without interpreting their frequency bands, while the Thinking worker provides detailed but query-divergent analysis.Source: §A.1 · p. 13, Figure 8 · p. 15

A.2 Scaling Behavior Across Backbones

• Qwen3-VL-2B surpasses Qwen2-VL-7B at less than one-third the parameters, while Qwen3-VL-4B outperforms Qwen2.5-VL-7B at roughly half the size.Within each of the three backbone families, performance improves monotonically with model size, and the framework transfers across architectures without architecture-specific tuning.Source: Figure 5 · p. 13, §A.2 · p. 14

A.3 Reasoning Gain Distribution

• The Proprietary worker has the highest median reasoning gain, whereas the Thinking worker has the widest bimodal spread, including both the highest individual gains and substantial negative values.This complementarity supports combining heterogeneous reasoning sources and weighting accepted samples by pair-aware alignment scores.Source: §A.3 · pp. 14–15
• The selection threshold is ϵ = −0.1, and the Thinking worker produces the largest number of filtered samples because many chains substantially harm query–target alignment.Effectiveness depends on relative weighting among accepted samples, not simply retaining most candidates.Source: Figure 6 · p. 13, §A.3 · pp. 14–15

A.4 Distribution of Reasoning Utility

• Roughly 60% of instances have positive reasoning utility, while 40% have negative utility, showing that always reasoning or never reasoning is suboptimal.The continuous, instance-dependent utility distribution motivates reinforcement learning rather than a hard threshold for learning the reasoning policy.Source: §A.4 · p. 15

MMEB-V1 results

• 74.8 overall on MMEB-V1, MMEmb-R1 (Qwen3-VL-4B) outperforms all baselines, including Embed-RL-4B and UME-R1-7B.The result covers 36 datasets and shows that the framework’s benefits extend beyond the video and document modalities introduced in MMEB-V2.Source: §A.5 · p. 15

MMEB-V2 statistical analysis

• MMEmb-R1 consistently outperforms the compared baselines across the 78 MMEB-V2 tasks.The analysis uses paired per-task scores, with the Wilcoxon signed-rank test as the primary test, a paired t-test as complementary analysis, and task-level bootstrap confidence intervals from 10k resamples.Source: §A.5 · pp. 15–16
• 3.08 points is MMEmb-R1’s mean improvement over Embed-RL-4B on 69 of 78 MMEB-V2 tasks, with a 95% bootstrap confidence interval of [2.13, 4.02].Wilcoxon signed-rank and paired t-tests are highly significant for all comparisons, and all corresponding confidence intervals remain strictly above zero.Source: §A.5 · pp. 15–16

• Increasing worker diversity and capability generally improves downstream performance, but adding three further Kimi/Claude/GPT workers reduces performance by complicating CoT-based optimization.Heterogeneous workers better approximate the latent reasoning space, whereas excessive diversity creates additional training difficulty.Source: §A.6 · p. 16
• 63.2 from the 2B Evaluator-Flash surpasses UME-R1’s 60.1 while reducing evaluator size and improving pipeline scalability.Evaluator-Flash is trained to mimic the original 32B evaluator’s scoring process.Source: §A.6 · p. 16
• Evaluation remains stable across capable evaluator models and prompting strategies, making worker quality more critical because workers generate the training reasoning data.This supports pair-aware selection as a mechanism for filtering and weighting high-quality reasoning supervision.Source: §A.6 · p. 16
• Using Qwen3-VL-32B as the CoT worker still underperforms the full framework, indicating that improvements arise mainly from the proposed methodology rather than stronger-model distillation.The comparison is reported under the evaluator-configuration experiments in Table 7.Source: §A.6 · p. 17, Table 7 · p. 17

A.7 Results Generalized to Other Domains

• The method consistently improves performance with pair-aware selection on both MMStar and out-of-domain ROCOv2, including CUI@5 evaluation for ROCOv2.These results support transfer to generative multimodal and medical multimodal tasks.Source: §A.7 · p. 17, Table 8 · p. 17

A.8 Query-Dependent Embeddings

• Removing question text drops MMEB-V2 performance from 65.0 to 52.8, while random reassignment lowers it further to 43.1.The target items remain unchanged, showing that query embeddings depend on specific question or instruction intent.Source: §A.8 · p. 17, Table 9 · p. 18

B.1 Multi-Worker Reasoning Path Generation

• The framework uses heterogeneous MLLMs to approximate the reasoning latent-space distribution by combining complementary reasoning styles and knowledge coverage.This multi-worker design is intended to provide a richer prior than any single model.Source: §B.1 · p. 18
• InternVL3-14B-Instruct generates structured, concise, factual, retrieval-oriented rationales and serves as a stable baseline in the candidate pool.Its prompt enforces a <reason>...<sum> format.Source: §B.1 · p. 18
• GLM-4.1V-Thinking produces longer, exploratory, higher-variance chains that occasionally yield uniquely high-gain candidates.Unlike the instruct-based worker, it uses native chain-of-thought without a rigid output format.Source: §B.1 · p. 18
• Doubao-Seed-1.6-Vision generates qualitatively richer rationales and achieves the highest median reasoning gain among worker types.Its advantage is attributed to broader world knowledge despite using the same prompt template as instruct-based models.Source: §B.1 · p. 18

B.2 Pair-Aware Evaluator Implementation

• Qwen3-VL-32B-Instruct with vLLM provides evaluator logit scores, whose gains across Instruct, Thinking, and Proprietary workers are softmax-normalized into weighted sampling probabilities.The evaluator uses two inference passes per training pair and stores the resulting weights alongside the rationales for training.Source: §B.2 · p. 18
• For each rationale, the evaluator computes a counterfactual gain Δr by subtracting baseline relevance log-odds from rationale-conditioned relevance log-odds.The evaluator compares first-token [YES] and [NO] log-probability ratios for raw and rationale-augmented query–target inputs; positive Δr indicates improved match confidence.Source: §B.2 · p. 18, Table 10 · p. 19
• 64.8 on MMEB-V2 and Kendall’s τ = 0.82 show that revised evaluator prompts produce comparable performance and highly consistent worker rankings.The revised 2B model was trained with weights recomputed under matched prompts, while the ranking agreement was measured against the original weights.Source: §B.2 · p. 18

B.3 Details of Joint Reasoning and Embedding Training

• Direct-embedding samples are introduced during joint training for low-selection-weight or very short inputs, replacing rationales with <empty> 10% of the time to reduce overreliance on reasoning.The vision encoder remains frozen while the multimodal projector and LLM backbone are updated.Source: §B.3 · pp. 19–20

B.4 Details of Adaptive Reasoning Control

• Adaptive control combines format, embedding-quality, and reasoning-cost rewards, with α = 0.2, c = 1 × 10−3, and early encouragement of the DIRECT action.The embedding reward evaluates positive-target ranking and positive–negative similarity gaps, averaged symmetrically across query-to-target directions.Source: §B.4 · p. 20

B.5 Discussion on the Implementation of Other Baselines

• MMEmb-R1’s training complexity is comparable to existing reasoning-driven baselines, while potentially becoming more efficient at larger data scales by avoiding manual prompts and heuristic filtering.TTE separately trains embedder and reasoner, whereas UME-R1 and EMB-RL use SFT-plus-RL pipelines.Source: §B.5 · pp. 20–21

C.1 Causal Inference and Causal Learning

• Counterfactual comparison of predictions with and without rationale is presented as conceptually related to causal intervention, while evaluator reweighting converts diverse rationale candidates into a more informative posterior.The discussion grounds this view in SCMs, do-operator interventions, and counterfactual reasoning.Source: §C.1 · p. 21

C.2 Group Relative Policy Optimization

• GRPO avoids PPO’s separate critic by sampling grouped candidates, normalizing rewards into relative advantages, and optimizing a clipped objective with KL regularization.Its stated benefits are lower memory use, stable group-normalized gradients, and straightforward integration with standard language-model training.Source: §C.2 · p. 21

C Background and Preliminaries

• Table 12 reports detailed baseline and MMEmb-R1 results across the full MMEB-V2 benchmark, organized by image, video, and VisDoc modalities.The supplied passage identifies the table scope and bolds the best result per row but provides no cell values.Source: Table 12 · p. 22

C.2 Group Relative Policy Optimization

• Table 13 reports MMEB-V1 results across 36 image-embedding tasks, distinguishing in-distribution from out-of-distribution datasets and incorporating some results from Embed-RL.The supplied passage does not provide numerical cell values.Source: Table 13 · p. 23