Source-linked AI summary
Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrieval
Lanyun Zhu, Deyi Ji, Tianrun Chen, Haiyang Wu, Shiqi Wang
TL;DR
Multimodal retrieval needs methods that improve reasoning without the instability and token costs of directly applying R1-style reinforcement learning. Retrv-R1 addresses this with retrieval-specific compression, details inspection, activation training, and curriculum-based reinforcement learning, achieving strong effectiveness, efficiency, and generalization across benchmarks.
Problem
Existing MLLM retrieval methods lack explicit reasoning, while direct reinforcement-learning training can be unstable and computationally costly for retrieval.
Method
Retrv-R1 combines retrieval-specific information compression and details inspection with synthetic-CoT activation training and curriculum-based reinforcement learning.
Results
Retrv-R1 achieves strong retrieval performance, inference efficiency, and generalization across multiple benchmarks and task settings.
Takeaways & Limitations
The results support using task-tailored reasoning and efficiency mechanisms to extend MLLM reasoning to universal multimodal retrieval.
Takeaways & Limitations
The Information Compression Module causes performance drops of -0.9% and -1.1% on two retrieval settings because of information loss from token compression.
Abstract
from arXiv · showhide
The success of DeepSeek-R1 demonstrates the immense potential of using reinforcement learning (RL) to enhance LLMs' reasoning capabilities. This paper introduces Retrv-R1, the first R1-style MLLM specifically designed for multimodal universal retrieval, achieving higher performance by employing step-by-step reasoning to produce more accurate retrieval results. We find that directly applying the methods of DeepSeek-R1 to retrieval tasks is not feasible, mainly due to (1) the high computational cost caused by the large token consumption required for multiple candidates with reasoning processes, and (2) the instability and suboptimal results when directly applying RL to train for retrieval tasks. To address these issues, Retrv-R1 introduces an information compression module with a details inspection mechanism, which enhances computational efficiency by reducing the number of tokens while ensuring that critical information for challenging candidates is preserved. Furthermore, a new training paradigm is proposed, including an activation stage using a retrieval-tailored synthetic CoT dataset for more effective optimization, followed by RL with a novel curriculum reward to improve both performance and efficiency. Incorporating these novel designs, Retrv-R1 achieves SOTA performance, high efficiency, and strong generalization ability, as demonstrated by experiments across multiple benchmarks and tasks.
1 Introduction
Retrv-R1 addresses the limitations of existing MLLM retrieval methods by combining explicit reasoning with task-specific architecture and training designs for universal multimodal retrieval.
- Universal multimodal retrieval seeks one model that handles diverse retrieval tasks and modalities, extending earlier fixed-format retrieval settings.
- Existing MLLM retrieval methods either rely on imprecise embedding similarity or generate results directly without explicit reasoning, limiting robustness on complex cases.
- SFT-based retrieval training can suffer from poor generalization and frequent hallucinations, motivating stronger reasoning and more effective training mechanisms.
- Naive GRPO training on retrieval datasets is difficult and unstable, often producing incorrect reasoning and suboptimal retrieval performance.
- Retrv-R1 is proposed as the first R1-style reasoning MLLM framework for universal multimodal retrieval.
- Retrv-R1 combines token-reduction architecture, details inspection, and curriculum-based rewards, with experiments spanning universal and out-of-domain benchmarks.
2 Related Work
Prior multimodal retrieval uses vision-language models and MLLMs, while Retrv-R1 adds retrieval-specific compression, details inspection, and reinforcement-learning-based reasoning.
- Multimodal Retrieval: Earlier multimodal retrieval spans text-image, composed-image, document, and instruction-based image retrieval, with CLIP-based methods achieving strong results.
- Multimodal Retrieval: Retrv-R1 is presented as the first R1-style MLLM for multimodal retrieval, using reinforcement-learned reasoning to improve retrieval effectiveness.
- MLLM: Compared with prior MLLM approaches, Retrv-R1 tailors token compression to universal retrieval and preserves retrieval-relevant information through task-specific design.
- MLLM: Its details inspection mechanism can incorporate full uncompressed candidate tokens when challenging candidates require closer analysis.
- LLM and MLLM Reasoning: Unlike fixed-format CoT prompting and general reasoning methods, Retrv-R1 uses synthesized retrieval CoT activation training and a curriculum-based token-efficiency reward.
3 Method
Retrv-R1 builds universal multimodal retrieval around coarse-to-fine candidate selection, token compression with selective inspection, and staged reasoning-oriented training. Its design addresses the instability and token cost of naive RL retrieval by combining ICM pretraining, synthetic-CoT SFT, and curriculum-based RL.
- Overall Framework: Retrv-R1 first selects top-K candidates by embedding similarity, then uses a second MLLM to identify the best match among them.The framework separates efficient coarse retrieval from fine-grained reasoning-based selection.
- Baseline Method and Shortcomings: Naive GRPO retrieval is unstable, often produces incorrect reasoning, and incurs high computational cost from multiple candidates and fine-grained CoT.These shortcomings motivate architectural compression and a staged training strategy.
- Information Compression Module: The Information Compression Module reduces each candidate to content and relationship tokens, preserving retrieval-relevant information while leaving context for reasoning.Content tokens summarize each candidate, while relationship tokens encode query-candidate similarities and differences.
- Self-Alignment for Pretraining ICM: ICM self-alignment pretraining freezes the language model and trains compressed representations to preserve content and relationship descriptions before retrieval fine-tuning.The alignment uses cross-entropy objectives with separate content and relationship instruction templates.
- Information Compression Module: Details inspection lets the model retrieve a challenging candidate’s full token sequence during CoT generation when compressed tokens are insufficient.Special inspection tokens trigger appending the selected candidate’s uncompressed sequence as supplementary information.
- Model Training with Activation and Enhancement: Training proceeds through synthetic-CoT SFT followed by RL, while a reward combines retrieval correctness with inspection frequency and gradually increases the efficiency weight.Synthetic reasoning includes negative-sample filtering, positive-sample verification, challenging-candidate inspection, and final result generation; λ_i = i/N_iter strengthens the efficiency constraint over training.
4 Experiments
Retrv-R1 achieves strong retrieval performance, efficiency, and generalization across multimodal benchmarks, while ablations support the contribution of its major components and curriculum reward design.
- Comparison of Effectiveness: Retrv-R1 consistently achieves state-of-the-art performance across 16 M-BEIR scenarios, with the 3B model outperforming larger baselines in most cases.It also performs especially strongly on challenging FIQ retrieval tasks.
- Comparison of Efficiency: Retrv-R1 reduces inference-time and GPU-memory consumption relative to comparison methods by combining information compression with an efficiency-constrained reward.Larger candidate sets improve retrieval but increase token use, computation, and memory pressure.
- Generalization: Retrv-R1 generalizes to unseen datasets, held-out retrieval tasks, and multimodal recommendation, achieving strong performance without fine-tuning and SOTA results after task-specific fine-tuning.These experiments cover broader data types and tasks beyond the training setting.
- Ablation Study: Removing any major component significantly degrades performance or efficiency, while the details inspection mechanism improves accuracy with only minor computational overhead.The ablated components include ICM, compressed tokens, self-alignment, DIM, SFT, and RL.
- Ablation Study: Replacing the curriculum efficiency constraint with a fixed λ causes a notable retrieval-accuracy drop, supporting adaptive reward weighting during training.The curriculum design balances accuracy and token efficiency as training progresses.
- RL Analysis: During RL, detailed inspections initially increase and later decline, while generated chain-of-thoughts become progressively more flexible than the synthetic four-step format.These observations suggest training changes both the model’s accuracy-oriented inspection behavior and its reasoning structure.
5 Conclusion
Retrv-R1 is presented as the first R1-style MLLM framework for multimodal universal retrieval, with task-tailored architectural and training designs. Extensive experiments report improvements in effectiveness, inference efficiency, and task generalization.
- Retrv-R1 is the first R1-style MLLM framework designed for multimodal universal retrieval tasks.
- Task-tailored designs span both the model architecture and training strategies.
- Extensive experiments across multiple benchmarks demonstrate improvements in model effectiveness, inference efficiency, and task generalization.
A.1 Details of Synthesizing Retrieval CoT Dataset
The synthetic retrieval CoT dataset activates MLLM reasoning before RL by decomposing retrieval into speculation, negative-sample filtering, challenging-candidate identification, and refined verification. These outputs are integrated into a complete step-by-step reasoning template.
- Step 1: Retrieval result speculation: Qwen2.5-VL-72B first generates a detailed description of what an ideal retrieval result should look like.This speculative description is denoted drrs.
- Step 2: Quick verification for negative samples: The synthesis process identifies candidates that can be directly classified as negative samples without extensive step-by-step reasoning.The model outputs their indexes as {ns1, ns2, ..., nsNns}.
- Step 3: Identification for challenging candidates: For remaining candidates, Qwen2.5-VL-72B identifies cases where compressed tokens are insufficient and full features are required.These challenging-candidate indexes are denoted {cc1, cc2, ..., ccNcc}.
- Step 4: Refined verification for positive samples: Candidates not classified as negative samples receive fine-grained reasoning that analyzes query relationships and compares candidates to identify the positive sample.The generated refined verification process is denoted rvps.
- Step 5: Integrating into a Complete CoT: Information from the preceding steps is combined into a complete chain-of-thought template for retrieval decisions.The template includes the ideal-result description, negative samples, and candidates requiring full token sequences.
A.2 Details of Applying Retrv-R1 to Multimodal Recommendation
Retrv-R1 is evaluated on an out-of-domain multimodal recommendation task using simple extensions without changing its model structure. Its information compression module generates content tokens from behavioral items, supporting strong reported generalization.
- Retrv-R1 is applied to an out-of-domain multimodal recommendation task using simple extensions without modifying the model structure.
- The information compression module generates a content token from each item in the human behavior data.
- The experiment reports outstanding performance, demonstrating strong generalization ability.
B More Experimental Results
Additional BEIR experiments test Retrv-R1 on text-only retrieval across eight subsets by reranking the top 100 passages. Retrv-R1 achieves the best average NDCG@10 performance, supporting generalization beyond multimodal retrieval.
- Retrv-R1 achieves the best performance across the evaluated BEIR text-only retrieval subsets.
- The evaluation reranks the top-100 retrieved passages across eight BEIR subsets: COVID, NFCorpus, Touché, DBPedia, SciFact, Signal, News, and Robust.
- The comparison uses average NDCG@10 scores and further demonstrates Retrv-R1’s generalization ability and versatility across multimodal and text-only retrieval tasks.
C Qualitative Examples
Retrv-R1 is illustrated through synthesized retrieval CoT examples and qualitative retrieval outputs, while the supplied tables indicate broader benchmark and ablation evaluations.
- Qualitative examples show synthesized retrieval CoT data used for SFT and retrieval results generated by Retrv-R1.The retrieval-result examples include reasoning processes and are intended to illustrate fine-grained reasoning.
- Table 9 reports a comparison on BEIR.
- Table 10 reports average ablation results across all tasks in M-BEIR.
D Discussion of Limitations
Retrv-R1’s main limitation is a small accuracy decrease from token compression, although the compression module substantially reduces inference time and details inspection helps mitigate information loss.
- Token compression causes accuracy drops of 0.9% for (qi, qt) →ci and 1.1% for (qi, qt) →(ci, ct).The paper attributes these drops to information loss caused by compression.
- Information compression reduces inference time by over 7×, making the accuracy trade-off acceptable to the authors.
- The details inspection mechanism identifies challenging samples and uses their full tokens when necessary to alleviate compression-related information loss.