UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning

Sule Bai; Mingxing Li; Yong Liu; Jing Tang; Haoji Zhang; Lei Sun; Xiangxiang Chu; Yansong Tang

Source-linked AI summary

UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning

Sule Bai, Mingxing Li, Yong Liu, Jing Tang, Haoji Zhang, Lei Sun, Xiangxiang Chu, Yansong Tang

arXiv:2505.14231v1May 20, 2025cs.CV

TL;DR

Traditional visual grounding is limited in handling implicit, complex instructions across multiple images and diverse multimodal contexts. UniVG-R1 addresses this gap with CoT cold-start supervision, GRPO reinforcement learning, and difficulty-aware weighting, achieving state-of-the-art MIG-Bench performance and broad zero-shot gains across four reasoning-guided benchmarks.

Problem
Existing visual grounding methods mainly address single-image scenarios with simple references, leaving complex implicit instructions across multiple images insufficiently covered.
Method
UniVG-R1 uses a two-stage framework combining CoT-supervised cold-start training with GRPO reinforcement learning and difficulty-aware weighting.
Results
9.1% improvement over the previous leading model on MIG-Bench and 23.4% average zero-shot improvement across four image and video reasoning-guided grounding benchmarks.
Takeaways & Limitations
UniVG-R1 provides a universal grounding approach that performs across complex multimodal contexts, single-image and multi-image scenarios, and image and video benchmarks.

Abstract

from arXiv · show

Traditional visual grounding methods primarily focus on single-image scenarios with simple textual references. However, extending these methods to real-world scenarios that involve implicit and complex instructions, particularly in conjunction with multiple images, poses significant challenges, which is mainly due to the lack of advanced reasoning ability across diverse multi-modal contexts. In this work, we aim to address the more practical universal grounding task, and propose UniVG-R1, a reasoning guided multimodal large language model (MLLM) for universal visual grounding, which enhances reasoning capabilities through reinforcement learning (RL) combined with cold-start data. Specifically, we first construct a high-quality Chain-of-Thought (CoT) grounding dataset, annotated with detailed reasoning chains, to guide the model towards correct reasoning paths via supervised fine-tuning. Subsequently, we perform rule-based reinforcement learning to encourage the model to identify correct reasoning chains, thereby incentivizing its reasoning capabilities. In addition, we identify a difficulty bias arising from the prevalence of easy samples as RL training progresses, and we propose a difficulty-aware weight adjustment strategy to further strengthen the performance. Experimental results demonstrate the effectiveness of UniVG-R1, which achieves state-of-the-art performance on MIG-Bench with a 9.1% improvement over the previous method. Furthermore, our model exhibits strong generalizability, achieving an average improvement of 23.4% in zero-shot performance across four image and video reasoning grounding benchmarks. The project page can be accessed at https://amap-ml.github.io/UniVG-R1-page/.

1 Introduction

Universal visual grounding extends beyond simple single-image references to complex, implicit instructions across multimodal contexts. UniVG-R1 combines cold-start reasoning data, GRPO reinforcement learning, and difficulty-aware weighting, achieving strong benchmark results and zero-shot generalization.

Motivation: Universal visual grounding must handle implicit instructions and complex visual contexts across multiple images, unlike conventional category-based or simple-reference grounding.The motivating example requires identifying furniture in one image based on objects in another.
Method: UniVG-R1 is a reasoning-guided multimodal model that combines GRPO training with cold-start initialization for universal grounding.The method targets reasoning over instructions and visual inputs.
Method: Difficulty bias in GRPO arises because reward normalization gives easy and hard samples similarly scaled updates, reducing training focus on harder examples as easy samples become prevalent.The proposed difficulty-aware strategy dynamically scales gradients by sample difficulty and yields additional improvements across tested difficulty metrics.
Results: 9.1% improvement over the previous leading model on MIG-Bench accompanies state-of-the-art results across all ten subtasks.The cited comparison is against Migician.
Results: 23.4% average zero-shot improvement across four image and video reasoning-guided grounding benchmarks demonstrates broad generalizability.The benchmark-specific gains are +27.8% on LISA-Grounding, +15.9% on LLMSeg-Grounding, +20.3% on ReVOS-Grounding, and +25.3% on ReasonVOS.
Method: A high-quality CoT grounding dataset with detailed reasoning chains provides the cold-start supervision for learning reasoning-based grounding.The dataset is introduced as a contribution to facilitate advanced reasoning-based grounding.

2 Related Work

Visual grounding research has progressed from standard referring-expression localization toward more challenging instruction and multimodal settings. Related work includes MLLM-based reasoning efforts, multi-image benchmarks, and the use of reinforcement learning for complex reasoning.

2.1 Visual Grounding: Visual grounding localizes a visual element from a linguistic query, with RefCOCO/+/g serving as a widely used benchmark.Early approaches use vision-language pretrained models such as CLIP for fine-grained understanding.
2.1 Visual Grounding: Newer grounding datasets require complex instruction comprehension, including queries that ask models to identify food rich in vitamins.These tasks extend beyond simple referring expressions such as “the left apple.”
2.1 Visual Grounding: Migician introduced free-form multi-image grounding across ten subtasks requiring multi-context understanding and grounding.Examples include static difference, common object, and correspondence.
2.2 Reasoning-Chain Guided Reinforcement Learning: Reinforcement learning has been used to enhance complex reasoning in large language models, including RLHF and GRPO-based approaches.DeepSeek-R1 is cited as employing GRPO, while OpenAI-o1 is cited for RLHF during fine-tuning.
2.2 Reasoning-Chain Guided Reinforcement Learning: UniVG-R1’s two-stage training is depicted as CoT-SFT followed by GRPO with difficulty-aware weighting, with multiple generated responses receiving distinct rewards.The figure separates data construction, difficulty-aware weighting, and the GRPO response-reward process.

3 Method

UniVG-R1 addresses universal visual grounding by combining cold-start CoT supervised fine-tuning with rule-based GRPO reinforcement learning for complex instructions and multi-image contexts. It also targets GRPO’s difficulty bias by weighting harder samples more heavily during training.

3.1 Overview: UniVG-R1 localizes objects from implicit, complex instructions using a target image and additional images.The model maps textual instruction T, target image I, and additional images V to a bounding box B.
Training Framework: The two-stage training process first uses CoT annotations for supervised fine-tuning, then applies rule-based GRPO to select correct reasoning chains.The first stage teaches structured reasoning trajectories; the second further enhances reasoning through verifiable rewards.
Cold-Start Data Construction: Pure reinforcement learning underperformed supervised fine-tuning with the same data amount, motivating cold-start data to guide multi-image grounding exploration.The authors attribute this result to limited grounding ability that makes reasoning-space exploration difficult without prior guidance.
Cold-Start Data Construction: 76k CoT samples were generated and selected with Qwen-VL-MAX, with 99.87% acceptance after manual verification of a random 10% sample.Each item includes a question, bounding-box coordinates, and reasoning formatted with <think> and <answer> tags.
Reinforcement Learning: GRPO combines IoU-based accuracy rewards with binary format rewards and updates the policy using normalized relative advantages among sampled responses.The accuracy reward encourages predicted boxes to match ground truth, while the format reward enforces the required reasoning-and-answer structure.
Difficulty-Aware Weight Adjustment Strategy: During training, easy samples increase while medium and hard samples decline, causing later GRPO updates to focus predominantly on easier instances.Easy, medium, and hard samples are defined by mIoU thresholds above 0.7, between 0.3 and 0.7, and below 0.3, respectively.
Difficulty-Aware Weight Adjustment Strategy: The difficulty-aware strategy dynamically scales sample weights using a coefficient negatively correlated with mIoU, assigning greater weight to difficult samples.The authors report that multiple difficulty metrics consistently provide additional performance improvements.

4 Experiments

UniVG-R1 is evaluated on multi-image, zero-shot, conventional grounding, and ablation settings, with results supporting its reasoning-guided training design. It achieves leading performance across the reported benchmarks and benefits from CoT-SFT, GRPO, and difficulty-aware weighting.

Main Results: UniVG-R1 achieves new state-of-the-art results across all 10 MIG-Bench subtasks, surpassing Migician by 9.1%.It uses 100k curated CoT samples versus Migician’s 1.2 million samples.
Main Results: UniVG-R1 averages 58.61% across zero-shot image and video reasoning-guided grounding benchmarks.It performs strongly in both single-image and multi-image scenarios.
Main Results: UniVG-R1 achieves the best average RefCOCO/+/g performance of 88.20%, including the strongest result on the more complex RefCOCOg split.The result supports its ability to comprehend intricate reference instructions.
Ablation Study: Pure RL underperforms CoT-SFT by 7.07% in average score on 21k samples, motivating the paper’s two-stage training approach.The authors attribute this gap to difficulty exploring reasoning in multi-image grounding contexts.
Ablation Study: CoT-SFT reaches 69.00% average performance versus 67.30% for coordinate-only SFT, with gains of 11.34% in “Reason” and 10.27% in “Co-Re”.Increasing CoT-SFT data from 21k to 90k samples improves average performance by 4.5%.
Ablation Study: GRPO improves average performance by 1.88% over continued CoT-SFT, while GRPO-Difficulty adds approximately 2.0% over standard GRPO.The difficulty-aware strategy dynamically emphasizes harder samples during reinforcement learning.
Ablation Study: Among tested difficulty functions, exp(1−mIoU) produces the highest average performance of 72.62%.The paper adopts this function as its default setting.
Ablation Study: GRPO substantially boosts the smaller Qwen2-VL-2B model, and difficulty-aware weighting further amplifies that gain, although 2B remains below 7B.The authors associate the larger improvement with the smaller model’s weaker post-SFT reasoning development.

5 Visualization

Qualitative comparisons show UniVG-R1 handling cross-image context together with implicit and complex instructions. It provides more accurate grounding results with explanations than the compared methods.

Visualization: UniVG-R1 understands multi-context information across images and handles implicit and complex instructions in qualitative examples.Examples include matching objects by functionality and identifying what can sweep away rain.
Visualization: Compared with Qwen2-VL-7B and Migician, UniVG-R1 provides more accurate results with explanations.The comparison covers MIG-Bench and LISA-Grounding.

6 Conclusion

UniVG-R1 is a reasoning-guided MLLM for universal visual grounding, trained with cold-start CoT supervision, GRPO-based reinforcement learning, and difficulty-aware weighting. It achieves state-of-the-art MIG-Bench performance and substantial zero-shot gains across reasoning-guided grounding benchmarks.

Training framework: UniVG-R1 uses cold-start supervised fine-tuning on a high-quality CoT dataset to learn structured reasoning trajectories.The framework then applies GRPO-based reinforcement learning to further enhance reasoning capabilities.
Training framework: A difficulty-aware weight adjustment strategy dynamically prioritizes harder samples during GRPO training.The strategy addresses difficulty bias by scaling sample gradients according to difficulty.
Experimental results: 9.1% improvement on MIG-Bench establishes UniVG-R1 as state of the art on the multi-image grounding benchmark.
Experimental results: UniVG-R1 achieves substantial zero-shot performance gains across multiple reasoning-guided grounding benchmarks.These results support its generalization across diverse multimodal grounding settings.