PaperlayerView exact revision on arXiv

Source-linked AI summary

MiCo: Multi-image Contrast for Reinforcement Visual Reasoning

Xi Chen, Mingkang Zhu, Shaoteng Liu, Xiaoyang Wu, Xiaogang Xu, Yu Liu, Xiang Bai, Hengshuang Zhao

arXiv:2506.22434v1cs.CV

TL;DR

VLMs can interpret single images but struggle to link visual cues across images, while rule-based reinforcement learning for complex comparisons is hard to supervise with curated QA pairs. MiCo trains on self-supervised image triplets and uses reinforcement learning to learn comparison reasoning. The paper reports generalization beyond comparison tasks and strong performance across multi-image and general vision benchmarks.

  • Problem

    VLMs struggle to link visual cues across images, and rule-based reinforcement learning for fine-grained cross-image reasoning is difficult to supervise with curated question-answer pairs.

    Source: §1 · p. 1

  • Method

    MiCo uses image constraints to form triplets of two augmented views of one image and a similar but distinct image, then trains comparison reasoning with rule-based reinforcement learning.

    Source: §1 · p. 2

  • Results

    Trained solely on image comparison, MiCo generalizes to broader multi-image scenarios and reports strong performance on multi-image benchmarks and improvements on general vision tasks.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Visual comparison training supports cross-image cue linking and fine-grained attention, with reported benefits extending to general vision tasks.

    Source: §1 · p. 2

    MiCo is less effective on specialized tasks such as face verification, visual math, and spatial understanding that require structured priors or domain-specific knowledge.

    Source: §5 · p. 9

Abstract

from arXiv · show

This work explores enabling Chain-of-Thought (CoT) reasoning to link visual cues across multiple images. A straightforward solution is to adapt rule-based reinforcement learning for Vision-Language Models (VLMs). However, such methods typically rely on manually curated question-answer pairs, which can be particularly challenging when dealing with fine grained visual details and complex logic across images. Inspired by self-supervised visual representation learning, we observe that images contain inherent constraints that can serve as supervision. Based on this insight, we construct image triplets comprising two augmented views of the same image and a third, similar but distinct image. During training, the model is prompted to generate a reasoning process to compare these images (i.e., determine same or different). Then we optimize the model with rule-based reinforcement learning. Due to the high visual similarity and the presence of augmentations, the model must attend to subtle visual changes and perform logical reasoning to succeed. Experiments show that, although trained solely on visual comparison tasks, the learned reasoning ability generalizes effectively to a wide range of questions. Without relying on any human-annotated question-answer pairs, our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks.

1 Introduction

VLMs struggle to link visual cues across images, while rule-based reinforcement learning for reasoning typically depends on QA pairs that are difficult to curate for fine-grained, complex comparisons. MiCo uses image-derived constraints for supervision and learns visual comparison reasoning that generalizes across multi-image and general vision tasks.

1 Introduction

  • VLMs struggle to link visual cues across images, even though multi-image reasoning is important for understanding actions, 3D views, and changes between states.The paper identifies cross-image cue linking as a gap despite VLMs’ promising single-image capabilities.Source: §1 · p. 1
  • Fine-grained cross-image reasoning is difficult to supervise with rule-based reinforcement learning because it requires manually constructed question-answer pairs with standard answers.The paper describes such QA construction as particularly challenging for fine-grained visual details and complex logic across images.Source: §1 · p. 1
  • MiCo uses inherent image constraints as supervision, constructing triplets of two augmented views of one image and a similar but distinct image for reasoning-based comparison.The model generates comparison reasoning and receives reinforcement learning advantages based on final-answer correctness.Source: §1 · p. 2
  • Trained only on image comparison, MiCo generalizes to broader multi-image scenarios and improves performance on general vision tasks.The paper highlights visual cue linking and fine-grained detail attention as abilities associated with this transfer.Source: §1 · p. 2

2 Related Work

Prior work extends language-model Chain-of-Thought reasoning and reinforcement learning into vision-language models, but often relies on curated training data. Multi-image models and benchmarks have expanded, yet evaluations report persistent difficulty linking fine-grained cues across images.

2 Related Work

  • Vision-language reasoning methods extend Chain-of-Thought approaches, but many rely on curated data generated by existing models or human annotations.MiCo is presented as an alternative that uses inherent constraints in visual data without explicit QA supervision.Source: §2 · p. 2
  • Multi-image models and benchmarks address varied settings, yet VLMs still struggle to link fine-grained cues, identify objects across views, and detect subtle state changes.MiCo targets visual comparison and logical analysis across images.Source: §2 · p. 2

3 Method

The method first probes current VLMs’ cross-image comparison difficulties, then builds contrastive image triplets and trains reasoning with Augmented GRPO. It uses weak-view rollouts to optimize on stronger augmentations, with rewards based on comparison correctness.

3.1 Pilot Study for Multi-image Understanding

  • Pilot examples show that VLMs can interpret images independently but hallucinate on cross-image camera movement, object identity, and semantic changes.These cases motivate visual comparison as a core skill for multi-image understanding.Source: §3.1 · p. 3, Figure 1 · p. 3

3.2 Multi-image Contrast

  • MiCo frames multi-image reasoning as visual comparison, aiming to distinguish similar images, establish correspondences, and support step-by-step analysis.The method targets the cross-image meta-skill rather than collecting data for each specific task.Source: §3.2 · p. 3
  • Contrastive pairs are selected for high structural similarity but subtle differences, using video frames or before-and-after image edits.Video pairs are sampled two seconds apart and filtered by SSIM; edited pairs are filtered by pixel-wise MSE.Source: §3.2 · p. 4
  • Random cropping and resizing create triplets with two views of one image and a similar but distinct third image, preserving content while adding visual variation.The examples include video and image-editing samples with subtle differences.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3 Augmented GRPO

  • The training prompts require comparisons among image pairs or triplet members and structured thinking and answer tags.Triplet accuracy reward is given only when all three pairwise comparisons are correct.Source: §3.3 · pp. 4–5
  • Augmented GRPO samples reasoning trajectories from weakly augmented inputs and optimizes the policy on stronger augmented contexts.The weak views make valid reasoning chains easier to produce, while the optimization context is harder.Source: §3.3 · pp. 4–5
  • The objective encourages higher likelihood for responses with higher relative rewards within each sampled group.Advantages are normalized from group rewards, and the reward combines output-format compliance with answer correctness.Source: §3.3 · p. 5

4 Experiments

MiCo's final model achieves the best average performance on VLM2-Bench, with its 7B model outperforming GPT-4o. Ablations examine training, data, augmentation, and prompt choices; broader evaluations show relational-task strengths alongside limits on some single-image and depth tasks.

4.1 Implementation Details

  • Experiments use Qwen2.5-VL-7B, OmniEdit edits and Vidgen-1M frames, GRPO, and VLM2-Bench tracks GC, OC, and PC with task-specific metrics.Evaluation follows Qwen2.5-VL default hyperparameters and uses VLMEvalKit.Source: §4.1 · pp. 5–6, §4.2 · p. 6

4.2 Result Analysis for Multi-image Comparison

  • MiCo's 7B model achieves the best average performance overall on VLM2-Bench and outperforms GPT-4o, without human- or model-annotated data.The table caption identifies VLM2-Bench as evaluating fine-grained visual-cue comparison and linking across images.Source: Table 1 · p. 6, §4.2 · p. 6

4.3 Ablation Studies

  • SFT and no-CoT RL yield limited gains; combining OmniEdit with VidGen helps, weak-sampling/strong-optimization rollout performs best, and pair-plus-triplet prompts outperform either format alone.Prompt variation helps prevent overfitting and saturates around 50 templates; random cropping and resizing were selected as the default augmentation.Source: §4.3 · pp. 7–8

4.4 Analysis on General Vision Tasks

  • MiCo improves over baselines and performs strongly on multi-image tasks, but trails task-guided models on complex single-image tasks; correspondence and spatial-relation tasks are strengths, while Relative Depth remains difficult.It excels at Visual Retrieval and Semantic Correspondence, but lags on Scene Understanding and Forensic Detection.Source: §4.4 · p. 8

4.5 Qualitative Analysis

  • Qualitative examples show MiCo identifying answer-relevant cues in each image, comparing fine-grained similarities and differences, then producing an answer.Figure 3 marks reasoning in gray and underlines key contents.Source: §4.5 · p. 9, Figure 3 · p. 9

5 Conclusion

MiCo uses image constraints and contrastive triplets to train VLMs for multi-image reasoning with Augmented GRPO. It generalizes beyond image comparison, but remains less effective on specialized tasks requiring structured priors or domain-specific knowledge.

5 Conclusion

  • MiCo combines image triplets and Augmented GRPO to train VLMs for multi-image reasoning.The framework uses inherent image constraints as supervision and samples rollouts from simpler examples while optimizing on harder ones.Source: §5 · p. 9
  • Despite training only on image comparison, MiCo generalizes across multiple benchmarks.The conclusion reports strong results across multiple benchmarks without specifying their scores here.Source: §5 · p. 9
  • MiCo is less effective on face verification, visual math, and spatial understanding, which require structured priors or domain-specific knowledge.The authors identify more efficient, domain-tailored data construction as a future direction.Source: §5 · p. 9

A More Implementation Details

The implementation details describe Augmented GRPO, prompt expansion, and filtering strategies for constructing informative visual comparisons. They also distinguish MiCo’s augmentation strategy from NoisyRollout’s hybrid clean/noisy rollouts.

A More Implementation Details

  • Prompt expansion varies question phrasing and queried image pairs, while balancing answer types to avoid label bias.It includes forward and reverse questions and different pairings among the three images.Source: A More Implementation Details · p. 13
  • Image-pair filtering removes edited pairs with a differing-pixel ratio above 0.8 and video-frame pairs with SSIM above 0.95.For edited images, pixels count as different when their grayscale difference exceeds 30.Source: A More Implementation Details · p. 13
  • Augmented GRPO samples trajectories with weak image augmentation, then uses them to optimize the model on stronger augmentations.The approach assumes weak augmentation makes it easier for the policy to produce high-quality reasoning trajectories, which can inform optimization on harder samples.Source: A More Implementation Details · p. 13
  • Unlike MiCo’s weak-to-strong augmentation optimization, NoisyRollout mixes clean and noisy-image trajectories and optimizes on clean images.NoisyRollout samples trajectories from both clean and moderately distorted images before calculating advantages.Source: A More Implementation Details · p. 13
  • Tables 5–7 report task analyses evaluating MiCo’s generalization on MMStar, MuirBench (12 sub-tasks), and BLINK (14 sub-tasks); Table 5 compares MiCo with the Qwen2.5VL baseline, while MuirBench assesses fine-grained generalization.Source: Table 5 · p. 14, Table 6 · p. 14, Table 7 · p. 14

B.1 More quantitative results

Across MMStar, MuirBench, and BLINK, MiCo improves on most reasoning-related tasks, while results show weaker performance on some fine-grained, absolute-matching, and viewpoint-sensitive tasks.

B.1 More quantitative results

  • MiCo consistently improves upon Qwen2.5VL across most reasoning-related tasks in MMStar, MuirBench, and BLINK.Overall, the reported results favor tasks involving reasoning, structure, and comparison, while identifying fine-grained low-level perception as a remaining direction for improvement.Source: §B.1 · p. 14
  • On MMStar, MiCo gains on instance reasoning, logical reasoning, and math, but slightly declines on fine-grained perception.The authors connect the latter decline to room for improvement in precise visual attribute understanding.Source: §B.1 · p. 14
  • On MuirBench, MiCo improves on 8 of 12 subtasks, including Diagram Understanding, Geographic Understanding, and Visual Retrieval.Attribute Similarity and Counting show marginal drops; the authors suggest these may rely more on absolute visual matching than relational reasoning.Source: §B.1 · p. 14
  • On BLINK, MiCo improves on five tasks requiring visual logic, spatial matching, or multi-view inference, but declines on Multi-view Reasoning and Relative Reflectance.The authors identify viewpoint shifts and subtle appearance variations as areas for further robustness work.Source: §B.1 · p. 14

B.2 Unsuccessful Attempts

The authors report that confidence-based reward reweighting and importance sampling did not improve performance. They offer possible explanations tied to unreliable confidence signals and potential disruption of GRPO’s trajectory ranking.

B.2 Unsuccessful Attempts

  • The explored alternative approaches did not improve performance.The section discusses confidence reweighting and importance sampling as unsuccessful attempts.Source: §B.2 · p. 14
  • Confidence-based reward reweighting failed to improve performance, possibly because token probabilities do not reliably measure reasoning correctness.The authors note that tokens such as punctuation or irrelevant words can receive high confidence despite being unrelated to answer correctness.Source: §B.2 · p. 15
  • Importance sampling did not improve performance and may distort GRPO’s within-group trajectory rankings.The authors also suspect visual-augmentation-related token-probability changes make importance weights noisy or poorly calibrated.Source: §B.2 · p. 15

C Qualitative Analysis

MiCo’s qualitative examples show detailed visual comparisons, jigsaw solving, and abstract reasoning across varied tasks. Together, they illustrate systematic multi-step reasoning grounded in visual understanding.

C Qualitative Analysis

  • MiCo identifies object, pose, structure, and scene-semantic differences in detailed comparisons and uses spatial consistency to solve jigsaw-like tasks.In toy comparisons, it also considers object counts, assembly, model type, and color configuration.Source: C Qualitative Analysis · p. 15
  • MiCo solves IQ-style symbol patterns, matches functional and spatial correspondences, and compares visual similarity using geometric and design attributes.The examples emphasize structural matching rather than reliance on superficial texture or color.Source: C Qualitative Analysis · p. 15
  • These examples characterize MiCo’s performance as systematic, multi-step reasoning grounded in visual understanding across low-level tasks and high-level abstract challenges.Source: C Qualitative Analysis · p. 15

D Potential Social Impact

MiCo’s reliance on intrinsic visual constraints may reduce data-curation needs and support development where curated datasets are scarce. Its fine-grained comparison ability also presents misuse risks, for which the paper advocates safeguards.

D Potential Social Impact

  • MiCo uses intrinsic visual constraints in a self-supervised, reinforcement-learning approach without human-annotated question-answer pairs, potentially easing development where curated datasets are scarce.Source: D Potential Social Impact · p. 15
  • Fine-grained cross-image comparisons could be misused in surveillance, misinformation, or unauthorized inference of user intent from visual data.Source: D Potential Social Impact · p. 15
  • The paper advocates transparency, consent, sensitive-content filtering, restrictions in high-stakes or privacy-sensitive settings, and human review for critical decisions.Source: D Potential Social Impact · p. 16

Related papers

Loading 2506.22434v1…