Learning to Zoom Efficiently with a Contrastive Curriculum

1 Introduction

• Zoom-in helps visual agents handle high-resolution images with small regions of interest and is linked to their grounding capacity.Source: §1 · p. 1
• RL makes tool learning sequential: models decide whether to zoom before deciding how, although skill acquisition calls for mastering the tool first.The paper focuses on learning how to use the tool, not explicitly pruning when to use it.Source: §1 · p. 1
• The proposed reward contrasts post-tool logits for a correct positive zoom with a nearby negative region, whose overlap hardness increases during training.Negative regions are generated from a silver-label positive sample.Source: §1 · pp. 1–2, Figure 1 · p. 2
• The intrinsic reward teaches zoom-in use without warm-start SFT or additional tool-use labels.Source: §1 · pp. 1–2
• M&C provides a scalable benchmark for zoom-in ability, while the method is competitive on general benchmarks and outperforms baselines when replacing SFT with continued pure RL.Source: §1 · p. 2

2.1 Multimodal Tool Use

• Prior tool-use approaches include warm-start SFT followed by RL, while RL-only methods add tool-use rewards that may rely on labels or heuristics.The review also notes methods that forego an explicit tool-use reward after extensive SFT.Source: §2.1 · pp. 2–3

2.2 Intrinsic Exploration Rewards

• Intrinsic exploration rewards support learning from experiences that may initially underperform, after which standard RL can refine the policy using external rewards.The paper draws on InfoNCE-style contrastive learning to develop an intrinsic reward for tool exploration without SFT data.Source: §2.2 · pp. 3–4

2.3 Evaluation Data

• Existing zoom-in benchmarks may lack suitable image size or RoI annotations; prior image-panel datasets also lack bounding boxes and use panels up to one megapixel.The paper proposes structured single-image haystacks with RoI annotations that can scale in image size and task complexity.Source: §2.3 · p. 4

3.1 Method description

• A correct answer makes the preceding tool call a silver positive, paired with an alternative call generated from its position.Source: §3.1 · p. 4
• The alternative tool execution creates a second image-region prefix, with its overlap IoU with the positive region set by τ.Source: §3.1 · p. 4
• The method rescored the existing second generation under the alternative prefix instead of sampling another, requiring only a forward pass.Source: §3.1 · p. 5

3.2 Interpretation as Contrastive Learning

• The reward is the negative InfoNCE loss shifted by ln(2), measuring whether the subsequent generation distinguishes positive from negative tool calls.Source: §3.2 · p. 5
• Constructed negatives let τ tune difficulty from easy (τ = 0) to hard (τ > 0), avoiding unconstrained mutual-information maximization that pushes diverse tool calls.Source: §3.2 · p. 5

3.3 Practical Considerations

• Because the second generation has 40+ tokens, the implementation sums token-level logits, clips per-token terms, and applies tanh to bound the reward.Source: §3.3 · p. 5
• For fixed sequence length, unclipped rtool conservatively maximizes rtilde; the final reward gates tool reward on answer correctness, and rtool = I{tool used} recovers Conditional.Source: §3.3 · p. 5

• Similarity between chihuahua snouts and eyes and muffin blueberries motivates M&C as a stress test for zooming into visually confusable details.Source: §4 · p. 5
• Gold RoI annotations let M&C assess zoom-call quality independently of downstream task performance.Source: §4 · p. 6
• Single-cell queries use balanced muffin/chihuahua labels and should need one targeted zoom, while finding the lone muffin likely requires many zooms.Source: §4 · p. 6
• M&C contains 3,600 samples in 36 splits across four image sizes, five grid sizes, and the single-cell-query and find-outlier tasks.Source: §4 · p. 6

5.1 Training Setup

• Training continues Qwen 2.5 VL 7B with GRPO and selective sample replay on 6,698 PR/video queries, flushing the buffer every other update.Source: §5.1 · p. 6

5.2 Evaluation

• Evaluation covers HRBench 4k and 8k, V*-Bench, and MME-RealWorld, alongside M&C task accuracy and RoI-overlap analyses.Source: §5.2 · pp. 6–7
• As grid size increases, single-cell-query performance moves steadily toward the 50% random baseline; only Conditional and our models consistently outperform No tool.Source: Table 2 · p. 7

5.3 Baselines and Models

• GRPO-trained baselines isolate no-tool access, curiosity reward, and answer-conditional tool reward; Ours uses curriculum negatives, while '+' variants train further on Visual Probe without added reward.Source: §5.3 · p. 7

6.1 General Benchmarks

• Ours+ is the strongest model overall, while Ours is competitive and only Mini o3 outperforms it before continued training.Source: §6.1 · p. 7
• Tool-use training improved tool-free evaluation only for Qwen2.5-VL; InternVL and Gemma tool-trained models underperformed their no-tool baselines in that setting.Source: §6.1 · pp. 7–8

6.2 Muffin&Chihuahua Dataset

• Only Conditional baselines gained an advantage from tool use on find outlier, while Mini o3’s 32 tool uses still did not navigate the task meaningfully.Source: Table 3 · p. 8, §6.2 · p. 9

6.1 General Benchmarks

• Continued training benefited tool-free and tool performance for exploration models, raising tool performance by 1–2 points; Ours+ surpassed Mini o3 and Curiosity+ reached parity.Source: §6.1 · p. 8
• Ours needs one tool use to outperform Mini o3’s six-use setup, and Ours+ is Pareto-optimal on the reported efficiency comparison.Source: §6.1 · p. 8

6.2 Muffin&Chihuahua Dataset

• On single-cell query, Conditional performed best, performance generally fell as grids grew, and the 1×1 case exposed difficulty when no zoom was needed.Source: §6.2 · p. 8
• Additional zooms help when they enable region comparison or correction, but these models performed better with single zoom than with multi-zoom approaches.Source: §6.2 · p. 9
• Recall was most indicative of final performance, supporting reliable coverage of the target region as the most effective zoom behavior.Source: §6.2 · p. 9, Table 4 · p. 9

• The proposed reinforcement-learning method teaches zoom use without warm-start SFT, drawing inspiration from mutual information maximization.Source: §7 · p. 9
• Ours+ achieves state-of-the-art performance within its model class on V*, HRBench 4k, and MME-RealWorld, using the tool once and transferring from Qwen2.5 to InternVL 3.5.Source: §7 · p. 9
• M&C provides 36 splits across grid and image size, and its annotations indicate that performance is best when zoom regions reliably cover the region of interest.Source: §7 · p. 9

8 Limitations

• Hard negatives mitigate reward-hacking cycles in which a high-reward second turn arises from factors other than a good tool call.Other factors include turn length, lexical overlap between turns, and round-valued tool coordinates.Source: §8 · p. 10
• Good performance requires τmax in a narrow [0.15, 0.2] corridor, with τmax = 0.175 best; using only easy negatives triggers reward hacking.For Gemma, the authors instead set teasy = 0 to obtain meaningful results.Source: §8 · p. 10
• The method depends on a dataset where the base model sometimes answers correctly while using the tool; otherwise it will not reliably learn tool use.The authors say this limitation also applies to conditional rewards and GRPO more broadly.Source: §8 · p. 10

9 Ethical considerations

• Before production use, the authors recommend another instruction-tuning round to reduce harmful generations; the M&C base-image licenses could not be found.The authors provide a script for re-downloading the images as part of the open-source code.Source: §9 · p. 10

B.1 Hyperparameters

• The IoU target stays at zero for the first 30% of training, then rises linearly to τmax = 0.175 by the end.The appendix uses varying τmax values for ablations; τmax = 0 means training uses only easy negatives.Source: Figure 5 · p. 14
• Training uses 382 PR\video steps over about 35 hours on 8×A100 GPUs; Visual Probe training takes 554 steps and 78 hours.The setup uses selective sample replay and a slightly off-policy procedure.Source: §B.1 · p. 14

B.2 Runtimes

• The extra forward pass required by the method adds little runtime because multi-turn trajectory generation dominates runtime.Runtime may vary as the model develops during reinforcement learning.Source: §B.2 · p. 14

C.1 Dataset Construction

• M&C grids combine muffin and chihuahua images; single-cell queries are balanced by label, while find-outlier grids contain exactly one muffin.The dataset also randomizes multiple-choice labels to counteract position bias.Source: §C.1 · pp. 14–15

C.2 Prompt templates

• The prompt templates ask either which object appears in a numbered cell or which cell contains the lone muffin.The initial prompt explains that cells are numbered left-to-right and top-to-bottom.Source: §C.2 · p. 15

C.3 Image sizes

• The image-size appendix compares evaluation datasets by median size and uses “M&C X” for all M&C samples of image size X.The category spans grid sizes and tasks.Source: §C.3 · p. 15, Table 6 · p. 15

C.4 Base image links

• The authors list links for the sixteen source images and provide Internet Archive backups if the original links are unavailable.They also include a re-download script in the open-source code.Source: §9 · p. 10, §C.4 · p. 15

D.1 General Benchmarks

• The benchmarks cover high-resolution general images, object relations and attributes, and real-world domains including driving, monitoring, diagrams, OCR, and remote sensing.HRBench includes 4k and 8k versions; the 8k set has 200 images and 800 samples after answer-option rotations.Source: §D.1 · pp. 15–16

D.2 Evaluated Models

• Evaluation generally resizes images to [0.392, 3.92] million pixels and uses greedy decoding with exact-match accuracy.Mini o3 instead uses temperature 1.0 and reports dataset-specific Avg@k values; its image resizing also differs.Source: §D.2 · p. 16
• The evaluated training variants include no-tool, Curiosity, Conditional, and Ours; Ours uses an IoU target that rises from zero to 0.175.The Curiosity variant uses absolute pixel coordinates and no warm-start SFT stage.Source: §D.2 · p. 16

E Tool Prompts

• The generic zoom-in tool prompt is adapted from prior work, and its DESCRIPTION and DTYPE parameters specify bounding-box details.The authors note that tested models are sensitive to bounding-box types.Source: E Tool Prompts · p. 16, §E.1 · pp. 16–17

E.2 User prompt

• The user prompt asks the model to assess whether visual tools would help, reason step by step, and put its final answer in boxed format.The same guiding text is appended during training and inference.Source: §E.2 · p. 17

E.3 Tool Reply

• Tool replies present the cropped image with its width and height in the model’s preprocessed-image frame, followed by image tokens.The reply uses a fixed cropped-image format.Source: §E.3 · p. 17

F.1 Ablations

• Good performance requires τmax in a narrow corridor, and the caption reports that more negatives seem to help models retain enhanced grounding for tool-free inference.The ablations vary the number of negatives, keep τ at zero throughout training, and test τmax values of 0.15 and 0.2 against 0.175.Source: Table 10 · p. 18, §F.1 · p. 18

F.2 Extension to multiple tool uses

• The proposed extension to multiple tool calls performed subpar, likely because Qwen 2.5 VL cannot easily learn multiple meaningful zoom-ins without SFT.The authors propose applying the additional reward to the last tool call before a correct answer.Source: §F.2 · p. 18

G.1 Reward Hacking

• In the HRBench example, the curriculum model makes a targeted crop covering 0.79% of the image, with IoU 0.28 and recall 1.0, and answers correctly.The annotated region of interest occupies 0.2% of the image.Source: §G.1 · p. 18, First Reasoning · p. 18

Second Reasoning

• Without the negative curriculum, the example model guesses first and uses the tool for verification; repeated words and inflated confidence can raise its reward despite inexact tool use.Its crop covers 20.95% of the image and has IoU 0.01 with the region of interest, while recall is 1.0.Source: Second reasoning · p. 19, First reasoning · p. 19

G.2 Excessive Tool Use

• On 1×1 grids, both models call the tool despite its being unnecessary; Ours+ can zoom into the cell index and lose the global context needed to answer.Curiosity+ selects a larger area that retains more global context in the described example.Source: §G.2 · p. 19, Table 11 · p. 19
• When the entire image is the region of interest (RoI), “good” IoU-based zoom-in does not help the task; a small constant negative tool-use reward caused our model to abandon the tool entirely.The authors describe tool calling as a learned reflex common across RL-trained zoom-in models.Source: §G.2 · p. 19

H Algorithms

• The appendix points to an algorithm for generating hard-negative bounding boxes and defines U and LogNormal as Uniform and Lognormal distributions.It also identifies B as Bernoulli.Source: H Algorithms · p. 19

I Cross Model Results

• After continual training, our method takes over the other baselines on InternVL 3.5 8B.The InternVL setup kept the tool call fixed and varied only tool execution.Source: I Cross Model Results · p. 19, Table 12 · p. 21
• Gemma 4 E4B results were poor across variants; Curiosity performed best and our method ranked last.Source: Table 13 · p. 21
• Frequent tool use during training received zero reward, a measure intended to prevent endless tool-use loops during evaluation.Source: I Cross Model Results · p. 21
• On Gemma, removing the initial 30% of easy negatives and using a linear curriculum led to tool-use abandonment at training’s end, despite frequent tool use during evaluation.The authors hypothesize that the initial easy negatives caused the standard curriculum’s deteriorated results.Source: I Cross Model Results · p. 21

J Additional Results

• InfographicVQA was excluded from the official results because its median image size was 2.25M pixels and the best PR\video-trained model used no tools.Source: J Additional Results · p. 21
• Precision, recall, and intersection-over-union (IoU) were used to analyze zoom-in behavior.Source: J Additional Results · p. 21
• Mini o3 sometimes produced no answer after generation was terminated at 32 tool uses.Source: J Additional Results · p. 21
• β = 0.01 was clearly the best value in the conditional-reward sweep, although results showed no monotonic trend.Source: J Additional Results · p. 21
• Ours recovered from late-training tool-use declines across three curricula without increasing second-generation length, unlike variants that increased it to hack reward.The easy-negative run also showed a tool-use drop around step 300.Source: Figure 7 · p. 22