Source-linked AI summary
Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, Yutong Cai, Siyuan Li, Huijia Zhu, Weiqiang Wang, Linghe Kong, Yue Wang, Zhuosheng Zhang, Weiran Huang
TL;DR
Fine-grained perception is limited when small decisive evidence is overwhelmed in global images, and inference-time zooming improves isolation at the cost of latency. The paper introduces Region-to-Image Distillation, which generates teacher supervision on micro-crops and distills it to full-image training for single-pass inference. Its models improve fine-grained perception and general multimodal cognition while narrowing the global–regional zooming gap, though spatial reasoning and multi-object perception remain under-evaluated.
Problem
MLLMs struggle with small decisive evidence in full images, while iterative visual zooming incurs repeated tool-use and encoding latency.
Method
Region-to-Image Distillation generates high-veracity VQA supervision on micro-crops and transfers it to full images using explicit grounding for single-pass inference.
Results
The models consistently improve fine-grained perception and general multimodal cognition, outperform tool-use baselines at lower inference cost, and narrow the zooming gap.
Takeaways & Limitations
Zooming benefits can be internalized during training when regional supervision is grounded back to the full image, reducing reliance on iterative inference-time tools.
Takeaways & Limitations
Spatial reasoning and multi-object perception are not widely incorporated or evaluated, including on benchmarks such as TreeBench.
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) excel at broad visual understanding but still struggle with fine-grained perception, where decisive evidence is small and easily overwhelmed by global context. Recent "Thinking-with-Images" methods alleviate this by iteratively zooming in and out regions of interest during inference, but incur high latency due to repeated tool calls and visual re-encoding. To address this, we propose Region-to-Image Distillation, which transforms zooming from an inference-time tool into a training-time primitive, thereby internalizing the benefits of agentic zooming into a single forward pass of an MLLM. In particular, we first zoom in to micro-cropped regions to let strong teacher models generate high-quality VQA data, and then distill this region-grounded supervision back to the full image. After training on such data, the smaller student model improves "single-glance" fine-grained perception without tool use. To rigorously evaluate this capability, we further present ZoomBench, a hybrid-annotated benchmark of 845 VQA data spanning six fine-grained perceptual dimensions, together with a dual-view protocol that quantifies the global--regional "zooming gap". Experiments show that our models achieve leading performance across multiple fine-grained perception benchmarks, and also improve general multimodal cognition on benchmarks such as visual reasoning and GUI agents. We further discuss when "Thinking-with-Images" is necessary versus when its gains can be distilled into a single forward pass. Our code is available at https://github.com/inclusionAI/Zooming-without-Zooming.
1 Introduction
Fine-grained perception remains difficult because decisive micro-evidence is overwhelmed by global visual context, while inference-time zooming improves isolation at substantial latency. Region-to-Image Distillation moves zooming into training, and experiments report stronger perception and cognition with lower-latency single-pass inference.
- MLLMs struggle to isolate tiny text and subtle attributes from thousands of visual tokens in cluttered full images.
- Thinking-with-Images reduces interference by iteratively isolating regions, but repeated tool calls and visual encoding passes limit real-time usage.
- Region-to-Image Distillation uses strong teachers on micro-crops, then transfers region-grounded supervision to full images with bounding-box grounding.
- ZoomBench contains 845 VQA samples spanning six perceptual dimensions and supports dual-view measurement of the global–regional zooming gap.
- Experiments report gains across fine-grained perception benchmarks, competitiveness with larger MLLMs, lower-latency performance than agentic models, and improved multimodal cognition.
2 Related Works
Prior work provides fine-grained perception data and inference-time visual tools, but existing synthesis pipelines and benchmarks remain constrained in scale, diversity, coverage, difficulty, or evaluation format. Region-to-Image Distillation addresses this gap with region-centric synthesis and full-image training.
- Fine-Grained Multimodal Perception: Thinking-with-Images methods acquire local visual information during inference instead of relying only on global image encoding.
- Multimodal Synthetic Data: Existing fine-grained datasets are constrained by limited size, low difficulty, or homogeneous image and task distributions due partly to manual annotation.
- Multimodal Synthetic Data: Prior synthetic-data methods largely use global-to-global teacher-to-student distillation, whereas Region-to-Image Distillation generates QA pairs from micro-crops and trains on full images.
- Perception Benchmarks: Existing perception benchmarks often have narrow task coverage, single protocols, templated questions, or specialization in particular capabilities.
3 Zooming without Zooming
Region-to-Image Distillation converts tool actions into training supervision: teachers generate fine-grained QA on meaningful micro-regions, then grounding transformations re-anchor that supervision to full images. ZoomBench applies the same region-centric construction to produce diverse, verified evaluations of fine-grained perception.
- 3.1 Region-to-Image Distillation: Region-to-Image Distillation internalizes tool-based zooming during training so students perform fine-grained perception from full images in one forward pass.
- 3.1 Region-to-Image Distillation: Object-centric proposal generates semantically meaningful bounding boxes, whose cropped and resized regions expose fine-grained evidence for teacher-generated questions.
- 3.1 Region-to-Image Distillation: Box-overlay grounding resolves referential ambiguity when crop-derived questions are transferred to the global image.
- 3.1 Region-to-Image Distillation: The synthesis pipeline applies a tool action, generates QA pairs on the altered observation, and inverse-transforms them into a distilled dataset for full-image training.
- 3.1 Region-to-Image Distillation: The framework can distill tool actions beyond zooming, including flipping, 3D grounding, and expert-model calls.
- 3.2 ZoomBench: ZoomBench combines automated synthesis with human verification and contains 845 diverse, challenging QA pairs across six fine-grained perception categories.
4 Experiments
Experiments evaluate whether Region-to-Image Distillation enables fine-grained perception in one forward pass, across perception, out-of-distribution, agentic, and real-world multimodal tasks. The results show consistent gains, strong data efficiency, and lower-latency performance than tool-based alternatives, while visual grounding and crop-based synthesis are important design choices.
- Experimental Settings: Training Qwen-VL backbones with DAPO on 74K synthetic samples produces single-glance ZwZ models without supervised fine-tuning.The evaluated backbones are Qwen3-VL-4B, Qwen2.5-VL-7B, and Qwen3-VL-8B.
- Main Results: ZwZ-8B raises the Qwen3-VL-8B average score from 62.86 to 68.12, including a ZoomBench increase from 37.87 to 58.11.ZwZ-4B and ZwZ-8B also surpass larger open-source models on average general perception benchmarks and remain competitive with Gemini-3.
- Comparison with Agentic Models: ZwZ-8B achieves the best overall average among listed agentic methods while using a single forward pass instead of iterative zooming.The comparison covers several perception benchmarks and contrasts single-pass models with agentic models.
- Efficiency: ZwZ attains higher accuracy at around 10× faster inference speed than agentic and tool-use baselines.Inference speed is defined as the inverse of average per-sample inference time on ZoomBench, while accuracy is the average Table 4 score.
- Ablation Study: Micro-crop supervision outperforms direct full-image synthesis across all evaluated benchmarks, while bbox-in-image grounding outperforms no-bbox and bbox-in-question variants.These ablations support using isolated regions for higher-quality supervision and explicit spatial guidance during distillation.
- Real-World Generalization: ZwZ-8B consistently outperforms Qwen3-VL-8B on AIGC detection and GUI-agent benchmarks, extending gains beyond fine-grained perception.The broader evaluations include LOKI, FakeCLUE, ScreenSpot Pro, and OSWorldG.
5 Deeper Analysis on ZoomBench
ZoomBench’s dual-view protocol separates recognition from the ability to locate fine-grained evidence in a full image. Vanilla MLLMs show substantial global–regional gaps, while ZwZ narrows the gap and concentrates attention more strongly on annotated key regions.
- Dual-View Evaluation: The dual-view protocol compares answers from the full image and corresponding micro-crop, defining the zooming gap as Global–Regional performance difference.Regional-View accuracy acts as an empirical upper bound, while Global-View accuracy measures retrieval and use of the same evidence in the full image.
- Zooming Gap Results: 25.21%: Qwen3-VL-8B falls from 63.08% Regional accuracy to 37.87% Global accuracy.The gap suggests failures often reflect perceptual oversight rather than insufficient recognition or reasoning.
- Zooming Gap Results: 15.26%: ZwZ-8B reduces the zooming gap to the smallest reported value among compared models.Larger vanilla models still exhibit notable gaps, indicating that parameter scaling alone does not eliminate the fine-grained retrieval bottleneck.
- Dimension-Wise Analysis: Counting remains the most challenging dimension, even when the evidence is shown in Regional-View.The passage attributes this difficulty to precise localization of densely packed objects and reasoning required to avoid over- or under-counting.
- Dimension-Wise Analysis: Structure and Material have the largest average zooming gaps across perceptual dimensions.These dimensions therefore show the greatest difference between regional evidence access and full-image evidence retrieval.
- Attention Map Coverage Analysis: ZwZ variants achieve higher relative-attention bounding-box coverage than corresponding Qwen-VL baselines across model scales.Higher coverage indicates greater concentration of question-relevant attention within annotated key regions and accompanies a narrower zooming gap.
6 Discussion and Future Direction
The discussion distinguishes image actions that add unpredictable external information from operations whose benefits can be internalized through training. This frames ZwZ as complementary to agentic Thinking-with-Images rather than a replacement for information-gain tools.
- Comparison with Thinking Paradigms: Prior multimodal thinking paradigms trade off test-time cost, data or engineering overhead, and adaptivity in different ways.Tool-based methods are flexible and interpretable but slower and require adaptive tool-use decisions; image-generation and latent approaches also address visual reasoning with different costs.
- Information-Gain Criterion: TwI is most valuable when an action produces unpredictable information gain, whereas predictable image operations are candidates for distillation.External search or retrieval remains essential because its outputs cannot be inferred from the current image view or internalized weights.
- Information-Gain Criterion: Zooming can provide new information when downsampling erases small details, but this setting is avoided here by encoding images at full resolution.Under full-resolution encoding, the paper treats many zooming benefits as information-neutral and suitable for training-time internalization.
- Agentic Overhead: Table 10 reports surprisingly small agentic-versus-direct performance gaps for DeepEyes and Thyme on perception benchmarks.The comparison suggests iterative tool-call overhead contributes marginally to final accuracy in those evaluated settings.
- Internalizing Workflows: Parallel workflows can be internalized when intermediate results serve only as auxiliary support rather than appearing in the final output.The discussion relates this principle to reflection-based reasoning and majority-voting benefits distilled into single forward passes.
- Limitation and Future Direction: Spatial reasoning and multi-object perception remain underrepresented and were not evaluated on benchmarks such as TreeBench.The authors state that the methodology could be extended using spatial-reasoning or multi-object-search tools.
- Limitation and Future Direction: A future unified agent policy could default to enhanced single-pass inference, invoke tools selectively, and prioritize information-gain actions.This would combine the efficiency of ZwZ with the open-world capability of agentic TwI.
7 Conclusion
Zooming without Zooming converts inference-time zooming into training-time Region-to-Image Distillation, teaching MLLMs to recover fine-grained evidence from global inputs in one pass. The method improves perception and broader multimodal cognition while reducing tool-use cost and narrowing the zooming gap.
- Conclusion: Region-to-Image Distillation synthesizes high-veracity micro-crop supervision and distills it to full images with explicit grounding.The framework targets recovery of fine-grained evidence from global inputs without inference-time zooming.
- Conclusion: The models consistently improve fine-grained perception across benchmarks and outperform agentic and official tool-use baselines at much lower inference cost.The reported gains extend to general multimodal cognition, including visual reasoning and real-world tasks.
- Conclusion: Attention analyses show stronger focus on task-relevant micro-regions and a substantially narrower global–regional zooming gap.The conclusion presents these analyses as evidence of improved fine-grained evidence use.
8 Implementation Details of Our Method
The implementation constructs region-grounded training and benchmark data from high-resolution images using proposal, cropping, teacher consensus, grounding, and rejection-sampling steps. ZoomBench additionally uses partitioning and human verification to validate generated question–answer pairs.
- Region-to-Image Distillation: Algorithm 2 initializes a distilled dataset and iterates over images, candidate boxes, crops, generated questions, teacher answers, grounding, and rejection sampling.The pipeline keeps only high-consensus answers before storing full-image training triplets.
- Region-to-Image Distillation: Candidate regions are selected from detection or segmentation proposals whose area is below a sparsity threshold, then cropped for region-answerable question generation.The resulting question and answer are mapped back to the full image with a spatial constraint.
- Data Sources: The image pool contains mostly high-resolution images above 800 × 800 pixels from several large-scale datasets.An object recognition and segmentation system supplies object-level annotations when source datasets lack them.
- Benchmark Construction: ZoomBench uses Gemini-2.5-Pro for question and answer generation, separates training and benchmark images and QA pairs, and adds human verification.Three paper authors independently check whether each question is unambiguous and each answer is correct under both full-image and cropped-region views.
9 Implementation Details of Experiments
The experiments use a tiered reward system for reinforcement learning and benchmark evaluation, combining deterministic matching with LLM judging when necessary. Training rewards increase steadily over time, indicating that the dataset is learnable.
- A two-stage tiered reward system prioritizes deterministic evaluation and uses heuristic or semantic methods only when necessary.The pipeline balances accuracy and computational efficiency.
- Reward modeling: Exact or symbolic answer matching assigns the maximum reward of 1 when the extracted answer matches the ground truth.
- Reward modeling: An efficient LLM judge provides a binary 0/1 reward when rules and numerical parsing cannot resolve correctness.Qwen3-8B is used for reinforcement-learning rewards, while Qwen3-30B is used for benchmark evaluation.
- Training behavior: A steady increase in Qwen3-VL-8B’s training reward indicates that the dataset is highly learnable.
- Inference setup: Direct-answer models use one full-image forward pass, while agentic baselines follow official crop or zoom configurations and report end-to-end wall-clock latency.
10 Prompts.
The prompts define procedures for judging answer equivalence and generating image-grounded VQA questions. Question generation requires concise, accurate questions answerable solely from the provided image.
- LLM-as-a-Judge: The LLM-as-a-Judge prompt evaluates whether a response expresses the same meaning as the ground-truth answer.It accepts equivalent phrasing and minor precision differences but rejects different objects, numbers, concepts, or interpretations.
- Question generation: The question-generation prompt asks an expert to produce three high-quality VQA questions based solely on the provided image.Answers must be identical, accurate, concise, short, factual, and concrete.
11 Case Study
The case study presents ambiguous examples in which crop-derived answers differ from answers obtained from the full image. It also illustrates benchmark questions across counting, OCR, color, structure, material, and identification.
- Ambiguous cases: Crop-derived questions can be unambiguous in isolation but ambiguous in the full image because cropping causes information loss.The training data therefore overlays the target bounding box to provide explicit visual grounding.
- Ambiguous cases: A tennis-ball question receives answer C for the cropped image but D for the full image.
- Ambiguous cases: An island-group question receives answer C for the crop, whereas the full-image answer is “A CANARIAS.”
- Ambiguous cases: A bicycle-fender brand question receives answer C for the crop but B for the full image.
- Benchmark cases: Representative benchmark categories include counting, OCR, color, structure, material, and identification.The examples are generated by MLLMs through Region-to-Image Distillation.
- Benchmark cases: The benchmark includes questions about fish counts, poster text, valve-handle color, statue headwear, and bottle material.
12 Relative Attention Map Computation
Relative attention maps combine answer-to-token and token-to-image attention to estimate which visual regions support an answer. The method normalizes question-specific attention against generic image-description attention and selects a fixed layer for coverage analysis.
- Setup and notation: The setup represents an image as an N × N grid of visual tokens, optionally resampled by a Transformer connector into T image tokens for the LLM.
- Attention construction: Answer-to-token attention is extracted from the starting answer token to the image tokens and averaged across attention heads.
- Attention construction: For connector-based models, token-to-image attention maps each image token back to the N^2 ViT visual grid and is averaged over heads.Without a Transformer connector, the mapping is treated as an identity mapping.
- Attention construction: Answer-to-image attention combines LLM answer-to-token and connector token-to-image attention through tensor-product-style matrix multiplication, producing a spatial map over the visual grid.The final dimension is reshaped into an N × N grid.
- Relative attention: Relative attention divides question-specific answer-to-image attention by attention from a fixed generic image-description instruction, using a small epsilon for numerical stability.
- Layer selection: The resulting tensor is reduced to one N × N map by selecting the relative attention from the 24th LLM layer and a fixed connector layer.
- Qualitative comparison: ZwZ-8B concentrates more relative attention on the annotated key region than base Qwen3-VL-8B, indicating improved localization of task-relevant evidence.