SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs

Niccolo Avogaro; Nayanika Debnath; Li Mi; Thomas Frick; Junling Wang; Zexue He; Hang Hua; Konrad Schindler; Mattia Rigotti

Source-linked AI summary

SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs

Niccolo Avogaro, Nayanika Debnath, Li Mi, Thomas Frick, Junling Wang, Zexue He, Hang Hua, Konrad Schindler, Mattia Rigotti

arXiv:2602.06566v3Feb 6, 2026cs.CVcs.AIcs.CL

TL;DR

SPARC addresses brittle visual reasoning caused by entangled perception and reasoning in long, costly contexts. It separates visual search from reasoning, enabling asymmetric scaling and selective optimization; the paper reports improved results over native “thinking with images” and efficient independent training of perception.

Problem
Visual-language models struggle with computationally expensive, brittle reasoning chains that mix visual and reasoning tokens, especially in visually heavy contexts and extended sequences.
Method
SPARC separates perception from reasoning through a two-stage pipeline that locates relevant visual regions before reasoning, enabling selective perceptual self-consistency and asymmetric compute allocation.
Results
SPARC enables independent perception scaling, with eight perceptual roll-outs boosting full-pipeline performance by up to 9.3%, while outperforming native “thinking with images.”
Takeaways & Limitations
The modular design supports efficient targeted optimization of perception without degrading pretrained reasoning capabilities, while aggregating lightweight visual searches into a high-confidence context.
Takeaways & Limitations
SPARC’s perception dataset used a relatively weak Molmo2-8B teacher, which likely provided insufficient supervision for the 4B student and limited gains.

Abstract

from arXiv · show

Despite recent successes, test-time scaling -- i.e., dynamically expanding the token budget during inference as needed -- remains brittle for vision-language models (VLMs). Unstructured visual reasoning chains entangle perception and reasoning, leading to long, disorganized contexts where small perceptual mistakes may cascade into completely wrong answers. Reasoning also requires expensive reinforcement learning with hand-crafted rewards. Here, we introduce SPARC (Separating Perception And Reasoning Circuits), a modular framework that explicitly decouples visual perception from reasoning. Inspired by sequential sensory-to-cognitive processing in the brain, SPARC implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer. This separation enables independent test-time scaling with asymmetric compute allocation (e.g., prioritizing perceptual processing under distribution shift), and supports selective optimization (e.g., improving the perceptual stage alone when it is the bottleneck for end-to-end performance). It also accommodates compressed contexts by running global search at lower image resolutions and allocating high-resolution processing only to selected regions, thereby reducing visual token count and compute. SPARC outperforms monolithic baselines and strong visual-grounding approaches across challenging visual reasoning tasks, such as improving Qwen3VL 4B on the $V^*$ VQA benchmark by 6.7 points and surpassing "thinking with images" by 4.6 points in an OOD setting with a $200\times$ lower token budget.

1. Introduction

SPARC addresses brittle, costly multimodal reasoning by separating visual perception from reasoning in a two-stage VLM inference pipeline. This modularity supports independent scaling, optimization, and efficient training of the perception stage.

Motivation: Long multimodal reasoning chains entangle visual and textual tokens, increasing computational cost and brittleness, especially for smaller VLMs.Visually heavy contexts and extended chains also amplify difficulty with fine visual details and bias-driven reasoning.
Method: SPARC first uses Implicit Relevance Detection to identify question-relevant image regions, then reasons over high-resolution crops to produce the answer.The framework is motivated by hierarchical sensory-to-cognitive processing, with perception separated from reasoning.
Contributions: SPARC enables reliable zero-shot test-time scaling of perception with small computational overhead.The approach is presented as an effective prompting scheme rather than requiring complex multimodal reasoning chains.
Contributions: SPARC supports asymmetric compute allocation by scaling perception independently from reasoning and applying targeted self-consistency to visual search.Eight IRD roll-outs with a shared KV-cache add only a few text tokens and an additional crop while boosting full-pipeline performance by up to 9.3%.
Contributions: The perception and reasoning stages can be trained separately, allowing targeted perception optimization without degrading the model’s generic reasoning capabilities.IRD training is efficient because it generates only a small number of crop-coordinate tokens instead of long multimodal reasoning chains.

2. Related Work

Related work positions SPARC within advances in VLM grounding and test-time scaling. Existing approaches increasingly combine visual grounding with interleaved or reinforced multimodal reasoning traces.

Vision-Language Models and Grounding: Modern VLMs connect visual encoders to LLM backbones, while fine-grained grounding enables outputs containing bounding boxes or points alongside text.Examples include Kosmos-2, Qwen3VL, and PaliGemma, with later systems refining segmentation or point-based grounding.
Test-Time Scaling of Large Language Models: Test-time scaling extends autoregressive inference with additional intermediate tokens, with Chain-of-Thought and Self-Consistency among its foundational techniques.Later methods also organize reasoning into non-linear structures such as trees and graphs.
Test-Time Scaling of Vision-Language Models: VLM test-time scaling methods such as “thinking with images” interleave visual operations with textual chains of thought, often using reinforcement learning to encourage grounded reasoning.Point-RFT and ViGoRL align reasoning traces with precise spatial references through reinforcement fine-tuning.

3. Test-Time Scaling of Perception

SPARC reframes perception scaling as structured visual context engineering: locate relevant regions first, then reason over compact high-resolution crops. Experiments indicate that accurate localization can recover reasoning performance while reducing reliance on global visual detail.

Motivation: Test-time scaling can improve detailed VLM perception by expanding the visual and textual information available before the final answer.SPARC focuses on handling task-relevant visual input efficiently and robustly.
Background: “Thinking with images” obtains richer visual representations through extended reasoning traces interleaved with zoom-in actions and high-resolution crops.The approach increases image-token usage as image understanding becomes more detailed.
Motivation: For primarily perceptual tasks, lengthy intermediate text and complex multi-turn handling are described as superfluous or counter-productive.The paper instead motivates structured, modular visual context containing only necessary high-resolution content.
Empirical Evidence: Accurate crops can compensate for reduced global visual detail: at high overlap, the 256px model converges to full-resolution performance.Reasoning accuracy generally declines as crop overlap decreases, especially at lower resolutions.
Empirical Evidence: On V* benchmark tasks, increasingly precise crops drive VQA performance toward the theoretical upper bound, showing that localization can unlock existing reasoning ability.The benchmark emphasizes small-object localization, sufficient inspection resolution, and complex spatial relationships.
Implications: The findings support offloading perceptual computation to specialized modules so the reasoning backbone can operate with a lower visual-processing burden.This layout is presented as analogous to dedicated perceptual processing in biological brains.

4. Two-Stage Architecture: Decoupling Perception and Reasoning

SPARC separates visual relevance detection from perceptual reasoning, using localized image regions to condition final answers. This modular design supports efficient inference across model families, benchmarks, resolutions, and computational budgets.

Two-stage pipeline: SPARC uses sequential prompting: relevance detection first localizes salient regions, then perceptual reasoning generates the answer conditioned on those regions.The two stages activate distinct functions for visual search and logical deduction while structurally managing the context window.
Two-stage pipeline: Implicit Relevance Detection requires inferring query-relevant regions rather than locating an explicitly named target, distinguishing it from standard referring expression comprehension.The task is defined around latent visual relevance from the question.
Experimental setup: SPARC is evaluated on Qwen3-VL bounding-box grounding and Molmo2 point-based grounding across multiple model sizes and spatial modalities.The evaluation includes Qwen3-VL and Molmo2 families with different grounding outputs.
Experimental setup: The evaluation aggregates in-domain performance across V* and HRBench suites and uses XLRS remote sensing as an out-of-distribution proxy, while also varying input resolutions and inference costs.These configurations support analysis of both accuracy and efficiency, including a Pareto frontier for Qwen3-VL 4B.
Findings: SPARC consistently outperforms native baselines, with especially strong gains in low-resolution settings where crops restore high-frequency information using approximately 0.1% of naive full-resolution visual tokens.The reported advantage is particularly pronounced for perceptually demanding scenarios and OOD remote sensing evaluation.
Efficiency: Shared visual KV-caches and truncated contexts reduce latency and enable asymmetric test-time scaling by concentrating additional computation on perception rather than full reasoning chains.The reasoning backbone processes selected regions instead of repeatedly processing complete multimodal chains.

5. Scaling via Perceptual Consistency

SPARC applies self-consistency selectively to perceptual localization, aggregating multiple lightweight crop hypotheses into a compact context for expensive reasoning. Weighted Box Fusion improves localization reliability while limiting downstream computation.

Asymmetric scaling: SPARC applies self-consistency to the perception branch, generating multiple inexpensive coordinate hypotheses while running the expensive reasoning backbone only once.This asymmetric allocation avoids the linear cost of full-chain self-consistency.
Crop aggregation: The pipeline prioritizes spatial recall by merging overlapping proposals and retaining distinct regions, because accuracy drops sharply when crop overlap with the target decreases.Additional background is accepted to preserve coverage of relevant features.
Weighted Box Fusion: N independent IRD rollouts at non-zero temperature explore diverse localization hypotheses before Weighted Boxes Fusion aggregates them.The example uses N = 8 and T = 0.7; boxes with at least 50% intersection over union are merged.
Experimental setup: Table 2 reports performance gains and average crop counts for WBF with N = 4 and N = 8 rollouts, linking perceptual scaling to reduced image-token processing during reasoning.The aggregation operates on crop proposals in text space.
Findings: Accuracy increases monotonically as initial rollouts grow from 1 to 8, indicating that stochastic aggregation denoises localization and improves downstream visual contexts.The reported trend holds across all evaluated models.
Efficiency: WBF improves accuracy without proportional linear downstream cost because deduplication keeps the final crop set lean despite eight perceptual rollouts.Broad exploration occurs in the cheaper perceptual stage while reasoning receives a compact context.
Resolution effects: Higher input resolutions produce fewer final crops, which the authors hypothesize reflects more confident IRD predictions and greater consensus across rollouts.WBF consequently merges more highly overlapping predictions.

6. Fine-Tuning for Pure Perception

SPARC improves perception through separately trained lightweight adapters rather than modifying the reasoning backbone. Supervised fine-tuning across resolutions generally improves downstream performance, with reduced-resolution training often outperforming native-resolution training.

Motivation and architecture: Explicitly training the perceptual module improves downstream VQA accuracy, complementing training-free test-time scaling.The objective is to strengthen perception as an independent computational component.
Motivation and architecture: A lightweight LoRA enables independent perception optimization while preserving the base model’s generic reasoning capabilities.This avoids the catastrophic forgetting risk of naively fine-tuning the full VLM on IRD.
Training method: Unlike “thinking with images,” SPARC uses standard supervised fine-tuning rather than complex reinforcement learning, custom reward shaping, and extensive process-supervision datasets.The approach reduces engineering overhead and instability associated with inducing latent reasoning traces.
Training data: Training data uses spatial relevance annotations generated synthetically from successful traces filtered by final-answer correctness, with approximately 23,000 Qwen3-VL samples and 14,000 Molmo2 samples.The annotation format is tailored to bounding-box or point-based grounding modalities.
Training method: Supervised fine-tuning runs for two epochs with autoregressive next-token prediction across three resolution scales.The multi-resolution design tests whether high-fidelity inputs are necessary for relevance detection.
Findings and limitation: Fine-tuning improves all evaluated dimensions except Molmo2-4B at lower resolutions, where performance is comparable to baseline.The authors attribute this plateau to insufficiently strong supervision from the Molmo2-8B teacher.
Resolution ablation: The SPARC SFT 256 adapter achieves the highest accuracy across most test settings, supporting reduced-resolution training as a regularizer against trivial high-resolution distillation.Lower-resolution training forces relevance inference from coarser signals.

7. Conclusion

SPARC decouples visual perception from reasoning to support efficient, asymmetric inference and targeted perceptual fine-tuning. Its current scope is limited to VLMs with native spatial grounding outputs.

Conclusion: SPARC separates region localization from question answering, reducing computational overhead and enabling asymmetric test-time scaling through lightweight visual-search aggregation.The framework is inspired by systems neuroscience and explicitly decouples perceptual and reasoning circuits.
Conclusion: The modular design permits targeted fine-tuning of perception without degrading the base model’s pretrained reasoning capabilities.This supports independent optimization of the perceptual circuit.
Limitation: SPARC currently relies on VLMs with native spatial grounding such as coordinate or point outputs.Applying it to purely text-generative VLMs would require an external object detection model.

Impact Statement

The authors report no societal consequences that require specific highlighting.

No potential societal consequence is identified as requiring specific emphasis.

A.1. Test-Time Scaling Generalization to Molmo2 Architecture

SPARC’s crop-precision benefits generalize from Qwen3-VL to Molmo2. At low resolutions, accurate crops recover performance toward the full-resolution baseline.

Molmo2’s 256px and 512px variants recover sharply as crop overlap with the target improves.The same resolution-compensation pattern observed for Qwen3-VL appears in Molmo2.
At approximately 1.0 overlap, Molmo2’s 256px and 512px baselines effectively close the gap with the full-resolution baseline.

A.2. Implicit Relevance Detection as an Ill-Posed Problem

Crop expansion has a non-monotonic effect: moderate context improves reasoning, but excessive expansion hurts resolution-constrained models.

Crop expansion initially improves performance across resolutions, peaking around 2.5× for the 256px model.
Excessive expansion hurts 256px and 512px models because resizing crops into fixed pixel buffers reduces effective visual detail.

A.3. Prompts

The appendix details SPARC’s prompting choices, scaling behavior, efficiency trade-offs, robustness, and evaluations across model sizes and task types.

A.3. Prompts: Strict JSON or point-coordinate output formats reduce syntax errors and hallucinations in the IRD prompts.
A.3. Prompts: Moderate crop expansion supplies missing context, whereas excessive expansion sharply degrades resolution-constrained models.
A.4. Comparison with “Thinking with Images” Approaches on V∗: SPARC reaches 91.2% with Qwen3-VL 8B, exceeding Qwen3-VL 235B-A22B at 91.1% and DeepEyes at 90.1%.The comparison uses supervised fine-tuning of the perception circuit.
A.5. Comparison with Bottom-Up Divide-and-Conquer Approaches: Molmo2 4B improves from 48.7% to 62.3% at 256 resolution with bottom-up slicing, but processes up to approximately 11 crops versus SPARC’s approximately 1.9.
A.6. WBF Scaling and Ablation: For V∗, 64 IRD rollouts produce 85 crops covering 1.78% of the image, while target objects occupy 0.20%.
A.6. WBF Scaling and Ablation: WBF performance peaks around eight rollouts and compresses 85 raw crops into approximately 2.4 deduplicated crops.
A.7. Robustness to Global Scene and Relational Queries: SPARC avoids systematic context collapse on global and relational queries by retaining both the original image and localized high-resolution crops.
A.8. Real-World Efficiency Metrics; A.10. Distilled SFT Baseline; A.11. Larger Model Scaling; A.12. Performance on General Reasoning Tasks; A.14. Qualitative Analysis: The appendix reports implementation-dependent TTFT and E2E latency measurements, frontier-scale evaluation, general-reasoning results, and qualitative WBF analyses.Distillation SFT underperforms at higher resolutions and uses 4× the GPU memory of SPARC SFT; SPARC maintains or improves MME Realworld performance and WBF deduplicates boxes at full resolution.