Source-linked AI summary
HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning
Shenzhi Wang, Shixuan Liu, Jing Zhou, Chang Gao, Xiong-Hui Chen, Binghai Wang, An Yang, Shiji Song, Bowen Yu, Gao Huang, Junyang Lin
TL;DR
Existing vision-language RLVR data insufficiently tests the visually grounded, multi-step reasoning required by long chain-of-thought inference, where intermediate errors can compound. HopChain synthesizes logically dependent, instance-grounded multi-hop queries with verifiable numerical answers for RLVR training, and improves performance on 20 of 24 benchmarks for both evaluated models. The results support multi-hop data as a broadly transferable training signal for vision-language reasoning, while the current pipeline excludes images without detectable segmentable instances.
Problem
Long-CoT vision-language reasoning can develop compounding perception, reasoning, knowledge, and hallucination errors, while existing RLVR data insufficiently requires visual evidence throughout the chain.
Method
HopChain synthesizes logically dependent, instance-grounded multi-hop queries that repeatedly require visual re-grounding and terminate in specific, unambiguous numerical answers suitable for RLVR.
Results
20 out of 24 benchmarks improve on both Qwen3.5-35B-A3B and Qwen3.5-397B-A17B when original RLVR data is augmented with HopChain data.
Takeaways & Limitations
HopChain data provides broad, generalizable gains in vision-language reasoning across diverse benchmarks without targeting any specific downstream benchmark.
Takeaways & Limitations
The current pipeline depends on successful instance segmentation and excludes images with no detectable SAM3-segmentable instances.
Abstract
from arXiv · showhide
Vision-language models (VLMs) show strong multimodal capabilities but still struggle with fine-grained vision-language reasoning. We find that long chain-of-thought (CoT) reasoning exposes diverse failure modes, including perception, reasoning, knowledge, and hallucination errors, which can compound across intermediate steps. However, most existing vision-language data used for reinforcement learning with verifiable rewards (RLVR) does not involve complex reasoning chains that rely on visual evidence throughout, leaving these weaknesses largely unexposed. We therefore propose HopChain, a scalable framework for synthesizing multi-hop vision-language reasoning data for RLVR training of VLMs. Each synthesized multi-hop query forms a logically dependent chain of instance-grounded hops, where earlier hops establish the instances, sets, or conditions needed for later hops, while the final answer remains a specific, unambiguous number suitable for verifiable rewards. We train Qwen3.5-35B-A3B and Qwen3.5-397B-A17B under two RLVR settings: the original data alone, and the original data plus HopChain's multi-hop data, and compare them across 24 benchmarks spanning STEM and Puzzle, General VQA, Text Recognition and Document Understanding, and Video Understanding. Although this multi-hop data is not synthesized for any specific benchmark, it improves 20 of 24 benchmarks on both models, indicating broad and generalizable gains. Consistently, replacing full chained queries with half-multi-hop or single-hop variants reduces the average score across five representative benchmarks from 70.4 to 66.7 and 64.3, respectively. Notably, multi-hop gains peak in long-CoT vision-language reasoning, exceeding 50 points in the ultra-long-CoT regime. These experiments establish HopChain as an effective, scalable framework for synthesizing multi-hop data that improves generalizable vision-language reasoning.
1 Introduction
HopChain addresses compounding long-CoT vision-language reasoning failures by synthesizing logically dependent, visually grounded multi-hop data for RLVR. The framework uses a scalable construction pipeline and produces broad gains across diverse benchmarks.
- Motivation: Long-CoT reasoning exposes compounding perception, reasoning, knowledge, and hallucination failures that existing vision-language RLVR data does not adequately address.Intermediate errors can persist through otherwise coherent reasoning and produce incorrect final answers.
- Approach: HopChain queries terminate in specific, unambiguous numerical answers, making their rewards directly verifiable while preserving dependence on intermediate reasoning.The chained structure usually requires intermediate steps to be correct to obtain the correct final number.
- Approach: The synthesis pipeline identifies categories, segments instances, generates chained queries, and verifies annotations with multiple annotators and difficulty calibration.Only queries whose independent solvers agree on the final numerical answer are retained as valid training examples.
- Approach: HopChain creates multi-hop queries in which earlier hops establish instances, sets, or conditions required by later hops, forcing repeated visual re-grounding.The framework formalizes perception-level and instance-chain hops for long-CoT RLVR training.
2 Preliminaries
RLVR for VLMs optimizes verifiable-answer rewards from image-query inputs and generated responses. SAPO replaces hard clipping with a temperature-controlled soft gate to address instability and inefficiency.
- Reinforcement Learning with Verifiable Rewards: RLVR for VLMs conditions policy-generated textual chain-of-thought and answers on an image and text query, with rewards based on answer equivalence.The sampled image, query, and ground-truth answer come from dataset D, while the response is generated by policy π.
- Soft Adaptive Policy Optimization: SAPO substitutes hard clipping in prior RLVR algorithms with a temperature-controlled soft gate.The method is introduced to mitigate potential instability and inefficiency associated with hard clipping.
- Soft Adaptive Policy Optimization: SAPO uses token-level policy ratios and advantages while distinguishing positive and negative token temperatures.The current policy, old rollout policy, token indices, and temperature parameters are defined in the optimization notation.
3 Diverse Failure Modes in Long Chain-of-Thought Reasoning
Long-CoT vision-language reasoning requires repeated visual grounding, but errors in perception, reasoning, knowledge, and hallucination can propagate through the chain. Multi-hop training is motivated by this broad, coupled failure structure.
- Failure modes: Long-CoT reasoning is fragile because models must repeatedly recover visual evidence and use it to determine subsequent reasoning steps.The failure modes include perception, reasoning, knowledge, and hallucination errors that can emerge along the chain.
- Error analysis: Perception errors are the largest sampled failure category, while reasoning, knowledge, and hallucination errors also occur.The breakdown analyzes 20 incorrect responses per benchmark from Qwen3.5-397B-A17B under RLVR w/o Multi-Hop.
- Qualitative examples: Representative failures include miscounting details, misjudging relations, misreading signs, charts, diagrams, and illustrated body parts.These examples span natural images, charts, and scientific diagrams.
- Error propagation: A faulty intermediate visual judgment can trigger downstream reasoning or unsupported-inference errors, whereas multi-hop training more often recovers correct evidence at each hop.The qualitative comparison contrasts RLVR w/o Multi-Hop with RLVR w/ Multi-Hop.
- Implication: The central challenge is maintaining reliable, step-wise reasoning over visual evidence across diverse scenarios, not merely producing longer textual CoT.The analysis connects this challenge to perception quality, repeated image-grounded observation, and revisiting visual regions.
4 Boosting Vision-Language Generalization by Synthesizing Multi-Hop Data
HopChain synthesizes multi-hop queries that tightly link visual instances and reasoning operations, forcing repeated image grounding while preserving numerically verifiable answers. Its pipeline combines automated generation with human verification and difficulty filtering.
- Query structure: HopChain defines Level 3 queries as chains combining single-object and multi-object perception steps with logically dependent hops.Earlier hops establish the instances, sets, or conditions required by later hops, and both perception-level and instance-chain links are used.
- Query constraints: Each synthesized query must combine both hop types, remain answerable from the original image, and terminate in a specific numerical answer.Queries must avoid references to segmentation masks, bounding boxes, and patch images that could provide auxiliary shortcuts.
- Data synthesis pipeline: The synthesis pipeline identifies image categories, localizes concrete instances with SAM3, and generates queries from combinations of 3–6 instances.Qwen3-VL-235B-A22B-Thinking identifies categories and generates questions using the image and design-time instance crops.
- Data synthesis pipeline: A representative six-hop query chains object localization, attribute counting, text reading, and arithmetic into one final-number question.The example links sheep and doll observations with a reference value, letter count, and toy-figure count.
- Ground-truth annotation and difficulty calibration: Stage 4 retains queries only when four annotators agree on the final numerical answer and removes those a weaker model solves with 100% accuracy.The resulting dataset contains verified ground truths and calibrated difficulty.
- Ground-truth annotation and difficulty calibration: The workflow scales from raw images to verified multi-hop queries while enforcing hop-wise dependency and repeated visual re-grounding beyond shallow RLVR paths.Human and model-based verification provide quality control for broad image collections with sufficient detectable instances.
5 Experiments
Experiments evaluate HopChain across model scales, benchmark families, training-query variants, response lengths, difficulty levels, and error types. Multi-hop augmentation produces broad gains, with full chained queries outperforming shortened variants and benefits remaining strongest for long reasoning.
- Experimental Setup: HopChain is evaluated across 24 benchmarks spanning STEM and puzzle reasoning, general VQA, text and document understanding, and video understanding.The experiments use Qwen3.5-35B-A3B and Qwen3.5-397B-A17B under multiple RLVR settings.
- Main Benchmark Results: 20 out of 24 benchmarks improve for both Qwen3.5-35B-A3B and Qwen3.5-397B-A17B when HopChain data augments the original RLVR data.Gains span all four benchmark families and are not targeted to any specific benchmark.
- Main Benchmark Results: 70.4, 66.7, and 64.3 are the average scores for full multi-hop, half-multi-hop, and single-hop training across five representative benchmarks.The per-benchmark ordering is consistent: full multi-hop performs best, followed by half-multi-hop and single-hop.
- Analysis by Reasoning Length: Multi-hop training retains an advantage across response-length bins and is often stronger in the ultra-long-response regime.Figure 6 aggregates Qwen3.5-397B-A17B benchmark results by response token count.
- Error-Type Analysis: Corrected cases cover perception, reasoning, knowledge, hallucination, and diverse subtypes rather than a single narrow failure type.Subtypes include chart and text misreads, object misidentification, spatial and counting errors, and logic, math, temporal, and causal reasoning errors.
6 Related Work
Prior work established strong multimodal models, RL-based reasoning, compositional and multi-hop benchmarks, and multimodal data synthesis, while revealing persistent visual-grounding failures. HopChain differs by generating benchmark-agnostic RLVR data that explicitly forces repeated visual grounding.
- VLMs integrate visual encoders with language models, but object hallucination, visual illusion, and language-prior errors remain persistent.
- RL methods improve language-model alignment and can refine vision-to-reasoning alignment, yet VLMs may retain weak visual grounding and textual over-reliance.
- HopChain generates a benchmark-agnostic proxy task that forces repeated visual grounding and targets broad, generalizable improvements beyond pre-existing task or benchmark data.
- Compositional visual reasoning, multi-hop question answering, and multimodal CoT provide established settings for chaining evidence across objects, relations, or passages.
- Existing multimodal synthesis pipelines combine VLMs with vision foundation models, extending earlier language-only instruction-data scaling strategies.
7 Conclusion
HopChain addresses compounding long-CoT vision-language errors with chained, visually grounded RLVR data and produces broad benchmark gains. Its current synthesis workflow excludes images without detectable, segmentable instances.
- 20 of 24 benchmarks improved on both evaluated Qwen3.5 models after adding HopChain-synthesized multi-hop data to the original RLVR data.The gains were benchmark-agnostic and accompanied by stronger long-CoT reasoning, preserved full-query benefits, and broad error correction.
- HopChain chains instance-grounded hops, repeatedly re-grounding reasoning in images while ending with specific numerical answers suitable for verifiable rewards.
- The pipeline excludes images with no detectable objects because successful instance segmentation is required for synthesis.The paper identifies complementary construction routes for such images as a next step.
A Full Prompt for Multi-Hop Query Design
The prompt specifies runtime-filled placeholders for synthesizing multi-hop vision-language reasoning queries. It is presented as the complete prompt used by the framework.
- Runtime placeholders include query-count, object-list, and target-hop-count fields that are filled during synthesis.
Multi-Hop Query Design Prompt (Full)
The query-design prompt requires self-contained, strictly multi-hop questions that chain perception levels and dependent instances while maximizing object coverage and preserving unambiguous references. It also constrains the available visual inputs and forbids patch or box references in query text.
- Multi-hop structure: Strict Level 3 queries chain multiple perception steps across perception levels and dependent instances, so each next instance depends on the previous one.
- Input constraints: Queries must use only the original image at answering time; instance patches are reference aids during question design and are unavailable for answering.
- Instance coverage: Queries must be designed for a specific object combination and should involve all or nearly all listed instances.
- Unambiguous references: Object references must be uniquely identifiable from the original image using spatial, contextual, functional, or visual attributes.
- Unambiguous references: Ambiguous references can make a question unsolvable or allow multiple valid interpretations.
- Core principles: Each query must remain self-contained and independent, with no reliance on another query’s information or answer.
- Multi-hop structure: The prompt requires genuine multi-hop chains whose hops are logically dependent, and rejects single-hop, overly simple, or instance-chain-free questions.
- Perception levels: Level 1 extracts single-object attributes, whereas Level 2 reasons over multiple objects and their relationships.
B Full Prompt for Image Filtering
The paper provides the complete prompt used to screen images before synthesizing multi-hop queries.
- The complete image-screening prompt is provided for the pre-synthesis filtering stage.
- The prompt is used before multi-hop query synthesis.
- The screening prompt supports image selection for the synthesis pipeline.
Image Filtering Prompt (Full)
The image-filtering prompt evaluates perceptual complexity and quality from a computer-vision perspective, identifies challenging objects or regions, and returns a strictly formatted JSON object. It emphasizes clear, recognizable content while treating technical degradation and uncountably dense scenes as low-quality cases.
- Image Filtering Prompt (Full): The prompt evaluates image complexity on a 1-to-10 scale, rates quality, and identifies objects or areas contributing to complexity.
- Image Complexity Factors: Complexity factors include occlusion, object count and density, and unusual poses or viewing angles.
- Image Quality Guidelines: The analysis focuses on clear, perceptible objects that are recognizable for standard computer-vision assessment.
- Image Quality Guidelines: Low quality covers blurry, noisy, severely overexposed, or underexposed images that make objects unrecognizable.
- Image Quality Guidelines: Clear but extremely dense scenes can also receive low quality when individual objects cannot be practically boxed or counted.
- Output Format Requirements: The output must be a single structured JSON object containing scores, quality, complexity analysis, and detailed complex-object entries.