PaperlayerView exact revision on arXiv

Source-linked AI summary

HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning

Shenzhi Wang, Shixuan Liu, Jing Zhou, Chang Gao, Xiong-Hui Chen, Binghai Wang, An Yang, Shiji Song, Bowen Yu, Gao Huang, Junyang Lin

arXiv:2603.17024v2cs.CVcs.AIcs.CL

TL;DR

VLMs still face compounding errors in long-chain reasoning, while existing vision-language RLVR data is insufficient for chains requiring visual evidence throughout. HopChain synthesizes logically dependent, visually grounded multi-hop queries with verifiable numerical answers; adding this data improves 20 of 24 benchmarks on both evaluated models.

  • Problem

    Long-CoT vision-language reasoning faces diverse, compounding errors, and existing vision-language RLVR data is insufficient to address this challenge.

    Source: §1 · p. 1

  • Method

    HopChain synthesizes logically dependent chains of visually grounded hops that end in specific numerical answers verifiable for RLVR.

    Source: §1 · pp. 2–3

  • Results

    20 of 24 benchmarks improve on both Qwen3.5-35B-A3B and Qwen3.5-397B-A17B when HopChain data augments the original RLVR data.

    Source: §7 · p. 14

  • Takeaways & Limitations

    The results support multi-hop RLVR training as a way to strengthen long-CoT vision-language reasoning beyond benchmark-specific targeting.

    Source: §1 · p. 3, §5.2 · p. 10

    The synthesis pipeline depends on successful instance segmentation, excluding images with no detectable, segmentable objects.

    Source: §7 · p. 14

Abstract

from arXiv · show

Vision-language models (VLMs) show strong multimodal capabilities but still struggle with fine-grained vision-language reasoning. We find that long chain-of-thought (CoT) reasoning exposes diverse failure modes, including perception, reasoning, knowledge, and hallucination errors, which can compound across intermediate steps. However, most existing vision-language data used for reinforcement learning with verifiable rewards (RLVR) does not involve complex reasoning chains that rely on visual evidence throughout, leaving these weaknesses largely unexposed. We therefore propose HopChain, a scalable framework for synthesizing multi-hop vision-language reasoning data for RLVR training of VLMs. Each synthesized multi-hop query forms a logically dependent chain of instance-grounded hops, where earlier hops establish the instances, sets, or conditions needed for later hops, while the final answer remains a specific, unambiguous number suitable for verifiable rewards. We train Qwen3.5-35B-A3B and Qwen3.5-397B-A17B under two RLVR settings: the original data alone, and the original data plus HopChain's multi-hop data, and compare them across 24 benchmarks spanning STEM and Puzzle, General VQA, Text Recognition and Document Understanding, and Video Understanding. Although this multi-hop data is not synthesized for any specific benchmark, it improves 20 of 24 benchmarks on both models, indicating broad and generalizable gains. Consistently, replacing full chained queries with half-multi-hop or single-hop variants reduces the average score across five representative benchmarks from 70.4 to 66.7 and 64.3, respectively. Notably, multi-hop gains peak in long-CoT vision-language reasoning, exceeding 50 points in the ultra-long-CoT regime. These experiments establish HopChain as an effective, scalable framework for synthesizing multi-hop data that improves generalizable vision-language reasoning.

1 Introduction

HopChain targets compounding long-CoT vision-language errors with logically dependent, visually grounded training queries whose final numerical answers are verifiable. Its RLVR training yields broad benchmark gains across two models and multiple domains.

1 Introduction

  • Long-CoT reasoning can compound perception, reasoning, knowledge, and hallucination errors, while existing vision-language training data often lacks chains requiring visual evidence throughout.The paper argues that training data should structurally require visual evidence seeking at each reasoning step rather than relying on or simply expanding existing RLVR data.Source: §1 · pp. 1–3
  • HopChain combines perception-level and instance-chain hops so earlier steps establish instances, sets, or conditions needed later, requiring repeated visual re-grounding and ending in a verifiable numerical answer.The queries combine single-object perception and multi-object relationship reasoning with explicit dependency chains, making intermediate reasoning necessary to reach the final answer.Source: §1 · pp. 2–3
  • A four-stage pipeline identifies image categories, segments instances, generates chained queries, and retains queries only when independent annotators agree on the final numerical answer.Category identification and query generation use Qwen3-VL-235B-A22B-Thinking, while instance segmentation uses SAM3.Source: §1 · pp. 2–3
  • Adding HopChain data improves 20 of 24 benchmarks on both Qwen3.5-35B-A3B and Qwen3.5-397B-A17B versus RLVR on original data alone.The gains span STEM and Puzzle, General VQA, Text Recognition and Document Understanding, and Video Understanding, although the synthesized data targets no specific benchmark.Source: §1 · p. 3
  • Multi-hop gains exceed 50 accuracy points in the ultra-long-CoT regime on Qwen3.5-397B-A17B.Repeated sampling found more than half of queries were partially solved, with outcomes distributed relatively evenly from fully incorrect to fully correct; corrected error types closely followed the original error distribution.Source: §1 · p. 3
  • On five representative benchmarks, replacing full chained queries with half-multi-hop or single-hop variants reduces average score from 70.4 to 66.7 and 64.3, respectively, on Qwen3.5-35B-A3B.Source: §1 · p. 3

2 Preliminaries

The preliminaries define VLM RLVR as image-and-query-conditioned chain-of-thought generation with a verifiable answer, then introduce SAPO’s soft-gating alternative to hard clipping.

2.1 Reinforcement Learning with Verifiable Rewards for Vision-Language Models

  • VLM RLVR conditions a textual chain-of-thought and verifiable answer on both an image and a query, with the image, query, and ground-truth answer sampled from dataset D.The generated response is produced by policy π conditioned on the image and query.Source: §2.1 · p. 3

2.2 Soft Adaptive Policy Optimization

3 Diverse Failure Modes in Long Chain-of-Thought Reasoning

Long chain-of-thought reasoning requires repeated visual grounding, making it vulnerable to diverse intermediate errors that can compound into incorrect answers. The analysis motivates methods that sustain step-wise reasoning over visual evidence throughout the chain.

3 Diverse Failure Modes in Long Chain-of-Thought Reasoning

  • Long CoT requires repeatedly recovering visual objects, attributes, relations, or numeric evidence to guide each next step, so one flawed intermediate can derail the final answer.The remaining reasoning may still appear coherent while relying on faulty evidence.Source: §3 · p. 4
  • In sampled incorrect responses from Qwen3.5-397B-A17B under RLVR w/o Multi-Hop, perception errors were the largest failure group, alongside reasoning, knowledge, and hallucination errors.Annotators classified 20 incorrect responses per benchmark by comparing outputs with ground-truth answers.Source: §3 · p. 4
  • Examples across natural images, charts, and scientific diagrams show faulty visual judgments propagating into later reasoning, while RLVR w/ Multi-Hop more often recovers the correct evidence and final prediction.The examples include miscounts, relation and sign errors, chart misreading, and incorrect selections in diagrams.Source: §3 · p. 5, Figure 3 · p. 5
  • The analysis identifies reliable step-wise reasoning over visual evidence across diverse scenarios—not simply longer textual CoT—as the central challenge for improving long-CoT reasoning.HopChain is introduced to force repeated grounding and use each step's result to locate, verify, or constrain the next.Source: §3 · p. 5

4 Boosting Vision-Language Generalization by Synthesizing Multi-Hop Data

HopChain synthesizes multi-hop vision-language queries that require repeated grounding in visual evidence and end in unambiguous numerical answers suitable for RLVR.

4.1 Multi-Hop Vision-Language Reasoning Definition

  • Queries chain single-object and multi-object perception through both perception-level transitions and instance dependencies, so earlier hops determine later entities or conditions and the numerical answer.The definition rules out loosely connected or bypassable pseudo-multi-hop queries.Source: §4.1 · p. 6

4.2 Data Synthesis Pipeline

  • The pipeline identifies image categories, localizes instances with SAM3, then uses Qwen3-VL-235B-A22B-Thinking to generate queries from combinations of 3–6 instances.The model receives the original image and instance crops, with crops used only during query design.Source: §4.2 · p. 6
  • Generated queries use answerable instances, describe only spatial, contextual, or visual attributes, avoid mask, box, and patch cues, and require logically dependent hops.Earlier hops must establish the instances, sets, or conditions needed later, forcing repeated recovery and use of visual evidence.Source: §4.2 · p. 8

4.3 Ground-Truth Annotation and Difficulty Calibration

  • Four annotators independently solve each query; only unanimous numerical answers are retained, and queries a weaker model answers perfectly across eight samples are removed.Queries with ambiguous references or other quality issues are discarded.Source: §4.3 · p. 8
  • The workflow scales from raw images to verified multi-hop queries for broad image collections with sufficient detectable instances, using human and model-based quality control.Its hop-wise dependencies enforce repeated visual re-grounding beyond shallow or loosely coupled paths typical of existing RLVR-style data.Source: §4.3 · p. 8

5 Experiments

HopChain augmentation improves benchmark performance across two model scales, while full hop chains outperform truncated alternatives. Gains also extend to long responses, varied query difficulties, and diverse error types.

5.1 Experimental Setup

  • Experiments compare pre-RLVR, original-data-only RLVR, and RLVR with HopChain data for two Qwen3.5 models across 24 benchmarks in four families.HopChain uses filtered images and difficulty filtering to construct about 6k–8k multi-hop RLVR samples per model.Source: §5.1 · pp. 8–10

5.2 Main Benchmark Results

  • 20 of 24 benchmarks improve for each model when HopChain data is added, with gains across all four benchmark families and 5 of 6 video tasks.The evaluation covers Qwen3.5-35B-A3B and Qwen3.5-397B-A17B; the synthesized data targets no specific benchmark.Source: Table 1 · p. 9, Table 2 · p. 10, §5.2 · pp. 10–11

5.3 Ablation on Hop Structure: Single-Hop vs. Half-Multi-Hop vs. Multi-Hop

  • 70.4 for RLVR w/ Multi-Hop falls to 66.7 for RLVR w/ Half-Multi-Hop and 64.3 for RLVR w/ Single Hop across five representative benchmarks.The same ordering holds individually on MathVision, MMMU Pro, RealWorldQA, ERQA, and VideoMMMU.Source: Figure 5 · p. 11, §5.3 · p. 11

5.4 Analysis

  • HopChain's advantage over RLVR w/o Multi-Hop persists across response-length bins and is often larger for ultra-long responses.Figure 6 analyzes Qwen3.5-397B-A17B, supporting gains in long-CoT reasoning rather than only short or easy responses.Source: §5.4 · p. 11, Figure 6 · p. 12
  • For both models, more than half of sampled multi-hop queries are partially correct, and query success rates span multiple buckets.Each query is measured from eight independently sampled responses, indicating broad difficulty coverage for RLVR training across model scales.Source: §5.4 · p. 12, Figure 7 · p. 12
  • Corrections span perception, reasoning, knowledge, hallucination, and other errors, including diverse visual and reasoning subtypes.Perception errors remain the largest corrected group and reasoning errors the second largest; subtypes include chart and text misreads, object misidentification, spatial and counting errors, and logic, math, temporal, and causal reasoning errors.Source: §5.4 · pp. 12–13, Figure 8 · p. 13

6 Related Work

Related work traces progress in VLMs, RL for language and vision-language models, visual reasoning, and multimodal data synthesis, while identifying persistent grounding challenges. HopChain is distinguished by benchmark-agnostic, real-image multi-hop data designed to encourage repeated visual grounding.

6.1 Vision-Language Models

  • Visual instruction tuning and scaled VLM architectures advanced multimodal understanding, but strong models remain vulnerable to hallucination and image-ungrounded errors during long reasoning (Liu et al., 2023b; Liu et al., 2025).The paper positions unreliable visual perception during multi-step reasoning as the bottleneck it targets.Source: §6.1 · p. 13

6.2 Reinforcement Learning for Language and Vision-Language Models

  • RL-based alignment progressed from RLHF/PPO to RLVR and VLM extensions, yet models retain weak visual grounding and often train on task-specific data (Ouyang et al., 2022; Shao et al., 2024; Zhao et al., 2026).The cited work also describes selective RL effects and visually grounded RL for document and video reasoning; HopChain instead generates a benchmark-agnostic proxy task for broad generalization.Source: §6.2 · pp. 13–14

6.3 Vision-Language Reasoning and Multi-Hop Reasoning

  • Unlike prior visual reasoning work, HopChain combines perception-level and instance-chain hops, uses multi-hop reasoning as a proxy for general VLM capability, and synthesizes data at scale on real images.Prior work includes compositional visual benchmarks and methods, multi-hop language QA, and multimodal CoT; other studies emphasize fine-grained observation, intermediate perception, and repeated image inspection.Source: §6.3 · p. 14

6.4 Scalable Data Synthesis for Model Training

  • Prior multimodal synthesis combines VLM detection with SAM/SAM2 segmentation or open-set detection to construct queries, whereas HopChain creates benchmark-agnostic proxy tasks for repeated grounding.Earlier instruction-data synthesis includes Self-Instruct, Alpaca, and ShareGPT4V; many pipelines aim to approximate the target task distribution.Source: §6.4 · p. 14

7 Conclusion

HopChain synthesizes chained, visually grounded multi-hop data for RLVR and yields broad benchmark gains, but its current synthesis pipeline depends on instance segmentation.

7 Conclusion

  • Adding benchmark-agnostic HopChain data improves 20 of 24 benchmarks on both Qwen3.5-35B-A3B and Qwen3.5-397B-A17B.Source: §7 · p. 14
  • HopChain addresses compounding long-CoT failures by having earlier hops establish instances, sets, or conditions for later hops, ending in specific numerical answers for verifiable rewards.Source: §7 · p. 14
  • Further analyses find that full chained queries matter, multi-hop data strengthens long-CoT reasoning, and the data spans broad difficulty while helping models correct diverse errors.Source: §7 · p. 14
  • The current pipeline excludes images without detectable SAM3-segmentable instances; future work should add complementary construction routes while retaining chained visual grounding.Source: §7 · p. 14

Appendix · A Full Prompt for Multi-Hop Query Design

The appendix provides the complete prompt used to synthesize multi-hop vision-language reasoning queries. Runtime-filled placeholders include {num_queries_word}, {object_list}, {num_queries}, and {target_hop_count_info}.

A Full Prompt for Multi-Hop Query Design

Multi-Hop Query Design Prompt (Full)

The prompt defines how to design high-difficulty, image-grounded queries that chain perception and instance reasoning over supplied objects into deterministic numerical answers, using uniquely identifiable visual references and objective criteria.

Multi-Hop Query Design Prompt (Full)

  • Each query must form a logically dependent chain across single-object and multi-object perception, with every hop required for the next.The next instance must be found using information about or a relationship to the previous instance; queries are independent of one another.Source: Multi-Hop Query Design Prompt (Full) · pp. 19–21
  • Queries must yield a specific, unambiguous number using only image evidence, with precise instance references and objective, deterministic criteria.The original image is available when answering, but instance patches are design aids only; query text must not refer to patches, boxes, or detection markers.Source: Multi-Hop Query Design Prompt (Full) · pp. 19–21
  • Every query must meaningfully involve all instances in the supplied combination, rather than using only a few when more are available.Each referenced instance must be listed in the output's involved_objects field.Source: Multi-Hop Query Design Prompt (Full) · p. 20
  • Each query must meet its specified hop count and combine nontrivial visual perception with chained instance reasoning and challenging logic.The prompt encourages operations such as set reasoning, conditional inference, counting under constraints, comparisons, and numerical computations.Source: Multi-Hop Query Design Prompt (Full) · pp. 20–21
  • The query set should cover color, spatial reasoning, counting, text and symbol recognition, and logical inference while varying reasoning paths.Conditional checks should have a balanced mix of true and false outcomes, and query wording must not disclose image text or numbers the model must recognize.Source: Multi-Hop Query Design Prompt (Full) · pp. 21–22

Examples of Well-Designed Multi-Hop Queries

  • A five-hop example chains sign reading, vehicle selection, distance comparison, conditional counting, and arithmetic across all five listed objects.The query multiplies the speed-limit value by a condition-dependent object count, then subtracts the wheels visible on the car and bicycle.Source: Multi-Hop Query Design Prompt (Full) · p. 24
  • Instance references must distinguish each object using multiple visual or spatial features, while spatial relations and counting rules must use explicit, measurable criteria.The prompt illustrates this with a uniquely described car and a count based on object centers within a specified pixel distance of its edge.Source: Multi-Hop Query Design Prompt (Full) · pp. 25–26
  • The prompt rejects trivial single-object queries, short or weak chains, independent parallel counts, ambiguous references, and subjective or undefined criteria.For example, “nearby” lacks a clear threshold, whereas a fixed distance from a defined object edge makes the criterion deterministic.Source: Multi-Hop Query Design Prompt (Full) · p. 25

Final Checklist Before Outputting

  • Each sub-query should meet the target hop count, cover all combination instances, require each hop for the next, include nontrivial perception and logic, and yield a computable number.The checklist also excludes detection-border references and requires queries to be answerable from the original image alone.Source: Multi-Hop Query Design Prompt (Full) · p. 26

B Full Prompt for Image Filtering

This section provides the complete prompt used to screen images before multi-hop query synthesis.

B Full Prompt for Image Filtering

Image Filtering Prompt (Full)

The prompt instructs an image analyst to score perceptual complexity and quality, identify the contributing objects or areas, and return the assessment as JSON. It defines complexity factors and distinguishes low-quality images from difficult but usable scenes.

Image Filtering Prompt (Full)

  • Rate overall perceptual complexity from 1 to 10, assess quality, and identify the objects or areas that contribute to complexity.The response must be a single JSON object with no explanatory text outside it.Source: Image Filtering Prompt (Full) · p. 27
  • Return JSON fields for the integer complexity score, quality rating, brief complexity analysis, and complex objects with generalized names and reasons selected from the listed factors.The output format requires no additional text.Source: Image Filtering Prompt (Full) · pp. 27–28
  • Assess complexity using occlusion, object count and density, unusual poses or angles, complex interactions, fine-grained recognition, and challenging lighting or shadows.The analysis should focus on clear, perceptible objects.Source: Image Filtering Prompt (Full) · p. 27
  • Rate unusable images as Low quality and assign a low complexity score of 1–3, whether unusability stems from technical flaws or impractical annotation of vast, uncountable objects.Technical flaws include blur, noise, or extreme exposure that makes objects unrecognizable; clear but excessively dense scenes can also lack verifiable ground truth.Source: Image Filtering Prompt (Full) · p. 27

Related papers

Loading 2603.17024v2…