PaperlayerView exact revision on arXiv

Source-linked AI summary

VQ-VA World: Towards High-Quality Visual Question-Visual Answering

Chenhui Gou, Zilong Chen, Zeyu Wang, Feng Li, Deyao Zhu, Zicheng Duan, Kunchang Li, Chaorui Deng, Hongyi Yuan, Haoqi Fan, Cihang Xie, Jianfei Cai, Hamid Rezatofighi

arXiv:2511.20573v1cs.CV

TL;DR

Open-source models lag at answering visual questions with images, a capability emerging in proprietary systems. VQ-VA WORLD builds a large targeted training corpus and a human-curated benchmark. Fine-tuning LightFusion-World raises its IntelligentBench score well above prior open-source baselines and narrows the gap to proprietary leaders.

  • Problem

    Open-source models underperform on VQ-VA, and the authors hypothesize that standard editing datasets underrepresent free-form generation requiring knowledge and multi-step reasoning.

    Source: §1 · p. 1

  • Method

    VQ-VA WORLD uses an agentic pipeline to construct targeted training data and introduces IntelligentBench, a human-curated benchmark of knowledge and reasoning demands.

    Source: §1 · pp. 1–2

  • Results

    53.06 on IntelligentBench for LightFusion-World, versus 7.78 for LightFusion and 1.94 for UniWorld-V1, substantially narrowing the gap to proprietary leaders.

    Source: §1 · p. 3

  • Takeaways & Limitations

    VQ-VA data is associated with improvements on VQ-VA, reasoning-based image editing, and standard image editing tasks.

    Source: §4.4 · p. 8

Abstract

from arXiv · show

This paper studies Visual Question-Visual Answering (VQ-VA): generating an image, rather than text, in response to a visual question -- an ability that has recently emerged in proprietary systems such as NanoBanana and GPT-Image. To also bring this capability to open-source models, we introduce VQ-VA World, a data-centric framework built around an agentic pipeline for large-scale, targeted data construction. Leveraging web-scale deployment, this pipeline crawls a massive amount of ~1.8M high-quality, interleaved image-text samples for model training. For evaluation, we further release IntelligentBench, a human-curated benchmark that systematically assesses VQ-VA along the aspects of world knowledge, design knowledge, and reasoning. Training with VQ-VA World data yields strong empirical gains: it helps LightFusion attain 53.06 on IntelligentBench, substantially surpassing the best prior open-source baselines (i.e., 7.78 from vanilla LightFusion; 1.94 from UniWorld-V1), and significantly narrowing the gap toward leading proprietary systems (e.g., 81.67 from NanoBanana; 82.64 from GPT-Image). By releasing the full suite of model weights, datasets, and pipelines, we hope to stimulate future research on VQ-VA.

1. Introduction

VQ-VA—answering a visual question with an image—has emerged in proprietary systems while open-source models lag, a gap the paper hypothesizes is driven by data scarcity. VQ-VA WORLD builds targeted training data and IntelligentBench to support and assess this capability.

1. Introduction

  • Open-source models lag on VQ-VA, which the authors hypothesize reflects training data that underrepresents free-form generation requiring knowledge and multi-step reasoning.Frontier systems such as GPT-Image and NanoBanana exhibit the emerging ability to answer visual questions with images.Source: §1 · p. 1
  • VQ-VA WORLD uses a five-module agentic pipeline to retrieve image pairs, generate and filter questions, diversify wording, and produce reasoning traces.The questions condition on the first image and use the second as the answer; reasoning traces explain the knowledge and transformations involved.Source: §1 · pp. 1–2
  • The web-scale pipeline curates about 1.8M high-quality interleaved image-text samples across world knowledge, design knowledge, and reasoning domains.The world-knowledge domain includes scientific, spatial, temporal, and other real-world topics.Source: §1 · p. 2
  • IntelligentBench is a human-curated benchmark sourced from web-interleaved documents, with items designed to probe specific knowledge and reasoning demands.GPT-4o and Gemini-2.5-Flash [9] serve as automatic judges for large-scale evaluation.Source: §1 · pp. 2–3
  • 53.06 on IntelligentBench for LightFusion-World, versus 7.78 for LightFusion and 1.94 for UniWorld-V1, substantially narrows the gap to proprietary leaders.Figure 1 also shows that training with VQ-VA WORLD improves VQ-VA performance.Source: Figure 1 · p. 2, §1 · p. 3
  • The authors release model checkpoints, training and evaluation sets, and pipelines to support future open research on VQ-VA.Source: §1 · p. 3

2. Related Work

Prior image-to-image datasets and benchmarks largely target standard editing, while VQ-VA requires generating new images from knowledge and reasoning. IntelligentBench differs from nearby reasoning-oriented benchmarks through its semantic demands and real-world, human-verified examples.

2. Related Work

  • Image-to-image models include single I2I models, unified multimodal understanding-and-generation models, and leading proprietary models; existing editing datasets emphasize standard pixel-level edits and underrepresent tasks requiring external knowledge and multi-step reasoning.EmuEdit covers 7 fixed editing types, MagicBrushEdit 9 predefined tasks, and ImageEdit 14 tasks.Source: §2 · pp. 3–4
  • Existing image-to-image datasets target standard pixel-level edits, whereas VQ-VA WORLD targets entirely new images requiring real-world knowledge and reasoning.Earlier datasets include manually annotated, scalable, hybrid, and specialist-model-based collections, but their tasks remain centered on editing.Source: §2 · pp. 3–4
  • RISEBench [47] and KRIS-Bench [39] emphasize reasoning and world knowledge, but still primarily reward pixel-level edits and rely heavily on synthetic images.IntelligentBench instead uses real-world web content, with each item manually verified and paired with a genuine reference answer image.Source: §2 · p. 4

3. Methods

VQ-VA WORLD turns knowledge-rich web-interleaved documents into VQ-VA training samples through preprocessing and a five-worker pipeline. IntelligentBench evaluates knowledge and reasoning demands using human-curated examples.

3.1. VQ-VA World Data Framework

  • The two-stage framework produces about 1.8M samples, with preprocessing retaining knowledge- and design-related documents before the agent pipeline constructs examples.The dataset comprises 24.35% reasoning, 30.37% design knowledge, and 43.69% world knowledge; preprocessing combines LLM labels, a FastText classifier, and LLM refinement.Source: §3.1 · p. 4, Figure 2 · p. 5
  • The retriever selects knowledge- and reasoning-rich image pairs, and the instruction generator asks a free-form question about one image whose answer is the other.Questions can target temporal or causal relations, spatial structures, and scientific or analytical phenomena.Source: §3.1 · p. 5
  • Filtering retains only triplets scoring 6 across question quality, answer quality, and context dependence, while rewriting diversifies questions and reasoning adds a transformation trace.The reasoner’s trace is combined with the question image, question text, and answer image as a training quadruplet.Source: §3.1 · pp. 5–6
  • The training strategy first uses the full dataset for continued pretraining, then applies stricter filtering to retain about 500k samples for supervised fine-tuning.A separate set of about 100k temporally related samples is constructed with Seedance [11].Source: §3.1 · p. 6

3.2. IntelligentBench

  • IntelligentBench contains 360 unanimously approved human-curated examples across world knowledge, design knowledge, and reasoning, and uses GPT-4o as its default VLM evaluator.The domains contain 171, 88, and 101 examples, respectively; GPT-4o achieved 80.6% agreement with human ratings versus 73.1% for Gemini-2.5-Flash, while human agreement averaged 82.5%.Source: §3.2 · pp. 6–7

4. Experiments

The experiments evaluate VQ-VA, reasoning-informed editing, and standard image editing. VQ-VA World data yields strong gains on knowledge- and reasoning-intensive tasks, while standard editing gains are more modest.

4. Experiments

  • Evaluation covers IntelligentBench, RISEBench, KRIS-Bench, GEdit-Bench, and ImgEdit-Bench; training incorporates VQ-VA WORLD at a 25% sampling ratio and fine-tunes the model for 45k total steps across two stages.Training comprises 30k continued-training steps and 15k supervised fine-tuning steps; the original 45M LightFusion data is mixed in both stages.Source: §4 · p. 7

4.1. Results on VQ-VA

  • LightFusion-World achieves the best performance among fully open-source models on IntelligentBench, surpasses Qwen-Image, and narrows—but does not close—the gap to leading proprietary models.Scores are normalized to 0–100 by domain and averaged; items without a generated image score zero.Source: §4.1 · p. 7

4.2. Results on Reasoning-Based Image Editing Benchmark

  • On RISEBench, LightFusion-World is comparable to BAGEL-Think with far less training data, gains substantially over vanilla LightFusion, and outperforms some large-data-trained models; on KRIS-Bench, it beats every fully open-source competitor.Table 3 reports results for RISEBench and KRIS-Bench.Source: §4.2 · pp. 7–8, Table 3 · p. 8

4.3. Results on Standard Image Editing Benchmark

  • LightFusion-World delivers consistent but modest gains over LightFusion on GEdit-Bench-EN and ImgEdit-Bench, smaller than its gains on VQ-VA and reasoning-centric editing.Table 4 reports standard image-editing benchmark results, with higher scores better.Source: §4.3 · p. 8, Table 4 · p. 8

4.4. Summarized Results

  • LightFusion-World improves on VQ-VA, reasoning-informed editing, and standard editing, while gains are largest on VQ-VA and reasoning-centric editing.The authors also report that open-source models still struggle on VQ-VA despite progress on standard editing.Source: §4.4 · p. 8

5. Conclusion

The paper presents VQ-VA World as a data-centric effort to bring visual question-answering through image generation to open-source models. Its web-scale data and human-curated benchmark accompany substantial gains for LightFusion on IntelligentBench.

5. Conclusion

  • VQ-VA is described as an emerging capability previously seen exclusively in leading proprietary models.The paper frames its work as bringing this capability to open-source models.Source: §5 · pp. 8–9
  • VQ-VA World uses a scalable agentic pipeline to curate approximately 1.8 million high-quality training samples and is paired with the human-curated IntelligentBench benchmark.The paper releases code, data, pipelines, and model checkpoints.Source: §5 · p. 9
  • Fine-tuning LightFusion on BAGEL-World raises its IntelligentBench score from 7.78 to 53.06, surpassing existing open-source models and substantially narrowing the gap to proprietary leaders.The conclusion presents these results as motivation for further research on VQ-VA and multimodal systems that answer with images.Source: §5 · p. 9

Supplementary Material

The supplementary material provides additional benchmark results, pipeline prompts, and visual comparisons. It includes full IntelligentBench visualizations and a further qualitative comparison on RISEBench.

Supplementary Material

  • The supplement reports complete ImgEdit-Bench results by subdomain and provides prompts for the Retriever, Instruction Generator, Question Score, Answer Score, Context Dependence Score, Rewriter, and Reasoner.It also describes the prompt details of the VQ-VA World framework.Source: Supplementary Material · p. 12, Table 5 · p. 13, Table 6 · p. 14

5.4. Qualitative Comparison on RISEBench

Loading 2511.20573v1…