Source-linked AI summary
Program-Verified Self-Evolution for Vision-Language Models
Ahmed Heakl, Sungik Choi, Moontae Lee, Salman Khan
TL;DR
Self-evolving vision-language models generate training questions without gold answers, so prior methods rely on imperfect proxy labels. VQS computes answers by running fixed programs over structured image parses and verifies the facts those programs read. It reduces answer errors and improves performance across ten benchmarks, outperforming the strongest self-evolving baseline at each scale.
Problem
Self-generated questions have no gold answers, and prior proxy labels can be wrong during self-evolution.
Source: §1 · p. 1Method
VQS parses images into structured records, uses fixed templates to generate questions and compute answers, and checks the visual facts the programs read.
Source: §1 · p. 2Results
VQS improves Qwen3-VL by up to 3.18 points across ten benchmarks, beats the strongest baseline at every scale, and reaches a 3.84-point gain at 2B after three cycles.
Source: §6 · p. 9Takeaways & Limitations
The same claim-level checks select training targets that improve the parser without labels.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Self-evolving vision-language models train on questions they generate from unlabeled images. Since these questions have no gold answers, prior methods label them by majority vote over sampled answers or by a model judge. In a human evaluation, we find that 24\% of majority-vote labels and 18\% of model-judge labels produced during self-evolution are wrong. To address this problem, we present Verifiable QA Generation for Self-Evolving Models (VQS), which changes how the model judges answers. Instead of voting on an answer, the model parses each image into a structured record, such as a scene graph, a chart table, or a diagram graph. Fixed programs then write a question from the record and compute its answer. The model still acts as a visual checker, but it only confirms the individual facts the program reads, one short claim at a time. These claim-level checks select the parser's training targets, so the parser also improves without labels. Human raters find 94\% of VQS answers correct, against 76\% for majority voting. Across ten benchmarks, VQS improves Qwen3-VL by up to 3.18 points at the 2B, 4B, and 8B scales and outperforms the strongest self-evolving baseline at each. Gains keep growing over three training rounds, reaching 3.84 points at 2B. Code is released at https://github.com/ahmedheakl/VQS
1 INTRODUCTION
Self-evolution lacks gold answers for its generated questions, leaving prior methods to rely on proxy labels that can be wrong. VQS computes answers from verified visual facts and improves benchmark performance across model scales.
1 INTRODUCTION
- Prior self-evolution methods rely on proxy labels, while answer verification remains difficult because gold answers are absent.Majority voting can reward a wrong answer when it is sampled most often, as Figure 2 illustrates.Source: §1 · p. 1, Figure 2 · p. 2
- Up to 3.18 points in average gain over ten benchmarks, VQS beats the strongest self-evolving method at every scale.It improves every benchmark shown for Qwen3-VL-2B.Source: Figure 1 · p. 1, §1 · p. 2
- 24% of majority-vote labels and 18% of model-judge labels were wrong in a human evaluation of generated questions.Prior methods use proxy labels because self-generated questions have no gold labels.Source: §1 · p. 1
- VQS parses images into structured records, uses fixed templates to generate questions and compute answers, and checks the underlying visual facts claim by claim.The claim-level checks also select parser training targets, allowing the parser to improve without labels.Source: §1 · p. 2
2 RELATED WORK
Prior self-evolving models use different reward proxies, while prior program-based question generation either relies on annotated or model-produced parses. VQS uses claim-level verification to select parses for label-free parser training.
2 RELATED WORK
- R-Zero, VisPlay, EvoLMM, and iReasoner use competing self-evolution strategies, and prior methods draw rewards from model outputs that can become less accurate during training.VisPlay’s estimated pseudo-label accuracy falls from 72% to 61% over three rounds, and R-Zero’s from 79% to 63%.Source: §2 · pp. 2–3
- CLEVR uses programs with rendered scene graphs, GQA uses human-annotated graphs, and ProVision and SpatialVLM generate from model-produced parses without verifying generated claims.VQS instead checks parse claims individually and retains high-scoring parses for parser training.Source: §2 · p. 3, Figure 3 · p. 3
3 METHOD
VQS builds questions and answers from structured image parses using fixed programs, while claim-level visual checks filter questions and select unlabeled parser-training targets. The solver is then trained on the retained questions using computed answers.
From images to questions
- Fixed domain schemas turn images into structured records, which templates traverse to generate questions, computed answers, and hop counts.The records capture objects and relations in photos, chart values and ticks, diagram nodes and arrows, or infographic text, numbers, and units.Source: §3 · pp. 3–4, Figure 4 · p. 5
Which questions enter the training data
- Questions are retained only after atomic fact checks, a text-only blind gate, and a difficulty filter based on the solver’s rollout correctness.The blind gate drops questions answerable without the image too often; the difficulty filter drops questions the solver always gets right or always gets wrong.Source: §3 · p. 4
Solver Training
- The solver receives an exact-match reward plus a format reward and is trained with GRPO using normalized group rewards and a clipped objective with a KL penalty.The reward weights exact answer matching at 0.9 and required answer format at 0.1.Source: §3 · pp. 4–5
Parser Training
- The parser is trained without labels by sampling parses, scoring the fraction of claims verified, keeping a sufficiently strong parse, and fine-tuning on retained targets.The best parse must beat the mean score of the sampled parses and meet a minimum claim count; retained image-parse pairs use cross-entropy training.Source: §3 · p. 5
4 EXPERIMENTAL SETUP
VQS is evaluated on ten benchmarks using Qwen3-VL models trained from unlabeled images, with no human questions, answers, or labels. It achieves the largest average gain at each of the three model scales.
Models and data
- Training uses Qwen3-VL-2B, 4B, and 8B Instruct models on Venkatraman et al. (2026) images, discarding the source questions, answers, and labels, with no human annotation at any stage.A target is retained when it beats the mean of all K parses by δ=0.10 and asserts at least m=4 claims; this turns 12,000 images into 4,792 targets.Source: §4 · pp. 5–6
Benchmarks
- The ten-benchmark evaluation uses lmms-eval and includes visual question answering, science, spatial, logic, and multimodal benchmarks.The listed benchmarks include GQA, OK-VQA, InfoVQA, ScienceQA, MMMU, MMBench, EmbSpatial, LogicVista, MMStar, and SEEDBench.Source: §4 · p. 5
Main Results
- +3.18 at 2B, +2.32 at 4B, and +2.43 at 8B are VQS’s average gains over each backbone’s base, exceeding the strongest published method’s +1.01, +1.04, and +0.66.The comparison spans ten benchmarks and three Qwen3-VL scales.Source: Table 1 · p. 6
5 RESULTS
VQS leads the self-evolving baselines across ten benchmarks, while human evaluation, parser tests, and multi-cycle training examine answer reliability and continued improvement.
Main Results
- +3.18 points over base at 2B, versus +1.01 for the strongest baseline; VQS lifts all ten benchmarks at each scale.At 2B, VQS gains +7.86 on MMMU versus +1.75 for the strongest baseline; VISE lowers LogicVista and MMStar scores while VQS raises both.Source: §5 · pp. 6–7
Model scale
- +3.18 at 2B, +2.43 at 8B; the share of benchmark answers changed by RL falls from 7.82% at 2B to 4.93% at 8B.All three model scales reach a training reward of about 0.80, but larger models transfer less of what they learn to benchmarks.Source: §5 · p. 7, Figure 5 · p. 7
Answer correctness
- 94.4% correctness among retained VQS questions, versus 76.4% for majority voting and 82.2% for a model judge; VQS retains 76%.Human raters compared the three sources on 500 shared questions across visual domains and template families.Source: §5 · p. 7, Table 2 · p. 7
Question paraphrasing
- The final question matches its program in 99.8% of cases at 2B, 100% at 4B, and 99.1% at 8B.All four 8B mismatches swap the two years in a yes/no comparison, making the stored answer wrong.Source: §5 · p. 7
Parser training
- Label-free parser training improves F1 by 5.0 points on GQA, 6.6 on ChartQA, and 7.4 on AI2D-RST.Per-claim target selection also beats whole-parse checking, raising parse precision from 85.0 to 87.6 and correct-answer accuracy from 94.0 to 96.1.Source: §5 · pp. 7–8, Table 3 · p. 7, Figure 6 · p. 8
Multiple cycles
- Average gain over base rises from 3.18 points after cycle 1 to 3.41 after cycle 2 and 3.84 after cycle 3.Nine of ten benchmarks improve in every cycle; the 2B third-cycle gain is led by ScienceQA (+8.53) and MMMU (+8.44).Source: §5 · p. 8, Figure 8 · p. 8
Filters and parser settings
- Removing parser training lowers solver accuracy from 62.4 to 58.0; removing fact-checking lowers it to 60.9.Raising candidate count or selection margin slightly improves parse precision, while raising the minimum claim count slightly lowers it.Source: §5 · pp. 8–9, Figure 7 · p. 8, Table 4 · p. 9
Questions and templates
- Gains grow from 1.9 points with 10 template families to 3.18 with all 70, while reasoning benchmarks gain more than perception benchmarks at every family count.Middle-difficulty families score 62.4, ahead of hard at 61.1 and easy at 60.8; within-family hop ordering reaches 62.4 versus 61.3 shuffled.Source: §5 · p. 9, Table 5 · p. 9
6 CONCLUSION
VQS uses structured image parses and fixed programs to generate questions and compute answers, while claim-level checks also select unlabeled parser training data. It improves Qwen3-VL across ten benchmarks and continues gaining over three cycles.
Question-design results
- Table 6 compares question-design choices at equal training budgets using solver accuracy averaged over ten benchmarks.Boldface marks the VQS default.Source: Table 6 · p. 9
Conclusion
- VQS computes answers with fixed programs over structured image parses and checks each fact those programs read.The framework covers photographs, charts, diagrams, and infographics; its checks also select parser training data without labels.Source: §6 · p. 9
- Across ten benchmarks, VQS improves Qwen3-VL by up to 3.18 points and beats the strongest baseline at every scale.At 2B, gains reach 3.84 points after three cycles.Source: §6 · p. 9
A APPENDIX
The appendix contents list supplementary results, settings, prompts, and qualitative examples.
Appendix contents
- The appendix includes training and evaluation settings, benchmark categories, and additional results.Source: A APPENDIX · p. 13
- It includes sections on checker independence and transfer to other model families.Source: A APPENDIX · p. 13
- Further materials cover image parsing, question rewriting, blind-gate and training prompts, model-written programs, qualitative examples, traced questions, and benchmark comparisons.Source: A APPENDIX · p. 13
A.1 TRAINING AND EVALUATION SETTINGS
The solver is trained with GRPO using shared settings across three scales, with micro-batch size varying by scale.
Solver training
- Solver training uses GRPO with a rank-64 LoRA adapter on linear layers, leaving vision modules frozen.Source: §A.1 · p. 13
Optimization
- Training uses AdamW at a constant 1e−5 learning rate, a KL penalty with β = 0.01, and 96 steps.Source: §A.1 · p. 13
Batching
- Each prompt has 8 rollouts; batches contain 256 prompts, with a global batch of 128 and 256-token response budget.Per-device micro-batch is 16 at 2B and 4 at 8B.Source: §A.1 · p. 13
A.2 BENCHMARK CATEGORIES
The ten benchmarks are grouped by the main skill each tests, and category and overall scores are computed as unweighted means.
A.2 BENCHMARK CATEGORIES
- Perception benchmarks test what is visible, including objects, relations, printed text, and spatial layout.Source: §A.2 · p. 13
- Knowledge & Reasoning benchmarks require information beyond the image, such as outside facts, subject knowledge, or logical reasoning.Source: §A.2 · p. 13
- General benchmarks mix visible-image questions with questions requiring outside information; category scores average their benchmarks equally, and the overall score averages all ten.Source: §A.2 · p. 13
A.3 MORE RESULTS
VQS transfers to three non-Qwen model families, improving all ten benchmarks in each. The checker audit finds that verification strength varies by model size and claim type.
A.3 MORE RESULTS
- 244 of 247 corrupted claims were rejected by VQS’s checker, while agreement with other checkers depended more on size than family.The two 8B checkers from different families agreed at κ=0.877, versus κ=0.837 for the 2B checker and Qwen3-VL-8B; SmolVLM2-2.2B accepted the most traps (5.7%).Source: §A.3 · p. 13, Table 9 · p. 15
- At least two other checkers rejected 26.7% of accepted diagram-arrow claims and 8.3% of chart-value claims, but none of the object-existence claims.The authors treat these disagreements as warning signs, not error rates, because another checker’s rejection is not ground truth.Source: §A.3 · p. 14
- VQS’s average gains on these three families were smaller than its +2.32 to +3.18 gains on Qwen3-VL.Source: §A.3 · p. 14
- +1.77 points on Gemma3-12B-It, +1.51 on InternVL3-8B-Instruct, and +1.84 on Llama-3.2-11B-Vision-Instruct; VQS improves all ten benchmarks for each family.Source: §A.3 · p. 14, Table 10 · p. 15
A.4 PROMPTS
The appendix prompts specify structured image parsing, constrained question rewrites, image-free answer checks, answer-format instructions, and the two answer-labeling baselines.
A.4 PROMPTS
- Brace-delimited fields such as {q} are Python format fields filled at call time; the baseline prompts are the only prompts that ask a model to vote on or judge an answer.Source: §A.4 · p. 15
A.4.1 PARSING AN IMAGE INTO A STRUCTURE
- Each domain uses a prompt paired with a constrained-decoding JSON schema, so valid JSON is enforced rather than requested only by instruction.The schemas require fields and disallow extra fields; illegible numeric values may be recorded as null and skipped by templates.Source: §A.4.1 · pp. 15–16
- Chart and infographic prompts record labelled data, preserving printed values while also supplying numeric values and units where applicable.Source: §A.4.1 · pp. 15–16
- Photo parses represent objects, attributes, boxes, and relations, while diagram parses represent labelled nodes and directed edges.The diagram prompt says to leave edges empty when no arrows are drawn and not to invent arrows.Source: §A.4.1 · pp. 15–17
A.4.2 REWRITING THE TEMPLATED QUESTION
- The model may rewrite a template question for natural wording, but cannot add or remove conditions or use words absent from the original question.The computed answer is neither shown nor changed during rewriting.Source: §A.4.2 · p. 18
A.4.3 THE BLIND GATE
- The blind gate asks the same model to answer the shipped question four times without seeing the image.Source: §A.4.3 · p. 18
A.4.4 TRAINING-TIME PROMPTS
- Training prompts pair the image and question with an answer-format suffix selected by the program’s answer type.Suffixes request a word or phrase, a number, or a multiple-choice option letter.Source: §A.4.4 · p. 18
A.4.6 BASELINE PROMPTS
- Both baselines start from the same free-form image-question prompt; majority voting keeps the plurality among five answers, while the judge returns a yes-or-no correctness verdict.Source: §A.4.6 · p. 19
A.4.5 MODEL-WRITTEN PROGRAMS (ABLATION)
The appendix contrasts model-written programs with fixed template executors, then illustrates generated questions, answer checks, and model responses on example images.
A.4.5 MODEL-WRITTEN PROGRAMS (ABLATION)
- In the model-written-program ablation, the model receives a JSON parse and writes a question, answer, and Python program; only program execution determines correctness.The program must compute from the JSON rather than hard-code the answer.Source: §A.4.5 · pp. 18–19
A.5 QUALITATIVE EXAMPLES
- The qualitative examples show real-image questions from twenty template families, with program-computed keys checked against their images.Source: §A.5 · p. 20, Table 11 · p. 20
A.5.1 TRACED QUESTIONS
- A traced chart question computes the 2019-versus-2018 Stayovers difference as 31,984, and its fact check confirms the read values and recomputed answer.Source: §A.5.1 · p. 21
A.5.2 THE PROGRAM BEHIND A TEMPLATE
- The ranked-arithmetic executor compares sums of the two highest and three lowest values, returning no question when values are insufficient or the sums tie.The fact check verifies the cells read by the executor.Source: §A.5.2 · p. 23
A.5.3 VQS, VISE AND THE BASE MODEL ON THE SAME BENCHMARK QUESTIONS
- On one shared benchmark question, the base model and VISE report 29,750, whereas VQS reports approximately 30,435 for Binders’ October–December revenue.These are example responses from the three models, not a stated comparison of their correctness.Source: §A.5.3 · pp. 23, 26