SIMIT: Self-Improving Vision-Language Models via Imagination at Test-Time

1 INTRODUCTION

• Test-time scaling can improve frozen-model predictions, while training methods update weights but may reinforce errors through supervision from the behavior being corrected.Open-ended multimodal answers complicate voting and require fine-grained visual verification; retrieval-based supervision requires labeled examples.Source: §1 · p. 1
• SIMIT synthesizes query-conditioned question–answer–image triplets with the model itself, combining native image generation and structured rendering skills without target labels or external models.It also checks consistency, filters by expected difficulty, and allocates synthesis budgets according to zero-shot confidence.Source: §1 · p. 2
• The synthetic samples support in-context learning, single-query or test-set fine-tuning, and a fine-tuning-plus-ICL combination.Test-set fine-tuning can benefit multiple test queries, whereas single-query adaptation treats each query separately.Source: §1 · p. 2
• 7.20% mean relative gain over zero-shot is reported for SIMIT-FT→ICL, which outperforms prior work across 17 benchmarks.SIMIT-ICL gains 6.98%, single-query SIMIT-FT 1.69%, and test-set SIMIT-FT 6.39%.Source: §1 · p. 2

2 RELATED WORK

• Test-time scaling uses longer reasoning or candidate selection, but leaves model parameters unchanged and may reinforce biases or lack consensus on open-ended tasks.Sequential reasoning and revision include chain-of-thought (Wei et al., 2022) and Self-Refine (Madaan et al., 2023); parallel selection includes Self-Consistency (Wang et al., 2023) and best-of-K (Cobbe et al., 2021).Source: §2 · pp. 2–3
• Label-free reinforcement-learning methods update parameters using consensus, response statistics, semantic clustering, or self-judging rewards, while SIMIT specifies answers before constructing and verifying visual inputs.The cited approaches include TTRL (Zuo et al., 2025), TTRV (Singh et al., 2026), EMPO (Zhang et al., 2026), and SRLM (Yuan et al., 2024).Source: §2 · p. 3
• Test-time training draws on retrieval, augmented demonstrations, or unlabeled-input objectives; SIMIT instead builds complete multimodal examples without a labeled retrieval corpus or supplied demonstrations.The closest cited comparison, TT-SI, fine-tunes on self-generated tool-use examples and validates calls.Source: §2 · p. 3

3.1 PROBLEM FORMULATION: TEST-TIME SELF-IMPROVEMENT

• The problem setting is target-conditioned and label-free: the target model is the only learned model available at test time, while deterministic tools may execute its specifications.Observed target queries determine what the model practices, without ground-truth answers or an auxiliary teacher, verifier, retriever, or generator.Source: §3.1 · p. 3

3.2 SELF-SUPERVISION BY IMAGINING SIMILAR PROBLEMS

• SIMIT constructs similar multimodal samples by proposing questions and intended answers before generating images that support those answers, rather than reusing the target prediction as supervision.The data-synthesis figure shows budget allocation, triplet generation, image realization, verification, and difficulty filtering before adaptation.Source: §3.2 · pp. 3–4, Figure 2 · p. 4
• The synthesis budget K is selected from zero-shot confidence, with less-confident queries receiving more samples and K decreasing monotonically as confidence rises.The allocation rule selects the smallest budget predicted to reach a desired correctness probability, subject to a maximum budget.Source: §3.2 · pp. 3–4
• Visualization routing sends natural-image requests to native generation and structured visuals to category-specific rendering skills that turn compact specifications into rendered images.The deterministic backend computes supported quantities, geometry, and layout and can provide explicit error feedback.Source: §3.2 · p. 4
• Verification checks image–description consistency and can trigger repair or regeneration, while difficulty filtering keeps examples whose intended-answer confidence falls within a selected band.The confidence band rejects examples that may contain residual label noise or exceed model capability, as well as examples with little learning value.Source: §3.2 · pp. 4–5

3.3 TEST-TIME UPDATE: USE CASES OF THE SELF-GENERATED SYNTHETIC DATA

• SIMIT-ICL uses synthetic examples as demonstrations; SIMIT-FT adapts weights per query or across the test set, and SIMIT-FT→ICL combines fine-tuning with demonstrations.Test-set fine-tuning lets supervision from one query benefit others; in-context learning requires inference access rather than parameter access.Source: §3.3 · p. 5

4.1 COMPARISON WITH STATE-OF-THE-ART TEST-TIME METHODS

• +7.20% mean relative gain over BAGEL-7B is achieved by SIMIT-FT→ICL, which outperforms state-of-the-art methods in every domain; training-free SIMIT-ICL gains +6.98%.SIMIT-ICL improves 16 benchmarks and ties one; Table 1 compares methods across five domains.Source: Table 1 · p. 6, §4.1 · p. 6

4.2 COMPARISON OF SELF-GENERATED SYNTHETIC DATA VS. REAL DATA

• For certain adaptation modes and domains, SIMIT’s synthetic data outperform real human-annotated data.The qualitative comparison notes that retrieved neighbors can be off-topic, while SIMIT’s demonstrations match the query subject.Source: §4.2 · p. 7, Figure 4 · p. 7

4.3 ABLATION STUDIES

• The full synthesis pipeline achieves the best ablation scores; adaptive budgeting and difficulty filtering improve efficiency and accuracy, while removing verification or diversity prompting reduces performance.Difficulty filtering alone can underperform without retuning its thresholds jointly with adaptive budgeting.Source: §4.3 · p. 8, Table 2 · p. 8
• Generating both questions and images broadens relevant visual content and question-answer relations around the target query, though some domains benefit from retaining the question.Fixed images restrict visual evidence, while unnecessary variation can mismatch synthetic examples and test queries.Source: §4.3 · p. 8, Table 3 · p. 8

4.4 WHY DOES SIMIT OUTPERFORM LFRL?

• SIMIT corrects more initially wrong queries than LFRL baselines, while consensus-based methods mainly stabilize majority-correct predictions.SIMIT corrects 14.4% of initially wrong queries and disrupts 2.0% of initially correct ones (∆=+12.4pp); TTRV corrects 9.3% at the same disruption rate.Source: §4.4 · p. 9, Figure 6 · p. 9
• SIMIT’s supervision has a lower error rate and weaker estimated alignment with base-model failures than LFRL supervision.Its supervision-error rate is 12% versus 33–38% for LFRL, and its δ is −6.5pp versus +22 to +79pp for LFRL.Source: §4.4 · p. 9, Figure 7 · p. 9

• SIMIT specifies answers before constructing corresponding images, enabling learning without first solving the target test query.Source: §5 · p. 9
• Combining finetuning with ICL achieves a +7.20% mean gain over the BAGEL-7B baseline without annotations or extra models.Source: §5 · p. 9
• The approach turns test-time compute into targeted practice, offering a route to more accurate automatic annotation and personalized agents learning from encountered tasks.Source: §5 · p. 9

• The authors release code so researchers can reproduce the findings and build on the work.Source: REPRODUCIBILITY STATEMENT · p. 10
• The experimental setup uses publicly accessible models, and the benchmarks are established and widely used standards.Source: REPRODUCIBILITY STATEMENT · p. 10
• Appendix G details prompts and hyperparameters, while Appendix A describes the benchmarks used.Source: REPRODUCIBILITY STATEMENT · p. 10

• Large language models were used solely for language polishing and coding assistance during paper preparation.Source: AUTHOR STATEMENT ON THE USE OF LARGE… · p. 10
• The authors state that LLMs were not used for research ideation, experiment design, analysis, or scientific writing.Source: AUTHOR STATEMENT ON THE USE OF LARGE… · p. 10
• Vision-language models are used in the method to generate synthetic training datasets.Source: AUTHOR STATEMENT ON THE USE OF LARGE… · p. 10

A DATASET STATISTICS

• Evaluation spans 17 benchmarks across five domains, with 50 validation and 500 evaluation examples per benchmark and benchmark-level relative improvements equally weighted in overall gains.Metrics vary by task: CIDEr for captioning, exact-match accuracy for VQA and AI2D, relaxed accuracy for ChartQA, ANLS for DocVQA and InfoVQA, and multiple-choice accuracy for MMBench and SEED-Bench.Source: A DATASET STATISTICS · p. 16

B EXTENDED RELATED WORK

• Unlike inference-only scaling, consensus-based training, or retrieval-dependent adaptation, SIMIT constructs and verifies multimodal practice examples with answers specified before images.The related-work taxonomy describes SIMIT as using no labels and adapting in context or weights, for single queries or across a test set.Source: Table 5 · p. 17, B EXTENDED RELATED WORK · p. 18

C FROM TARGET-RISK MINIMIZATION TO SYNTHESIS PRINCIPLES

• SIMIT’s synthesis principles target accuracy, learner-relative complexity, and diversity constrained to the test-query domain.The framework links accuracy to reduced supervision error, diversity to reduced finite-sample error, and target similarity to reduced distribution mismatch.Source: C FROM TARGET-RISK MINIMIZATION TO… · pp. 19–20

D ADAPTIVE BUDGET ALLOCATION: DERIVATION

• Adaptive allocation selects the smallest demonstration budget predicted to meet a correctness threshold, using calibrated zero-shot confidence and a precomputed accuracy-budget relationship.When the estimated accuracy improvement is non-decreasing with budget, more confident queries receive fewer demonstrations.Source: D ADAPTIVE BUDGET ALLOCATION: DERIVATION · pp. 20–21
• The budget rule’s population-level prediction is not a per-query guarantee, and a single zero-shot confidence estimate avoids the decoding cost of repeated rollout-based estimation.The authors note that agreement can mislead on open-ended tasks and when systematic errors recur.Source: D ADAPTIVE BUDGET ALLOCATION: DERIVATION · pp. 20–21

E CONFIDENCE-BASED DIFFICULTY FILTERING

• Confidence-band filtering rejects examples that are too uncertain to trust or too easy to provide sufficient learning value, while accounting for adaptation cost and context capacity.Under the concrete assumptions of calibrated confidence and learning value v(p) = 1 − p, the positive-utility region is an interval.Source: E CONFIDENCE-BASED DIFFICULTY FILTERING · p. 22

F CONSENSUS-BASED LFRL UNDER SINGLE-QUERY ADAPTATION

• Consensus-based adaptation preserves the initial modal answer under exact KL-regularized updates, so an incorrect initial majority cannot be corrected by selecting the adapted model’s most probable answer.The result concerns episodic, isolated single-query adaptation; for population majority-vote rewards, an incorrect majority also suppresses the correct answer’s probability below its initial value. Shared-parameter cross-sample transfer is outside this result’s guarantee.Source: F CONSENSUS-BASED LFRL UNDER… · pp. 23, 24, 26
• A modal answer with initial probability at least 0.75 and 32 rollouts per round has at least 98% probability of remaining modal throughout ten adaptation rounds.For finite rollouts, the probability of a change is bounded by min{1, T(K −1) exp(−Nδ0²/2)}, where δ0 is the initial gap between the modal answer and its nearest competitor.Source: F CONSENSUS-BASED LFRL UNDER… · pp. 24–26

G IMPLEMENTATION DETAILS

• BAGEL-7B is the only learned model used throughout synthesis and adaptation, while parsers, renderers, a sandboxed interpreter, and a headless browser execute its specifications.Reported runtimes use a single NVIDIA H100 GPU and include synthesis, SIMIT-FT training where applicable, and evaluation.Source: G IMPLEMENTATION DETAILS · p. 26

G.1 IMPLEMENTATION DETAILS OF SIMIT

• SIMIT routes natural images to BAGEL’s native generator and structured visuals to rendering skills, then validates specifications and uses BAGEL as a visual critic to verify realized images.The prompts and skills are generic and shared across benchmarks; the skills encode rendering formats rather than task knowledge. Images passing verification score at least 50/100, and triplets that fail after up to five repair rounds are discarded.Source: §G.1 · pp. 27–28
• Breadth-first realization scheduling gives candidates an initial attempt before retries, reducing model calls per filled slot by 1.9–3.5× relative to depth-first scheduling across generation logs from four benchmarks.Source: §G.1 · p. 29
• SIMIT-ICL prepends generated examples to the test query, while single-query SIMIT-FT trains and resets a LoRA adapter per query; test-set SIMIT-FT pools examples into one adapter.Test-set finetuning updates only the language-model understanding expert, leaving the vision encoder, connector, and image-generation components frozen.Source: §G.1 · pp. 29–30

G.2 TTS BASELINES

• TTS baselines use longer thinking, majority selection across sampled responses, or iterative self-critique and answer revision.Self-consistency uses 8, 16, or 32 responses; self-refinement is evaluated at 1, 3, or 5 iterations.Source: §G.2 · p. 32

G.3 LFRL BASELINES

• +4.35%: with ABA and DF disabled, SIMIT still exceeds every tuned LFRL baseline, despite removing all seven synthesis hyperparameters.Baseline tuning uses the same search budget and 50 labeled validation examples per benchmark; these labels score configurations only, not demonstrations, training targets, or verification signals.Source: §G.3 · p. 33
• TTRL uses majority-vote pseudo-labels, EMPO promotes semantic concentration, TTRV rewards response frequency with an entropy term, and SRLM trains on self-judged preference pairs.Reported EMPO results use lexical-similarity soft matching; SRLM uses DPO on the highest- and lowest-scoring responses.Source: §G.3 · p. 33

Ablation Studies

• Ablations use SIMIT-FT (Test-Set), first removing ABA and DF, then individually removing DP, SL, or V while keeping LoRA settings and 150 fine-tuning steps fixed.Differences are therefore attributed to data construction and selection.Source: §G.4 · p. 33
• The full adaptive pipeline achieves the highest mean gain and a lower runtime than every ablated variant except w/o V.The fixed-budget comparison disables ABA and DF; the variants differ in synthesis strategy, and retained sample counts can vary after validation.Source: §G.4 · p. 34

Data Quality

• The external, anonymized triplet audit reports the percentage judged correct, excluding ambiguous items, and receives no feedback to synthesis or adaptation.The same protocol measures supervision correctness in Sec. 4.4.Source: §G.4 · p. 34

Stratified Gain Analysis

• Stratified Gain Analysis labels samples using benchmark-specific correctness rules and counts only transitions between confident wrong and correct buckets.For CIDEr benchmarks, scores ≥0.8 count as correct and scores <0.2 as wrong; intermediate scores are ambiguous and excluded. Benchmark probabilities are averaged equally.Source: §G.4 · pp. 34–35

Agentic Evaluation Protocol

• The triplet judge marks an item ambiguous only when a reasonable person genuinely cannot decide, and ambiguous items are excluded from the statistics.Difficulty or uncertainty alone is not sufficient reason to flag an item.Source: §G.4 · p. 36

H DETAILED PER-BENCHMARK RESULTS

• +7.20% is the combination’s highest mean relative improvement over the zero-shot baseline, although it does not lead every benchmark.SIMIT-ICL improves on 16 of 17 benchmarks and ties on one; compared with test-set SIMIT-FT, it leads on eight benchmarks while SIMIT-FT leads on nine.Source: H DETAILED PER-BENCHMARK RESULTS · p. 36
• Table 8 provides per-benchmark results for SIMIT variants, with each score’s adjacent small number giving its relative percentage improvement over BAGEL-7B.The breakdown complements domain-level averages by showing variation across individual benchmarks.Source: H DETAILED PER-BENCHMARK RESULTS · p. 36, Table 8 · p. 37

Experimental Setup

• With ABA and DF disabled, mean relative gain is 4.35%, compared with approximately 6.4% for the full pipeline; these fixed-count results are not full-pipeline performance.The comparison disables adaptive budgeting and difficulty filtering to fix sample counts.Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 37

Does Adding More Samples Consistently Help?

• Unfiltered synthetic ICL typically peaks at K = 23 and then plateaus or declines, whereas finetuning on the same underlying samples generally continues to improve.These curves disable ABA and DF and therefore do not represent SIMIT’s full scaling behavior; the proposed explanation for ICL’s pattern is not isolated by the curves.Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 38, Figure 9 · p. 38
• At Ntrain = 2000, synthetic data remains below real-data results on all six benchmarks with synthetic gains above 2%, recovering 7% to 79% of their real-data gains.The other 11 benchmarks have small gains, so these experiments establish neither a stable synthetic-to-real exchange rate nor whether larger budgets would close the gap.Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 38

Do Two Sources Benefit the Same Benchmarks?

• r = 0.67 pooled correlation across 3 × 17 observations shows that real- and synthetic-data gains are positively associated at matched sample counts.The setting-specific correlations are r = 0.79 for ICL, r = 0.65 for single-query finetuning, and r = 0.74 for test-set finetuning (p < 10^-6).Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 38, Figure 10 · p. 38

Real/Synthetic Gain Analysis

• r = 0.83 pooled residual correlation shows that benchmarks benefiting more than predicted from real data also tend to benefit more than predicted from synthetic data.The adjustment covers 12 benchmarks with fixed score ceilings; within-setting residual correlations are r = 0.79 for ICL, r = 0.93 for test-set finetuning, and r = 0.77 for single-query finetuning (all p < 0.01).Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 39

Where Do the Sources Differ?

• Synthetic data outperforms real data on Math & Science under both finetuning settings, while real data generally gains more, particularly on document and infographic tasks.Real-data gains are larger overall, but the ordering reverses on Math & Science for finetuning.Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 39
• Synthetic single-query adaptation has more consistent benefits than TTT-NN across benchmarks, with a smaller worst-case decline and wins over its real-data counterpart on more benchmarks than in the other settings.TTT-NN has large gains on some benchmarks and substantial regressions on others; SIMIT-FT (Single-Query) can also regress.Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 39

Implications

• Real examples provide authentic content but depend on pool coverage and retrieval quality, while synthetic examples can target queries and vary in difficulty but may contain realization errors or redundancy.The analyses interpret the sources as complementary rather than treating sample count alone as a measure of valid, informative supervision.Source: I SELF-GENERATED SYNTHETIC VS. REAL… · p. 39

Experimental Design

• The transfer study evaluates 50 source-target pairs, split evenly between same-domain and cross-domain pairs, and compares each transferred checkpoint with zero-shot and target-specific checkpoints.To control for source-data quality, both transfer settings reuse the same 14 source benchmarks and source-trained checkpoints.Source: J CROSS-DATASET GENERALIZATION · pp. 39–40

Cross-Dataset Transfer

• +4.3% average within-domain transfer and +0.5% cross-domain transfer are both positive, while target-specific finetuning averages +6.4%.Target-specific finetuning outperforms transferred checkpoints on most evaluated pairs.Source: Figure 12 · p. 39, J CROSS-DATASET GENERALIZATION · p. 39
• The results support target-conditioned data over data conditioned on a different benchmark, but do not directly evaluate fully generic, unconditioned synthesis.Synthetic datasets can still be useful outside their intended benchmark, especially for related benchmarks.Source: J CROSS-DATASET GENERALIZATION · p. 41

K CROSS-MODEL GENERALIZATION

• +2.21% mean relative gain on SenseNova-U1-8B and +3.97% on Lance-3B show SIMIT-ICL benefits beyond BAGEL-7B, with the largest gains in different domains.SenseNova-U1-8B gains most in VQA and image understanding and captioning; Lance-3B gains most in multi-discipline reasoning.Source: K CROSS-MODEL GENERALIZATION · pp. 41–42, Table 9 · p. 41
• Lance-3B, the weakest zero-shot model of the three, gains +3.97% overall, concentrated in Multi-discipline reasoning (MM-Bench CN +13.8%, MM-Bench EN +11.7%), while captioning barely moves.This suggests SIMIT-ICL’s benefit scales with zero-shot headroom on a given task, rather than model size or architecture, and generalizes beyond the model used to develop the method.Source: K CROSS-MODEL GENERALIZATION · p. 42

L LABEL-NOISE, MODEL-NOISE ALIGNMENT: ADDITIONAL ANALYSES

• +38.9pp net correct-minus-incorrect supervision on hard queries makes SIMIT the only method with a positive net rate there; every method nets positive on easy queries.δ alone is silent about queries with no positive supervision, so the audit also reports availability and query outcomes by base-model correctness.Source: L LABEL-NOISE, MODEL-NOISE ALIGNMENT:… · p. 43, Table 10 · p. 43, Figure 13 · p. 43

M SKILL LIBRARY ABLATION: ADDITIONAL SAMPLES

• Native image generation can produce structured visuals with incorrect text or relations, while deterministic rendering improves fidelity to specified content and structure.The examples include an unrelated portrait generated instead of a Gantt chart.Source: M SKILL LIBRARY ABLATION: ADDITIONAL… · p. 44, Figure 14 · p. 44

N QUALITATIVE RESULTS

• Self-generated samples can yield more accurate answers than the pretrained model in the selected qualitative comparisons.The examples compare retrieved real samples with SIMIT-ICL and SIMIT-FT (SQ), using ABA and DF disabled and a fixed sample budget of K = 2.Source: N QUALITATIVE RESULTS · p. 45, Figure 15 · p. 45
• Fine-tuning can absorb irrelevant details from mismatched retrieved samples, whereas in-context learning can disregard them.Examples include an unnecessary color attribute copied from a retrieved sample and a retrieved material label repeated as an art form.Source: N QUALITATIVE RESULTS · p. 45
• Synthetic supervision can contain unsupported attributes, yet both SIMIT variants still answer the target task correctly in the illustrated case.The synthetic answer calls an animal’s belly pink although it is not clearly visible; the passage says this noise need not reinforce the target prediction error or prevent successful adaptation.Source: N QUALITATIVE RESULTS · p. 45

O LIMITATIONS AND FUTURE WORKS

• SIMIT-FT→ICL uses the test-set fine-tuned checkpoint in the ICL procedure without additional hyperparameter tuning or dedicated training on ICL-formatted synthetic samples; both directions can improve accuracy further.Source: O LIMITATIONS AND FUTURE WORKS · p. 55
• SIMIT is described as orthogonal to LFRL, and combining the approaches could yield further gains in suitable domains.Source: O LIMITATIONS AND FUTURE WORKS · p. 55
• Query-conditioned transductive adaptation can specialize models to a target test set, but this specialization can reduce performance on other benchmarks.The passage suggests multi-teacher on-policy distillation across specialists as a possible route to a generalist model.Source: O LIMITATIONS AND FUTURE WORKS · p. 55