PaperlayerView exact revision on arXiv

Source-linked AI summary

VISA: Agentic Self-Evolving Data Synthesis for Multimodal Instruction Following

Min Zeng, Guanxin Tan, Libin Cen, Yawei Wen, Rui Hu, Liuyang Bian, Xiaolong Chen, Xiaoxin Chen

arXiv:2608.26013v1cs.CL

TL;DR

Multimodal instruction synthesis needs accurate, diverse, verifiable, and challenging data, but one-pass pipelines often discard feedback from failures and model errors. VISA creates a self-evolving loop with image-aware constraint selection, adaptive memory, verification, recovery, and target-model probing. It improves instruction following while preserving general multimodal capability, and its verifier contracts also supply reinforcement-learning rewards without a separate reward model.

  • Problem

    Existing multimodal synthesis pipelines are largely one-pass systems that often discard feedback from failed samples, verifier outcomes, repeated templates, and target-model errors.

  • Method

    VISA uses a self-evolving agent loop that selects and discovers image-grounded constraints, verifies and repairs samples, probes target-model difficulty, and updates persistent memory.

  • Results

    VISA consistently improves multimodal instruction following while preserving general multimodal capability across seven public benchmarks.

  • Takeaways & Limitations

    VISA-SFT-15k surpasses the base model and data-synthesis baselines on average MM-IFEval, while VISA-RL-15k further improves the average score and general benchmark performance.

  • Takeaways & Limitations

    VISA adds overhead through state updates, iterative recovery, and target-aware probing compared with static one-shot pipelines.

Abstract

from arXiv · show

Multimodal instruction-following models require training data that is accurate, diverse, verifiable, and challenging. Existing synthesis pipelines typically follow a one-pass generate-and-filter paradigm, discarding feedback from failed samples, verifier outcomes, and target-model errors. We present VISA (Visual Instruction Synthesis Agent), an agentic framework that reformulates multimodal instruction synthesis as a self-evolving loop. At each round, VISA analyzes an image to filter incompatible constraints and discover new verifiable ones, samples diversity- and difficulty-aware constraint sets from persistent memory, generates candidate instructions, and verifies the resulting samples with executable tools and structured large language model judges. Failed samples trigger diagnostic-guided recovery, while accepted samples are probed against the target model to estimate difficulty. The resulting verifier signals and target-model failure profiles are written back to memory, allowing subsequent rounds to adaptively expand the constraint space, reduce template repetition, and focus on unresolved model weaknesses. The same verifier contracts further provide reward signals for reinforcement learning without a separately trained reward model. Experiments on MM-IFEval show that VISA consistently improves multimodal instruction following over strong baselines, while preserving general multimodal capability across seven public benchmarks.

1 Introduction

Multimodal instruction following requires data that satisfies diverse visual and textual constraints, but prevailing synthesis pipelines generate and filter samples in one pass. VISA replaces this stateless process with a self-evolving loop that uses verification and target-model feedback to produce more accurate, diverse, and challenging data.

  • Reliable multimodal instruction following requires satisfying constraints involving formatting, keywords, style, structure, and visually grounded reasoning.
  • Most existing synthesis systems discard or locally use feedback from failed samples, repeated templates, missing verifiers, and target-model errors.
  • VISA combines planning, execution, reflection, and persistent memory in a closed-loop synthesis framework.It verifies samples, repairs failures diagnostically, probes accepted samples for difficulty, and stores iteration outcomes for subsequent planning.
  • VISA steers later synthesis toward data that is more accurate, diverse, and challenging to the current target model without additional human annotation during self-evolution.
  • The framework unifies feedback-driven synthesis, verification and recovery, target-aware difficulty probing, and verifiable reward signals extendable to reinforcement learning.

2 Related Work

Prior work improves multimodal instruction following through synthetic data, structured constraints, hybrid verification, and iterative rewriting. VISA extends these directions by binding constraints to verifiers during synthesis and retaining reflection outcomes across rounds.

  • Existing multimodal synthesis methods scale diversity and response quality through large-scale generation, structured constraints, and hybrid verification.
  • Self-evolving instruction methods increase complexity through iterative rewriting or evolutionary generation.
  • VISA extends programmatic constraint verification from evaluation into synthesis by binding each constraint to a verifier at generation time.
  • Unlike agentic systems that operate in isolated episodes, VISA preserves reflection outcomes in persistent memory to reshape future planning.

3 Method

VISA formulates synthesis as image-conditioned constraint satisfaction and executes an iterative loop that filters and discovers visual constraints, selects diverse difficult instructions, verifies and repairs samples, and updates persistent memory.

  • Problem Formulation: Each synthesized sample pairs an image-grounded instruction and response with embedded constraints and a code-based or structured-LLM verification procedure.The constraint registry can expand when new types are proposed and validated during perception.
  • Perception: Perception prunes constraints that cannot be grounded in the current image and proposes new image-specific constraint types with candidate verifiers.This keeps downstream instructions visually meaningful while expanding the constraint space.
  • Planning & Execution: VISA samples constraints using persistent coverage and difficulty signals, while embedding-space selection reduces repetition among candidate instructions.The planner generates candidates first and passes only the selected instruction to response generation.
  • Reflection: Reflection uses executable tools for deterministic constraints and structured LLM judges for semantic or visual constraints, returning verdicts and localized diagnostics.Failed samples receive local edits, instruction rewrites, or constraint simplification instead of immediate discard.
  • Reflection: Accepted samples are probed against the target model to compute difficulty and identify per-constraint failures that indicate informative unresolved weaknesses.
  • Memory Update: Persistent memory stores accepted samples, constraint-verifier bindings, embeddings, diversity signals, and target-model failure profiles to guide every subsequent round.Later iterations shift sampling toward unresolved constraints, avoid redundant instructions, and incorporate newly discovered types.

4 Reinforcement Learning with Verifiable Rewards

VISA reuses its verifier contracts as reinforcement-learning rewards, replacing a separately trained reward model with constraint-based signals. The reward directly measures instruction adherence and can identify specific failure sources.

  • Reinforcement learning with verifiable rewards uses rule-based or programmatically checkable signals instead of separately trained reward models.
  • VISA evaluates each response against the bound verifier for every embedded constraint and defines reward as the resulting constraint pass rate.
  • The verifier-derived reward measures the fraction of instruction constraints followed and reuses verification logic already maintained by the synthesis agent.It can trace failures to formatting errors, missing keywords, or incorrect visually grounded descriptions.

5 Experimental Setup

VISA is evaluated by fine-tuning multimodal models on approximately 15k synthesized samples and measuring both multimodal instruction following and general capability. MM-IFEval reports compose-level and perception-level accuracy, while seven additional benchmarks assess broader perception and reasoning.

  • Dataset: VISA synthesizes approximately 15k samples, each containing an image, instruction, response, and associated verification method.
  • Models: Qwen3.5-27B serves as the synthesis agent, while Qwen3.5-4B is the target model for SFT and RL.
  • Training: Experiments fine-tune only the LLM backbone and alignment module, with frozen visual encoder layers, using 128 L40s GPUs and bfloat16 precision.
  • Evaluation: MM-IFEval measures compose-level constraints on outputs and perception-level cases grounded in input images, reporting C-Level, P-Level, and average accuracy.
  • Evaluation: General multimodal capability is evaluated on MMBench, MMStar, MM-Vet, HallusionBench, MathVista, OCRBench, and AI2D.

6 Results

VISA improves multimodal instruction following over the evaluated baselines and its components contribute through complementary recovery and constraint-selection mechanisms. The gains extend to general multimodal capability across seven benchmarks, with RL producing the strongest overall result.

  • Main Results: 63.9 average score is achieved by VISA-SFT-15k, surpassing the Qwen3.5-9B reference model’s 63.0 with less than half the parameters and 15k training samples.
  • Main Results: 64.9 average score is achieved by VISA-RL-15k on MM-IFEval, including a P-Level score of 59.0.
  • Main Results: P-Level evaluation depends partly on visual perception and grounding, yet VISA-RL-15k reaches 59.0, outperforming all 4B-scale baselines and approaching Qwen3.5-9B at 60.0.
  • Ablation Study: The static pipeline improves C-Level but degrades P-Level, whereas reflection raises average performance from 60.2 to 61.2 under SFT.
  • Ablation Study: The full VISA system reaches 67.2 C-Level, 54.0 P-Level, and 63.9 average, with reflection improving local sample quality and memory guiding global constraint selection.
  • General Capability Evaluation: 72.9 average score is achieved by VISA-RL-15k across seven general multimodal benchmarks, improving over the base model on five of seven benchmarks.

7 Analysis

VISA expands and analyzes a persistent constraint-verifier registry while producing diverse, jointly constrained, and difficulty-targeted instruction data. The registry grows rapidly before saturating, and target-model failures identify constraint types that remain challenging.

  • Dynamic Constraint Expansion: VISA expands a registry from 76 hand-crafted constraint types by discovering image-grounded types and binding each to code or structured-LLM verification.New types are validated before entering the registry.
  • Dynamic Constraint Expansion: Most discoveries occur in early rounds, after which the registry approaches saturation; the final 310 types are concentrated in visual-content and structural categories.Later additions are comparatively rare and focus less on text-only rules.
  • Diversity and Difficulty: All 90 off-diagonal category pairs appear in the synthesized data, while instruction embeddings form a continuous cloud with mean pairwise cosine distance 0.977.Object-grounded, counting, and spatial constraints cooccur with attribute in 85–92% of samples.
  • Diversity and Difficulty: 82.6% of samples are easy, 15.9% medium, and 1.5% hard, while verifier failures concentrate in counting (11.6%), lexical (10.2%), and format (8.1%) constraints.Approximately 2,730 medium or hard examples contain at least one verifier-detected failure.
  • Efficiency Trade-off: VISA adds synthesis-time computation, but parallel candidate generation and batched verification partly amortize this offline data-construction cost.State updates, recovery, and target-aware probing remain additional overheads relative to static pipelines.

8 Conclusion

VISA turns multimodal instruction synthesis into an adaptive, feedback-driven process using perception, planning, execution, reflection, and persistent memory. Experiments show improved instruction following while preserving broad multimodal capabilities.

  • Conclusion: VISA couples perception, planning and execution, reflection, and memory update to replace static generate-and-filter synthesis with adaptive feedback-driven data construction.Persistent memory records verifier outcomes, accepted samples, diversity signals, and target-model difficulty profiles.
  • Conclusion: VISA-SFT-15k surpasses the base model and data-synthesis baselines on average MM-IFEval, while VISA-RL-15k raises the average MM-IFEval score to 64.9.The verifier contracts also provide RL reward signals without a separately trained reward model.
  • Conclusion: VISA-RL-15k improves the average score across seven general multimodal benchmarks from 70.5 to 72.9, preserving broad perception and reasoning ability.The model improves on five of seven benchmarks.

Limitations

VISA prioritizes synthesis quality and difficulty adaptation over aggressive efficiency optimization, accepting additional overhead during offline data construction.

  • Limitations: State updates, iterative recovery, and target-aware probing introduce overhead compared with static one-shot pipelines, despite parallelizable synthesis stages.The authors regard this trade-off as acceptable for offline data construction.

Ethics Statement

Synthetic data generated from source images and MLLMs may inherit biases, unsafe associations, or privacy-sensitive content. VISA’s verification and reflection mechanisms are not complete safety or fairness filters, so deployment requires auditing and license compliance.

  • Ethics Statement: VISA data may inherit biases, unsafe associations, or privacy-sensitive content from source images and the synthesis model.The framework is intended for research and should be audited before deployment, especially in high-stakes settings.
  • Ethics Statement: Image-aware constraint selection, verification, and reflection improve constraint satisfaction but do not constitute complete safety or fairness filtering.Source images and generated artifacts must comply with underlying dataset licenses.

A Additional Analysis

VISA’s additional analysis describes an evolving constraint registry, image-conditioned planning and instruction composition, verifier contracts, and efficiency trade-offs in synthesis. The framework combines executable and LLM-based checks with bounded recovery and structured prompts to produce jointly satisfiable, verifiable samples.

  • Constraint registry: The final registry separates broad constraint categories from atomic constraint types, with “Seed” and “+Disc.” tracking initial and dynamically discovered types.The registry evolves through image-conditioned discovery, verifier binding, validation, and reuse across synthesis rounds.
  • Efficiency: Approximately 20 seconds per accepted sample is the reported average synthesis time, while 10,230 of 15,668 accepted samples (65.3%) pass verification without recovery.The implementation uses four candidate instructions per image, concurrency of 16, parallel generation and verification, and bounded recovery; a manual audit found approximately 95% agreement between LLM judges and human annotations.
  • Verification: Each code-verifier tool maps a response and typed parameters to a standardized pass/fail record, with metadata, schemas, and test cases checked before registry admission.The contract applies to both seed verifiers and verifiers attached to dynamically discovered constraint types.
  • Perception and discovery: Perception analyzes each image, filters unsupported registered types, and may propose new image-compatible constraint types with verifier information.The prompt returns a structured scene profile, identifies incompatible vision-grounded types, and proposes new types when existing parameterized types are insufficient.
  • Planning and composition: The planner selects a budgeted set of mutually consistent, jointly satisfiable, and verifiable constraints before the agent rewrites them as one natural-language instruction.Vision-grounded constraints refer to objects without revealing answers, while text-only constraints state the required response property directly.
  • Recovery: Coverage self-checks identify constraints missing from the composed instruction and return repair suggestions before full verification.A constraint counts as covered only when explicitly referenced, not merely implicit or conventional.
Loading 2608.26013v1…