PaperlayerView exact revision on arXiv

Source-linked AI summary

Innovator-VL: A Multimodal Large Language Model for Scientific Discovery

Zichen Wen, Boxue Yang, Shuang Chen, Yaojie Zhang, Yuhang Han, Junlong Ke, Cong Wang, Yicheng Fu, Jiawang Zhao, Jiangchao Yao, Xi Fang, Zhen Wang, Henxing Cai, Lin Yao, Zhifeng Gao, Yanhui Hong, Nang Yuan, Yixuan Li, Guojiang Zhao, Haoyi Tao, Nan Wang, Han Lyu, Guolin Ke, Ning Liao, Xiaoxing Wang, Kai Chen, Zhiyu Li, Feiyu Xiong, Sihan Hu, Kun Chen, Yanfeng Wang, Weinan E, Linfeng Zhang, Linfeng Zhang

arXiv:2601.19325v1cs.CVcs.AI

TL;DR

Scientific multimodal tasks remain difficult, while existing approaches often depend on costly specialized data and opaque pipelines. Innovator-VL introduces a transparent, reproducible training framework and achieves 61.83% across 37 benchmarks, outperforming similar-scale compared models.

  • Problem

    Existing multimodal models, particularly open-source systems, still struggle with demanding scientific tasks, while many approaches require costly specialized data and opaque pipelines that limit reproducibility.

  • Method

    Innovator-VL uses a principled, fully transparent, end-to-end reproducible training framework emphasizing data efficiency, reproducibility, and balanced generalization.

  • Results

    61.83% average across 37 benchmarks, Innovator-VL-8B-Thinking achieves SOTA performance among compared similar-scale models spanning general, mathematical, and scientific domains.

  • Takeaways & Limitations

    The transparent, reproducible recipe provides a foundation for further community exploration of scientific multimodal models.

  • Takeaways & Limitations

    The scientific fine-tuning data are not yet publicly available because of ongoing data compliance review.

Abstract

from arXiv · show

We present Innovator-VL, a scientific multimodal large language model designed to advance understanding and reasoning across diverse scientific domains while maintaining excellent performance on general vision tasks. Contrary to the trend of relying on massive domain-specific pretraining and opaque pipelines, our work demonstrates that principled training design and transparent methodology can yield strong scientific intelligence with substantially reduced data requirements. (i) First, we provide a fully transparent, end-to-end reproducible training pipeline, covering data collection, cleaning, preprocessing, supervised fine-tuning, reinforcement learning, and evaluation, along with detailed optimization recipes. This facilitates systematic extension by the community. (ii) Second, Innovator-VL exhibits remarkable data efficiency, achieving competitive performance on various scientific tasks using fewer than five million curated samples without large-scale pretraining. These results highlight that effective reasoning can be achieved through principled data selection rather than indiscriminate scaling. (iii) Third, Innovator-VL demonstrates strong generalization, achieving competitive performance on general vision, multimodal reasoning, and scientific benchmarks. This indicates that scientific alignment can be integrated into a unified model without compromising general-purpose capabilities. Our practices suggest that efficient, reproducible, and high-performing scientific multimodal models can be built even without large-scale data, providing a practical foundation for future research.

1 Introduction

Scientific multimodal tasks remain challenging because they demand precise cross-modal understanding, multi-step reasoning, and domain knowledge, while existing solutions often require costly specialized data and opaque pipelines. Innovator-VL addresses these limitations with a transparent, reproducible, data-efficient framework that preserves broad multimodal capabilities.

  • Scientific tasks remain underexplored and especially difficult because they require precise understanding, rigorous multi-step reasoning, and cross-modal domain knowledge [Zhang et al., 2023].
  • Existing scientific multimodal approaches show promise but often depend on large-scale or highly specialized datasets, limiting reproducibility and accessibility.
  • Innovator-VL introduces a fully transparent, end-to-end reproducible training framework designed to improve data efficiency, reproducibility, and balanced generalization.
  • Innovator-VL achieves competitive performance across diverse scientific tasks using fewer than five million curated scientific samples without large-scale scientific-domain pretraining.
  • Innovator-VL also performs competitively on general vision, multimodal reasoning, and scientific benchmarks among similarly sized models, without compromising general-purpose capabilities.

2 Model Architecture

Innovator-VL uses a three-stage vision encoder–projector–language model architecture that integrates region-aware visual representations, token compression, and a scientific reasoning language model.

  • 2 Model Architecture: Innovator-VL follows a three-stage vision encoder–projector–language model architecture to bridge visual and textual modalities.
  • 2 Model Architecture: RICE-ViT captures holistic and localized visual cues through a specialized Region Transformer layer within a single forward pass.It is designed for region-aware representation learning, unlike CLIP [Radford et al., 2021] and SigLIP [Zhai et al., 2023, Tschannen et al., 2025], which primarily focus on global patch interactions.
  • 2 Model Architecture: PatchMerger learns to compress many Vision Transformer patch embeddings into fewer representative tokens, reducing downstream sequence length, computational complexity, and memory usage.The module addresses the quadratic cost of processing dense visual tokens.
  • 2 Model Architecture: Qwen3-8B-Base [Yang et al., 2025a] serves as Innovator-VL’s language modeling and reasoning component for scientific multimodal tasks.Its strengths include STEM, logical reasoning, and long-context understanding, alongside open-source availability and mature tooling support.

3 Pre-training

Innovator-VL uses two multimodal pre-training stages: projector-based language-image alignment followed by full-parameter high-quality mid-training. The design avoids additional scientific-text pre-training and instead emphasizes efficient knowledge injection while preserving broad visual capabilities.

  • 3 Pre-training: Pre-training comprises language-image alignment followed by high-quality mid-training with full-parameter training across all modules.The first stage trains the projector, while the second updates all modules to inject new knowledge efficiently.
  • 3 Pre-training: The alignment stage trains the projector on LLaVA-1.5 558k [Liu et al., 2024b] to map visual features into the LLM’s word-embedding space.
  • 3 Pre-training: The mid-training stage uses approximately 85 million high-quality image-text pairs from LLaVA-OneVision-1.5-Mid-Training [An et al., 2025], including 65 million English and 20 million Chinese samples.
  • 3 Pre-training: The authors omit continued scientific-text pre-training because Qwen3-8B-Base already absorbed substantial scientific knowledge, while scarce high-quality data could cause domain bias and overfitting.This choice also supports maintaining general capabilities grounded in robust visual understanding.

4 Post-training

Innovator-VL uses staged supervised fine-tuning and reinforcement learning to improve instruction following, multistep reasoning, and scientific multimodal reasoning. Its post-training combines diverse instructional and scientific data with discrepancy-guided RL selection and sequence-level optimization, although a referenced version is not publicly released due to compliance review.

  • 4 Post-training: Staged post-training combines supervised fine-tuning and reinforcement learning to improve instruction following, multistep reasoning, and scientific multimodal reasoning.The objectives span general visual instruction following, complex reasoning, and scientific problem solving.
  • Limitations: A referenced high-quality scientific corpus version is not publicly released because data compliance review procedures remain ongoing.This limits immediate public access to that version of the corpus.
  • Supervised Fine-tuning: SFT uses general multimodal instructions, chain-of-thought and multistep reasoning annotations, and scientific understanding data to support complex scientific reasoning.The general instruction data cover categories including captioning, charts and tables, code and mathematics, VQA, grounding and counting, and OCR.
  • Scientific Data Construction: Scientific training data are built with domain-specific pipelines emphasizing systematic quality control across in-the-wild OCSR, literature-based reaction understanding, and electron-micrograph characterization.The pipelines use synthetic bootstrapping and active learning for OCSR, hierarchical PDF-based reaction benchmarks, and cleaned EM data with dense segmentation and structured descriptions.
  • Reinforcement Learning: RL training uses Innovator-VL-RL-172K, selecting medium-difficulty samples through Pass@N–Pass@1 discrepancy analysis and reward-based filtering before standardizing reasoning formats.The corpus normalizes textual structures and stepwise outputs to reduce training instability across heterogeneous sources.
  • Reinforcement Learning: GSPO aligns importance sampling and clipping with sequence-level rewards, reducing token-level mismatch and supporting robust, efficient RL convergence for long-context reasoning.Its objective maximizes group response rewards under a sequence-level trust region using length-normalized sequence likelihood ratios.

5 Infrastructure

Innovator-VL uses an optimized distributed training pipeline spanning pre-training, supervised fine-tuning, and asynchronous reinforcement learning. Transformer-level optimizations, offline data packing, and decoupled rollout-learning improve computational and training efficiency.

  • 5 Infrastructure: The pre-training and SFT pipeline uses AIAK-Training-LLM, derived from Megatron-LM [Shoeybi et al., 2019], with mixed-precision acceleration, distributed optimizer parallelism, and selective activation recomputation.AIAK-Training-LLM is described as an enterprise-grade Megatron-LM optimization provided by Baidu Cloud.
  • 5 Infrastructure: Offline data packing reduces padding overhead by pre-optimizing multimodal data layout, improving context-window utilization, GPU utilization, training throughput, and computational efficiency [An et al., 2025].The strategy follows LLaVA-OneVision-1.5 [An et al., 2025] and addresses redundancy from variable sequence lengths.
  • 5 Infrastructure: AReaL [Fu et al., 2025] addresses synchronous reinforcement-learning bottlenecks by decoupling rollout from learning in a non-blocking producer-consumer architecture.The framework enables dedicated inference workers to continuously produce rollout data while learning proceeds separately.

6 Evaluation

Innovator-VL is evaluated against similarly scaled multimodal models across 37 general, mathematical, and scientific benchmarks. Innovator-VL-8B-Thinking achieves the best overall average, while the Instruct and Thinking variants show strong general, reasoning, scientific, and efficiency performance.

  • Evaluation Setup: The evaluation uses representative multimodal benchmarks grouped into general, math & reasoning, and science dimensions, with deterministic decoding in lmms-eval.Innovator-VL is compared against similarly scaled 7B–9B state-of-the-art multimodal models using temperature 0.0 and top-p 1.0.
  • Overall Evaluation: 61.83% average across 37 benchmarks makes Innovator-VL-8B-Thinking SOTA, surpassing all compared models of similar scale.The evaluation spans general, mathematical, and scientific domains.
  • Reasoning Efficiency: Innovator-VL-8B-Thinking achieves 1.4×–2× higher accuracy-to-token ratios than MiMo-VL-7B-RL and 3.9×–4.3× higher than Intern-S1-mini.Figure 5 reports significantly shorter reasoning chains alongside these efficiency gains.
  • General Vision: 74.50% average makes Innovator-VL-8B-Instruct comparable to Qwen3-VL-8B at 74.71%, while achieving the best results on AI2D and RealWorldQA.It also demonstrates superior scene understanding on MME-RealWorld and significantly outperforms InternVL3.5-8B and LLaVA-OV-1.5-8B.
  • Math & Reasoning: 55.41% average makes Innovator-VL-8B-Thinking SOTA in Math & Reasoning, improving 4.54 percentage points over Innovator-VL-8B-Instruct.The results attribute this gain to reinforcement learning, which enhances complex reasoning performance.
  • Science: Innovator-VL secures the top two Science-category averages, 50.13% and 49.79%, while exceeding 57% on OpenRxn and 64% on MolParse.All other baselines remain below 17% on OpenRxn and MolParse.

7 On Token Efficiency of Reasoning

Innovator-VL-8B-Thinking reasons more efficiently than Intern-S1-mini [Bai et al., 2025b] and MiMo-VL-7B-RL [Xiaomi, 2025], using fewer tokens while achieving higher accuracy-to-token ratios. This efficiency results from reinforcement learning that jointly optimizes correctness and conciseness by removing redundant reasoning steps.

  • Token consumption: Innovator-VL-8B-Thinking uses approximately 62% to 66% fewer tokens than Intern-S1-mini [Bai et al., 2025b] and 18% to 48% fewer than MiMo-VL-7B-RL [Xiaomi, 2025] across three benchmarks.Figure 5 (a) reports the reasoning-token comparison.
  • Reasoning efficiency: Innovator-VL-8B-Thinking achieves approximately 1.4× to 2× higher accuracy-to-token ratios than MiMo-VL-7B-RL [Xiaomi, 2025] and 3.9× to 4.3× higher than Intern-S1-mini [Bai et al., 2025b] across all benchmarks.Higher accuracy-to-token ratios indicate that each consumed token yields more useful information rather than redundant or irrelevant content.
  • Efficiency mechanism: Reinforcement learning jointly optimizes correctness and conciseness, teaching the model to identify critical reasoning steps and bypass redundant computations.The resulting compact reasoning pathways reduce inference costs and may limit error accumulation in scientific applications.

8 Conclusion and Future Works

Innovator-VL achieves strong performance across diverse scientific domains while maintaining excellent general vision capabilities through a fully transparent and reproducible training pipeline. The work highlights that competitive scientific reasoning can be achieved with fewer than five million carefully curated samples.

  • Conclusion: Innovator-VL achieves strong performance across diverse scientific domains while maintaining excellent general vision capabilities.The model is presented as a scientific multimodal large language model.
  • Conclusion: A fully transparent and reproducible training pipeline supports Innovator-VL’s scientific multimodal intelligence.
  • Conclusion: Fewer than five million carefully curated samples can yield competitive scientific reasoning.The conclusion frames this as a key principle for building scientific multimodal intelligence.

A More Results · A.1 Qualitative Case Study

The qualitative case study presents representative evaluation cases showing that Innovator-VL answers correctly where baseline models fail under the same setting, with full responses and associated visual inputs reported for inspection.

  • A.1 Qualitative Case Study: The study uses curated representative cases sampled from the evaluation benchmarks to illustrate practical strengths beyond aggregate benchmark scores.These cases complement the paper’s aggregate performance reporting.
  • A.1 Qualitative Case Study: For every presented case, the trained model produces the correct answer under the evaluation setting.
  • A.1 Qualitative Case Study: Other baseline models fail on the same cases and under the same evaluation conditions.
  • A.1 Qualitative Case Study: The cases are drawn from evaluation benchmarks rather than being presented as unrelated illustrative examples.
  • A.1 Qualitative Case Study: Each case reports the original question together with an associated image or figure when applicable.
  • A.1 Qualitative Case Study: The full model responses are included to expose how the model’s capabilities align with the task requirements.

A.1.1 General Visual Tasks … Qwen3-VL-8B

The supplied examples span general visual recognition, visual counting and directional reasoning, and scientific molecule representation. Responses vary in reasoning detail and correctness, while the E-SMILES example identifies a standard SMILES with an empty extension.

  • A.1.1 General Visual Tasks: The visual-task examples ask models to identify handbag and clock colors, determine direction, and report that the bike rider has a blue backpack.These prompts require direct answers from visual inputs, including option letters or a single word/number.
  • Innovator-VL-8B-Thinking: Innovator-VL-8B-Thinking counts 5 triangles by identifying two in the bow, one in the skirt, and two in the feet.Its step-by-step analysis distinguishes the bow, dress, body, and feet before summing the identified triangles.
  • MiniCPM-V4.5 (8B): MiniCPM-V4.5 (8B) counts 2 triangles, attributing them to the pink bow and the lower part of the green dress.It excludes facial features, limbs, belt, and hair from the triangle count.
  • InternVL3.5-8B: InternVL3.5-8B counts 10 triangles by including the bow, dress top, skirt, arms, legs, and feet.The response repeatedly sums these components as 2 + 1 + 1 + 2 + 2 + 2 = 10.
  • MiMo-VL-7B-RL: MiMo-VL-7B-RL also reports 8 total whistles, based on its identified left/right orientations and per-sparrow counts.It states that sparrows 1 and 3 look left, sparrows 2, 4, and 5 look right, then concludes that all whistle 8 times.
  • A.1.3 Scientific Tasks; Case 1: The scientific task asks for an Extended SMILES representation, whose format combines an RDKit-compatible SMILES string, a <sep> delimiter, and structured extension annotations.The extensions can annotate atom, ring, and abstract repeating-ring positions using <a>, <r>, and <c> tags, with <dum> marking a connection point.
  • SMILES String; E-SMILES; Intern-S1-mini (9B): For the given molecule, InternVL3.5-8B provides the SMILES CN1N=C2CCCC2=C1C(=O)N1CCN(C(=O)[C@@H]2C[C@H]2[N+](=O)[O-])CC1 followed by an empty extension.It concludes that no explicit substituents or abstract rings require additional annotations, while Intern-S1-mini (9B) presents a different SMILES variant and discusses adding extension tokens.

Intern-S1-mini (9B) … MiMo-VL-7B-RL

The section presents multimodal answers spanning molecular structure, organic reactions, chemical transformations, microscopy, spectroscopy, RNA, astronomy, transient classification, temperature anomalies, and tidal amphidromy. It also records differing model outputs for the White Sea amphidromy question, with Qwen3-VL-8B and MiMo-VL-7B-RL identifying the northern section.

  • Intern-S1-mini (9B): The q2 image compound is represented by the ESMILES string COC(=O)C1=C(I)C=CC2=C1OC(C(=O)O)C=C2.The string is explicitly identified as the structure of the compound shown in the q2 image.
  • Case 3: Scheme 4 converts 2-boryl-1,3-diene 2f into deborylated product 3, arylated product 4, unsaturated ketone 5, and allylic alcohol 6 through four derivatization pathways.Reported yields are 54%, 74%, 54%, and 40%, respectively, with high stated Z:E or E:Z selectivities.
  • Case 4: The reaction question describes phenylacetylene and N-oxide imine 1 reacting under Cu catalysis to form a four-membered-ring compound 3, with Et3N or pyridine specified in the alternatives.Another alternative describes formation of a bridged-ring compound 3 with Et3N.
  • Case 5; Case 6; Case 7: The supplied questions also cover particle-area uniformity in electron microscopy, sulfur-atom counting from MS/MS isotope patterns, and possible RNA secondary structures including hairpin, interior, and bulge loops.The microscopy criterion defines non-uniformity using a maximum-to-minimum diameter mean above 30%.
  • Case 8; Case 9; Case 10: Additional multimodal questions ask for galaxy morphology, a multi-band light-curve type, and the approximate regions of highest and lowest global surface-temperature anomalies.The galaxy options include unbarred tight or loose spirals, cigar-shaped smooth galaxies, and edge-on galaxies without a bulge; the light-curve options include KN, SNII, and SNIbc.
  • Qwen3-VL-8B; MiMo-VL-7B-RL: Qwen3-VL-8B explains its northern-section answer through phase contours increasing radially from a central point, while MiMo-VL-7B-RL likewise attributes the result to phase lines converging there.Both explanations characterize the pattern as indicating degenerate amphidromy.

B Evaluation Details … Generation

The evaluation covers diverse multimodal, scientific, mathematical, chemistry, and real-world vision benchmarks, with task-specific prompts enforcing concise answer formats, exact matching, or structured reasoning. It also includes generation and property-prediction tasks spanning molecular descriptions, SMILES, chemical reactions, and land-use classification.

  • B.2 Evaluation Prompts: Evaluation prompts require task-appropriate outputs, including option letters, single words or phrases, exact matches, step-by-step reasoning enclosed in <think>/<answer> tags, and JSON responses.Prompts also support image-grounded questions, multiple-choice formats, hints, English and Chinese variants, and benchmark-specific instructions.
  • ClinTox: ClinTox asks whether a molecule causes clinical toxicity and requires a strict “Yes” or “No” prediction.The prompt takes a molecular input and returns a binary clinical-toxicity prediction.
  • Side Effect: Side Effect evaluation includes binary side-effect prediction plus numerical aqueous-solubility and lipophilicity estimation from molecular structures.The prompts request logSol as a floating-point value and logP as a numerical value only.
  • Forward Synthesis: Forward Synthesis evaluates reaction-product prediction, retrosynthesis, SMILES–IUPAC conversion, and molecular-formula determination using text-only chemical outputs.Outputs are constrained to product SMILES, reactant SMILES, IUPAC names, or molecular formulas without extra explanation.
  • IUPAC to Molecular Formula: IUPAC to Molecular Formula converts an IUPAC molecular name into its molecular formula and requests only the formula as output.The prompt supplies the input name and explicitly excludes explanation or extra text.
  • Captioning: Captioning asks the model to describe a molecule’s structure and chemical classification in detail from the provided input.The task is open-ended and requests a detailed structural and chemical description.
  • Generation: Generation includes molecule-description-to-SMILES generation and image-based land-use classification with either one or multiple correct choices.The land-use prompts distinguish settings with and without land-use classification and require answer letters, separating multiple choices by spaces.
Loading 2601.19325v1…