PaperlayerView exact revision on arXiv

Source-linked AI summary

Qwen3-VL Technical Report

Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, Ke Zhu

arXiv:2511.21631v2cs.CVcs.AI

TL;DR

Advancing multimodal reasoning and long-context understanding must not erode a language model’s linguistic proficiency. Qwen3-VL combines high-quality multimodal training data with architectural improvements for vision–language alignment and temporal grounding. The authors report leading performance across multimodal benchmarks while maintaining strong pure-text capabilities.

  • Problem

    Vision–language models face the challenge of advancing multimodal reasoning and long-context understanding while preserving their underlying language model’s linguistic proficiency.

    Source: §1 · p. 2

  • Method

    Qwen3-VL combines high-quality multimodal training data with enhanced interleaved-MRoPE, DeepStack vision–language alignment, and text-based temporal grounding.

    Source: §6 · p. 25

  • Results

    The authors report leading performance across multimodal benchmarks while maintaining strong pure-text capabilities.

    Source: §6 · p. 25

  • Takeaways & Limitations

    256K-token interleaved contexts support reasoning over long documents, image sequences, and videos.

    Source: §6 · p. 25

    Video benchmark comparisons cannot guarantee full fairness because resource and API limitations imposed different input-frame budgets across models.

    Source: §5.9 · p. 20

Abstract

from arXiv · show

We introduce Qwen3-VL, the most capable vision-language model in the Qwen series to date, achieving superior performance across a broad range of multimodal benchmarks. It natively supports interleaved contexts of up to 256K tokens, seamlessly integrating text, images, and video. The model family includes both dense (2B/4B/8B/32B) and mixture-of-experts (30B-A3B/235B-A22B) variants to accommodate diverse latency-quality trade-offs. Qwen3-VL delivers three core pillars: (i) markedly stronger pure-text understanding, surpassing comparable text-only backbones in several cases; (ii) robust long-context comprehension with a native 256K-token window for both text and interleaved multimodal inputs, enabling faithful retention, retrieval, and cross-referencing across long documents and videos; and (iii) advanced multimodal reasoning across single-image, multi-image, and video tasks, demonstrating leading performance on comprehensive evaluations such as MMMU and visual-math benchmarks (e.g., MathVista and MathVision). Architecturally, we introduce three key upgrades: (i) an enhanced interleaved-MRoPE for stronger spatial-temporal modeling across images and video; (ii) DeepStack integration, which effectively leverages multi-level ViT features to tighten vision-language alignment; and (iii) text-based time alignment for video, evolving from T-RoPE to explicit textual timestamp alignment for more precise temporal grounding. Under comparable token budgets and latency constraints, Qwen3-VL achieves superior performance in both dense and Mixture-of-Experts (MoE) architectures. We envision Qwen3-VL serving as a foundational engine for image-grounded reasoning, agentic decision-making, and multimodal code intelligence in real-world workflows.

1 Introduction

Qwen3-VL combines a model family supporting up to 256K-token contexts with architectural, data, and training upgrades that strengthen multimodal capabilities while preserving language proficiency. It is presented as a foundation for applications integrating perception, reasoning, and action.

1 Introduction

  • 256K-token context windows support long-context understanding across Qwen3-VL’s four dense models (2B/4B/8B/32B) and two MoE models (30B-A3B / 235B-A22B).Built on Qwen3, the family preserves underlying language proficiency through optimized training data and strategy. Both non-thinking and thinking variants are released, with thinking variants demonstrating stronger multimodal reasoning on complex tasks.Source: §1 · p. 2
  • Interleaved MRoPE, DeepStack (Meng et al., 2024), and explicit video timestamps improve positional representation, cross-layer vision–language fusion, and temporal representation.Interleaved MRoPE distributes temporal and spatial dimensions uniformly across frequency bands; DeepStack routes multi-level visual tokens into corresponding LLM layers through lightweight residual connections without extending context. Explicit timestamp tokens replace positional absolute-time alignment, while square-root-normalized per-token loss replaces per-sample loss to better balance text and multimodal training contributions.Source: §1 · p. 2
  • The training-data overhaul strengthens quality, diversity, and structure across perception, grounding, reasoning, and agent interaction.Upgrades cover caption supervision, omni-recognition, OCR, normalized grounding, 3D/spatial reasoning, code, long documents, and temporally grounded video. Chain-of-thought reasoning and diverse, high-quality GUI-agent interaction data connect perception, reasoning, and action.Source: §1 · p. 2
  • 8K, 32K, and 256K sequence lengths progressively expand context during full-parameter pretraining, following merger-only alignment.The alignment warm-up freezes all parameters except the vision–language projection layers. Post-training proceeds through long-chain-of-thought supervised fine-tuning, knowledge distillation from stronger teachers, and reinforcement learning.Source: §1 · p. 2
  • Qwen3-VL is presented as a perception–reasoning–action platform, motivated by expanding multimodal applications and the need to retain language proficiency.Motivating applications include long-context understanding, STEM reasoning, GUI comprehension and interaction, and agentic workflows; multimodal advances are expected to match or surpass text-only counterparts on language benchmarks.Source: §1 · p. 2

2 Model Architecture

Qwen3-VL combines a vision encoder, an MLP-based vision–language merger, and a Qwen3 language backbone. Its architectural upgrades balance spatial–temporal positional frequencies, inject multi-level visual features into the language model, and encode video timestamps as text.

2 Model Architecture

  • 235B total parameters, with 22B activated per token, power Qwen3-VL-235B-A22B, which outperforms most VLMs across multimodal tasks and its text-only counterpart on most language benchmarks.The Qwen3-backed family includes dense 2B/4B/8B/32B models and MoE 30B-A3B/235B-A22B models.Source: §2 · pp. 2–3
  • SigLIP-2 (Tschannen et al., 2025) encodes dynamic-resolution inputs using 2D-RoPE and interpolated absolute position embeddings following CoMP (Chen et al., 2025).The encoder continues training from official pretrained checkpoints with dynamic resolutions. SigLIP2-SO-400M is the default, while 2B and 4B language models use SigLIP2-Large (300M).Source: §2 · p. 3
  • 2 × 2 visual features are compressed into one visual token by a two-layer MLP merger aligned with the language model’s hidden dimension.The overall architecture comprises a vision encoder, this vision–language merger, and an LLM, following Qwen2.5-VL.Source: §2 · pp. 2–3

2.1 Interleaved MRoPE

  • Interleaved MRoPE distributes temporal, horizontal, and vertical components across low- and high-frequency bands, mitigating spectral bias and improving long-range video positional modeling.Original MRoPE in Qwen2-VL (Wang et al., 2024c) assigned distinct frequencies to separate axis subspaces, creating an imbalanced spectrum associated with degraded long-video understanding.Source: §2.1 · p. 3

2.2 DeepStack

  • Building on DeepStack (Meng et al., 2024), Qwen3-VL injects intermediate ViT features into the first three LLM layers, preserving low- to high-level visual information.Unlike the original approach using multi-scale visual inputs, dedicated vision–language mergers project features from three encoder levels into visual tokens added directly to corresponding LLM hidden states.Source: §2 · p. 3, §2.2 · p. 4

2.3 Video Timestamp

  • Textual timestamps prefix each video temporal patch (Chen et al., 2024b), improving temporal precision for video grounding and dense captioning.The prior absolute-time scheme produced large, sparse temporal IDs for long videos and required extensive sampling across frame rates, increasing data-construction cost. Training uses seconds and HMS (hours:minutes:seconds) formats, with a modest context-length increase.Source: §2.3 · p. 4

3 Pre-Training

Qwen3-VL pre-training progresses through four stages, from vision–language alignment to ultra-long-context adaptation. Curated and synthetic data support document understanding, grounding, spatial reasoning, coding, video comprehension, STEM reasoning, and agentic interaction.

3.1 Training Recipe

  • 262,144-token sequences cap a four-stage curriculum progressing from merger-only alignment to full-model multimodal training and long-context adaptation.The architecture combines a dynamically trained SigLIP-2 vision encoder, an MLP merger, and a Qwen3 backbone. S0 trains only the merger on approximately 67B tokens at length 8,192; S1 jointly trains all components on approximately 1T mixed text–multimodal tokens at 8,192; S2 uses approximately 1T tokens at 32,768 with more text, video, and agent data; S3 uses 100B tokens emphasizing long videos and documents.Source: §3.1 · pp. 4–5

3.2.1 Image Caption and Interleaved Text-Image Data

  • Image–text curation combines detailed recaptioning, text-only semantic deduplication, targeted visual diversification, and quality-filtered interleaved documents.A specialized Qwen2.5-VL-32B recaptions predominantly Chinese–English web pairs, while visual-embedding clustering (Douze et al., 2024; Diao et al., 2025) identifies underrepresented concepts for augmentation. A Qwen-based scorer removes low-value web documents; Qwen2.5-VL-7B parses books into aligned text–image sequences of up to 256K tokens, with page-count, alignment, and image-to-text quality controls.Source: §3.2.1 · p. 5

3.2.2 Knowledge; 3.2.5 Spatial Understanding and 3D Recognition

  • Entity knowledge and spatial datasets support visual concept recognition, relational reasoning, action planning, and monocular 3D grounding.Knowledge curation samples prominent entities more heavily while retaining rare concepts, filters misaligned samples, and replaces sparse captions with descriptions of attributes, context, and interactions. Spatial data express relations, affordances, and action queries relative to objects or scene frames; single-view 3D grounding uses filtered 9-DoF boxes, rich referring expressions, and a virtual camera coordinate system following Omni3D (Brazil et al., 2023).Source: §3.2.2 · p. 6, §3.2.5 · p. 7

3.2.3 OCR, Document Parsing and Long Document Understanding

  • 30 million in-house OCR samples anchor document training that also covers multilingual recognition, layout-aware parsing, and cross-page visual question answering.OCR pseudo-labels refined by Qwen2.5-VL require no human annotation; expansion adds 29 languages, approximately 30 million synthetic multilingual samples, and over 1 million real multilingual images. Parsing uses 3 million Common Crawl PDFs and 4 million internal documents, combining layout predictions, region recognition, synthetic and pseudo-labeled data, QwenVL-HTML element boxes, and QwenVL-Markdown image/table localization; merged-page parsing sequences and balanced multi-page VQA teach reasoning across heterogeneous document elements.Source: §3.2.3 · p. 6

3.2.4 Grounding and Counting

  • Box- and point-based grounding support localization and three counting formulations, using normalized coordinates for resolution and aspect-ratio robustness.Public annotations are augmented with synthesized boxes and fine-grained points; box synthesis extracts candidates, localizes them with detectors and Qwen2.5-VL, and filters inaccurate annotations. Counting includes direct, box-based, and point-based tasks, while coordinates use [0, 1000] rather than the Qwen2.5-VL convention.Source: §3.2.4 · pp. 6–7

3.2.6 Code; 3.2.9 Agent

  • Coding and agent data connect visual understanding with program generation, cross-platform GUI interaction, multimodal function calling, and factual search.Qwen3 and Qwen3-Coder corpora supply text-only coding foundations, while multimodal tasks include screenshot-to-HTML/CSS generation, visual coding questions, and diagram-to-code transcription. GUI data combine perception annotations with audited, self-evolving trajectories and planning rationales; function-calling trajectories synthesize queries, definitions, calls, and responses without executable functions, while image/text search trajectories teach lookup of unfamiliar entities.Source: §3.2.6 · p. 8, §3.2.9 · p. 9

3.2.7 Video

  • Video training combines timestamp-interleaved caption synthesis, spatio-temporal grounding, source balancing, and sequence-length-adaptive frame sampling.Short-to-long synthesis produces coherent story descriptions alongside event summaries and segment details, while grounding annotations cover objects, actions, and persons. Metadata guides balancing across video sources, and adaptive frame rates and frame limits preserve visual information under different context budgets.Source: §3.2.7 · p. 8

3.2.8 Science, Technology, Engineering, and Mathematics (STEM)

  • 1 million point-grounding samples, 2 million perception VQA pairs, and 6 million diagram captions develop STEM perception alongside multimodal and linguistic reasoning.Programmatically rendered diagrams and ensemble-generated, verified captions train perception separately before integration with reasoning. Reasoning data include over 60 million cleaned and reformulated exercises, over 12 million image-paired long-CoT samples validated through rules and models and filtered toward challenging problems, plus linguistic reasoning data from Qwen3.Source: §3.2.8 · pp. 8–9

4 Post-Training

Qwen3-VL post-training combines supervised fine-tuning, strong-to-weak distillation, and reinforcement learning to develop instruction following, reasoning, and human-preference alignment. Specialized data curation and tool-integrated training support long-context understanding, multimodal reasoning, and visual-agent capabilities.

4.1 Training Recipe

  • Post-training proceeds through supervised fine-tuning, strong-to-weak distillation, and reinforcement learning, with separate training formats for thinking and non-thinking models.SFT first develops instruction following and reasoning at 32k context length, then extends to 256k using long-document and long-video data; thinking models receive explicit Chain-of-Thought formats. The final stage separates Reasoning RL from General RL and covers text and multimodal domains including mathematics, OCR, grounding, and instruction following.Source: §4.1 · pp. 9–10

4.3 Strong-to-Weak Distillation

  • Text-only strong-to-weak distillation fine-tunes the LLM backbone, improving lightweight students’ reasoning across text-centric and multimodal tasks.Off-policy distillation first trains students on teacher-generated responses to establish fundamental reasoning abilities. On-policy distillation then uses student-generated sequences and minimizes KL divergence between student and teacher logits.Source: §4.1 · p. 9, §4.3 · p. 11

4.2.1 SFT Data

  • Approximately 1,200,000 SFT samples comprise one-third text-only data and two-thirds image-text or video-text pairs, including multilingual, conversational, and interleaved agentic examples.Curated from open-source datasets and web resources, these samples expand spatial reasoning, image-grounded reasoning, video tracking, and long-document understanding; training uses one epoch at 32K followed by one at 256K, interleaving shorter inputs with documents and videos up to two hours long. Query filtering checks verifiability, clarity, substance, difficulty, and relevance, while response filtering combines rules against defective or harmful outputs with Qwen2.5-VL reward-model assessments of answer quality and visual grounding.Source: §4.2.1 · p. 10

4.2.2 Long-CoT Cold Start Data

  • An approximate 1:1 vision-language-to-text-only sample ratio balances the Long-CoT cold-start dataset used to bootstrap advanced multimodal reasoning.Multimodal data emphasizes STEM and agentic workflows alongside VQA, OCR, grounding, and video analysis, while text data covers mathematics, coding, logic, and STEM. Curation favors challenging problems, excludes visual mathematics solvable without images by Qwen3-30B-nothink, and removes incorrect final answers, repetition, language mixing, and unsupported guessing.Source: §4.2.2 · p. 11

4.4.1 Reasoning Reinforcement Learning

  • Approximately 30K reasoning-RL queries cover deterministically verifiable text and multimodal tasks, with easy queries exceeding a 90% pass rate filtered out.Preparation discards multimodal queries when all 16 preliminary Qwen3-VL-235B-A22B responses fail, removes low-improvement sources, and samples 16 responses per query during model-specific filtering before constructing mixed-task batches. SAPO (Gao et al., 2025) provides consistent improvements across tasks, sizes, and architectures; task-specific rewards use format prompts rather than explicit format rewards and penalize response-language mismatches.Source: §4.4.1 · pp. 11–12

4.4.2 General Reinforcement Learning

  • General RL combines rule-based and model-based rewards to strengthen instruction following, preference alignment, generalization, and operational robustness across SFT tasks.Verifiable counter-intuitive counting and clock tasks target flawed SFT priors, while dedicated prompts enable frequent penalties for language mixing, repetition, and formatting errors. Rule-based rewards provide precise feedback and mitigate reward hacking; Qwen2.5-VL-72B-Instruct or Qwen3 judges assess reference-grounded response quality, accommodating valid unconventional answers while optimizing helpfulness, factual accuracy, style, and explicit constraints.Source: §4.4.2 · p. 12

4.5 Thinking with Images

  • Approximately 120k multi-turn agentic interactions support Qwen3-VL’s visual-agent training, with tool-calling rewards countering degeneration into single-call reward hacking.An initial Qwen2.5-VL-32B agent receives SFT on approximately 10k grounding examples and tool-integrated RL, then generates the larger dataset used alongside synthesized data for Qwen3-VL’s SFT and RL. Both stages reward final-answer accuracy, coherent interpretation of tool feedback, and tool-call counts matched to expert-estimated task complexity.Source: §4.5 · pp. 12–13

4.6 Infrastructure

  • 10,000 GPUs are the reported upper scale for hybrid-parallel pretraining, while vLLM or SGLang support local deployment and performance evaluation.Training runs on Alibaba Cloud’s PAI-Lingjun service, using Megatron-LM with tensor, pipeline, context, expert, and ZeRO-1 data parallelism to balance computation and communication. vLLM uses PagedAttention for memory-efficient, high-throughput inference, while SGLang supports structured generation and complex prompts.Source: §4.6 · p. 13

5 Evaluation · 5.1 General Visual Question Answering · 5.2 Multimodal Reasoning

Qwen3-VL achieves leading results across general visual question answering and multimodal reasoning benchmarks, with competitive performance extending from flagship models to medium and small variants.

5.1 General Visual Question Answering

  • 89.3/88.9 on MMBench and 79.2 on RealWorldQA are the highest non-reasoning-mode scores, achieved by Qwen3-VL-235B-A22B-Instruct.Source: §5.1 · p. 13
  • 78.7 on MMStar gives Qwen3-VL-235B-A22B-Thinking the highest score in the thinking-mode comparison.Gemini-2.5-Pro (Comanici et al., 2025) delivers the best overall thinking-mode performance, with Qwen3-VL-235B-A22B-Thinking close behind.Source: §5.1 · p. 13
  • 89.5/89.5 on MMBench and 79.4 on RealWorldQA place Qwen3-VL-32B-Thinking first among medium-sized models.Qwen3-VL-32B-Instruct is reported at 79.0 on RealWorldQA. The accompanying claim that it outperforms Thinking conflicts with the reported Thinking score of 79.4.Source: §5.1 · pp. 13–14
  • 79.9 to 85.3 on MMBench-EN and 68.1 to 75.3 on MMStar mark thinking-mode gains from Qwen3-VL-2B to Qwen3-VL-8B.Within the smaller-model comparison, Qwen3-VL-8B performs best across all five benchmarks.Source: §5.1 · p. 14

5.2 Multimodal Reasoning

  • Qwen3-VL-235B-A22B achieves leading multimodal reasoning results in both Instruct and Thinking modes.Among non-thinking or low-thinking-budget models, Instruct leads on MathVistamini, MathVision, MathVersemini, DynaMath, ZeroBench, VLMsAreBlind, VisuLogic, and VisualPuzzlesDirect. Thinking achieves state-of-the-art results on MathVistamini, MathVision, MathVersemini, ZeroBench, LogicVista, and VisuLogic.Source: §5.2 · p. 14
  • Qwen3-VL-32B consistently outperforms Gemini-2.5-Flash and GPT-5-mini on reasoning tasks and surpasses the previous-generation Qwen2.5-VL-72B.The Qwen3-VL-30B-A3B MoE model also delivers competitive results.Source: §5.2 · p. 14
  • Qwen3-VL-8B leads overall in the small-model comparison against GPT-5-Nano, while Qwen3-VL-4B scores highest on DynaMath and VisuLogic.The smallest Qwen3-VL-2B model also exhibits strong reasoning capabilities.Source: §5.2 · p. 14

5.3 Alignment and Subjective Tasks · 5.4 Text Recognition and Document Understanding

Qwen3-VL-235B-A22B reports leading instruction-following, OCR, and long-document results. Smaller variants remain competitive, while multilingual OCR support expands to 39 languages.

5.3 Alignment and Subjective Tasks

  • Alignment evaluation examines multimodal instruction following and image-level hallucinations through multi-turn judging, image-context reasoning, and complex compositional instructions.MM-MT-Bench (Agrawal et al., 2024) uses multi-turn LLM judging; HallusionBench diagnoses image-context reasoning. MIA-Bench (Qian et al., 2024) tests complex instructions, including character limits and compositional requirements.Source: §5.3 · p. 14
  • 3.0, 1.0, and 6.3 points are Qwen3-VL-235B-A22B-Thinking’s HallusionBench gains over Gemini-2.5-pro (Comanici et al., 2025), GPT-5 (OpenAI., 2025), and Claude opus 4.1 (Anthropic., 2025), respectively.It also achieves the best overall MIA-Bench score, exceeding GPT-5-high-thinking by 10.0 points on math and 5.0 points on textual subtasks.Source: §5.3 · p. 14, Table 2 · p. 15

5.4 Text Recognition and Document Understanding

  • Qwen3-VL-235B-A22B-Instruct establishes state-of-the-art results on OCR parsing and comprehensive OCR benchmarks, marginally outperforming its Thinking counterpart.Evaluations cover CC-OCR, OmniDocBench (Ouyang et al., 2024), OCRBench (Liu et al., 2024), and OCRBench_v2 (Fu et al., 2024b).Source: §5.4 · pp. 14, 17, Table 2 · p. 15
  • 57.0% and 56.2% overall accuracy on MMLongBench-Doc (Ma et al., 2024) establish state-of-the-art long-document performance for Qwen3-VL-235B-A22B-Instruct and Qwen3-VL-235B-A22B-Thinking, respectively.These results come from the comparison among flagship models.Source: Table 2 · p. 15, §5.4 · p. 17
  • Qwen3-VL-235B-A22B-Thinking outperforms Instruct on CharXiv reasoning (Wang et al., 2024g), trailing only GPT5-thinking and Gemini-2.5-Pro-Thinking, while both variants perform comparably on OCR-related VQA.Comparable results cover DocVQA (Mathew et al., 2021b), InfoVQA (Mathew et al., 2021a), AI2D (Kembhavi et al., 2016), ChartQA (Masry et al., 2022), and CharXiv descriptions (Wang et al., 2024g). CharXiv reasoning requires deep chart comprehension and multi-step reasoning.Source: Table 2 · p. 15, §5.4 · p. 17
  • Qwen3-VL’s 30B-A3B and 32B variants outperform Gemini-2.5-Flash and GPT-5-mini across most evaluation metrics and surpass comparably sized models on MIA-Bench, demonstrating strong multimodal instruction following.The 2B/4B/8B variants remain competitive on OCR parsing, visual question answering, and comprehensive benchmark suites. They also show a negligible performance drop, especially on MIA-Bench.Source: §5.3 · p. 14, §5.4 · p. 17
  • Over 70% accuracy on 32 of 39 supported languages is reported for Qwen3-VL on a newly constructed in-house multilingual OCR dataset.Support expands from Qwen2.5-VL’s 10 non-English/Chinese languages to 39 languages in Qwen3-VL.Source: Figure 2 · p. 17, §5.4 · p. 17

5.5 2D and 3D Grounding

Qwen3-VL-235B-A22B achieves state-of-the-art results across 2D grounding and counting benchmarks and outperforms other closed-source models across multiple 3D grounding datasets. Smaller variants also show competitive performance in both settings.

2D Grounding

  • The 2D evaluation covers referring expression comprehension, open-vocabulary object detection, and counting.Benchmarks are RefCOCO/+/g (Mao et al., 2016), ODinW-13 (Li et al., 2022), and CountBench (Paiss et al., 2023), respectively. ODinW-13 evaluation uses mAP with confidence fixed at 1.0 and all dataset categories supplied simultaneously in the prompt.Source: §5.5 · pp. 17, 19
  • 48.6 mAP on ODinW-13 is achieved by Qwen3-VL-235B-A22B, which reports state-of-the-art results across 2D grounding and counting benchmarks.Smaller-scale variants also exhibit competitive 2D visual grounding performance.Source: §5.5 · p. 19

3D Grounding

  • 5.2 points separate Qwen3-VL-235B-A22B-Thinking from Gemini-2.5-Pro in its favor on SUN RGB-D.The flagship consistently outperforms other closed-source models across multiple 3D datasets; smaller variants also show competitive 3D object grounding performance.Source: §5.5 · p. 19
  • 3D object localization is evaluated on Omni3D (Brazil et al., 2023) using image-text inputs and test-set mAP@0.15.Each textual prompt specifies an object category, and detection confidence is fixed at 1.0. Comparisons use models of comparable scale.Source: §5.5 · p. 19

5.6 Fine-grained Perception … 5.9 Video Understanding

Qwen3-VL achieves state-of-the-art tool-assisted fine-grained perception and leading or competitive results in multi-image, embodied, spatial, and video understanding. Video comparisons are qualified by unequal input-frame limits.

5.6 Fine-grained Perception

  • 93.7 on V*, 85.3 on HRBench-4k, and 82.3 on HRBench-8k establish tool-augmented Qwen3-VL-235B-A22B’s state-of-the-art fine-grained perception performance.The series substantially improves over Qwen2.5-VL-72B on V* (Wu &Xie, 2024), HRBench-4k (Wang et al., 2024e), and HRBench-8k (Wang et al., 2024e).Source: §5.6 · p. 19
  • ∼5 points on V* (Wu &Xie, 2024) are consistently gained by adding tools within Qwen3-VL, with tool integration outperforming model-size increases.The report identifies tool-integrated multimodal agentic learning as a promising direction.Source: §5.6 · p. 19

5.7 Multi-Image Understanding

  • 80.1 on MuirBench (Wang et al., 2024a) places Qwen3-VL-235B-A22B-Thinking ahead of all compared models in multi-image understanding.Qwen3-VL-235B-A22B-Instruct performs comparably to Gemini-2.5-pro across the multi-image evaluation. Training incorporates multi-image referring grounding, visual correspondence, and multi-hop reasoning.Source: §5.7 · p. 19

5.8 Embodied and Spatial Understanding

  • Qwen3-VL rivals Gemini-2.5-Pro, GPT-5, and Claude-Opus-4.1 on embodied and spatial tasks following training with high-resolution spatial annotations and spatio-temporal perception data.84.3 on EmbSpatial (Du et al., 2024), 69.9 on RefSpatial (Zhou et al., 2025), and 73.9 on RoboSpatialHome (Song et al., 2025a) are reported for Qwen3-VL-235B-A22, whose spatial training includes fine-grained pointing, relative-position annotations, and QA pairs. 52.5 on ERQA (Team et al., 2025) and 60.0 on VSIBench (Yang et al., 2025b) are reported for Qwen3-VL-235B-A22B, with embodied training incorporating pointing, grounding, and spatio-temporal perception.Source: §5.8 · pp. 19–20

5.9 Video Understanding

  • 8B Qwen3-VL achieves video performance competitive with Qwen2.5-VL 72B through interleaved MRoPE, textual timestamps, and scaled temporally dense video captions.The report attributes improved video understanding to both expanded training data and architectural enhancements.Source: §5.9 · p. 20
  • 256K-token context enables Qwen3-VL-235B-A22B-Instruct to match or surpass Gemini-2.5-Pro on long-video tasks, notably MLVU (Zhou et al., 2024).On standard video-understanding benchmarks, the flagship matches leading models including Gemini 2.5 Pro with a thinking budget of 128 and GPT-5 minimal.Source: §5.9 · p. 20
  • 512 frames for Gemini 2.5 Pro, 256 for GPT-5, and 100 for Claude Opus 4.1 limit fairness of video comparisons.Resource and API limitations constrained these proprietary models’ input-frame counts, preventing guarantees of fully fair comparisons.Source: §5.9 · p. 20
  • 2,048 frames and 224K video tokens cap Qwen3-VL evaluation inputs across general understanding, temporal grounding, reasoning, and long-form video tasks.Per-frame token limits are 768 for VideoMMMU and MMVU and 640 elsewhere; sampling uses 4 fps for Charades-STA and 2 fps elsewhere. VideoMMMU uses a model-based judge because rule-based scoring was insufficiently accurate.Source: §5.9 · p. 20

5.10 Agent

Qwen3-VL achieves state-of-the-art GUI grounding across multiple tasks, while Qwen3-VL 32B surpasses current foundation VLMs in online OSWorld and AndroidWorld evaluations.

5.10 Agent

  • 41 on OSWorld (Xie et al., 2025c;b) and 63.7 on AndroidWorld (Rawles et al., 2024): Qwen3-VL 32B surpasses current foundation VLMs.These online environment evaluations assess decision-making abilities.Source: §5.10 · p. 20
  • Qwen3-VL-235B-A22B achieves state-of-the-art GUI grounding across multiple tasks covering desktop, mobile, and PC interfaces.UI perception evaluations use ScreenSpot (Cheng et al., 2024), ScreenSpot Pro (Li et al., 2025b), and OSWorldG.Source: §5.10 · p. 20
  • Qwen3-VL demonstrates strong planning, decision-making, and reflection as a GUI agent, while smaller models remain highly competitive on these benchmarks.Source: §5.10 · p. 20

5.11 Text-Centric Tasks

Qwen3-VL delivers competitive text-centric performance across instruct and thinking variants, with flagship strengths in mathematics and coding. Its smaller models also outperform corresponding baselines, supporting lightweight model development through Strong-to-Weak Distillation.

Qwen3-VL-235B-A22B

  • Qwen3-VL-235B-A22B-Instruct matches or surpasses DeepSeek V3 0324, Claude-Opus-4 (without thinking), and Qwen3-235B-A22B-Instruct-2507, particularly excelling in mathematics and coding.The text-centric evaluation covers knowledge, reasoning, coding, alignment, agent, and multilingual tasks.Source: §5.11 · pp. 20–21, Table 5 · p. 21
  • Qwen3-VL-235B-A22B-Thinking outperforms OpenAI o3 (medium) and Claude-Opus-4 (with thinking) on AIME-25 and LiveCodeBench v6, while remaining competitive overall.Thinking outputs are capped at 32,768 tokens, extended to 81,920 for AIME-25, HMMT-25, and LiveCodeBench v6.Source: §5.11 · p. 21, Table 6 · p. 22

Qwen3-VL-32B / 30B-A3B

  • Qwen3-VL-32B-Instruct and Qwen3-VL-30B-A3B-Instruct improve across all benchmarks over Qwen3-32B and Qwen3-30B-A3B, respectively.Qwen3-VL-30B-A3B-Instruct also achieves comparable or better results than Qwen3-30B-A3B-2507, particularly on AIME-25 and HMMT-25.Source: §5.11 · p. 21, Table 7 · p. 22
  • Qwen3-VL-32B-Thinking and Qwen3-VL-30B-A3B-Thinking surpass baselines on most benchmarks, with the latter also achieving performance comparable to Qwen3-30B-A3B-2507.The corresponding text-only baselines include Qwen3-32B and Qwen3-30B-A3B.Source: §5.11 · p. 21, Table 8 · p. 23

Qwen3-VL-8B / 4B / 2B

  • Qwen3-VL-2B, Qwen3-VL-4B, and Qwen3-VL-8B outperform their baselines, supporting Strong-to-Weak Distillation for building lightweight models with reduced costs and effort.The 2B and 8B models are compared with Qwen3-1.7B and Qwen3-8B, respectively; the 4B model is compared with Qwen3-4B and Qwen3-4B-2507.Source: §5.11 · pp. 22–23, Table 9 · p. 23, Table 10 · p. 24

5.12 Ablation Study

Ablations show performance gains from Qwen3-ViT and DeepStack, while video needle retrieval achieves 100% accuracy up to 256K tokens and 99.5% after extension to 1M tokens.

5.12.1 Vision Encoder

  • Qwen3-ViT remains competitive on standard zero-shot benchmarks, substantially improves OmniBench, and outperforms the SigLIP-2-based baseline across key downstream tasks.Downstream comparisons use the same 1.7B Qwen3 language model trained for 1.5T tokens. Qwen3-ViT also remains significantly ahead on OmniBench, an in-house evaluation of world knowledge integration under diverse, challenging conditions.Source: Table 11 · p. 24, §5.12.1 · p. 24

5.12.2 DeepStack

  • DeepStack improves overall benchmark performance by integrating rich visual information, strengthening fine-grained visual understanding on tasks such as InfoVQA and DocVQA.The ablation uses an internal 15B-A2B language model pretrained on 200 billion tokens, with direct validation-set evaluation and no post-training.Source: Table 12 · p. 24, §5.12.2 · p. 24

5.12.3 Needle-in-a-Haystack

  • 100% accuracy is achieved on videos up to 30 minutes (256K tokens), and 99.5% on approximately two-hour videos (1M tokens).Qwen3-VL-235B-A22B-Instruct locates evidence frames inserted at varying temporal positions and answers corresponding questions, using 1 FPS sampling and dynamically adjusted resolution to maintain a constant visual token budget. The 1M-token evaluation uses YaRN-based positional extension.Source: Figure 3 · p. 25, §5.12.3 · p. 25

6 Conclusion

Qwen3-VL combines data and architectural improvements with native long-context support to advance multimodal performance while maintaining strong pure-text capabilities. Future directions emphasize embodied agents and unified understanding-generation architectures.

6 Conclusion

  • Qwen3-VL achieves state-of-the-art multimodal performance through high-quality multimodal data iteration and architectural innovations while maintaining strong pure-text capabilities.Architectural innovations include enhanced interleaved-MRoPE, DeepStack vision-language alignment, and text-based temporal grounding.Source: §6 · p. 25
  • 256K-token native interleaved sequences support robust reasoning over long, complex documents, image sequences, and videos.This capability targets applications requiring high-fidelity cross-modal comprehension.Source: §6 · p. 25
  • Dense and Mixture-of-Experts variants accommodate diverse latency and quality requirements, while post-training includes non-thinking and thinking modes.Source: §6 · p. 25
  • Future work targets embodied agents through interactive perception, tool-augmented reasoning, and real-time multimodal control.Envisioned agents would interact with users, manipulate digital interfaces, and guide robotic systems through context-aware actions. The authors are also exploring unified understanding-generation architectures that use visual generation to enhance overall intelligence.Source: §6 · p. 25
  • The entire model family is openly released under Apache 2.0 to encourage community-driven innovation toward integrated multimodal agents.Source: §6 · p. 25

A Benchmarks

Qwen3-VL’s evaluation suite covers multimodal reasoning, visual question answering, instruction following, document and spatial understanding, video, coding, GUI agents, and text-centric tasks.

Multimodal Reasoning

General Visual Question Answering; Subjective Experience and Instruction Following

  • General visual question answering, subjective experience, and complex instruction following are evaluated through separate benchmark suites.General VQA includes MMBench-V1.1 (Liu et al., 2023b) and RealWorldQA (xAI, 2024). Subjective experience and instruction following use HallusionBench, MM-MT-Bench (Agrawal et al., 2024), and MIA-Bench (Qian et al., 2024).Source: A Benchmarks · p. 33

Document Understanding

2D/3D Grounding and Spatial Understanding

Video Understanding

Coding

GUI Agent

Text-Centric Tasks

B Evaluation Prompts

The appendix provides the complete evaluation prompts, consistently applied during inference to support reproducibility, fairness, and comparability. The supplied templates cover multiple-choice answering, iterative visual research, and constrained computer interaction.

B Evaluation Prompts

  • Evaluation prompts were applied consistently during inference to maintain fairness and comparability and are provided to support reproducibility and future research.Source: B Evaluation Prompts · p. 34

B.1 STEM & Puzzle

  • STEM and puzzle prompts present a question and answer options, then instruct the model to select the correct answer.Source: §B.1 · p. 34

B.7 Multi-Image

  • Multi-image prompts present a question and answer options, then instruct the model to select the correct answer.Source: §B.7 · p. 38

B.9 Perception with Tool

  • Perception-with-tool prompts require iterative image inspection, tool-based research, and review until research is complete, followed by a synthesized answer.Models describe the image and distinguish visible observations from information requiring lookup.Source: §B.9 · p. 40
  • The image_zoom_in_tool crops image regions using bounding-box coordinates, an object label, and a zero-based image index.The function signature requires bbox_2d, label, and img_idx, with bbox_2d containing [x1, y1, x2, y2]. Calls return function names and arguments as JSON inside <tool_call> tags.Source: §B.9 · p. 40

B.11 Agent

  • Agent prompts restrict computer_use to mouse_move and left_click with pixel coordinates, requiring failure reporting when targets cannot be found or tasks are infeasible.The cursor tip should be centered on targets, avoiding edges unless requested; infeasible tasks return a terminate action with failure status.Source: §B.11 · p. 42

Related papers

Loading 2511.21631v2…