Source-linked AI summary
Are We on the Right Way for Evaluating Large Vision-Language Models?
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, Feng Zhao
TL;DR
Existing LVLM benchmarks may include visually unnecessary or training-leaked samples, limiting what their scores reveal about multimodal ability. The paper addresses this with MMStar, a human-curated benchmark and two diagnostic metrics. Across 16 LVLMs, high-resolution GPT-4V ranks first on MMStar with 57.1% accuracy.
Problem
Existing LVLM evaluations may include visually unnecessary samples and unintentional LLM or LVLM training-data leakage, obscuring actual multimodal performance gains.
Method
MMStar combines automated filtering, LVLM-based difficulty assessment, and strict human review to curate 1,500 samples spanning 6 capabilities and 18 axes, alongside MG and ML metrics.
Results
57.1% accuracy makes high-resolution GPT-4V the top-ranked model among 16 evaluated LVLMs on MMStar.
Takeaways & Limitations
MMStar provides a benchmark intended to evaluate LVLM multimodal capacities using vision-dependent, balanced, and purified samples.
Takeaways & Limitations
MMStar’s curation assumes samples should require visual understanding, minimize training leakage, and demand advanced multimodal capabilities across selected difficulty levels.
Abstract
from arXiv · showhide
Large vision-language models (LVLMs) have recently achieved rapid progress, sparking numerous studies to evaluate their multi-modal capabilities. However, we dig into current evaluation works and identify two primary issues: 1) Visual content is unnecessary for many samples. The answers can be directly inferred from the questions and options, or the world knowledge embedded in LLMs. This phenomenon is prevalent across current benchmarks. For instance, GeminiPro achieves 42.9% on the MMMU benchmark without any visual input, and outperforms the random choice baseline across six benchmarks over 24% on average. 2) Unintentional data leakage exists in LLM and LVLM training. LLM and LVLM could still answer some visual-necessary questions without visual content, indicating the memorizing of these samples within large-scale training data. For example, Sphinx-X-MoE gets 43.6% on MMMU without accessing images, surpassing its LLM backbone with 17.9%. Both problems lead to misjudgments of actual multi-modal gains and potentially misguide the study of LVLM. To this end, we present MMStar, an elite vision-indispensable multi-modal benchmark comprising 1,500 samples meticulously selected by humans. MMStar benchmarks 6 core capabilities and 18 detailed axes, aiming to evaluate LVLMs' multi-modal capacities with carefully balanced and purified samples. These samples are first roughly selected from current benchmarks with an automated pipeline, human review is then involved to ensure each curated sample exhibits visual dependency, minimal data leakage, and requires advanced multi-modal capabilities. Moreover, two metrics are developed to measure data leakage and actual performance gain in multi-modal training. We evaluate 16 leading LVLMs on MMStar to assess their multi-modal capabilities, and on 7 benchmarks with the proposed metrics to investigate their data leakage and actual multi-modal gain.
1 Introduction
The paper identifies visual irrelevance and training-data leakage as widespread problems in LVLM evaluation, then introduces MMStar to measure genuinely multimodal capability more accurately.
- Motivation: Existing benchmarks often contain samples answerable from textual world knowledge or clues in the question, making visual input unnecessary.This can reduce LVLM evaluation to testing the textual capabilities of their LLM backbones.
- Motivation: 42.9% of MMMU is achieved by GeminiPro without visual input, exceeding random choice by over 24% across six benchmarks.
- Motivation: Visual-necessary questions can also be answered without images when samples are memorized during LLM or LVLM training.Sphinx-X-MoE reaches 43.6% on MMMU without images, 17.9% above its LLM backbone.
- MMStar: MMStar contains 1,500 human-selected samples spanning 6 core capabilities and 18 detailed axes to evaluate purified, vision-dependent multimodal performance.
- MMStar: MMStar samples are filtered with LLM inspectors, difficulty-assessed by LVLMs, and manually reviewed for visual dependency, minimal leakage, and advanced multimodal requirements.The benchmark also introduces multimodal gain and multimodal leakage metrics.
- Results: 57.1% accuracy makes high-resolution GPT-4V the top model among 16 evaluated LVLMs on MMStar.
2 Related Work
Prior work developed broad LVLM benchmarks to assess general multimodal perception and reasoning, but existing evaluation procedures do not ensure visual necessity or account for training-data leakage.
- LVLM evaluation: Early single-task benchmarks such as VQA, MS-COCO, and OK-VQA do not holistically assess general multimodal perception and reasoning.
- LVLM evaluation: Comprehensive benchmarks were introduced to cover wider evaluation axes as LVLMs developed.
- Evaluation gap: Existing procedures can overlook whether samples are answerable without visual content and whether evaluation data leaked during multimodal training.
- Evaluation gap: Such omissions can produce unfair comparisons and misjudgments of the gains attributable to multimodal training.
3 Two Overlooked Issues for Evaluating LVLMs
Current LVLM evaluations face two problems: many samples do not require visual content, and some visual-necessary samples may be answered through training-data leakage. These issues can distort assessments of multimodal performance and motivate more reliable evaluation practices.
- Visual content is unnecessary: Many benchmark samples can be answered without images, reducing LVLM evaluation to the textual abilities of their LLM backbones.Answers may come from world knowledge or textual cues that directly reveal the answer.
- Visual content is unnecessary: 57.2% of ScienceQA samples and 46.2% of AI2D samples show abnormal hit rates under the study’s LLM-based measure.The abnormal hit rate counts samples correctly answered without visual input by at least six of eight evaluated LLMs.
- Unintentional data leakage: Some visual-necessary questions are answered without images by LLMs or LVLMs, suggesting inadvertent memorization during large-scale training.The study evaluates this issue across 22 models and six popular benchmarks.
- Unintentional data leakage: Figure 3 compares 16 LLM–LVLM-text pairs and counts benchmark samples answered by more than half of the LVLM-text models.The comparison highlights cases where the original LLM fails but its corresponding image-free LVLM succeeds.
- Implications: These inappropriate samples and leakage can misjudge actual multimodal gains and potentially misguide LVLM research.The paper argues that evaluation should account for leakage so measured performance reflects integrating and interpreting multimodal data rather than memorizing samples.
4 MMStar
MMStar is a vision-dependent benchmark curated through automated filtering and expert review, with balanced coverage of six core capabilities and eighteen detailed axes. It also introduces metrics to separate multi-modal gain from leakage.
- Data Curation: MMStar contains 1,500 high-quality samples selected from 22,401 initial samples through coarse filtering and manual review.The curation process retained 11,607 candidates before final selection.
- Data Curation: Each curated sample must require visual understanding, minimize data leakage, and demand advanced multi-modal abilities.Three experts review samples for visual grounding, capability coverage, and difficulty.
- Core Capabilities: MMStar organizes evaluation around six core capabilities and eighteen detailed axes, with relatively balanced coverage across dimensions.The benchmark assigns 250 samples to each core capability.
- Core Capabilities: The six capabilities include coarse perception, fine-grained perception, instance reasoning, logical reasoning, science and technology, and mathematics.These dimensions cover broad visual interpretation, detailed recognition, relational reasoning, logic, scientific knowledge, and quantitative reasoning.
- MG/ML Metrics: The proposed multi-modal gain and multi-modal leakage metrics are designed to measure actual training gains and inadvertent data leakage separately.The metrics address limitations of vanilla evaluation in revealing multi-modal benefits and enabling fair comparisons.
5 Experiments
Experiments evaluate LLMs and LVLMs on MMStar and assess multi-modal gain and leakage across seven benchmarks. Results show that MMStar is visually dependent, while current LVLMs still face substantial capability gaps and varying training effects.
- MMStar Results: LLMs perform close to random choice on MMStar, supporting the benchmark’s visual dependency and minimal leakage from LLM training.The evaluation covers two closed-source and twenty open-source LLMs.
- MMStar Results: 57.1% accuracy is achieved by high-resolution GPT4V, the best average MMStar score among evaluated LVLMs.Increasing GPT4V’s resolution and image-token count raises its score from 46.1% to 57.1%.
- MMStar Results: No LVLM reaches the 60% passing average in fine-grained perception, logical reasoning, science and technology, or mathematics.These four core capabilities remain particularly challenging for existing LVLMs.
- MG/ML Analysis: InternLM-XComposer2 records the highest average multi-modal gain among open-source LVLMs at 28.1, while LLaVA-1.5-7B records the lowest at 14.8.LLaVA-1.5-7B also exhibits minimal multi-modal leakage despite its lower gain.
- MG/ML Analysis: MMBench has the highest average multi-modal gain at 50.1, whereas MMMU has the lowest at 5.8 across evaluated benchmarks.The authors relate these differences to the degree of overlap between benchmark domains and LVLM training data.
6 Conclusion
The paper identifies visual irrelevance and training-data leakage as evaluation problems, then introduces MMStar and metrics for more accurate LVLM assessment.
- MMStar is a vision-dependent benchmark with 1,500 manually reviewed samples spanning 6 core capabilities and 18 detailed axes.
- Two proposed metrics measure data leakage and actual performance gain from multimodal training.
- Across MMStar, even the best evaluated model scores under 60 on average.
A Cases of Lacking Visual Dependency
Figure 6 highlights existing benchmark samples whose visual content is unnecessary for evaluation.
- Figure 6 highlights benchmark cases where evaluation samples lack the visual information necessary for answering.
B Cases of Data Leakage in LLMs’ Training Data
Figure 7 highlights existing benchmark samples that leaked into LLM training data.
- Figure 7 highlights benchmark cases where evaluation samples are leaked into LLMs’ training data.
C Cases of Data Leakage in LVLMs’ Multi-Modal Training Data
Figure 8 highlights existing benchmark samples that leaked into LVLM multimodal training data.
- Figure 8 highlights benchmark cases where evaluation samples are leaked into LVLMs’ multimodal training data.
D Detailed Evaluation Results of LVLMs on Six Multi-modal Benchmarks
Table 7 evaluates various LVLMs across six popular multi-modal benchmarks under aligned LLM, LVLM-text, and image-accessing evaluation strategies.
- The evaluation covers various LVLMs across six popular multi-modal benchmarks.
- The LLM strategy evaluates each model's corresponding LLM base using 0-shot inference.
- The LVLM-text strategy evaluates LVLMs without accessing images.