PaperlayerView exact revision on arXiv

Source-linked AI summary

Both Text and Images Leaked! A Systematic Analysis of Data Contamination in Multimodal LLM

Dingjie Song, Sicheng Lai, Mingxuan Wang, Shunian Chen, Lichao Sun, Benyou Wang

arXiv:2411.03823v3cs.CVcs.AIcs.CLcs.MM

TL;DR

Because MLLM training data are partly obscured and unimodal contamination detectors do not address multimodal data and multi-stage training, the paper develops MM-DETECT to analyze leakage across VQA tasks. Across twelve MLLMs and five benchmarks, it reports significant contamination, including origins in unimodal pretraining as well as multimodal training.

  • Problem

    Existing contamination detectors focus on LLMs and are limited for MLLMs because multimodal data and multi-stage training complicate detection.

    Source: §1 · p. 1

  • Method

    MM-DETECT analyzes multimodal contamination by distinguishing unimodal from cross-modal exposure and testing performance before and after semantic-irrelevant perturbations.

    Source: §2.1 · p. 2, §3 · p. 3

  • Results

    Across twelve MLLMs and five VQA datasets, the authors report significant contamination from multimodal training and unimodal pretraining, with older benchmarks disproportionately affected.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings indicate that multimodal evaluation reliability and fairness require attention to contamination originating in both unimodal pretraining and multimodal fine-tuning.

    Source: §7 · p. 9

    The study covers visual modalities and representative multiple-choice and caption datasets, not audio, video, open-ended generation, or cloze questions.

    Source: Limitations · p. 9

Abstract

from arXiv · show

The rapid advancement of multimodal large language models (MLLMs) has significantly enhanced performance across benchmarks. However, data contamination-unintentional memorization of benchmark data during model training-poses critical challenges for fair evaluation. Existing detection methods for unimodal large language models (LLMs) are inadequate for MLLMs due to multimodal data complexity and multi-phase training. We systematically analyze multimodal data contamination using our analytical framework, MM-Detect, which defines two contamination categories-unimodal and cross-modal-and effectively quantifies contamination severity across multiple-choice and caption-based Visual Question Answering tasks. Evaluations on twelve MLLMs and five benchmarks reveal significant contamination, particularly in proprietary models and older benchmarks. Crucially, contamination sometimes originates during unimodal pre-training rather than solely from multimodal fine-tuning. Our insights refine contamination understanding, guiding evaluation practices and improving multimodal model reliability.

1 Introduction

Multimodal model performance is difficult to assess fairly because training data are partly obscured and existing contamination detectors are not suited to MLLMs. The paper introduces MM-DETECT to analyze contamination forms, severity, and training-stage origins, then reports significant leakage across models and benchmarks.

1 Introduction

  • Existing LLM contamination detectors are inadequate for MLLMs because multimodal data and multi-stage training complicate detection.The paper identifies the resulting need for a multimodal-specific analytical framework.Source: §1 · p. 1
  • MM-DETECT distinguishes unimodal from cross-modal contamination and applies separate methods to multiple-choice and caption-based VQA tasks.The methods are Option Order Sensitivity Test and Slot Guessing for Perturbed Caption, respectively.Source: §1 · p. 2
  • Evaluations across twelve MLLMs and five VQA datasets found significant contamination, including contamination from unimodal pretraining and disproportionate impact on older benchmarks.The authors also report contamination in both multimodal training data and unimodal pretraining.Source: §1 · p. 2
  • The paper’s stated contributions include quantifying benchmark-leakage effects on performance metrics and characterizing model and dataset susceptibility.It presents this as the first systematic analytical examination of multimodal data contamination.Source: §1 · p. 2

2 Preliminaries

The preliminaries frame multimodal contamination as exposure through either unimodal or multimodal training data and explain why detecting it requires attention to MLLM-specific methods and training stages.

2 Preliminaries

  • The preliminaries formally define multimodal contamination and set out the challenges of detecting it in MLLMs.The definition distinguishes exposure through unimodal and multimodal data sources.Source: §2 · p. 2, §2.1 · p. 2

2.1 Definition of Multimodal Data Contamination

  • Multimodal contamination can enter through either unimodal text pretraining or multimodal training data, unlike contamination confined to a single modality.The paper distinguishes unimodal contamination, where text input or text-label pairs occur in pretraining, from cross-modal contamination, where image-text inputs or image-text-label examples occur in multimodal training.Source: §2.1 · p. 2

2.2 Challenges in Multimodal Detection

  • Unimodal retrieval, logits, masking, and output-comparison methods each encounter MLLM-specific detection problems.Retrieval struggles with multimodal information; logits differences shrink after instruction tuning, image clues distort masking tests, and caption outputs have low similarity.Source: §2.2 · p. 2
  • Contamination may enter during pretraining, multimodal fine-tuning, or image-text alignment, motivating a detector tailored to VQA and a heuristic for tracing training-phase origins.Benchmark question text may appear in pretraining corpora, while benchmark samples or internet-scale mixed image-text data may appear in later stages.Source: §2.2 · p. 3

3 Detection Framework: MM-DETECT

MM-DETECT detects contamination by measuring model-performance discrepancies before and after semantically irrelevant perturbations. It adapts perturbations to multiple-choice and caption-based tasks, then analyzes contamination at dataset and instance levels.

3 Detection Framework: MM-DETECT

  • MM-DETECT detects unusual performance discrepancies before and after semantically irrelevant perturbations, using two task-specific perturbations and predefined metrics.The framework analyzes contamination at both dataset and instance levels.Source: §3 · p. 3

3.1 Option Order Sensitivity Test

  • For multiple-choice VQA, Option Order Sensitivity Test shuffles answer choices while keeping the image and question unchanged; sensitivity to order suggests possible memorization of a canonical order.The test compares prompts before and after shuffling the correct answer’s position.Source: §3.1 · pp. 3–4

3.2 Slot Guessing for Perturbed Caption

  • For caption-based VQA, Slot Guessing masks extracted keywords in original and back-translated captions, testing whether the model predicts missing content despite paraphrasing.The authors’ intuition is that predicting a missing part of the original caption but failing on its back-translated version may indicate prior exposure.Source: §3.2 · p. 4

3.3 Detection Metrics

  • At dataset level, MM-DETECT uses the reduction in performance after perturbation, ∆, as an indicator of familiarity with the original benchmark relative to the perturbed set.A significant negative ∆ suggests potential extensive leakage and perturbation-sensitive performance.Source: §3.3 · pp. 4–5
  • At instance level, Φ is the share of cases correct before but incorrect after perturbation; a larger Φ indicates a higher likelihood of instance leakage.The framework also highlights performance drops after perturbation and is described as lower-overhead, sensitive, and applicable in black-box settings.Source: §3.3 · p. 5

4 Evaluating MM-DETECT with Intentional Contamination

The intentional-contamination experiments test MM-DETECT’s effectiveness, sensitivity to leakage levels, and ability to diagnose evaluation bias. Across these tests, contamination raises performance metrics, while MM-DETECT’s contamination-sensitive metrics detect changes in leakage.

4.1 MM-DETECT is An Effective Detector

  • ScienceQA-contaminated models increased average CR and PCR by 9.0% and 5.9%, while NoCaps-contaminated models increased them by 5.2% and 1.1% versus baseline.All contaminated models also showed a marked decrease in Δ.Source: §4.1 · p. 5, Table 1 · p. 5

4.2 MM-DETECT is Sensitive and Fine-grained

  • As contamination increased from 10% to 50% to 100%, CR and PCR increased and Δ grew progressively, distinguishing leakage levels.The experiments used reduced contamination amounts inserted late in training, relative to a fully contaminated baseline.Source: §4.2 · pp. 5–6, Figure 5 · p. 5

4.3 MM-DETECT Diagnoses Evaluation Bias from Training-set Leakage

  • On ScienceQA, the leaked model exceeded the clean model by 2.9% in CR and 2.3% in PCR, with Δ of -0.5; COCO showed Δ of -3.2.These comparisons indicate training-set leakage inflates performance and MM-DETECT detects it.Source: §4.3 · p. 6, Table 2 · p. 6
  • MM-DETECT’s Δ remained unaffected by prompt changes, supporting robustness to prompt variation.The section also notes that both training- and test-set leakage can result in unfairness.Source: §4.3 · p. 6

5 Assessing the Extent of Contamination in MLLMs

The assessment examines contamination across MLLMs and five VQA benchmarks. It finds contamination in both model types and dataset types, with leakage patterns varying by benchmark and training versus test split.

5.1 Setup

  • The analysis evaluates nine open-source and three proprietary MLLMs across two multiple-choice and three caption datasets, using validation sets for COCO-Caption2017 and NoCaps.The newer MMStar and Vintage datasets serve as contrasts to older benchmarks.Source: §5.1 · p. 6

5.2 Main Results

  • On multiple-choice datasets, both open-source and proprietary models show contamination; training-set leakage is generally more pronounced than test-set leakage, and older ScienceQA shows more leakage than newer MMStar.Claude-3.5-Sonnet has Δ = −5.3 on ScienceQA training versus −2.4 on its test set.Source: §5.2 · p. 6
  • Caption-dataset leakage varies by benchmark: open-source models show more contamination on NoCaps, while proprietary models show more on COCO-Caption2017.Both model types exhibit contamination on caption datasets.Source: §5.2 · p. 7, Table 4 · p. 7
  • Contamination appears at dataset and instance levels across open-source and proprietary MLLMs on multiple-choice and image-caption datasets.The evaluation covers twelve models and five benchmarks.Source: §5.2 · pp. 6–7

6 Identifying the Origin of Contamination in MLLMs

The section investigates whether contamination originates in unimodal pre-training as well as multimodal fine-tuning. Its analyses find evidence consistent with both sources contributing to contamination in MLLMs.

6.1 Heuristic Detection of Unimodal Pre-training Contamination

  • Correct answers to image-required MMStar questions without image input, with guessing inhibited, serve as a heuristic signal of possible unimodal pre-training leakage.The evaluated underlying LLMs showed varied contamination rates, and higher LLM contamination correlated with increased MLLM leakage; some contamination may therefore originate in pre-training.Source: §6.1 · p. 8

6.2 Analyzing Cross-modal Contamination in Multimodal Fine-tuning

  • The cross-modal analysis compares benchmark construction with MLLM visual-instruction-tuning data, but models labeled green may still risk overlap from large-scale or undisclosed training sources.The examined sources include interleaved image-text datasets, online-derived datasets, and in-house data.Source: §6.2 · p. 8, Table 6 · p. 8

6 Identifying the Origin of Contamination in MLLMs

  • The paper concludes that contamination may stem from both cross-modal and unimodal sources, and that both can significantly impact overall performance.The source analyses cover unimodal pre-training and cross-modal overlap during visual instruction tuning.Source: §6.2 · p. 8, §6 · p. 8

7 Conclusion and Future Work

The conclusion presents MM-DETECT as a framework for quantifying contamination and identifying performance bias. It identifies both unimodal pre-training and multimodal fine-tuning as contamination sources, and outlines two future-work directions.

7 Conclusion and Future Work

  • MM-DETECT quantifies varying contamination degrees and reveals performance biases from benchmark leakage, with contamination arising from both unimodal pre-training and multimodal fine-tuning.The authors link these sources to the reliability and fairness of multimodal evaluations.Source: §7 · p. 9
  • Future work proposes standardizing multimodal dataset use and reporting contamination impacts to minimize contamination and improve data consistency and quality.These are stated as a first future-work priority.Source: §7 · p. 9
  • The authors also propose a continuously updated benchmarking system for ongoing evaluation of multimodal model performance.They state that this would support advances and broader applications in the field.Source: §7 · p. 9

Limitations

The work is limited to visual modalities and selected VQA dataset types. Its detectors also face possible underestimation and option-order sensitivity under specified conditions.

Limitations

  • The analysis covers visual modalities, not audio or video.Source: Limitations · p. 9
  • Evaluation uses representative multiple-choice and caption datasets, not open-ended generation or cloze questions.The authors speculate that Slot Guessing for Perturbed Caption may apply to other image-feature-analyzing benchmarks.Source: Limitations · p. 9
  • Option Order Sensitivity Test can be undermined by option shuffling, which may improve performance but is computationally expensive and may increase training cost.Source: Limitations · p. 9
  • MM-DETECT may underestimate contamination when a model generalizes well enough to answer perturbed questions correctly; eliminating all such false negatives remains open.Dataset-level evaluations reduce this risk.Source: Limitations · p. 9

A Inefficiency of Unimodal Methods

The section examines unimodal contamination methods and caption-perturbation procedures alongside task-specific contamination-degree thresholds. The supplied passages describe why established detection approaches face mismatches with MLLMs.

A.1 Logits-base

  • Instruction fine-tuning can reduce token-probability differences, limiting logits-based contamination detection on MLLMs.LLaVA-1.5-13b exhibits extremely low perplexity on multimodal benchmark datasets.Source: §A.1 · p. 12, Table 7 · p. 12

A.2 Masking-base

  • Images may contain masked sentence portions, giving MLLMs answers and exaggerating masking-based contamination detection success.LLaVA-1.5-13b has a high Exact Match probability when predicting masked words.Source: §A.2 · p. 12, Table 8 · p. 12

A.3 Comparison-base

  • Data augmentation can make MLLM outputs diverge from benchmark labels, hindering output-similarity detection; CDD reports 0% contamination across the multimodal benchmarks.Source: §A.3 · p. 13, Table 9 · p. 13

B.1 Back-Translation

  • Back-translation generates a paraphrased caption by translating the original into Chinese and back into its original language.Source: §B.1 · p. 13

B.2 Keyword Extraction

  • Caption keywords are extracted from both original and paraphrased captions using Stanford POS tagging for nouns, adjectives, and verbs.Source: §B.2 · p. 13

B.3 Keyword Masking

  • The masking procedure randomly selects an extracted keyword and replaces its first occurrence in the caption with [MASK].Source: §B.3 · p. 13

C Contamination Degree Analysis

D.1 Semantic & Lexical Similarity After Back-Translation

Back-translation largely preserves caption semantics while changing wording. The uncertainty instruction reduces apparent accuracy in an image-free test, and prompt variations leave the contamination-sensitive metric largely stable.

D.1 Semantic & Lexical Similarity After Back-Translation

  • 0.914 SBERT similarity on the VINTAGE split marks the strongest semantic preservation; all three datasets score close to 0.9.Source: §D.1 · p. 14, Table 10 · p. 14
  • Pearson correlations of 0.38-0.42 show only weak positive coupling between semantic similarity and BLEU-4 lexical overlap.The authors report comparatively low BLEU-4 values, consistent with wording changes despite preserved meaning.Source: §D.1 · p. 14, Table 10 · p. 14

D.2 Sanity Check for the “I don’t know” Instruction

  • 19.2% drop (44.8% to 25.6%) in LLaMA2-7B apparent accuracy with the uncertainty instruction; authors attribute nearly half of uninstructed correct answers to lucky guesses.Source: §D.2 · p. 14, Table 11 · p. 14

D.3 Prompt Consistency

E Detailed Dataset Overlap Analysis

Determining benchmark overlap with MLLM training data is difficult because complete training datasets are often unavailable and exact matching is complicated by augmentation and images.

E Detailed Dataset Overlap Analysis

Loading 2411.03823v3…