PaperlayerView exact revision on arXiv

Source-linked AI summary

MM-LIMA: Less Is More for Alignment in Multi-Modal Datasets

Lai Wei, Xiaozhe Li, Zihao Jiang, Weiran Huang, Lichao Sun

arXiv:2308.12067v3cs.LGcs.AIcs.CLcs.CV

TL;DR

Low-quality examples are common in multimodal instruction-tuning data, while comprehensive quality-assessment methods are lacking. MM-LIMA combines quality indicators with a trainable data selector to choose 200 instructions, and reports better results than MiniGPT-4 across multimodal evaluations.

  • Problem

    Multimodal instruction-tuning data can contain incorrect or irrelevant responses, and comprehensive methods for assessing vision-language data quality are lacking.

    Source: §1 · p. 2

  • Method

    MM-LIMA combines multimodal instruction-quality indicators with a trainable selector to identify and filter low-quality examples for fine-tuning.

    Source: §1 · pp. 2–3

  • Results

    Across multimodal benchmarks, MM-LIMA outperforms MiniGPT-4; using 200 instructions yields a +23 score enhancement on MME.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The findings indicate that vision-language instruction-data quality can outweigh quantity in multimodal model fine-tuning.

    Source: §1 · p. 3, §6 · p. 11

    The indicator–quality-label correlation analysis is limited to fixed MiniGPT-4-7B; other model sizes and LLaVA are proposed for future study.

    Source: D Limitation and Future Works · p. 18

Abstract

from arXiv · show

Multimodal large language models are typically trained in two stages: first pre-training on image-text pairs, and then fine-tuning using supervised vision-language instruction data. Recent studies have shown that large language models can achieve satisfactory results even with a limited amount of high-quality instruction-following data. In this paper, we introduce MM-LIMA, which is fine-tuned on a small dataset comprising only 200 examples, amounting to approximately 6% of the instruction-following data used in the alignment dataset for MiniGPT-4. To achieve this, we first propose several metrics to access the quality of multimodal instruction data. Based on these metrics, we present an effective and trainable data selector to automatically identify and filter low-quality vision-language data. By employing this method, MM-LIMA outperforms the original MiniGPT-4 on various evaluations. Overall, our findings demonstrate that less but high-quality instruction tuning data is efficient in enabling multimodal large language models to generate better output. Our code is available at https://github.com/waltonfuture/InstructionGPT-4.

1 Introduction

Multimodal large language models use image-text pre-training and vision-language instruction tuning, but instruction data can be low quality and lacks comprehensive quality-assessment guidelines. MM-LIMA introduces data-quality indicators and a trainable selector, then reports better benchmark performance using 200 instructions.

1 Introduction

  • The work addresses low-quality instruction examples and the lack of comprehensive methods for evaluating vision-language data quality.Prior LIMA results motivate testing whether reduced, high-quality instruction data can improve multimodal alignment.Source: §1 · p. 2
  • The proposed indicators include CLIP Score, GPT Score, Reward Score, Length Score, and multimodal features; the table describes their roles in assessing image-caption fit, caption quality, and data characteristics.Source: §1 · p. 2, Table 1 · p. 3
  • The selector predicts instruction-data quality from indicators and multimodal features, using performance on validation data as genuine quality labels and clustering to preserve data diversity.It is applied to the original 3.4K MiniGPT-4 fine-tuning examples before selecting the smaller subset.Source: §1 · p. 2
  • Using 200 instructions (6% of MiniGPT-4's data), MM-LIMA improves over MiniGPT-4 by +23 on MME, +1.55 on MMBench, and +1.76% on VQA datasets.The evaluations include MME, MMBench, and VQA datasets from LVLM-eHub.Source: §1 · pp. 2–3
  • MM-LIMA outperforms MiniGPT-4 on 8 of 14 MME tasks and 13 of 20 MMBench abilities, and excels on all four LVLM-eHub VQA datasets.Source: §1 · p. 3

2 MM-LIMA

MM-LIMA selects 200 high-quality instructions from MiniGPT-4’s 3.4K set [1] using diversity and quality principles. Its selector relates subset embeddings to validation-score labels and ranks samples within clusters.

2 MM-LIMA

  • MM-LIMA fine-tunes on 200 high-quality instructions selected from the 3.4K instructions used for MiniGPT-4’s second-stage training [1].The paper presents high-quality instruction selection as the core of MM-LIMA.Source: §2 · p. 3

2.1 Selecting Principle

  • The selection principles are diversity and quality: spectral clustering divides image embeddings into ten categories, while quality reflects data’s ability to steer response generation.The paper describes most MLLM knowledge as coming from pre-training, motivating diverse instruction data for alignment.Source: §2.1 · p. 4

2.2 Indicators and Genuine Quality Labels

  • A subset’s genuine quality label is the average validation score of an MLLM fine-tuned on that subset.The paper proposes this performance-based label because directly evaluating data quality by training MLLMs across datasets can be inefficient.Source: §2.2 · p. 4
  • Each sample embedding combines CLIP image-response matching, response length, reward and GPT-4 scores with reduced frozen image-text features.The paper motivates dimensionality reduction because high-dimensional image-text features can make fitting prone to overfitting.Source: §2.2 · pp. 4–5

2.3 Data Selector

  • The selector is trained on subset-level embeddings paired with genuine quality labels, using an architecture such as self-attention to fit the relationship.The described procedure splits the dataset into subsets, evaluates models trained on each subset, and aligns their scores with concatenated indicators.Source: §2.3 · p. 5, Figure 2 · p. 5
  • At selection time, the method clusters images, ranks triplets by predicted quality within groups, and combines selected groups into the final dataset.The selected dataset S has a target size α.Source: §2.3 · pp. 5–6

3 Experimental Setup

The selector is trained and applied on cc_sbu_align, the 3,439-triplet dataset used for MiniGPT-4’s second-stage fine-tuning [1]. Evaluation spans VQA datasets, MMBench, MME, and GPT-4-judged LLaVA-Bench.

3.1 Implementation Details

  • K-means++ partitions cc_sbu_align into 30 subsets of 114 examples; the self-attention selector has two residual-connected layers, and the selected set contains 200 examples.The concatenated multimodal feature size is 6; the source dataset contains 3,439 image-instruction-response triplets.Source: §3.1 · p. 6

3.2 Evaluation

  • GQA, IconQA, ScienceQA and OKVQA evaluate models tuned on different subsets and provide the genuine quality-label metrics.The paper also evaluates zero-shot VQA performance on DocVQA, TextVQA, STVQA and VizWiz.Source: §3.2 · p. 6
  • MMBench and MME assess complex multimodal tasks, while GPT-4 judges MiniGPT-4 and MM-LIMA responses on LLaVA-Bench against reference answers.Figure 3 presents a comparison of MM-LIMA and MiniGPT-4 on MMBench.Source: §3.2 · p. 6, Figure 3 · p. 7

4 Experimental Results

Across benchmark, GPT-4, and demonstration evaluations, MM-LIMA performs better than MiniGPT-4 despite using a small selected dataset. Ablations identify useful indicators, self-attention, clustering, and task-dependent data and feature sizes.

4.1 Benchmark Scores

  • +23 score on MME, +1.55 score on MMBench, and +1.76% on VQA datasets versus MiniGPT-4; MM-LIMA also leads on all four LVLM-eHub VQA datasets.It outperforms MiniGPT-4 in 8 of 14 MME tasks and 13 of 20 MMBench abilities.Source: §4.1 · p. 7, Table 2 · p. 8, Table 4 · p. 8

4.2 GPT-4 Evaluation

  • Across 60 GPT-4-judged questions, MM-LIMA wins 26, loses 16, and ties 18 against MiniGPT-4.Both response orders were evaluated to address position bias.Source: §4.2 · pp. 7–8, Figure 4 · p. 7

4.3 Demonstrations

  • In a challenging image-understanding and conversation example, MM-LIMA outperforms MiniGPT-4 on complex image reasoning despite its smaller instruction dataset.The examples are described in Tables 9 and 10 in Appendix B.2.Source: §4.3 · p. 8

4.4 Ablation Study

  • Each tested indicator—CLIP, Length, Reward, GPT, and multimodal features—selects data yielding better performance than random sampling.The study evaluates each indicator separately on 200 selected examples.Source: §4.4 · p. 8, Table 5 · p. 9
  • Self-attention selectors perform best among linear, MLP, and self-attention structures, and two attention layers are the most suitable tested configuration.Self-attention enables feature and score interaction within embeddings; Figure 5 reports the structure comparison, while Table 5 summarizes layer counts.Source: §4.4 · p. 9, Figure 5 · p. 9, Table 5 · p. 9
  • Clustering improves performance across selector structures, feature size 6 performs best, and the required selected-data size depends on evaluation scope.Fifty examples suffice for VQA evaluation, whereas 200 are needed to comprehensively outperform MiniGPT-4 on MME and MMBench; Figure 6 distinguishes feature sizes from curated-data sizes.Source: §4.4 · pp. 9–10, Figure 6 · p. 10

5 Related Works

Related work covers visual instruction tuning and the curation of instruction data. MM-LIMA is presented as a multimodal data-selection approach that differs from prior single-modal curation and instruction-mining methods.

Visual Instruction Tuning

  • Visual instruction tuning adapts language models for multimodal tasks, with prior work using linear projection bridges and either self-instruct or existing multimodal datasets.Examples include MiniGPT-4 [1], LLaVA [3], LLaMA-Adapter [39], and InstructBLIP [9].Source: §5 · p. 10

Instruction Curation

  • Prior curation work filters low-quality data or constructs curated examples, while MM-LIMA presents a trainable selector for multimodal instruction data.LIMA [14] uses 1,000 human-curated examples; other work [15] [16] [26] evaluates instruction quality, and Instruction Mining [16] uses a linear selection rule.Source: §5 · p. 10
  • MM-LIMA is distinguished from the cited single-modal work by its multimodal data selection and from Instruction Mining [16] by its broader quality-evaluation pipeline.Source: §5 · p. 10

6 Conclusion

The conclusion emphasizes selecting fewer, higher-quality multimodal instructions and evaluating the resulting selector and model. The included appendix passages describe implementation context, GPT-4 rating prompts, and genuine quality-label construction.

6 Conclusion

  • The conclusion argues that carefully selected, smaller amounts of high-quality multimodal instruction data can improve model performance.It also reports analysis of the selector and evaluations of coherent and accurate outputs.Source: §6 · p. 11

A Implementation Details of MM-LIMA

A.1 GPT Prompt

A.2 Genuine Quality Labels

  • Genuine quality labels are formed by fine-tuning a model on 30 clustered subsets of 114 examples each and scoring them on four validation datasets.The validation datasets are GQA [28], IconQA [29], OKVQA [31], and ScienceQA [30].Source: §A.2 · p. 14

A.3 Selection Algorithm Implementation Details

The appendix describes clustered, per-sample scoring and ranking for data selection, alongside settings used in the experiments. The selection procedure distinguishes clustering for label construction from clustering used to diversify selected data.

A.4 Experimental Setting Details

  • The experimental setting excludes instructions from triplet indicators because dataset instructions have fixed formats, and uses PCA to reduce frozen ViT and Llama2 multimodal features.Source: §A.4 · p. 15
  • Selector training uses 20 epochs and learning rate 0.01; instruction tuning uses pre-trained 7B MiniGPT-4 with its original fine-tuning hyperparameters.Indicator ablations sort samples after clustering using CLIP, Reward, or GPT scores, and also test length ranges and a self-attention selector using multimodal features.Source: §A.4 · pp. 15–16

A.3 Selection Algorithm Implementation Details

  • For selection, image embeddings are spectrally clustered into 10 groups; each sample receives indicator and multimodal-feature scores, a predicted quality label, and a top-ranked selection within its group.The algorithm combines selected subsets into the returned dataset; the appendix distinguishes this clustering from the earlier clustering used to create genuine quality labels.Source: §A.3 · p. 14, §A.4 · p. 16

A.5 Evaluation Benchmarks

The evaluation benchmarks cover perception and cognition, multidimensional multiple-choice abilities, several VQA datasets, and judged image-based responses. Together, they assess model performance across distinct multimodal tasks and prompt settings.

A.5 Evaluation Benchmarks

  • MME measures perception and cognition across 14 subtasks, using concise yes-or-no questions and descriptions to limit prompt-engineering effects.Its overall score is 2800, and each subtask has a maximum score of 200.Source: §A.5 · p. 16
  • MMBench uses 2,974 multiple-choice questions across 20 ability dimensions organized into six levels, with ChatGPT matching predictions to answer choices.The passage describes a single correct answer for each question.Source: §A.5 · p. 16
  • The VQA evaluation uses GQA, IconQA, ScienceQA, and OKVQA as quality-label datasets, and tests zero-shot ability on DocVQA, TextVQA, STVQA, and VizWiz.Top-1 accuracy is used for these tasks.Source: §A.5 · p. 16
  • LLaVA-Bench uses GPT-4 to compare MiniGPT-4 and MM-LIMA responses against reference answers, assessing robustness to different prompts.Its 24 images have 60 questions spanning conversation, detailed description, and complex reasoning.Source: §A.5 · p. 16

B.1 MMBench Results

The MMBench appendix identifies the abilities covered by its performance comparison, while additional demonstrations describe MM-LIMA responses as more comprehensive and fluent than MiniGPT-4's.

B.1 MMBench Results

  • MMBench assesses approximately 3,000 questions across 20 ability dimensions and six levels; Table 8 illustrates the abilities assessed.The passage gives no specific performance values or comparison outcome.Source: §B.1 · p. 17, Table 8 · p. 17

B.2 Additional Demonstrations

  • MM-LIMA responses in Tables 9 and 10 are described as more comprehensive and fluent than MiniGPT-4 responses.The passages do not provide numerical scores for these demonstrations.Source: §B.2 · p. 17, Table 9 · p. 18
  • MM-LIMA's image-based story is described as vivid and detailed, while its food-image recipe demonstrates step-by-step chain-of-thought responses.The examples appear in Table 9 and Table 10, respectively.Source: §B.2 · p. 17, Table 9 · p. 18

C Selected Samples Versus Original Samples

The selected 200 samples are described as higher quality than the original data, with more comprehensive, detailed, and error-free language that aligns with human preferences.

C Selected Samples Versus Original Samples

  • The selected 200 data points exhibit more comprehensive, detailed, and error-free language than the original data.The authors state that this comparison demonstrates the selector can choose data aligned with human preferences.Source: C Selected Samples Versus Original… · p. 18
  • The selected data are described as having no apparent grammatical or syntactical errors.This is part of the reported qualitative comparison with the original dataset.Source: C Selected Samples Versus Original… · p. 18
  • The authors interpret the comparison as evidence that the data selector can choose samples aligned with human preferences.The passage points to Figure 7 when describing this interpretation.Source: C Selected Samples Versus Original… · p. 18

D Limitation and Future Works

The experiments examine indicator-quality correlations only on MiniGPT-4-7B. The authors identify broader model coverage and a more generalized selector as directions for future work.

D Limitation and Future Works

  • The reported indicator–quality-label correlation experiments are limited to the fixed MiniGPT-4-7B model.The authors suggest extending analysis to MiniGPT-4-13B and LLaVA.Source: D Limitation and Future Works · p. 18
  • The authors identify a more generalized data selector, including more evaluation indicators, as a future direction.This is presented as a potential avenue for exploration.Source: D Limitation and Future Works · p. 18
  • The authors also propose multimodal instruction mining as a future research direction.They state this may provide further insights and opportunities in the domain.Source: D Limitation and Future Works · p. 18
Loading 2308.12067v3…