PaperlayerView exact revision on arXiv

Source-linked AI summary

MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding

Xin Jin, Siyuan Li, Siyong Jian, Kai Yu, Huan Wang

arXiv:2510.23479v3cs.CV

TL;DR

SFT and RL each have limitations for MLLM alignment, and augmentation-based preference pairs leave control over data quality unresolved. MergeMix uses token-merge mixup to create aligned image-label samples and clean/augmented preference pairs, then reports gains across image classification and MLLM benchmarks.

  • Problem

    SFT requires high-quality annotations and does not explicitly model relative preferences, while RL-based alignment can require a reward model that may be biased or exploited; controlling preference-pair quality also remains open.

    Source: §1 · p. 1

  • Method

    MergeMix uses token merging to generate mixed images with aligned labels, treats clean and augmented images as preferred and dispreferred pairs, and applies mixed SimPO with a soft preference margin.

    Source: §1 · p. 2, §4.1 · p. 5, §4.2 · p. 6

  • Results

    MergeMix improves image-classification performance and shows beneficial alignment and generalization on MLLM benchmarks; it reports a +2.88% average gain over Qwen2.5-VL-Instruction.

    Source: §5.5 · p. 9, §6 · p. 10

  • Takeaways & Limitations

    The authors present MergeMix as a promising scalable and robust training paradigm for multimodal systems.

    Source: §6 · p. 10

    MergeMix currently augments only the image modality, leaving text untouched, and its token-merging policy is static and unlearned.

    Source: §6 · p. 10

Abstract

from arXiv · show

Vision-language alignment in multi-modal large language models (MLLMs) relies on supervised fine-tuning (SFT) or reinforcement learning (RL). To align multi-modal large language models (MLLMs) in the post-training stage, supervised fine-tuning (SFT) is a stable choice but requires human annotations and lacks task generalizations, while Reinforcement Learning (RL) searches for better answers from reward signals but suffers from computational overhead and instability. To achieve balance among scalability, efficiency, and alignment generalizations, we propose MergeMix, a unified paradigm that bridges SFT and RL with an efficient Token Merge based Mixup augmentation. As for the Mixup policy, we generate contextual aligned mixed images with the corresponding labels according to the merged attention maps with cluster regions. Then, we enhance the preference-driven paradigm for MLLMs by building preference pairs with raw images and MergeMix-generated ones and optimizing the soft preference margin with the mixed SimPO loss. Extensive experiments demonstrate that MergeMix not only achieves dominant classification accuracy as an augmentation method but also improves generalization abilities and alignment of MLLMs, providing a new learning paradigm for preference alignment with training efficiency and stability.

1 INTRODUCTION

SFT and RL offer different strengths for MLLM alignment, while existing augmentation-based preference pairs leave control over loser quality unresolved. MergeMix uses token-merge mixup to generate aligned samples and preferences, bridging SFT and RL with a mixed SimPO loss.

1 INTRODUCTION

  • SFT needs high-quality instruction–response annotations and does not explicitly model relative preferences, while RL-based methods use reward models that may be biased or exploited.The introduction frames these as complementary limitations in post-training alignment.Source: §1 · p. 1
  • Existing augmentation-built preference pairs leave open how to control pair quality; random augmentations and data-independent DPO can produce low-quality losers and discard training data.The cited SeVa example selects hard negatives, which the passage says can harm some visual question answering tasks.Source: §1 · p. 1
  • MergeMix generates contextual mixed samples with token merging and aligned labels, then treats augmented images as losers and clean images as winners for preference tuning.Its mixed SimPO loss uses the mixing ratio as a soft preference margin for adaptive optimization.Source: §1 · p. 2
  • The introduction reports that MergeMix achieves competitive performance in image classification and MLLM benchmarks with favorable efficiency.It also characterizes the approach as improving alignment with human preferences while preserving SFT's stability and scalability.Source: §1 · p. 2

2 RELATED WORK

The related work organizes image mixup into static and adaptive approaches and reviews token compression methods for MLLMs by where they reduce tokens. MergeMix is positioned among efforts to make augmentation data-dependent and training more efficient.

2 RELATED WORK

  • The section surveys existing image-classification mixup and MLLM token-compression work as the two relevant research areas.It introduces these areas in terms of augmentation and efficient training or inference.Source: §2 · p. 2

Mixup Augmentations

  • Image mixup methods are grouped into static approaches using priors or randomness and adaptive approaches guided by data-dependent metrics.The section gives examples ranging from MixUp and CutMix to attention- and gradient-guided methods.Source: §2 · p. 3

Token Compression in MLLMs

  • MLLM token compression methods reduce tokens in either the encoder or decoder to improve efficiency or acceleration.Encoder methods include pruning, clustering, and TopK selection; decoder methods include staged dropping and adaptive pruning.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries describe mixup as combining sampled training examples through a policy that supplies a mask and mixing ratio. They also define preference optimization in terms of contrasting preferred and dispreferred responses, including DPO's direct policy optimization.

Reformulation of Mixup Augmentation

  • Mixup samples two training example–label pairs and a Beta-distributed λ, then applies a policy-generated mask and optimized ratio to produce mixed images and labels.Without optimization, the adjusted ratio retains the initial sampling ratio; MergeMix can recompute it from the mask values.Source: §3 · p. 3

Preference Tuning for MLLMs

  • Preference optimization contrasts preferred and dispreferred responses using a scoring function to reflect model preference.The general formulation denotes the responses as y+ and y−.Source: §3 · p. 4
  • DPO directly optimizes the policy model with preference pairs rather than training a separate reward model.It encourages higher likelihood for preferred responses than dispreferred ones while staying close to a frozen reference model.Source: §3 · p. 4

4 MERGEMIX TRAINING PARADIGM

MergeMix uses token merging and attention recovery to create mixed images, then adapts the mixing ratio within a preference-training objective for MLLMs. Its pipeline connects image-space augmentation to SFT and preference ranking.

4.1 IMAGE MIXING VIA TOKEN MERGE

  • MergeMix uses the relationship between merge and mixing ratios to generate mixed images and labels, then mixes samples for training.The merge ratio measures raw-sample information, while the mixing ratio balances information between samples.Source: §4.1 · pp. 4–5
  • ToMeAttention merges visual tokens and provides attention maps, condensed feature tokens, and a source map relating them to the raw sequence.The merge count r controls how many tokens are merged.Source: §4.1 · pp. 4–5
  • The source map guides recovery of the merged attention map to the original sequence length, restoring spatial dependencies before generating a mixing mask.The passage contrasts this recovery with TopK selection, which discards low-ranked tokens and their spatial relationships.Source: §4.1 · p. 5

4.2 A UNIFIED AUGMENTATION PARADIGM: FROM IMAGE CLASSIFICATION TO MLLMS

  • A Gaussian-based policy refines the mixing ratio using merged-token information and mask values to control the sampling mean and standard deviation.The method accounts for information integration as well as spatial ratios.Source: §4.2 · p. 6
  • MergeMix replaces DPO’s ranking component with mixed SimPO, using λ as an image-similarity signal to adjust preference strictness.More similar mixes receive a lower γ, while more dissimilar mixes receive a higher γ.Source: §4.2 · p. 6

5 EXPERIMENTS

Experiments evaluate MergeMix across image classification and MLLM benchmarks, using varied datasets, baselines, and training configurations. The reported results include gains in classification accuracy and MLLM benchmark averages, alongside calibration and ablation analyses.

5.1 STATE-OF-THE-ART METHODS

  • The experiments compare image-mixing methods and ViT-specific approaches, alongside SFT and token-reduction baselines for LLaVA and RL training for Qwen2.5-VL-Instruction.Classification reports top-1 test accuracy over the final 10 training epochs; LLaVA uses its official code and Qwen evaluation uses lmms-eval.Source: §5.1 · pp. 6–7

5.2 DATASETS

  • The classification evaluation covers CIFAR100, ImageNet-1K, CUB200, FGVC-Aircrafts, and Stanford-Cars; MLLM evaluation spans 16 datasets across VQA, understanding, and reasoning.The selected classification datasets represent small-scale, large-scale, and fine-grained settings.Source: §5.2 · p. 7

5.3 IMPLEMENTATIONS

  • CIFAR100 ViT experiments use 224×224 images, AdamW, and 200 or 600 epochs; LLaVA experiments use Vicuna-v1.5 7B, a pretrained CLIP encoder, and SFT on llava-v1.5-mix665k.Qwen2.5-VL-Instruction checkpoints are fine-tuned for 0.1 epoch on the same training set.Source: §5.3 · pp. 7–8

5.4 RESULTS OF IMAGE CLASSIFICATION

  • CIFAR100 gains over TranMix are +2.15% and +2.51% on DeiT models and +2.87%, +2.88%, and +4.79% on ViT models; Stanford-Cars accuracies are 88.42% and 92.20%.On ImageNet-1K, MergeMix gains +0.27% and reduces FLOPs by 0.68G versus TransMix, with throughput of 1591.66 TP/s.Source: Table 1 · p. 7, Table 2 · p. 7, Table 3 · p. 7

5.5 RESULTS OF MLLM BENCHMARKS

  • LLaVA gains +0.83% with full vision tokens; across settings, MergeMix averages 66.84%, +1.27% over LLaVA, while Qwen2.5-VL-Instruction gains +2.88% on average.MMMU and MMMU-Pro results are reported as on par with methods targeting reasoning improvements.Source: Table 4 · p. 8, Table 5 · p. 8, §5.5 · p. 9

5.6 RESULTS OF CALIBRATION

  • MergeMix obtains CIFAR100 calibration results of 6.7% and 9.7%, reported as best among the compared ViT-specific mixup methods.For LLaVA, SFT reduces ECE when tuning the vision encoder, and adding augmentation and ranking loss can improve it further.Source: Figure 4 · p. 9, Table 6 · p. 9, §5.6 · p. 10

5.7 ABLATION STUDY

  • Ablations report gains over TopK sampling from token merge and further gains from rescaling the mixing ratio; unfreezing the vision encoder and adding augmentation with ranking loss outperform their stated alternatives.The study also examines sensitivity to merged tokens and feature-layer choice.Source: §5.7 · p. 10, Figure 6 · p. 10, Table 7 · p. 10

6 CONCLUSION

MergeMix applies token-merge mixup to image classification and MLLM alignment, using preference pairs to bridge SFT and RL. The authors identify image-only augmentation and static token merging as limitations for MLLMs.

6 CONCLUSION

  • MergeMix uses token merging and mixed-image/raw-image preference pairs to address both image classification and MLLM alignment.The conclusion reports improved classification performance and beneficial MLLM alignment and generalization on benchmarks.Source: §6 · p. 10
  • For MLLMs, augmentation currently affects images only, and the token-merging policy is static and unlearned.The authors identify text-modality mixup and learnable token merging as future directions.Source: §6 · p. 10

A DECLARATION OF LLM USAGE

B.1 DATASETS

The appendix describes five image-classification datasets and 16 MLLM evaluation datasets, covering varied tasks and data scales.

B.1 DATASETS

  • The image-classification suite comprises CIFAR100, ImageNet-1K, CUB200, FGVC-Aircrafts, and Stanford-Cars, with small-scale, large-scale, and fine-grained settings represented.The passage gives class counts, train/test sizes, and image resolutions for these datasets.Source: §B.1 · p. 17
  • The MLLM benchmark suite contains 16 datasets spanning visual question answering, multimodal understanding, and reasoning, including college-level exam and visual-math tasks.LLaVA benchmark images are typically cropped or resized to 336 × 336 pixels; Qwen2.5-VL dynamically scales images.Source: §B.1 · p. 17

B.2 IMPLEMENTATIONS

The implementation specifies training settings for image classification and MLLMs, then gives MergeMix algorithms for classification and preference tuning. It identifies text mixing and learnable token merging as future directions.

B.2 IMPLEMENTATIONS

  • Classification settings include CIFAR100 and ImageNet-1K training, fine-grained dataset fine-tuning, and LLaVA- and Qwen2.5-VL-based MLLM experiments.For fine-grained datasets, DeiT-Small and ViT-Base are fine-tuned for 200 epochs; the MLLM settings use LLaVA-v1.5-mix665k.Source: §B.2 · pp. 17–18

B.3 ALGORITHMS

  • For image classification, the algorithm samples a mixup ratio from Beta(α, α), creates a mixed image, and trains with a label-weighted cross-entropy loss.An optional consistency regularizer is added to the mixed-sample loss.Source: §B.2 · p. 19
  • MergeMix for MLLMs pairs each raw image with a mixed image for the same prompt, using raw-image output as winner and mixed-image output as loser.The total loss combines supervised fine-tuning on the winner with a SimPO preference loss.Source: §B.3 · p. 19

B.4 FUTURE WORK

  • Future work identifies raw text inputs and static, unlearned token merging as shortcomings, and proposes text mixing and learnable merging as directions to explore.It also proposes sampling losers from multiple augmentations and investigating image-editing methods for constructing preference pairs.Source: §B.4 · p. 19

C.1 EXTENSIVE RESULTS

The extensive results report strong classification performance across models, robustness experiments, and transfer to another backbone and audio datasets. The supplied passages also describe occlusion and calibration evaluations.

C.1 EXTENSIVE RESULTS

  • MergeMix achieves state-of-the-art results on many models across the reported CIFAR100 and fine-grained classification results.The text also reports improved speed of overfitting and attributes useful information gathering and redundancy reduction to token merging.Source: §C.1 · pp. 19–20

C.2 ROBUSTNESS EXPERIMENTS OF MIXUP AUGMENTATIONS

  • The calibration study evaluates seven mixup augmentations across CIFAR100 and ImageNet-1K settings using ViT-based models and varying training epochs.The provided passage describes the evaluation settings but does not report its calibration results.Source: §C.2 · p. 20, Table A3 · p. 20
  • MergeMix is evaluated against other mixup methods under image occlusion on CIFAR100 and ImageNet-1K, with results described in Figure A1 and Tables A4–A6.The supplied text identifies the robustness evaluation and its visuals but does not state comparative outcome values.Source: Figure A1 · p. 19, Table A4 · p. 20, Table A5 · p. 20

C.3 FURTHER RESULTS ON DIFFERENT BACKBONE AND DATA MODALITY.

  • On CIFAR100, MergeMix ranks second-best with CAFormer-S12, while on ESC-50 and UrbanSound8K it significantly improves over multiple shuffling baselines.The audio experiments use HuBERT-Base and treat audio data as one-dimensional sequences.Source: §C.3 · p. 21, Table A7 · p. 21, Table A8 · p. 21

D.1 CALIBRATION RESULTS OF MLLM

The MLLM calibration experiments find that augmentation can mitigate calibration degradation under token reduction, though calibration remains an open question. Separately, judge rewards rise with larger mixing ratios across nearly all evaluators.

D.1 CALIBRATION RESULTS OF MLLM

  • Token reduction moderately degrades calibration, while effective augmentation strategies significantly mitigate the drop across the evaluated settings.The study covers full, 50%, and 25% vision-token settings across four short-response tasks.Source: §D.1 · p. 22, Table A9 · p. 22
  • With full vision tokens, CutMixrl has the lowest GQA calibration error (6.09), while ResizeMixrl has the best SEED result (36.08).At 50% tokens, CutMixrl and MergeMixrl remain competitive; at 25%, CutMixrl retains relatively balanced performance.Source: §D.1 · p. 22, Table A9 · p. 22

D.2 RELATIONSHIP BETWEEN MIXING RATIOS AND REWARDS

  • VLM-as-the-judge rewards consistently rise with larger mixing ratios across nearly all evaluators, supporting λ as a preference-strength control signal.The evaluators include frontier MLLMs and a human-expert baseline.Source: §D.2 · p. 22, Figure A2 · p. 22

D.3 RESULTS OF DIFFERENT VISION TOKEN RATIOS ON INFERENCE

MergeMix improves LLaVA benchmark performance across vision-token merge ratios, and the ranking-loss ablation reports gains over vanilla SimPO. The supplied figure caption describes performance across inference merge ratios.

D.3 RESULTS OF DIFFERENT VISION TOKEN RATIOS ON INFERENCE

  • Across inference merge ratios, MergeMix brings gains ranging from +0.83 to +0.06, while other methods are described as performance-unstable due to randomness.The passage refers to Table A10 for the gains.Source: §D.3 · p. 23

D.4 RESULTS OF DIFFERENT RANKING LOSS

  • The mixed ranking loss brings more improvement than vanilla SimPO in the reported ablation.Figure A3 compares LLaVA-v1.5-7B performance under different inference-time merge ratios for several methods.Source: §D.4 · p. 23, Figure A3 · p. 24

D.3 RESULTS OF DIFFERENT VISION TOKEN RATIOS ON INFERENCE

  • 66.84% average gain is reported with token merge in training, 0.43% above training and inference without token merge and +1.27% above original LLaVA average performance.Table A11 reports these comparisons; MergeMix also maintains strong performance across a wide range of settings.Source: §D.3 · p. 23, Table A11 · p. 24

E.1 RESULTS OF DIFFERENT VISION TOKEN RATIOS ON INFERENCE

Token merging reduces FLOPs and improves throughput in ViT-L and LLaVA-v1.5-7B, while visualizations show how MergeMix samples change with mixing ratios. They also show that higher mixing can yield answers less related to the question.

E.1 RESULTS OF DIFFERENT VISION TOKEN RATIOS ON INFERENCE

  • ViT-L throughput rose from 122.83 to 201.07 (+63.7%) as FLOPs fell from 59.57G to 34.93G.The comparison uses different merged ratios and was evaluated on an Nvidia A100 GPU.Source: §E.1 · p. 24, Table A14 · p. 26
  • At r = 0.5, LLaVA-v1.5-7B throughput rose from 45.96 to 47.28 and TTFT fell from 86.44 ms to 66.74 ms.At r = 0.75, TTFT further decreased to 62.25 ms; Token Merge overhead was 0.97 ms.Source: §E.1 · p. 25, Table A15 · p. 26

F VISUALIZATION AND CASE STUDY

  • As the mixing degree increases, LLaVAv1.5-7B answers become more wrong or unrelated to the question.The figure marks correct answers in blue and wrong answers in green.Source: Figure A4 · p. 26
  • On ImageNet-1K, larger merge ratios group similar regions better, while increasing λ preserves the first image’s most distinguishable tokens and gradually adds less important regions.GradCAM visualizations show top-1 and top-2 classes for mixed samples.Source: F VISUALIZATION AND CASE STUDY · p. 26, Figure A5 · p. 27, Figure A6 · p. 28

Related papers

Loading 2510.23479v3…