PaperlayerView exact revision on arXiv

Source-linked AI summary

Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities

Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, Dacheng Tao

arXiv:2408.07666v5cs.LGcs.AIcs.CLcs.CV

TL;DR

Model merging offers a data- and computation-efficient way to combine expert models, but the literature lacks a systematic review of its rapidly expanding methods and applications. This survey introduces a two-stage taxonomy, synthesizes theory and empirical analysis, covers foundation models and over ten machine-learning subfields, and identifies open challenges and research directions.

  • Problem

    Existing reviews provide limited coverage of model-merging methods and miss recently published work, motivating a comprehensive understanding of methods, theories, applications, and future trends.

  • Method

    The survey proposes a two-stage taxonomy, reviews theoretical and empirical analyses, and surveys applications in foundation models and more than ten machine-learning subfields.

  • Results

    The survey comprehensively covers model-merging methods and theories, applications across foundation models and more than ten subfields, and remaining challenges and research directions.

  • Takeaways & Limitations

    Model merging is presented as an efficient and modular model-enhancement approach for combining diverse capabilities across foundation models and machine-learning applications.

  • Takeaways & Limitations

    Current merging effectiveness depends heavily on pretrained-finetuning conditions, and a significant gap from independent models remains, especially as model or task counts increase.

Abstract

from arXiv · show

Model merging is an efficient empowerment technique in the machine learning community that does not require the collection of raw training data and does not require expensive computation. As model merging becomes increasingly prevalent across various fields, it is crucial to understand the available model merging techniques comprehensively. However, there is a significant gap in the literature regarding a systematic and thorough review of these techniques. This survey provides a comprehensive overview of model merging methods and theories, their applications in various domains and settings, and future research directions. Specifically, we first propose a new taxonomic approach that exhaustively discusses existing model merging methods. Secondly, we discuss the application of model merging techniques in large language models, multimodal large language models, and more than ten machine learning subfields, including continual learning, multi-task learning, few-shot learning, etc. Finally, we highlight the remaining challenges of model merging and discuss future research directions. A comprehensive list of papers about model merging is available at https://github.com/EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications.

1 Introduction

This introduction frames model merging as a data- and computation-efficient way to combine expert models, then surveys its methods, applications, analyses, and open challenges.

  • Model merging combines expert-model parameters into a universal model without original training data or expensive computation.
  • The survey addresses gaps left by prior works that cover few methods, treat model merging narrowly, or miss recently published research.
  • Its taxonomy divides methods into pre-merging techniques that improve conditions and during-merging techniques that address conflicts and interference.
  • The paper surveys applications in foundation models and over ten machine-learning subfields, including continual, multi-task, few-shot, and adversarial learning.
  • The survey also reviews theoretical and empirical analyses and identifies performance gaps, memory costs, trustworthy-merging concerns, and future directions.

2 Advanced Model Merging Methods

Advanced model-merging methods are organized into two stages: methods before merging establish suitable conditions, while methods during merging resolve conflicts before parameter combination.

  • The section presents advanced techniques through a structured overview, with Table 1 summarizing each category’s primary purpose.
  • Before Merging Methods: Before-merging methods provide better prior conditions for subsequent model merging.
  • During Merging Methods: During-merging methods resolve task conflicts or interference through varied strategies and then perform parameter merging.

2.1 Notation and Model Merging Problem Definition

The problem definition considers merging multiple same-architecture models into one parameterized model, with weighted averaging as a basic solution whose effectiveness can be limited by alignment and interference issues.

  • The setup contains T models of the same architecture, trained from scratch or fine-tuned from a shared pretrained model.
  • The survey focuses on parameter-wise merging, producing Θ_merge by applying a merge operation to the models’ parameters.
  • Weighted averaging provides a straightforward parameter-merging solution.
  • Weighted averaging may perform poorly or be infeasible when models lack suitable merging conditions or exhibit conflicts and interference.

2.2 Pre-Merging Methods

Pre-merging methods prepare expert models by improving disentanglement, architecture compatibility, or weight alignment so that parameter-level merging becomes more feasible.

  • Pre-merging preparation includes linear or sparse fine-tuning, architectural transformation, and weight rebasining or alignment.
  • Merge-Friendly Fine-Tuning: Merge-friendly fine-tuning produces models that are easier to merge than standard fine-tuning, with minimal performance degradation after merging.
  • Architectural Transformation: Architectural transformation converts heterogeneous models into a common architecture before direct parameter-level merging.
  • Weight Re-basin or Alignment: Weight alignment permutes one model’s parameters to align with another before merging or interpolation.

2.3 During Merging Methods

During-merging methods combine expert models through basic, weighted, subspace, optimization, or routing strategies, differing in granularity, data dependence, and whether merging is static or input-dependent.

  • Method categories: Model merging methods are categorized as basic, weighted-based, subspace-based, optimization-based, routing-based, and other methods.These categories describe approaches for merging well-trained expert models.
  • Basic merging: Task Arithmetic defines task vectors as fine-tuning updates and uses their addition, subtraction, or analogy for multitask learning, forgetting, and zero-shot tasks.A task vector is the difference between a fine-tuned model and its pretrained model.
  • Weighted-based merging: Weighted merging learns coefficients for models or task vectors, while finer-grained variants assign separate coefficients to layers or modules.Evolutionary search, Bayesian optimization, gradient descent, and entropy minimization are used to determine coefficients.
  • Subspace-based merging: Subspace methods sparsify or factorize models into low-rank representations to remove redundant or conflicting parameters before merging.TIES-Merging retains the top 20% of parameters by magnitude and addresses sign conflicts; low-rank methods use matrix factorization.
  • Optimization-based merging: Optimization-based methods formulate data-free objectives that reduce interference or information loss, including orthogonalization and constrained projected gradient descent.AWD links reduced interference to mutually orthogonal task vectors, while DOGE optimizes task vectors interpreted as accumulated gradients.
  • Routing-based merging: Static methods produce one merged model for all inputs, whereas routing-based methods dynamically merge models or layers according to samples or tasks.Dynamic merging can adapt model behavior to input differences but changes the merging pattern during inference.

2.4 Summary and Discussion

Model merging reduces storage and management requirements, but methods trade off deployment simplicity, data use, inference overhead, granularity, and performance. No single method is optimal across all scenarios, so selection depends on requirements and available resources.

  • Key observations: Model merging can significantly reduce stored and managed parameters compared with maintaining independent experts, especially as task count N grows.The comparison concerns the total parameter burden of merged versus independent expert models.
  • Key observations: Training-free methods are simpler to deploy, whereas training-based methods require additional computation and greater implementation complexity.The survey presents both approaches as having distinct advantages.
  • Data dependence: Most methods require no extra merging data, but data-free merging may limit performance; data-dependent approaches use labeled, validation, or unlabeled test data.Data can guide coefficient estimation or the merging process, but adds a data requirement.
  • Efficiency and granularity: Static merging avoids inference overhead, while dynamic merging adds sample- or task-dependent computation and may require storing additional expert parameters.Finer-grained weighting often performs better but typically relies on extra data.
  • Selection trade-offs: No single perfect merging method fits every scenario; the appropriate choice depends on performance requirements, implementation complexity, data availability, and computational resources.The survey frames method selection as a context-dependent trade-off.

3 Theoretical and Experimental Analysis of Model Merging

Theoretical analyses explain model merging through trajectory averaging, same-task connectivity, and cross-task task-vector structure, while experiments compare methods across vision, language, LLM, and multimodal settings. Results indicate that advanced methods can reduce conflicts, but merged models generally remain below individually trained models.

  • 3.1 Theoretical Analysis: Model-merging theory is organized into single-trajectory averaging, same-task merging, and cross-task merging analyses.These categories differ by the source models and the relationships among their training trajectories, tasks, or task vectors.
  • 3.1 Theoretical Analysis: Linear mode connectivity explains why weight alignment can place same-task models in a shared parameter basin with higher merging feasibility.The account applies primarily to independently fine-tuned models that start from the same pretrained model.
  • 3.1 Theoretical Analysis: Weight disentanglement is identified as a necessary precondition for effective task-vector merging across different datasets.Each task vector contributes to its task-specific component, while a separate component captures task-agnostic behavior.
  • 3.1 Theoretical Analysis: A threshold bounds the number of models that can be effectively merged; beyond it, further merging no longer improves performance.The reported limitation concerns the number of models combined in the merging process.
  • 3.2.1 Benchmark Comparison: Existing benchmarks provide partial coverage of model-merging methods and modalities, motivating broader comparative evaluation.MergeKit and FusionBench are described as covering only limited sets of representative methods or model types.
  • 3.2.2 Comparison of Results: Experiments across five backbones show that optimization-based, dynamic, subspace-based, and weighted methods can outperform simple strategies, but all remain below individual-model performance.The evaluation covers visual classification, NLP, text generation, and multimodal tasks; multimodal results also show that naive averaging can severely degrade performance.

4 Application of Model Merging in Foundation Models

Model merging is presented as a promising approach for challenges in foundation models, including harmful outputs, multimodal information fusion, and mixed-style image generation. The survey summarizes applications across large language, multimodal, and visual generative models.

  • 4 Application of Model Merging in Foundation Models: Foundation-model applications span large language models, multimodal large language models, and visual generative models.The survey frames these applications around challenges including harmful content, multimodal information fusion, and mixed-style image generation.
  • 4 Application of Model Merging in Foundation Models: Model merging is presented as a promising solution for addressing several challenges in foundation models.The stated challenges include harmful content in LLMs, multimodal fusion difficulties in MLLMs, and mixed-style image generation.

4.1 Model Merging in Large Language Models (LLMs)

Model merging is applied to LLM alignment, detoxification, unlearning, training efficiency, inference efficiency, reasoning efficiency, and domain specialization. Across these uses, models or task vectors are combined to modify capabilities without retraining from scratch.

  • 4.1.1 Human Preference Alignment for LLMs: Model merging supports human-preference alignment by combining aligned and unaligned models or reward-aligned models.ExPO reports performance comparable to full-preference-data alignment when merging a model aligned on 10%/20% preference data with an SFT model.
  • 4.1.2 Detoxification of LLMs: Model merging can reduce toxic generation while preserving language fluency on a control task.Task Arithmetic negates task vectors from a model fine-tuned on toxic data to reduce toxic classifications with little change in WikiText-103 fluency.
  • 4.1.3 Knowledge Unlearning of LLMs: Anti-expert task vectors enable machine unlearning by subtracting task-specific knowledge from a target model.Reported applications include forgetting copyrighted content, reducing privacy issues, and removing undesirable behavior without requiring additional data to maintain old knowledge.
  • 4.1.4 Training Efficiency: Checkpoint merging along training trajectories offers a way to accelerate LLM training or enhance training performance.The motivation is the high computational cost of repeated training on massive datasets.
  • 4.1.5 Inference Efficiency: Model merging is discussed for reducing inference costs in mixture-of-experts systems by combining specialist expertise while activating only a subset of experts per input.The passage describes MoE architectures as an efficiency-oriented setting for large models.
  • 4.1.6 Reasoning Efficiency: Reasoning-oriented LLMs improve complex-task performance but can consume substantially more tokens than conventional models.The passage uses QwQ-32B-Preview as an example of high token use even for simple questions.
  • 4.1.7 Domain Specialization: Combining parameters from expert LLMs is proposed as a way to create composite models with knowledge across vertical domains.The motivation is that pretrained LLMs often lack specialized domain knowledge and require separate corporate fine-tuning.

4.2 Model Merging in Multimodal Large Language Models (MLLMs)

MLLM model merging combines modality-specific models into a parameter-efficient unified model without paired multimodal data, while supporting cross-modal interactions and new modalities. Applications include multimodal fusion, unified architectures, and transfer from high-resource to low-resource modalities.

  • Motivation and benefits: Multimodal model merging targets a single modality-agnostic model that encodes multiple modalities, learns cross-modal interactions, and retains performance comparable to independent specialists.The approach can avoid labeled paired multimodal examples, integrate new modalities, and support cross-modal knowledge transfer.
  • Multimodal fusion methods: JAM merges specialized text-to-image and text-only autoregressive decoder-only transformers to generate multimodal outputs.
  • Multimodal fusion methods: DAMC fuses multimodal LLMs across image, audio, video, and point-cloud modalities using parameter decoupling and adjusted modality fusion coefficients.
  • Empirical analysis: VL-Merging evaluates multimodal merging through independent modality fine-tuning, multimodal merging, and downstream task fine-tuning across initializations, methods, and architectures.
  • Unified architectures: UnIVAL provides a unified architecture for image, video, audio, and language by converting tasks and objectives into sequence-to-sequence next-token prediction.Its weight-space linear interpolation produces a general model performing well on seen and unseen tasks.
  • Cross-modal transfer: MAM transfers attention knowledge across modalities and reduces classification error by 18.42% on ESC-50 audio classification versus standard fine-tuning.

4.3 Model Merging in Visual Generative Models

Visual generative model merging addresses the cost of training models for mixed styles and updated data. The surveyed approaches combine pretrained generative models or intermediate training trajectories to expand styles and reduce training requirements.

  • Motivation: Visual generative models struggle to generate multiple style combinations flexibly, while training generative models remains costly.
  • Motivation: Mixed-style generation is expensive because collecting additional data is costly, while fine-tuning can cause forgetting of existing capabilities.
  • Style composition: Model merging can combine pretrained generative models to flexibly produce mixed styles, with recent work focusing mainly on diffusion models.
  • Training-cost reduction: 23× and 7× improvements on CIFAR-10 were reported for consistency distillation and consistency training, respectively, by combining intermediate diffusion-model training trajectories.LCSC required only a few iterations or a small batch size while achieving image quality comparable to a fully trained diffusion model.

5 Application of Model Merging in Different Machine Learning Subfields

Model merging is applied across continual learning, multitask and multi-objective learning, domain and out-of-distribution generalization, and other machine learning settings. These applications use merging to address forgetting, reduce joint-training costs, support transfer, and improve robustness or generalization, while important scalability and trust challenges remain.

  • Overview: Model merging is surveyed across continual learning, multitask learning, domain generalization, federated learning, few-shot learning, adversarial defense, and other subfields.
  • Continual learning: Continual-learning merging addresses catastrophic forgetting while avoiding additional parameters and inference costs.Methods merge models for old tasks with updated models for new tasks, including pretrained and fine-tuned model combinations.
  • Continual learning: Parameter-efficient continual-learning methods merge task-specific LoRA-derived task vectors, while other approaches merge convolutional layers or partition-trained submodels.These methods target incremental class discovery, preservation of pretrained knowledge, or reduced data bias.
  • Multitask and related learning: Merging independently trained task models offers a direct route to multitask learning without jointly training on all task data.The survey also covers model merging for multi-objective trade-offs, multi-domain conflicts, and auxiliary-task transfer.
  • Generalization: Out-of-distribution and domain-generalization applications target performance on unseen data or domains under distribution shifts such as corruption and seasonal changes.
  • Generalization: SWA averages checkpoints to improve training stability and OOD performance, while adaptive variants regulate averaging timing to avoid underfitting or overfitting.Other methods merge independently trained models whose diversity can improve OOD performance.
  • Zero-shot learning: Merging can support zero-shot transfer across languages, hybrid image styles, and modalities, including combined visual-audio input processing.

6 Remaining Challenges and Future Directions

The survey identifies unresolved challenges in model merging spanning reliability, efficiency, heterogeneity, fine-tuning, theory, and broader applications. It proposes future work to close performance gaps, strengthen theoretical guarantees, reduce costs, merge heterogeneous models, design merger-friendly fine-tuning, and expand interdisciplinary use.

  • Closing the Performance Gap Between the Merged and Independent Models: Future research should investigate model merging under relaxed conditions, including independently trained models, and extend applications across additional machine-learning subfields.The survey specifically highlights adapting strategies across subfields and exploring new application scenarios.
  • In-depth Theoretical Analysis for Model Merging: Model merging remains largely empirical, with limited theoretical analysis and few guarantees beyond models trained on shared trajectories or datasets.Future work should develop more comprehensive theory to improve the success and reliability of model merging.
  • Trustworthy Model Merging: Trustworthy merging must address intellectual-property disputes, model-owner defenses, verification, and malicious injections such as backdoors or poisoning attacks.The challenge concerns both protecting model ownership and preventing harmful content from authorized model sets.
  • Effective and Efficient Model Merging: Merging 72 fine-tuned ViT-B/32 models requires more than 200GB of memory, while coefficient heuristics and learnable methods add evaluation, data, or training costs.The survey calls for methods that avoid training, additional data, GPUs, and large memory requirements.
  • Merge Heterogeneous Models: Existing methods primarily merge homogeneous models; heterogeneous-model approaches often require knowledge distillation using original-task data and costly training.Future methods should merge heterogeneous architectures without expensive architectural transformations.
  • Develop Merger-Friendly Fine-tuning Strategies: Most methods mitigate conflicts only during merging, leaving merger-friendly fine-tuning strategies as an underexplored way to reduce performance degradation.Such strategies should avoid significantly increasing computation or imposing strict developer constraints.

7 Conclusions

The survey presents model merging as a modular model-enhancement approach, reviews its methods and theories, examines applications across foundation models and more than ten machine-learning subfields, and identifies seven future research directions.

  • 7 Conclusions: The survey reviews advanced model-merging methods and theories, applications across LLMs, MLLMs, and more than ten subfields, and seven future research directions.It frames model merging as an efficient and modular model-empowerment solution for practical scenarios.
Loading 2408.07666v5…