PaperlayerView exact revision on arXiv

Source-linked AI summary

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging

Zhenyi Lu, Chenghao Fan, Wei Wei, Xiaoye Qu, Dangyang Chen, Yu Cheng

arXiv:2406.15479v2cs.CLcs.AIcs.LG

TL;DR

Model merging faces interference among task experts and heterogeneous test inputs, limiting its performance relative to fine-tuned models. Twin-Merging separates shared and exclusive knowledge and dynamically combines them; experiments report gains over existing methods across discriminative and generative tasks.

  • Problem

    Model merging can fall short of fine-tuned experts because of interference between models and the challenge of using one static model for heterogeneous test inputs.

    Source: §1 · p. 2

  • Method

    Twin-Merging modularizes knowledge into shared and exclusive components, compresses exclusive knowledge, and uses a router to dynamically combine task-specific and shared knowledge based on test inputs.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    Across 12 datasets, Twin-Merging improved over the strongest baseline by an average of 28.34% in normalized scores for discriminative tasks and 3.86% for generative tasks on scaled Qwen-14B.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The reported benefits extend to strong scaled models, additional tasks and merging methods, and generalization to unseen data.

    Source: §5 · p. 10

    For unseen datasets, scores cannot be normalized against an expert upper bound because the corresponding expert is unavailable, narrowing the score ranges for some tasks.

    Source: §4.1 · p. 7

Abstract

from arXiv · show

In the era of large language models, model merging is a promising way to combine multiple task-specific models into a single multitask model without extra training. However, two challenges remain: (a) interference between different models and (b) heterogeneous data during testing. Traditional model merging methods often show significant performance gaps compared to fine-tuned models due to these issues. Additionally, a one-size-fits-all model lacks flexibility for diverse test data, leading to performance degradation. We show that both shared and exclusive task-specific knowledge are crucial for merging performance, but directly merging exclusive knowledge hinders overall performance. In view of this, we propose Twin-Merging, a method that encompasses two principal stages: (1) modularizing knowledge into shared and exclusive components, with compression to reduce redundancy and enhance efficiency; (2) dynamically merging shared and task-specific knowledge based on the input. This approach narrows the performance gap between merged and fine-tuned models and improves adaptability to heterogeneous data. Extensive experiments on $20$ datasets for both language and vision tasks demonstrate the effectiveness of our method, showing an average improvement of $28.34\%$ in absolute normalized score for discriminative tasks and even surpassing the fine-tuned upper bound on the generative tasks. Our implementation is available in \url{https://github.com/LZY-the-boys/Twin-Merging}

1 Introduction

Twin-Merging addresses model interference and heterogeneous test inputs by separating shared from task-specific knowledge and dynamically combining them. Experiments report gains over existing merging methods across discriminative and generative tasks.

1 Introduction

  • Model merging can lose task performance through model interference, while a static one-size-fits-all solution may underperform on heterogeneous test inputs.The introduction identifies redundancy, sign discrepancies, and task-distribution gaps among the sources of interference.Source: §1 · p. 2
  • Twin-Merging modularizes expert knowledge into shared and exclusive components, compresses exclusive knowledge, and uses a router to combine it with shared knowledge according to test inputs.The shared model provides the foundation, with task-specific knowledge conditionally injected.Source: §1 · p. 2, Figure 1 · p. 2
  • On scaled Qwen-14B, Twin-Merging improves over the strongest baseline by an average of 28.34% in normalized scores for discriminative tasks and 3.86% for generative tasks.The comparison covers 12 datasets; Figure 2b illustrates task-level comparisons.Source: §1 · p. 2, Figure 2 · p. 3
  • The method is described as requiring minimal hyperparameters, avoiding expert retraining, combining with other merging methods, and scaling efficiently in task count and storage.The authors also report that combining it with other merging methods can yield further gains.Source: §1 · p. 3

2 Related Work

Related work reviews methods for merging fine-tuned task experts, from averaging and interpolation to conflict-reduction strategies. Twin-Merging is positioned as modularizing shared and task-specific knowledge while dynamically handling heterogeneous test data.

2 Related Work

  • FisherMerging and RegMean use weight averaging but require extra data and computation, while Task-Arithmetic offers finer control but can suffer model interference.Source: §2 · p. 3
  • Linear-mode-connectivity methods interpolate models in a shared low-loss basin, using weight permutations or optimal transport, though LMC may not always hold for fine-tuned models.Source: §2 · p. 3
  • Ties-Merging, AdaMerging, and DARE address conflicts through parameter redundancy or sign disagreements, merging-coefficient search, and weight-density reduction, respectively.Source: §2 · p. 3
  • Jiang et al. use task-specific knowledge assuming test tasks are known, whereas Twin-Merging targets unpredictable test distributions with dynamic merging.Source: §2 · p. 3

3 Methodology

The analysis attributes merging difficulty to interference and distinguishes shared from exclusive task knowledge. Twin-Merging modularizes these components, compresses exclusive vectors, and dynamically combines them at inference.

3.1 Analysis of the Performance Gap in Model Merging

  • Merging can lag single-task experts because parameter redundancy or sign discrepancies and task differences create interference; both settings in Table 1 show performance degradation.The table caption distinguishes merging without parameter interference from merging similar tasks and notes that the experiments use different datasets.Source: Table 1 · p. 4, §3.1 · p. 4

3.2 Interpreting Interference From the Perspective of Knowledge

  • Increasing fine-tuning epochs reduces shared knowledge and worsens merging; with exclusive weights sparsified, performance stays stable through 90%, and 99% sparsity still beats multi-model merging.Figure 3 presents the shared-knowledge and exclusive-knowledge ratio analysis.Source: Figure 3 · p. 4, §3.2 · pp. 4–5

3.3 Twin Merging

  • Preprocessing forms a shared expert from the pre-merged model, subtracts it from each task model to isolate exclusive knowledge, then retains top-r SVD components.The compressed representation stores the corresponding singular vectors and values.Source: §3.3 · p. 5
  • At inference, a router uses shared-expert token embeddings to weight task-specific vectors per input, composing the model from shared and weighted exclusive knowledge.The resulting model produces an output for each input.Source: §3.3 · pp. 5–6

4 Experiments

Across language and vision benchmarks, Twin-Merging improves on merging baselines and approaches or surpasses fine-tuned performance. The experiments also examine unseen-task generalization, ablations, task scaling, routing, compression, and speed.

4.1 Merging Experiment

  • On GLUE, Twin-Merging improves absolute normalized score over Task Arithmetic by 28.34%, and it surpasses fine-tuned upper bounds on generative tasks.On generative tasks, it outperforms Task Arithmetic and DARE Merging by 5.77% and 3.86%, respectively.Source: Table 2 · p. 6, §4.1 · p. 7
  • On eight ViT-B/32 vision tasks, Twin-Merging achieves 95.33% vs. 94.04% accuracy, with lower time and storage costs than AdaMerging and Surgery.The reported costs are 47m22s vs. 215m01s and 5.0GB vs. 32.4GB.Source: §4.1 · p. 7, Table 3 · p. 7

4.2 Unseen Generalization

  • On unseen tasks, Twin-Merging outperforms traditional baselines without corresponding task-specific experts, including on GLUE and MMLU combinations.For MMLU, it combines knowledge from TruthfulQA, BBQ, and CNN-DailyMail domains.Source: Table 5 · p. 7, §4.2 · p. 8

4.3 Ablation Studies

  • Removing dynamic experts drops performance from 96.14 to 67.80, while replacing the shared expert drops it from 96.14 to 81.47.Dynamic merging directly on a pretrained model scores 85.90 vs. 96.14 for Twin-Merging.Source: §4.3 · p. 8, Table 6 · p. 8
  • Twin-Merging exceeds the direct-route oracle on generative tasks and outperforms static and pretrained-dynamic combinations.The reported comparisons are 102.38 vs. 100.0 for generative tasks, 102.38 vs. 96.61 for static combination, and 102.38 vs. 97.03 for pretrained dynamic merging; removing dynamic experts affects RoBERTa more than Qwen-14B.Source: §4.3 · p. 8

4.4 Scale to More Tasks

  • As task count grows, Twin-Merging maintains performance near fine-tuned models and achieves nearly 100% normalized accuracy while balancing storage.Conventional merging methods degrade notably as tasks increase, while storing all task-specific models requires excessive storage.Source: §4.4 · p. 9, Figure 4 · p. 9

4.5 Router Analysis

  • Routing adapts expert use to task knowledge: QNLI draws on entailment and question-answering expertise, while overlapping experts may receive similar weights.When vQNLI is available, the router relies on QNLI-specific knowledge; Qwen-14B may assign similar weights to vTruthfulQA and vMMLU.Source: §4.5 · p. 9, Figure 5 · p. 9

4.6 Compression and Speed Analysis

  • Twin-Merging retains 86.4% performance at 99.8% compression, and SVD retains task-specific information better than Magnitude or Bernoulli Dropout.The method uses 90% sparsity in its experiments; SVD is applied during preprocessing, while dynamic composition slightly reduces inference speed and uses a small router-training budget.Source: §4.6 · p. 10, Figure 6 · p. 10, Table 7 · p. 10

5 Conclusions

Twin-Merging combines modularized shared and task-specific expertise to improve merging performance, generalization, and storage efficiency. The paper also describes how the method applies to LoRA and contrasts its approach with multi-task learning and mixture-of-experts methods.

5 Conclusions

  • Twin-Merging approaches fine-tuned performance across settings and domains; with SVD it retains 86% performance using 0.1% of the parameters.The conclusion also reports benefits on Qwen-72B and better generalization on unseen data.Source: §5 · p. 10

A Twin Merge on LoRA

  • For LoRA, the base model remains fixed and Twin-Merging on model parameters is equivalent to applying it to the LoRA modules.The task-specific model is represented as the base model plus its fine-tuned LoRA module.Source: A Twin Merge on LoRA · p. 17

B More relative research

  • Multi-task learning integrates diverse knowledge by simultaneously optimizing task-specific objectives, while this paper focuses on settings without multi-task data integration or further expert updates.Prior work addresses task conflicts and forgetting through parameter sharing, objective adjustment, task weighting, and minimizing negative transfer.Source: B More relative research · p. 17
  • Unlike MoE methods that typically need substantial multi-task data and costly joint training, Twin-Merging reuses task-specific experts and trains a small router for fusion weights.The passage describes this as enabling natural expert specialization across domains.Source: B More relative research · p. 17

C The Merging Interference and Limited Generalization

The section illustrates why fixed merging coefficients are difficult to choose: task performance patterns vary across tasks, and coefficients that help some can hurt others. The authors propose training a router to merge experts dynamically.

C The Merging Interference and Limited Generalization

  • Task Arithmetic is evaluated by grid-searching coefficients from −2 to 2 for COLA and SST-2 experts, with only those two tasks seen and six GLUE tasks unseen.The experiment uses eight discriminative GLUE tasks.Source: C The Merging Interference and Limited… · p. 17
  • At γCOLA = γSST-2 = 1, performance drops on MRPC and QNLI but improves on MNLI, QQP, and RTE, illustrating task-dependent effects of fixed weights.Figure 7 maps normalized performance across eight GLUE tasks against the two expert weights; blue denotes improvement and red deterioration over the pretrained model.Source: C The Merging Interference and Limited… · p. 18, Figure 7 · p. 18
  • Complex, non-single-cluster performance patterns in STS-B and QNLI make one optimal weight difficult to find, while entropy minimization is limited to classification tasks and has limited adaptability.Even similar tasks such as MNLI and QNLI can have different merging-performance patterns.Source: C The Merging Interference and Limited… · p. 18
  • The authors recast model fusion as supervised learning and train a router to dynamically merge task-specific experts.This is proposed in response to the limits of fixed coefficients on unpredictable and unseen data.Source: C The Merging Interference and Limited… · p. 18

D.1 Compute Resources Used and Runtimes

The reported experiments use A100 GPUs, with merging evaluations and inference taking minutes or less per stated workload.

D.1 Compute Resources Used and Runtimes

  • Experiments ran on Nvidia A100 GPUs with 80GB RAM; reported single-task training ranged from 15 minutes per GLUE task to 10 hours for Qwen-72B on one GPU.Qwen-14B single-task LoRA training took 1–2 hours per generative task; the multitask vector took around 10 hours on single GPUs.Source: §D.1 · p. 18
  • The router is a three-layer linear network trained for 10 epochs on at most 1,000 validation items per task.Training uses a 5e-4 learning rate, and the validation set is drawn from a split of the task training data.Source: §D.1 · p. 18
  • Less than 2 minutes are required for merge evaluations; inference takes under 4 minutes per 1,000 generative items and under 30 seconds per 1,000 discriminative items.The passage says detailed training- and inference-cost comparisons appear in Table 7.Source: §D.1 · p. 19

D.2 Employed Datasets and Associated Licences

The appendices detail the evaluated datasets and model settings, a non-overlapping merging experiment, sparsification options, baseline definitions, and detailed result tables.

D.2 Employed Datasets and Associated Licences

  • The discriminative evaluation uses eight GLUE tasks, including STS-B regression, and treats the original validation data as test data after reserving 10% of training data for validation.The listed GLUE dataset licenses include CC-BY-SA, MIT, Apache 2.0, OANC, and CC BY 4.0; the passage states they are available for non-commercial research.Source: §D.2 · p. 19
  • Generative evaluation uses few-shot HELM benchmarks with Exact-Match for MMLU, TruthfulQA, and BBQ and ROUGE-2 for CNN-DailyMail; vision datasets span scene, vehicle, remote-sensing, satellite, digit, sign, and texture classification.The listed dataset licenses are available for non-commercial research.Source: §D.2 · pp. 19–20

D.3 Language Model Backbone

  • RoBERTa-base is fine-tuned for discriminative tasks, while Qwen-14B uses LoRA and Qwen-72B uses QLoRA for generative tasks.The passage specifies separate task fine-tuning and gives rank, batch-size, learning-rate, and epoch settings for these models.Source: §D.3 · p. 20

D.4 Non-Overlapping Merging

  • The non-overlapping experiment places task-specific LoRA knowledge in separate Qwen modules, then compares the merged checkpoints with standard merging.One task uses the w1 weights and the other c_proj, minimizing parameter-wise interference.Source: §D.4 · p. 20

D.5 Sparsification Methods Details

  • Sparsification compares magnitude pruning and Bernoulli dropout with truncated SVD, which reduces task-specific parameter dimensionality from m × (dout + din + 1) to r × (dout + din + 1).The decompressed matrices are discarded after the merged matrix is obtained, supporting efficient storage.Source: §D.5 · p. 20

D.6 Baselines Details

  • The baselines include individual fine-tuned models as task-specific upper bounds, weight averaging as a lower bound, and Task Arithmetic and Ties-Merging variants.The DARE variants apply 70% Bernoulli-dropout sparsification before Task Arithmetic or Ties-Merging.Source: §D.6 · p. 21
  • AdaMerging adapts layer coefficients through unsupervised test-set training, whereas Surgery inserts task-specific adapters when test-data IDs are accessible.Task Arithmetic and Ties-Merging coefficients are selected by a small validation grid search; DARE Merging uses coefficient 0.7.Source: §D.6 · p. 21

D.7 Detail Results

  • Detailed per-task results are reported in Tables 8 and 9, while Table 8 covers eight discriminative tasks and the paper also reports superior performance on fully fine-tuned LLaMA 7B generative experiments.The comparison section presents average normalized scores, with detailed statistics separated by discriminative and generative tasks.Source: §D.7 · p. 21, Table 8 · p. 21

D.8 More Advantages of Our Method

Twin-Merging is presented as outperforming AdaMerging and Surgery while adapting to distribution shifts and supporting deployment with continuous, heterogeneous inputs. Its design moves router training and knowledge modularization to preprocessing and uses SVD to reduce storage needs.

D.8 More Advantages of Our Method

  • Dynamic merging adapts to test inputs without requiring specific validation datasets for each in-domain expert, supporting diverse inputs.The method assumes that combining in-domain knowledge helps address out-of-domain inputs.Source: §D.8 · p. 22
  • Twin-Merging targets continuous data streams without gradient updates, with router training and knowledge modularization shifted to preprocessing and SVD used to reduce storage.The authors describe the method as addressing unpredictable, heterogeneous streaming inputs.Source: §D.8 · p. 22
  • The method is presented as applicable across NLP and CV tasks, including generative tasks with Qwen-14B and models up to 72B.The supplied captions identify performance tables for LLaMA-7B and four generative tasks but do not report their values.Source: §D.8 · p. 22, Table 10 · p. 22, Table 9 · p. 22

D.9 Compatibility of Twin-Merging with Other Merging Methods

Twin-Merging can be combined with primary model-merging techniques to improve performance. The reported GLUE example shows a large gain for Weight Average, and the authors note particular compatibility with Ties-Merging.

D.9 Compatibility of Twin-Merging with Other Merging Methods

  • 52.26 to 96.23 on GLUE: combining Twin-Merging with Weight Average improves the baseline score, and the authors report especially strong complementarity with Ties-Merging.Table 11 presents the method's extensibility to other model-merging methods.Source: Table 11 · p. 22, §D.9 · p. 23
  • The experiments combine different techniques for creating a shared expert with dynamically merged twin vectors.The authors describe the resulting integration with primary merging techniques as seamless.Source: §D.9 · p. 23
  • The authors connect Ties-Merging's complementarity with better isolation of shared knowledge.They suggest that improved isolation enhances Twin-Merging's overall performance.Source: §D.9 · p. 23

E Inference Efficiency

The efficiency analysis reports performance gains alongside modest added inference time and lower training-time and storage costs than comparison approaches. It also describes storage estimates and a group-wise extension for processing inputs.

E Inference Efficiency

  • Twin-Merging's estimated storage is 2TkPa + 2P + Pr bytes, versus 2(TPa + Pf) bytes for fine-tuned models and 2P bytes per pretrained model.Here P = Pf + Pa, with Pf frozen and Pa activated; the estimate assumes 16-bit floats and router parameters Pr.Source: E Inference Efficiency · p. 23
  • The paper estimates extra FLOPs for routing and merging, while noting that inference typically involves hundreds of forward passes and generation requires L tokens.The authors describe the additional consumption as negligible in generation tasks under their stated conditions.Source: E Inference Efficiency · p. 23
  • 0.039 seconds per sample is the reported added time cost, which the authors describe as small relative to the performance improvements.The paper attributes added computation mainly to routing and dynamic merging.Source: E Inference Efficiency · p. 23
  • 28.34% absolute normalized improvement for RoBERTa and 18.83% on ViT-B-32 compared to Task Arithmetic are reported alongside lower time and storage costs than comparison approaches.The reported costs are 47m22s vs. 215m01s and 5.0GB vs. 32.4GB.Source: E Inference Efficiency · p. 23
  • The current method routes batches but merges inputs sequentially; a group-wise extension bins router outputs by argmax, clusters within bins, and averages weights.Table 12 identifies the reported group-wise variant.Source: E Inference Efficiency · p. 23, Table 12 · p. 24

F Limitations and Future Work

The paper identifies limits in the theory and scope of model merging, and notes that its knowledge decomposition leaves other knowledge types unexplored. It also describes intended benefits and a risk from merging biased models.

F Limitations and Future Work

  • The theory behind when and why weight interpolation works remains incompletely understood, and merging is limited to models with the same architecture.The authors also note difficulty finding suitable fine-tuned models with specific capacities.Source: F Limitations and Future Work · p. 24
  • The paper has not investigated “evil knowledge,” which it describes as task-useless knowledge that distracts the model and obscures critical knowledge.By contrast, the authors report that dropping 90% of parameters retains most fine-tuned performance, validating the existence of irrelevant knowledge.Source: F Limitations and Future Work · p. 24

G Broader Impacts

  • The authors warn that improperly merging biased models may contaminate the merged model, while suggesting de-bias experts or sparsity techniques to minimize the impact.Source: G Broader Impacts · p. 24
Loading 2406.15479v2…