PaperlayerView exact revision on arXiv

Source-linked AI summary

A Broader Look at Model Merging: Rethinking Implicit Regularization Induced by Task Arithmetic

Sin-Han Yang, Shih-Cheng Huang, Chieh-Yen Lin, Yun-Nung Chen, Shao-Hua Sun, Hung-yi Lee

arXiv:2610.07990v1cs.LGcs.CLcs.CV

TL;DR

Model merging commonly uses auxiliary data to tune task-vector coefficients, which restricts candidate weights to the task-vector subspace. This paper tests direct optimization beyond that subspace and finds that it improves common merging methods across benchmarks, including with scarce auxiliary data. The results motivate exploring the broader weight space and reconsidering the regularization induced by task arithmetic.

  • Problem

    Task arithmetic’s coefficient search restricts candidate models to a task-vector subspace, leaving open whether this implicit regularization helps or excludes better multi-task weights.

    Source: §1 · p. 2

  • Method

    The paper applies gradient descent to merged-model weights beyond the task-vector subspace and also tests sampling weights outside that subspace.

    Source: §1 · p. 2

  • Results

    Across standard benchmarks, weight-space gradient descent consistently outperforms coefficient optimization, including when auxiliary data are scarce.

    Source: §3.2 · pp. 4–5

  • Takeaways & Limitations

    The findings call for exploring beyond the task-vector subspace and considering generalization when deciding how strongly to apply regularization.

    Source: §6 · p. 9

Abstract

from arXiv · show

Model merging aims to build a multi-task model cheaply by combining the weights of individual task-specific models. To perform well across multiple tasks, most existing merging methods use an additional dataset to find the coefficients for the best linear combination of task-specific weight updates. However, we identify an implicit regularization in this standard practice: searching over coefficients restricts the candidate models to a subspace spanned by task-specific weight updates. In this work, we investigate whether this regularization is actually useful. Surprisingly, empirical results show that optimizing merged-model weights without this regularization significantly boosts the performance of common merging methods across multiple architectures, domains, and even in an extremely data-limited scenario where only one instance is available per class. Moreover, directly optimizing the pretrained model weights even outperforms some existing merging methods. Analysis shows that better multi-task weights exist outside the subspace and can be found using multiple methods. We study different strategies for using the additional dataset, discussing their practical use and implications for model merging. Overall, this work calls for revisiting the existing model-merging pipeline, motivating a broader exploration of the weight space and a reconsideration of the implicit regularization induced by task arithmetic.

1 INTRODUCTION

Model merging combines task-specific model weights to build a multi-task model without retraining on all training data. This paper asks whether task arithmetic’s implicit restriction to the task-vector subspace helps or excludes better weights, and finds that optimizing beyond it improves performance.

1 INTRODUCTION

  • Coefficient optimization restricts candidate weights to the task-vector subspace, raising the question of whether that implicit regularization excludes better multi-task models.The question matters because merged models still trail individual task-specific models.Source: §1 · p. 2
  • Direct weight-space gradient descent consistently improves common merging methods across vision and language benchmarks, including when auxiliary data are scarce.The conclusion holds in a downsampled setting with only one sample per class, and the paper reports that pretrained-model optimization can outperform some merging methods.Source: §1 · p. 2
  • Merged models provide stronger starting points for gradient descent than the pretrained model, while the paper identifies no one-size-fits-all merging method.The paper also identifies generalization as important when deciding how strongly to apply the regularization.Source: §1 · p. 2
  • Gradient descent leaves the task-vector subspace, and S-RandOpt shows that sampling outside it can outperform coefficient optimization with one additional degree of freedom.These results support the existence of better multi-task weights outside the restricted subspace and show that finding them need not rely on gradient descent.Source: §1 · p. 2

2 OVERVIEW OF MODEL MERGING METHODS

Model merging combines models fine-tuned from a shared pretrained model, commonly by adding weighted task vectors. Existing methods transform those vectors or tune their coefficients, while auxiliary data and computational cost shape how the merging pipeline is used.

2 OVERVIEW OF MODEL MERGING METHODS

  • Model merging builds a multi-task model from task-specific models that share an architecture and pretrained starting point, rather than retraining on all training data.The setting assumes that the task-specific models are already available.Source: §2 · pp. 2–3
  • Task Arithmetic represents each fine-tuning update as a task vector, and merging methods may transform these vectors through sparsification, low-rank decomposition, or data-based estimates.The resulting variants are called advanced task vectors.Source: §2 · p. 3
  • Coefficient selection ranges from grid and diagonal search to gradient descent and Bayesian optimization, but optimizing coefficients confines weights to the task-vector subspace.Grid search over T task coefficients can require extensive inference, motivating shared-coefficient diagonal search when merging many tasks.Source: §2 · p. 3

2.1 COMPUTE AND DATA USAGE OF EXISTING MODEL MERGING METHODS

  • Auxiliary data includes hold-out or unlabeled test data for coefficient optimization and data used to construct advanced task vectors; this paper investigates using it to optimize model weights.Computational cost and data usage are practical considerations because model merging aims to avoid retraining on the full training set.Source: §2.1 · p. 3

3 DOES OPTIMIZING WITHIN TASK-VECTOR SUBSPACE SUFFICE?

Across vision and language tasks, the experiments compare weight-space gradient descent with coefficient optimization within the task-vector subspace. Weight optimization outperforms the subspace best, including in data-limited tests, while coefficient tuning and initialization choices also affect performance.

3.1 EXPERIMENTAL SETUP

  • The evaluation uses nine vision tasks, four language tasks, three CLIP-ViT variants, and Qwen3 and Llama language models.The auxiliary data are 10% of the training set in the original setup.Source: §3.1 · p. 4

3.1.1 TWO WAYS FOR OPTIMIZING MERGED MODELS

  • The study compares four merging methods and two optimization choices: coefficient search within the subspace and gradient descent on model weights initialized from the naive baseline.The main weight-optimization results use AdamW, full-weight fine-tuning for vision, and LoRA fine-tuning for language models.Source: §3.1 · p. 4, §3.1.1 · p. 4

3.2 RESULTS AND DISCUSSIONS

  • Optimizing coefficients improves every tested merging method, and optimization beyond diagonal search can further increase performance.This includes TSV-M, despite its relative insensitivity to coefficient choices.Source: §3.2 · p. 4
  • For vision tasks, gradient descent from the pretrained model beats TA, TIES, and DARE with optimized coefficients, though gradient descent from merged models performs better.The result indicates that pretrained-model optimization is competitive, while merged models remain stronger initializations.Source: §3.2 · p. 4
  • Weight-space gradient descent consistently outperforms coefficient search and the subspace best across merging methods, vision tasks, and language tasks.The comparison covers TA, TIES, DARE, and TSV-M.Source: §3.2 · p. 4, Figure 2 · p. 5
  • With downsampled auxiliary data, weight-space gradient descent still beats task-vector-subspace optimization, and its performance continues improving as more data are available.Coefficient optimization quickly saturates; the paper reports the conclusion holds in a setting with only one sample per class.Source: Figure 3 · p. 5, §3.2 · p. 5

4 LEAVING THE TASK-VECTOR SUBSPACE IMPROVES PERFORMANCE

The section tests whether gradient descent’s gains require leaving the task-vector subspace, then evaluates random sampling beyond that subspace as an alternative. Its results show that useful multi-task weights can be found outside the subspace without relying on gradient descent.

4.1 ANALYSIS OF GRADIENT DESCENT FOR MODEL MERGING

  • 9 to 20 points: the full gradient-descent update improves accuracy, while its in-subspace component alone yields no gain.The update’s orthogonal component dominates, indicating that the improvement requires leaving the task-vector subspace.Source: Table 2 · p. 6, §4.1 · p. 6
  • Gradient descent leaves the task-vector subspace as accuracy increases, across runs initialized on that subspace.The visualization uses two axes spanning the subspace and a third for the update’s orthogonal component.Source: Figure 4 · p. 6, §4.1 · p. 6

4.2 S-RANDOPT: A SIMPLE METHOD TO BOOST MODEL MERGING

  • S-RandOpt extends the task-vector subspace using an auxiliary-data gradient direction, then samples and selects weights using auxiliary data.It is designed to find multi-task weights beyond the subspace without depending on gradient descent for the final search.Source: §4.2 · p. 6
  • S-RandOpt frequently outperforms the best task-vector-subspace weights, and many sampled weights outside the subspace outperform the subspace best.The results support the possibility of finding better weights through sampling with one additional degree of freedom.Source: Table 3 · p. 7, §4.2 · p. 7, Figure 5 · p. 7

5 ANALYSIS FOR PRACTICAL USE

The section examines compute costs, initialization, generalization, and test-time adaptation when auxiliary data are used for model merging. It finds trade-offs across strategies, including stronger performance from weight updates in many settings but a role for regularization when data are scarce.

5.1 COMPUTATIONAL USAGE

  • Gradient descent trades compute for better performance and uses slightly more GPU memory than coefficient search in the standard implementation.For vision models, time and FLOPs rank Subspace GD above Weight GD above coefficient search; language-model search methods can be slower because they require repeated autoregressive inference.Source: §5.1 · pp. 7–8, Figure 6 · p. 8

5.2 MERGING METHODS PROVIDE STRONG INITIALIZATION

  • Merging methods with higher initial accuracy tend to achieve better post-gradient-descent performance, and merged-model starts outperform pretrained or random starts.Random initialization yields weak performance, so merging methods provide useful initialization points for gradient descent.Source: §5.2 · p. 8, Figure 7 · p. 8

5.3 EXTREME CASES AND UNDERLYING CONCEPTS

  • With extremely limited auxiliary data, gradient descent can underperform coefficient search; more data reverses this, while regularization helps in low-data settings.Regularization slightly decreases performance when data are sufficient; the authors frame the choice as a generalization trade-off.Source: §5.3 · p. 9

5.4 TEST-TIME ADAPTATION

  • For Task Arithmetic, gradient descent on unlabeled test data achieves better accuracy across models than adapting task-wise coefficients.The comparison concerns test-time adaptation methods applied on top of Task Arithmetic.Source: §5.4 · p. 9

6 CONCLUSION

The conclusion argues that task-vector-subspace optimization can limit model-merging performance, while weight optimization or sampling can improve it. It also emphasizes that the appropriate strategy depends on the models, auxiliary data, and scenario.

6 CONCLUSION

  • In the 2-task setting, gradient descent can trail coefficient search with downsampled data but outperform it as more data are provided; regularization helps in low-data cases.The figure notes that this setting has 53 classes, compared with 856 in the 9-task setting.Source: Figure 8 · p. 9
  • The task-vector-subspace restriction often limits performance, while gradient descent or sampling can improve merged models by relaxing it.The paper finds no one-size-fits-all method and identifies generalization as important when choosing how strongly to regularize.Source: §6 · p. 9

A.1.1 VISION MODELS

  • Vision experiments fine-tune CLIP image encoders on nine classification tasks, keeping the text encoder and zero-shot classification heads frozen.Only the image encoder is updated, so task vectors share a parameter space.Source: §A.1.1 · p. 14

A.1.2 LANGUAGE MODELS

  • Language experiments fully fine-tune four base models on Banking77, DDXPlus, IFEval, and Usefulness Judge using supervised fine-tuning.The reported setup selects checkpoints by held-out evaluation accuracy.Source: §A.1.2 · pp. 14–15

A.2 DATA DETAILS AND SPLIT

The appendix describes data splits and validation use, then reports learning-rate and optimizer checks alongside implementation and hyperparameter details. It also notes additional related-work discussion.

A.2 DATA DETAILS AND SPLIT

  • Validation data are the only split available to merging strategies; training fine-tunes individual models and test data are reserved for final evaluation.Vision uses a held-out portion of official training data for validation, while language validation and test sets are split from a held-out pool.Source: §A.2 · p. 15

A.3 SUBSPACE GD LEARNING RATE SWEEP AND INITIALIZATION POINT

  • Subspace gradient-descent accuracy is stable from learning rates 10^-4 to 10^-1, and 10^-2 is selected to balance accuracy and speed.The reported Subspace GD starts from the coefficient-search best because it generally performs better.Source: Table 8 · p. 16, §A.3 · p. 16

A.4 WEIGHT GD LEARNING RATE SWEEP AND INITIALIZATION POINT

  • Weight gradient descent mostly performs best at learning rate 10^-5, and better coefficient-search initialization improves its performance.The main paper reports gradient descent from the Naive baseline so its result does not depend on coefficient search.Source: §A.4 · p. 16, Table 9 · p. 16, Table 10 · p. 17

A.5 DIFFERENT OPTIMIZERS

  • SGD does not reduce performance much relative to AdamW, and the reported weight-gradient-descent results mostly outperform coefficient search and Subspace GD.Practitioners with limited GPU memory could choose SGD.Source: §A.5 · p. 17, Table 11 · p. 17

A.6 LLM EFFICIENCY IMPLEMENTATION DETAILS

  • Gradient checkpointing reduces activation memory at the cost of an extra forward pass, while fused linear cross-entropy avoids materializing the vocabulary-logit tensor without extra FLOPs.Both techniques are applied equally to gradient-based strategies, reducing their memory requirements while leaving their relative ordering unchanged.Source: §A.6 · p. 18

A.7 HYPERPARAMETERS OF STRATEGIES

B.1 VARIANTS OF TASK-WISE MERGING

The section compares ways to optimize within or beyond task-vector subspaces and examines how auxiliary data and model choices affect merging. It reports that weight-space gradient descent can outperform coefficient-based strategies, while small datasets can make regularization important.

B.1 VARIANTS OF TASK-WISE MERGING

  • TIES and TSV-M use one advanced task vector and one coefficient, so coefficient optimization confines candidates to a one-dimensional task-vector subspace.Their global-vector form can be rewritten as task-wise merging.Source: §B.1 · p. 20
  • Layer-wise merging uses L × T coefficients, more than task-wise merging but fewer than full-weight fine-tuning, and restricts candidates to a larger task-vector subspace.Each coefficient weights a task-vector component at a particular layer.Source: §B.1 · p. 20
  • With scarce auxiliary data, full-weight fine-tuning can underperform task-wise or layer-wise merging, and the appropriate regularization strength depends on the dataset.The cited comparison also notes the importance of gradient-descent initialization.Source: §B.1 · p. 20

B.2 DETAILED DISCUSSION REGARDING COMPUTE AND DATA USAGE

  • Backpropagation through model weights can trade additional computation for better performance; auxiliary data can also improve merging methods without tunable coefficients.Merged models can provide strong gradient-descent initialization points, and further optimization may narrow the gap with individual task-specific models.Source: §B.2 · p. 20

C.1 FINDING SUBSPACE BEST PERFORMANCE

  • The subspace comparison uses training data to seek its best performance, separating optimization capacity from train-test generalization, and also reports the best test performance among the tested methods.The tested approaches are coefficient gradient descent, fine-grained diagonal search, and Bayesian Optimization.Source: §C.1 · p. 21
  • On vision, Bayesian Optimization and Subspace GD perform similarly; on language, Bayesian Optimization generally scores higher, while Subspace GD has the lowest training loss.The authors attribute the accuracy difference to a mismatch between cross-entropy optimization and accuracy, more severe for language models.Source: §C.1 · p. 21, Table 14 · p. 21, Table 15 · p. 21

C.2 FULL-WEIGHT VS. LORA FINE-TUNING AND THE EFFECT OF INITIALIZATION POINTS

  • LoRA fine-tuning usually outperforms full-weight fine-tuning on merging tasks, which the authors attribute to its smaller parameter count and reduced overfitting risk on small auxiliary datasets.The comparison uses rank-16 LoRA and two task-vector-subspace initializations; full-weight fine-tuning was not evaluated on Qwen3-4B because of resource constraints.Source: §C.2 · p. 22, Table 16 · p. 22

C.3 MORE DATA SCALING RESULTS

  • For TIES, weight-space gradient descent has higher accuracy than coefficient search and subspace GD across auxiliary-data sizes.The figure reports three downsampling seeds with shaded error bars; the trends are consistent with the Task Arithmetic result.Source: §C.3 · p. 23, Figure 9 · p. 23

C.4 MAIN RESULTS IN TABLES

The tables report that weight-space gradient descent outperforms optimization within the task-vector subspace on the evaluated merging results. The section also describes the bounded sample space used for S-RandOpt and finds that merged models are strong gradient-descent initializations.

C.4 MAIN RESULTS IN TABLES

  • Gradient descent on model weights outperforms optimization within the task-vector subspace, with consistent findings on Llama-3.2-1B and the reported data-scaling results for Task Arithmetic and TIES.Table 17 provides numerical values for Figure 2; Tables 18 and 19 report the Task Arithmetic and TIES data-scaling results.Source: §C.4 · p. 23, Table 17 · p. 23, Table 18 · p. 24

C.5 S-RANDOPT IMPLEMENTATION DETAILS

  • S-RandOpt samples a bounded region around an initialization, varying task-vector coefficients and adding a direction orthogonal to their subspace.The task-vector coefficients vary within [−α, α], while the additional descent-direction coefficient varies within [0, β]; the bounds differ between vision and language models.Source: §C.5 · p. 24

D.1 COMPLETE ANALYSIS OF INITIALIZATIONS FOR GRADIENT DESCENT

  • Across the evaluated models, merged models provide stronger gradient-descent initializations than pretrained-model or random initializations.For vision, Figure 10 reports average test accuracy and averages the results over three random initialization points.Source: §D.1 · p. 25, Figure 10 · p. 25

D.2 2-TASK SETTING

In two-task settings, coefficient search and subspace gradient descent struggle to fit training data, while weight-space gradient descent can overfit when data are scarce. A class-downsampling ablation indicates that dataset size, rather than task diversity, is the main factor in which strategy best uses auxiliary data.

D.2 2-TASK SETTING

  • In both two-task scenarios, gradient descent and regularized gradient descent fit training data perfectly across data sizes, while coefficient search and subspace GD struggle even with more data.Weight-space gradient descent may overfit when data are scarce, with less risk when data are sufficient; the suitable regularization strength is important for the bias-variance trade-off.Source: §D.2 · p. 26, Figure 11 · p. 26

D.2.1 ABLATION STUDY: TASK DIVERSITY

  • After downsampling classes in the nine-task setting to match the two-task dataset sizes, GD and coefficient optimization perform similarly, and GD can perform worse.Across the tested class-selection seeds, the results point to dataset size, rather than task diversity, as the main factor determining which strategy best leverages auxiliary data.Source: §D.2.1 · pp. 26–27, Figure 12 · p. 27

D.3 TEST-TIME ADAPTATION FULL RESULTS

Test-time adaptation methods differ in their objectives and trainable parameters, and their results illustrate a bias–variance tradeoff when using auxiliary data.

D.3 TEST-TIME ADAPTATION FULL RESULTS

  • AdaMerging minimizes prediction entropy, whereas DivMerge minimizes Jensen–Shannon divergence between merged-model and task-specific-model predictions on unlabeled test data.Both learn merging coefficients and are evaluated with task-wise and layer-wise merging.Source: §D.3 · p. 28
  • For gradient descent, task-specific model predictions supervise merged-model optimization as either softmax outputs or one-hot labels, using 1,000 unlabeled test samples per task.Source: §D.3 · p. 28
  • Layer-wise DivMerge achieves the best performance in ViT-L-14, while weight-based gradient descent performs best in smaller models.Layer-wise DivMerge has more trainable parameters than its task-wise version but fewer than full-weight fine-tuning; the authors identify bias–variance tradeoff as key to choosing regularization strength.Source: Table 20 · p. 28, §D.3 · p. 28

D.4 LLM PERFORMANCE ON CONTROL TASKS

The section tests whether gradient descent on merged LLM weights affects general capabilities on tasks not targeted by expert models. Results vary by model size, and coefficient search can also coincide with substantial forgetting.

D.4 LLM PERFORMANCE ON CONTROL TASKS

  • The control tasks are GSM8K for mathematical reasoning and MBPP for code generation, neither targeted by the expert models.Source: §D.4 · p. 29, Table 21 · p. 29
  • On Qwen3-1.7B, gradient descent improves GSM8K 64.8→74.5 and MBPP 41.7→49.8; overall, it does not systematically cause more forgetting than coefficient search.The improved control-task averages reach levels comparable to the pretrained model.Source: Table 21 · p. 29, §D.4 · p. 29
  • On Qwen3-0.6B, gradient descent improves GSM8K 40.7→50.6 but degrades MBPP 30.7→25.7.Source: Table 21 · p. 29, §D.4 · p. 29
  • On Qwen3-1.7B, task arithmetic reaches 21.7 on MBPP, while gradient descent from the same merged model reaches 50.0.The authors conclude that remaining within the task-vector subspace does not guarantee general-capability retention.Source: Table 21 · p. 29, §D.4 · p. 29
Loading 2610.07990v1…