PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Objective Bayesian Optimization for Model Merging

Utkarsh Agarwal, Vamshi Bonagiri, Raul Astudillo, Monojit Choudhury

arXiv:2608.14264v1cs.LGcs.AI

TL;DR

Choosing merge parameters is difficult when evaluations are expensive and capabilities trade off. MOBO-Merge applies multi-objective Bayesian optimization to find Pareto-optimal merges, achieving higher mean hypervolume than random search in 11 of 12 held-out comparisons.

  • Problem

    Selecting merge coefficients is expensive, noisy, and nontrivial, especially when multiple model capabilities must be preserved simultaneously.

  • Method

    MOBO-Merge treats merge-parameter selection as black-box multi-objective optimization and uses Bayesian optimization to approximate Pareto fronts across merge operators.

  • Results

    11 of 12 held-out comparisons achieved higher mean hypervolume than random search, with largest gains in several TIES, block-wise, and three-model searches.

  • Takeaways & Limitations

    MOBO-Merge is most useful for efficiently exploring expressive merge spaces under limited evaluation budgets.

  • Takeaways & Limitations

    Scaling MOBO-Merge to higher-dimensional merge spaces and many competing objectives may require more expressive surrogate models and improved acquisition strategies.

Abstract

from arXiv · show

Model merging combines trained models directly in weight space, offering a compute-efficient alternative to additional fine-tuning. Selecting merge parameters is nevertheless difficult because downstream evaluations are expensive, gradients are unavailable, and source capabilities can conflict. We formulate merge-parameter selection as a black-box multi-objective optimization problem and introduce MOBO-Merge, a merge-operator agnostic framework that uses multi-objective Bayesian optimization to approximate the Pareto front under a limited evaluation budget. We evaluate Qwen3-4B and Llama-3.1-8B in two-model instruction-math and three-model instruction-math-code settings using Linear, SLERP, TIES, and block-wise merge operators. On held-out benchmark partitions, MOBO-Merge obtains higher mean hypervolume than random search in 11 of 12 reported comparisons. The gain is small for one-dimensional Linear interpolation but substantially larger for several TIES, block-wise, and three-objective searches. No merge operator is uniformly best: TIES leads in three of four family-setting combinations, whereas Block-Linear 4x is strongest for the Llama three-model merge. These results show that multi-objective Bayesian optimization is valuable as a search layer for expressive merge parameterizations.

1 Introduction

Model merging is an expensive black-box multi-objective problem because specialized capabilities can conflict and evaluations are costly, noisy, gradient-free, and budget-limited. MOBO-Merge addresses this by using merge-operator-agnostic multi-objective Bayesian optimization to approximate diverse Pareto-optimal merges, outperforming random search in most evaluated settings.

  • Introduction: Model merging combines specialized pretrained or fine-tuned models directly in weight space, enabling modular capability reuse without additional training.Common operators include linear interpolation, block-wise mixing, and interference-aware methods.
  • Introduction: Conflicting source capabilities make merge quality a trade-off across objectives, naturally characterized by the induced Pareto front rather than one optimum.Improving one capability can degrade another, and no single configuration is optimal across all criteria.
  • Introduction: Costly noisy evaluations, unavailable gradients, and limited budgets make exhaustive grid search infeasible and cause random or heuristic strategies to scale poorly.These properties place merge-parameter selection in the expensive black-box optimization regime.
  • Introduction: MOBO-Merge formulates merge-weight selection as black-box multi-objective optimization and uses Bayesian optimization to efficiently approximate diverse Pareto-optimal merges.The framework is merge-operator agnostic and explicitly seeks a set of trade-off solutions rather than a single merged model.
  • Introduction: 11 of 12 settings show higher mean hypervolume than random search, with larger gains in several TIES, block-wise, and three-objective searches.The evaluation covers Qwen3-4B and Llama-3.1-8B in two-model instruction-math and three-model instruction-math-code settings.
  • Introduction: TIES leads in three of four family-setting combinations, while Block-Linear 4x performs best for the Llama three-model merge.No merge operator is uniformly best, and Linear search remains competitive in one dimension.

2 Problem Formulation

Model merging is formulated as expensive black-box multi-objective optimization over a low-dimensional parameter space controlling how shared-architecture source models are combined. Because objectives can conflict and evaluations are costly, the goal is to approximate the Pareto front under a limited evaluation budget.

  • 2 Problem Formulation: Source models θ1, . . . , θK ∈Θ share a common parameterization and may be pretrained or fine-tuned for different purposes.
  • 2 Problem Formulation: A merge operator maps low-dimensional parameters x ∈X ⊂R^D to a merged model, treating the operator as a black box.
  • 2 Problem Formulation: The merge space can encode global mixture weights, block-wise coefficients, or other structured controls.
  • 2 Problem Formulation: External benchmark objectives map merge parameters to task-level outcomes and may be expensive or noisy to evaluate.
  • 2 Problem Formulation: Conflicting objectives mean solution quality is defined by Pareto optimality rather than a single optimal merge.
  • 2 Problem Formulation: The Pareto set contains nondominated merge parameters, whose objective-space image forms the Pareto front to approximate efficiently.
  • 2 Problem Formulation: Evaluations require merged-model instantiation and downstream benchmarks, provide no gradients, and are feasible only in small numbers.
  • 2.1 The Canonical Case: Merging Specialized Models: In the canonical specialized-model case, the number of objectives matches the number of source models, M = K, but the general formulation permits arbitrary objective counts and criteria.

3 Method: MOBO-Merge

MOBO-Merge treats merge-parameter selection as expensive black-box multi-objective Bayesian optimization, using uncertainty-aware surrogates and hypervolume-based acquisition to approximate the Pareto front under a limited evaluation budget. It is merge-operator agnostic and iteratively evaluates selected configurations to update the surrogate and return nondominated solutions.

  • Framework overview: MOBO-Merge applies the same multi-objective Bayesian optimization procedure to any parameterized merge operator, modeling its vector-valued objectives as an expensive black box.The framework uses probabilistic surrogate models and a hypervolume-based acquisition function to approximate the Pareto front under a limited evaluation budget.
  • Optimization loop: At each sequential iteration, the highest-scoring merge parameters are evaluated, their objective values are added to the dataset, and the probabilistic surrogate is refit.After the evaluation budget is exhausted, the method returns nondominated solutions among the evaluated configurations.
  • Surrogate modeling: The surrogate captures epistemic uncertainty and observation noise, guiding future merge-parameter selection from previously evaluated configurations.The framework permits any uncertainty-aware probabilistic regression model; experiments use independent Gaussian process surrogates for each objective.
  • Acquisition function: MOBO-Merge uses noisy Expected Hypervolume Improvement (NEHVI) to score candidates by expected improvement to the current Pareto-front approximation.NEHVI directly targets Pareto-front quality through dominated hypervolume under a limited evaluation budget.

4 Experiments and Analysis

Experiments across two model families, multiple merge operators, and two- versus three-model settings show that MOBO-Merge generally improves held-out multi-objective performance over random search, especially in expressive search spaces. Its advantage is small for one-dimensional Linear interpolation but larger for TIES, block-wise merging, and three-objective searches.

  • Experimental setup: MOBO-Merge is evaluated on Qwen3-4B and Llama3.1-8B using instruction-following, mathematical-reasoning, and code-generation objectives.The objectives use IFEval strict accuracy, GSM8K exact match, and humaneval_instruct pass@1.
  • Experimental setup: The study compares sequential NEHVI with uniform random search over identical feasible regions, using 100 acquisition steps or baseline evaluations across seeds 42–51.Initial BO design sizes vary with search dimension and model count.
  • Merge operators: The main comparison covers Linear, TIES, and four-block Block-wise Linear Merge, with TIES providing a 4-parameter two-model search space and Block-Linear 4x four independent AB coefficients.The block-wise method uses four equally sized layer blocks; its ABC setting has eight free parameters.
  • Three-model results: In ABC experiments, MOBO-Merge reaches 0.0429 versus 0.0055 for TIES and 0.0634 versus 0.0289 for Block-Linear 4x on Llama-3.1-8B.On Qwen3-4B, it also exceeds random search for Linear, TIES, and Block-Linear 4x; TIES is not uniformly the best operator.
  • Held-out results: 11 of 12 comparisons favor MOBO-Merge on mean held-out hypervolume; the only reversal is Qwen AB with Linear, where random search leads by 0.0005 (0.7% relative).The results indicate that gains are small in one-dimensional spaces and more substantial in higher-dimensional or three-objective searches.

5 Related Work

Model merging combines specialized models in weight space without additional training, but selecting coefficients remains expensive and difficult. Prior Bayesian-optimization work covers single- and multi-objective settings, while this study uses MOBO as a common search layer across heterogeneous operators.

  • Model merging: Model merging combines specialized models post hoc in weight space, offering an alternative to additional pretraining or joint multi-task fine-tuning without increasing inference cost.The paper defines model merging as training-free composition through weight-space operations, distinguishing it from broader model fusion.
  • Model merging: Improved operators and parameterizations mitigate interference but leave merge-coefficient selection expensive, noisy, and nontrivial when preserving multiple capabilities.Examples include TIES-Merging, DARE, and structured block-wise coefficients.
  • Bayesian optimization: Bayesian optimization targets expensive, noisy, gradient-free evaluations under limited budgets, motivating its use for selecting merge weights.Related work applies BO to single-objective checkpoint merging and structured post-hoc model combination.
  • Multi-objective search: Prior multi-objective methods establish Pareto-based search but generally couple optimization to a particular operator, parameterization, or target trade-off.The present study instead evaluates multi-objective Bayesian optimization as a common search layer across heterogeneous model-merging operators using a fixed protocol.

6 Limitations

MOBO-Merge remains limited by costly benchmark evaluations, its focus on homogeneous weight-space merging, and dependence on benchmark and objective choices. Scaling to higher-dimensional merge spaces with many objectives may require more expressive surrogates and improved acquisition strategies.

  • Evaluation cost: Repeated downstream evaluations remain a practical bottleneck, limiting how many merge configurations can be explored despite improved sample efficiency.The burden is especially pronounced for objectives requiring long-running inference or exhibiting high variance.
  • Scope and evaluation dependence: The framework targets models with a common architecture and parameterization, leaving heterogeneous models and non–weight-based composition as open challenges.Recovered Pareto-front quality also depends on evaluations and objectives that may not capture all model behavior or generalization.
  • Scaling: Scaling beyond low- to moderate-dimensional merge spaces and few objectives may require more expressive surrogate models and improved acquisition strategies.The framework’s demonstrated performance is confined to settings with a small number of objectives.

7 Conclusion

MOBO-Merge frames model merging as an expensive black-box multi-objective optimization problem, using Bayesian optimization to find high-quality Pareto-optimal merges under limited evaluation budgets. Across held-out comparisons, it outperforms random search in mean hypervolume in 11 of 12 settings, with especially pronounced gains in higher-dimensional searches.

  • Conclusion: MOBO-Merge replaces ad hoc merge-weight tuning with multi-objective Bayesian optimization that targets Pareto-optimal solutions under limited evaluation budgets.The framework exposes controllable trade-offs between competing model capabilities while remaining merge-operator agnostic.
  • Conclusion: 11 of 12 settings show higher mean hypervolume for MOBO-Merge than random search across held-out comparisons.The gains are most pronounced in several higher-dimensional TIES, block-wise, and three-model searches.
  • Conclusion: Future directions include adaptive merge parameterizations, early stopping or multi-fidelity evaluation, and extensions to higher-dimensional merge spaces or more objectives.These directions aim to learn which layers or components require independent weights and reduce evaluation costs.
  • Conclusion: The work applies existing multi-objective optimization techniques to pretrained-model merging without introducing new optimization methods or model capabilities beyond the source checkpoints.Its stated goal is a systematic, compute-efficient exploration of trade-offs that enables more flexible reuse of existing models.

A Appendix · A.1 Ablation on Acquisition Function Choice

The acquisition-function ablation finds no uniform winner between NEHVI and qNParEGO on the Qwen AB setting. Both guided methods generally outperform random search, supporting robustness of MOBO-Merge to acquisition choice.

  • A.1 Ablation on Acquisition Function Choice: No uniform winner emerges between NEHVI and qNParEGO on the Qwen AB setting.
  • A.1 Ablation on Acquisition Function Choice: qNParEGO achieves higher final mean validation hypervolume for Linear, TIES, Block-Linear 2x, and Block-Linear 6x.
  • A.1 Ablation on Acquisition Function Choice: NEHVI achieves higher final mean validation hypervolume for SLERP and Block-Linear 4x.
  • A.1 Ablation on Acquisition Function Choice: 0.0004 to 0.0029: the absolute NEHVI-qNParEGO gaps span this range.
  • A.1 Ablation on Acquisition Function Choice: Both NEHVI and qNParEGO generally outperform random search.
  • A.1 Ablation on Acquisition Function Choice: The ablation supports MOBO formulation robustness to acquisition-function choice.

A.2 Models

The study uses architecture-compatible Qwen3-4B and Llama-3.1-8B checkpoint families, with shared base references for TIES and source models assigned to instruction, math, and code objectives. AB merges Models A and B, while ABC merges add Model C.

  • Merge composition: The base checkpoint is a shared TIES reference and is not an additional source in either AB or ABC merges.Models A, B, and C denote the source checkpoints associated with the three experimental objectives.
  • Qwen3-4B models: The Qwen3-4B family uses Qwen3-4B-Base as the Base/TIES reference, with Instruct-2507, Draconis-Qwen3_Math-4B-Preview, and CodeScout-4B as Models A, B, and C.These checkpoints represent instruction following, mathematical reasoning, and code generation, respectively.
  • Llama-3.1-8B models: The Llama-3.1-8B family uses meta-llama/Llama-3.1-8B as the Base/TIES reference, with Llama-3.1-8B-Instruct, OpenMath2-Llama3.1-8B, and Llama-3.1-Swallow-8B-v0.5 as Models A, B, and C.The three source checkpoints correspond to instruction following, mathematical reasoning, and code generation objectives.
  • Merge composition: AB experiments merge Models A and B, whereas ABC experiments add Model C.The checkpoint labels identify each model’s experimental role and corresponding evaluation objective.

A.3 Compute Cost · A.4 Reproducibility

Each candidate requires FP16 checkpoint materialization and evaluation on two or three tasks, with representative Linear runtimes ranging from approximately 4 to 9 minutes on single RTX 6000 Ada GPUs. Reproducibility uses fixed seeds, deterministic partitions, temperature-zero decoding, aggregated hypervolume reporting, and retained release artifacts.

  • A.3 Compute Cost: Approximately 4 minutes per Qwen AB candidate and 6.5 minutes per Qwen ABC candidate were required for representative end-to-end Linear runs.Experiments ran on single RTX 6000 Ada GPUs with vLLM-backed evaluation.
  • A.3 Compute Cost: Approximately 6 minutes per Llama AB candidate and 9 minutes per Llama ABC candidate were required for representative end-to-end Linear runs.Runtime varies with the merge operator, generation lengths, and hardware.
  • A.3 Compute Cost: Each candidate required materializing an FP16 merged checkpoint before evaluation on two tasks in AB or three tasks in ABC.The evaluation backend was vLLM.
  • A.4 Reproducibility: Search seeds 42-51 and benchmark partition split seed 4203 were used for every model, operator, and search method.This fixed the experimental partitioning across comparisons.
  • A.4 Reproducibility: Decoding used temperature zero, while hypervolume curves reported across-seed aggregates and held-out results reported means across 10 completed complementary-split re-evaluations.The held-out table therefore aggregates all 10 completed re-evaluations.
  • A.4 Reproducibility: Configurations, per-candidate objective values, optimization-to-held-out mappings, code, and remaining experimental artifacts were retained for release.The retained mappings connect optimization results to held-out evaluations.
Loading 2608.14264v1…