Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning

Taebong Kim; Youngsik Hong; Minsik Kim; Sunyoung Choi; Jaewon Jang; Junghoon Shin; Minseo Kim

Source-linked AI summary

Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning

Taebong Kim, Youngsik Hong, Minsik Kim, Sunyoung Choi, Jaewon Jang, Junghoon Shin, Minseo Kim

arXiv:2605.14386v1May 14, 2026cs.NEcs.AI

TL;DR

The paper asks whether reasoning performance can be improved without additional training by reorganizing capabilities in existing checkpoints. Darwin uses diagnostic-guided evolutionary merging with adaptive genomes, MRI-Trust Fusion, and architecture mapping. Darwin-27B-Opus reaches 86.9% on GPQA Diamond and ranks #6 among 1,252 evaluated models, while the framework reports gains across scales and generations.

Problem
The paper addresses whether reasoning performance can improve without further training, given the cost and reproducibility challenges of conventional post-training pipelines.
Method
Darwin performs gradient-free weight-space recombination using a 14-dimensional genome, MRI-Trust Fusion, and an Architecture Mapper for tensor-level alignment.
Results
86.9% on GPQA Diamond places Darwin-27B-Opus at #6 among 1,252 evaluated models, while ablations report a +2.5pp GPQA improvement over the no-MRI baseline.
Takeaways & Limitations
Darwin supports training-free evolutionary merging across 4B–35B models, recursive generations, and released model and tooling resources for independent verification.
Takeaways & Limitations
Darwin cannot create capabilities absent from both parents, and high-performing models currently generally require parents sharing a common pretrained base.

Abstract

from arXiv · show

We present Darwin Family, a framework for training-free evolutionary merging of large language models via gradient-free weight-space recombination. We ask whether frontier-level reasoning performance can be improved without additional training, by reorganizing latent capabilities already encoded in existing checkpoints. Darwin introduces three key ideas: (i) a 14-dimensional adaptive merge genome enabling fine-grained component- and block-level recombination; (ii) MRI-Trust Fusion, which adaptively balances diagnostic layer-importance signals with evolutionary search through a learnable trust parameter; and (iii) an Architecture Mapper that enables cross-architecture breeding between heterogeneous model families. Empirically, the flagship Darwin-27B-Opus achieves 86.9% on GPQA Diamond, ranking #6 among 1,252 evaluated models, and outperforming its fully trained foundation model without any gradient-based training. Across scales from 4B to 35B parameters, Darwin models consistently improve over their parents, support recursive multi-generation evolution, and enable a training-free evolutionary merge that combines Transformer- and Mamba-based components. Together, the Darwin Family demonstrates that diagnostic-guided evolutionary merging is a practical and reproducible alternative to costly post-training pipelines for reasoning-centric language models.

1 Introduction

The paper asks whether reasoning performance can improve without further training by reorganizing capabilities already encoded in pretrained checkpoints. It motivates diagnostic-guided model merging because post-training can be costly and may not reliably preserve reasoning robustness or transfer.

Expensive instruction tuning, reinforcement learning, and distillation pipelines have driven strong LLM reasoning performance but are difficult to reproduce or adapt.
Instruction tuning can improve task accuracy while degrading reasoning faithfulness, robustness, or transfer, especially in chain-of-thought settings.
Reasoning-critical computation is unevenly distributed across layers and remains relatively invariant under post-training or fine-tuning.
These findings motivate selective interventions over uniform parameter modification and raise whether latent pretrained capabilities can be reorganized without further training.
Static weight averaging and fixed linear combinations can suffer task interference because specialized models are not uniformly mergeable.
Existing evolutionary merging methods optimize merge configurations without gradients but remain diagnostically blind, motivating adaptive diagnostic guidance.

2 Related Work

Related work spans layer-aware diagnostics, static and evolutionary model merging, and hybrid architectures. Together, these strands motivate structurally informed, training-free recombination across model components and architectures.

Knowledge acquisition and reasoning ability appear partially decoupled, while alignment can improve answer accuracy without reliably improving multi-step reasoning fidelity.
Probing studies show that linguistic and reasoning functions differ across layers, supporting layer-aware and component-specific diagnostics.
Static merging uses fixed coefficients but can degrade heterogeneous-specialist performance because of representational incompatibility and interference.
Model merging is positioned as an alternative to expensive multi-task training pipelines, with structural and representational considerations remaining important.
Evolutionary optimization supports black-box, gradient-free exploration of high-dimensional merge configurations.
Hybrid attention and state-space architectures provide precedent for combining complementary inductive biases through cross-architecture recombination.

3 The Darwin Framework

Darwin combines MRI diagnostics, evolutionary genome search, and tensor-level merge operations in weight space. MRI-Trust Fusion controls the balance between diagnostic priors and evolutionary exploration, while architecture-aware matching supports heterogeneous parents.

Darwin recombines frozen parent checkpoints in weight space instead of using gradient-based training.
MRI estimates tensor relevance from static statistics and probe responses, while a low-dimensional genome encodes candidate merge configurations.
Parent parameters are decomposed around a shared pretrained base, and Darwin recombines their task-specific deviations to construct a merged model.
Tensor-specific scalar mixing coefficients enable selective recombination of parent parameters without gradient-based optimization.
MRI combines static tensor statistics with probe-based functional responses, with α fixed to 0.5 in all experiments.
An Architecture Mapper scores tensor compatibility using functional role, dimensional consistency, and parameter-shape similarity before constrained greedy matching.
MRI-Trust Fusion uses τ to balance diagnostic ratios against genome-derived ratios, allowing evolutionary optimization to correct diagnostic noise while retaining structured priors.
The 14-dimensional genome controls global, component-level, sparsification, block-level, MRI-trust, and merge-kernel parameters, optimized through structural screening and empirical evaluation.

4 Experiments and Analysis

Experiments show that Darwin-27B-Opus improves reasoning performance through diagnostic-guided evolutionary merging, while analyses support adaptive trust, DARE-TIES, stable genome structure, and generalization across scales and architectures.

Main Results: 86.9% on GPQA Diamond places Darwin-27B-Opus at #6 among 1,252 evaluated models, outperforming its strongest parent without gradient-based training.The flagship result also surpasses several substantially larger fully trained models with a small evolutionary-search budget.
Main Results: Darwin achieves higher accuracy and reduced variance than static merging, and higher peak performance with more reliable convergence than unguided evolutionary merging.These comparisons support diagnostic guidance as a mechanism for navigating the merge space more effectively.
Merge Dynamics: τ converges to 0.35–0.55 across scales, indicating that Darwin balances diagnostic priors with evolutionary optimization rather than relying exclusively on either.This balance lets search compensate for diagnostic noise and inter-layer interactions.
Merge Dynamics: DARE-TIES consistently outperforms linear interpolation and SLERP by mitigating destructive interference through drop-and-rescale behavior.SLERP provides smoother early exploration but reaches lower peak accuracy, while DARE-TIES is especially suited to heterogeneous or specialized parents.
Ablation Studies: The full adaptive MRI-Trust variant improves GPQA by +2.5pp over genome-only merging, including a +1.2pp static-MRI gain and a further +0.9pp adaptive gain over fixed τ = 0.7.The ablation varies only τ while holding the other genome parameters constant across n = 30 runs.
Generalization: Across 4B–35B models, recursive generations and mixed architectures preserve attention selectively while recombining feed-forward components more aggressively.Darwin-4B-Genesis merges Transformer attention with Mamba-style components without retraining and outperforms both parents on targeted reasoning benchmarks.

5 Limitations and Future Work

Darwin’s benefits depend on the capabilities and compatibility of its parent models, while evolutionary search still requires evaluation compute and the largest variants remain incompletely verified.

Darwin cannot recover a skill or knowledge domain absent from both parent models because it reorganizes pretrained capabilities rather than creating new ones.
High-performing models generally require parents sharing a common pretrained base, and general cross-base merging at scale remains an open challenge.Limited cross-architecture recombination is possible through architecture-aware alignment.
Evolutionary search is cheaper than training or fine-tuning but still requires a compact evaluation set, while verification of the largest variants is ongoing.Mid-scale models have been independently verified on public leaderboards.
Future work targets 100B-scale sharded evaluation, improved cross-base alignment, and integration with test-time or inference-time interventions.

6 Conclusion

The Darwin Family presents training-free evolutionary merging as a scalable framework spanning multiple model sizes, with Darwin-27B-Opus providing the primary frontier-reasoning result and released artifacts supporting reproducibility.

The Darwin Family contains eight evolutionarily merged language models spanning 4B to 35B parameters.
Darwin-27B-Opus ranks #6 on the GPQA Diamond Leaderboard and outperforms its Father Qwen3.5-27B by +1.4pp.
The framework combines a 14-dimensional adaptive genome, MRI-Trust Fusion with learnable τ, and an Architecture Mapper for cross-architecture breeding.
Training-free evolutionary merging is presented as a practical and reproducible route to frontier-scale reasoning at three to six orders of magnitude lower compute cost than conventional pretraining.The models, V6 codebase, and MRI tooling are released under Apache 2.0 to support independent verification.

A.1 Data and reprodctibility site

Darwin provides public artifacts, reproducible implementation details, and documented experimental settings for MRI-guided evolutionary merging across model scales. The method uses a 14-dimensional genome, MRI-informed initialization, and recurring asymmetric component patterns in evolved models.

Data and reproducibility: Public releases include the model collection, an interactive evolution studio, community quantizations, and Apache 2.0 code.The codebase contains approximately 13,771 lines across 15 Python files.
Evolutionary settings: Phase 1 used a CMA-ES population of 50 for 20 generations, while Phase 2 used 5–10 generations depending on model size.Mutation standard deviation started at 0.01 and decayed by 0.95 per generation, with top-5 elite preservation.
MRI calibration: The MRI calibration probe set contains 123 samples across six categories with approximately balanced Korean–English coverage.Probe-conditional hidden states are collected at each transformer-layer output, using GENERIC as the baseline anchor for cosine-distance importance measurement.
Genome representation: Darwin’s 14-dimensional genome combines component ratios, sparsification densities, six MRI-derived block ratios, and fusion parameters.The core parameters include global, attention, feed-forward, and embedding ratios plus Father and Mother density controls.
Observed genome patterns: Across 4B–35B models, evolved genomes show recurring parameter ranges and systematically preserve Father attention while recombining Mother feed-forward layers.The reported pattern is presented as a recurring architectural regularity discovered across scales rather than a hard constraint.

C Architecture Mapper and Merge Kernels (Extended)

Darwin separates structural tensor alignment from numerical merging, then searches aligned merge configurations through a two-phase, gradient-free evolutionary procedure. Its primary DARE-TIES kernel sparsifies and recombines parent deltas, while the framework supports cross-architecture and multi-generation evolution.

Architecture Mapper: The Architecture Mapper establishes tensor correspondences across parent models using type, dimensionality, and positional compatibility without modifying parameter values.Numerical tensor combination occurs only after this structural alignment step.
Optimization procedure: Phase 1 screens candidate genomes with a lightweight proxy objective, and Phase 2 directly evaluates a small set of promising merged models on reasoning benchmarks.The merge kernels are invoked after tensor alignment by the Architecture Mapper.
DARE-TIES kernel: DARE-TIES computes parent deltas relative to a shared base, applies genome-controlled Bernoulli masks, rescales surviving entries, and performs weighted recombination.The component-specific mixing weight can be γ, αattn, αffn, or αemb.
DARE-TIES kernel: The drop-and-rescale procedure is reported to mitigate destructive interference and produce more stable performance than uniform averaging or linear interpolation.Darwin therefore prioritizes DARE-TIES for benchmark-driven fitness evaluation.
Alternative kernel: SLERP provides a lower-overhead exploration kernel, but it consistently underperforms DARE-TIES in Phase 2 and is excluded from final model selection.SLERP is used during early evolutionary exploration rather than final benchmark selection.
Comparison and scope: Darwin is described as uniquely combining a double-digit genome, learnable MRI-trust guidance, cross-architecture breeding, and demonstrated multi-generation evolution among surveyed prior methods.The comparison emphasizes genome dimensionality, diagnostic usage, architectural support, and heritable gains.

D.3 Structured Training-Free Merging with Parameter Selection

Structured training-free merging improves on static heuristics by selecting parameters and aligning representations, while Darwin adds adaptive diagnostic guidance to evolutionary search for more flexible recombination.

D.3 Structured Training-Free Merging with Parameter Selection: Training-free merging methods use structured parameter selection, alignment constraints, or dual-space consistency to improve merged performance without gradient updates.These approaches explicitly model consistency between weight space and activation space.
D.3 Structured Training-Free Merging with Parameter Selection: Fixed selection rules and objectives are not adaptive to downstream reasoning behavior and generally do not extend naturally to iterative multi-generation composition.The limitation applies despite their advance over static heuristics.
D.3 Structured Training-Free Merging with Parameter Selection: Evolutionary optimization searches merge configurations as a black-box problem without gradients and can discover recipes that outperform manually designed heuristics.This provides a flexible complement to structured parameter-selection methods.
D.3 Structured Training-Free Merging with Parameter Selection: Existing evolutionary merging methods remain diagnostically blind, often using low-dimensional or uniform parameters that make exploration inefficient and solutions difficult to interpret.They treat components as symmetrically mutable despite differing functional roles.
D.3 Structured Training-Free Merging with Parameter Selection: Darwin combines a multi-level adaptive merge genome with diagnostic priors and MRI-Trust Fusion, balancing guided merging against evolutionary exploration.The learnable trust parameter lets the framework interpolate between heuristic guidance and unconstrained search.
D.3 Structured Training-Free Merging with Parameter Selection: Darwin occupies a unified design point that combines training-free operation, diagnostic selectivity, and evolutionary adaptability.This contrasts with static heuristics, structured methods, and unguided evolutionary approaches.

E Failure Modes and Negative Results

Darwin’s non-improving parent pairs reveal structural boundary conditions: gains depend on complementary, compatible parents and informative diagnostics rather than search alone.

E Failure Modes and Negative Results: Non-improving parent pairs expose recurring structural failure modes that clarify when diagnostic-guided evolutionary merging is expected to succeed.The analysis treats these cases as operational boundaries rather than contradictions of the framework’s effectiveness.
E Failure Modes and Negative Results: Parents with similar capabilities and error patterns provide little benefit because recombination redistributes redundant structure instead of composing complementary functions.The resulting improvement is negligible or absent when neither parent contributes a distinct dominant capability.
E Failure Modes and Negative Results: Poorly aligned internal representations can disrupt reasoning-critical pathways faster than evolutionary optimization can recover them, causing early saturation of gains.This can occur even for models nominally derived from the same base architecture.
E Failure Modes and Negative Results: Weak, noisy, or inconsistent MRI signals make diagnostic guidance less informative, so evolutionary compensation yields smaller and less stable gains.The limitation is especially relevant when reasoning-related activations are diffuse rather than localized.
E Failure Modes and Negative Results: When parent pairs already approach a local optimum on target reasoning benchmarks, search converges quickly but further improvement is limited by missing latent complementarity.The constraint is structural rather than caused by search inefficiency.
E Failure Modes and Negative Results: Darwin is most effective for heterogeneous but compatible parent models with partially complementary reasoning structure.This summarizes the operational scope implied by the analyzed failure cases.