Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning

• Expensive instruction tuning, reinforcement learning, and distillation pipelines have driven strong LLM reasoning performance but are difficult to reproduce or adapt.Source: §1 · p. 1
• Instruction tuning can improve task accuracy while degrading reasoning faithfulness, robustness, or transfer, especially in chain-of-thought settings.Source: §1 · p. 1
• Reasoning-critical computation is unevenly distributed across layers and remains relatively invariant under post-training or fine-tuning.Source: §1 · p. 2
• These findings motivate selective interventions over uniform parameter modification and raise whether latent pretrained capabilities can be reorganized without further training.Source: §1 · p. 2
• Static weight averaging and fixed linear combinations can suffer task interference because specialized models are not uniformly mergeable.Source: §1 · p. 2
• Existing evolutionary merging methods optimize merge configurations without gradients but remain diagnostically blind, motivating adaptive diagnostic guidance.Source: §1 · p. 2

• Knowledge acquisition and reasoning ability appear partially decoupled, while alignment can improve answer accuracy without reliably improving multi-step reasoning fidelity.Source: §2.1 · p. 2
• Probing studies show that linguistic and reasoning functions differ across layers, supporting layer-aware and component-specific diagnostics.Source: §2.2 · p. 2
• Static merging uses fixed coefficients but can degrade heterogeneous-specialist performance because of representational incompatibility and interference.Source: §2.3 · p. 2
• Model merging is positioned as an alternative to expensive multi-task training pipelines, with structural and representational considerations remaining important.Source: §2.3 · p. 3
• Evolutionary optimization supports black-box, gradient-free exploration of high-dimensional merge configurations.Source: §2.4 · p. 3
• Hybrid attention and state-space architectures provide precedent for combining complementary inductive biases through cross-architecture recombination.Source: §2.5 · p. 3

• Darwin recombines frozen parent checkpoints in weight space instead of using gradient-based training.Source: §3 · p. 3
• MRI estimates tensor relevance from static statistics and probe responses, while a low-dimensional genome encodes candidate merge configurations.Source: §3 · p. 3
• Parent parameters are decomposed around a shared pretrained base, and Darwin recombines their task-specific deviations to construct a merged model.Source: §3.1 · p. 3
• Tensor-specific scalar mixing coefficients enable selective recombination of parent parameters without gradient-based optimization.Source: §3.2 · p. 4
• MRI combines static tensor statistics with probe-based functional responses, with α fixed to 0.5 in all experiments.Source: §3.3 · p. 4
• An Architecture Mapper scores tensor compatibility using functional role, dimensional consistency, and parameter-shape similarity before constrained greedy matching.Source: §3.4 · p. 4
• MRI-Trust Fusion uses τ to balance diagnostic ratios against genome-derived ratios, allowing evolutionary optimization to correct diagnostic noise while retaining structured priors.Source: §3.5 · p. 4
• The 14-dimensional genome controls global, component-level, sparsification, block-level, MRI-trust, and merge-kernel parameters, optimized through structural screening and empirical evaluation.Source: §3.5 · p. 4

Main Results

• 86.9% on GPQA Diamond places Darwin-27B-Opus at #6 among 1,252 evaluated models, outperforming its strongest parent without gradient-based training.The flagship result also surpasses several substantially larger fully trained models with a small evolutionary-search budget.Source: §4.2 · p. 5
• Darwin achieves higher accuracy and reduced variance than static merging, and higher peak performance with more reliable convergence than unguided evolutionary merging.These comparisons support diagnostic guidance as a mechanism for navigating the merge space more effectively.Source: §4.2 · p. 5

Merge Dynamics

• τ converges to 0.35–0.55 across scales, indicating that Darwin balances diagnostic priors with evolutionary optimization rather than relying exclusively on either.This balance lets search compensate for diagnostic noise and inter-layer interactions.Source: §4.3 · pp. 5–6
• DARE-TIES consistently outperforms linear interpolation and SLERP by mitigating destructive interference through drop-and-rescale behavior.SLERP provides smoother early exploration but reaches lower peak accuracy, while DARE-TIES is especially suited to heterogeneous or specialized parents.Source: §4.2 · p. 5, §4.3 · p. 6

Ablation Studies

• The full adaptive MRI-Trust variant improves GPQA by +2.5pp over genome-only merging, including a +1.2pp static-MRI gain and a further +0.9pp adaptive gain over fixed τ = 0.7.The ablation varies only τ while holding the other genome parameters constant across n = 30 runs.Source: §4.4 · p. 6, Table 2 · p. 6

Generalization

• Across 4B–35B models, recursive generations and mixed architectures preserve attention selectively while recombining feed-forward components more aggressively.Darwin-4B-Genesis merges Transformer attention with Mamba-style components without retraining and outperforms both parents on targeted reasoning benchmarks.Source: §4.5 · pp. 6–7

• Darwin cannot recover a skill or knowledge domain absent from both parent models because it reorganizes pretrained capabilities rather than creating new ones.Source: §5 · p. 7
• High-performing models generally require parents sharing a common pretrained base, and general cross-base merging at scale remains an open challenge.Limited cross-architecture recombination is possible through architecture-aware alignment.Source: §5 · p. 7
• Evolutionary search is cheaper than training or fine-tuning but still requires a compact evaluation set, while verification of the largest variants is ongoing.Mid-scale models have been independently verified on public leaderboards.Source: §5 · p. 7
• Future work targets 100B-scale sharded evaluation, improved cross-base alignment, and integration with test-time or inference-time interventions.Source: §5 · p. 7

• The Darwin Family contains eight evolutionarily merged language models spanning 4B to 35B parameters.Source: §6 · p. 7
• Darwin-27B-Opus ranks #6 on the GPQA Diamond Leaderboard and outperforms its Father Qwen3.5-27B by +1.4pp.Source: §6 · p. 7
• The framework combines a 14-dimensional adaptive genome, MRI-Trust Fusion with learnable τ, and an Architecture Mapper for cross-architecture breeding.Source: §6 · p. 7
• Training-free evolutionary merging is presented as a practical and reproducible route to frontier-scale reasoning at three to six orders of magnitude lower compute cost than conventional pretraining.The models, V6 codebase, and MRI tooling are released under Apache 2.0 to support independent verification.Source: §6 · p. 7

Data and reproducibility

• Public releases include the model collection, an interactive evolution studio, community quantizations, and Apache 2.0 code.The codebase contains approximately 13,771 lines across 15 Python files.Source: §A.1 · p. 9

Evolutionary settings

• Phase 1 used a CMA-ES population of 50 for 20 generations, while Phase 2 used 5–10 generations depending on model size.Mutation standard deviation started at 0.01 and decayed by 0.95 per generation, with top-5 elite preservation.Source: §A.2 · p. 9

MRI calibration

• The MRI calibration probe set contains 123 samples across six categories with approximately balanced Korean–English coverage.Probe-conditional hidden states are collected at each transformer-layer output, using GENERIC as the baseline anchor for cosine-distance importance measurement.Source: §A.3 · p. 10

Genome representation

• Darwin’s 14-dimensional genome combines component ratios, sparsification densities, six MRI-derived block ratios, and fusion parameters.The core parameters include global, attention, feed-forward, and embedding ratios plus Father and Mother density controls.Source: §B.1 · p. 10, §B.2 · p. 11, §B.3 · p. 11

Observed genome patterns

• Across 4B–35B models, evolved genomes show recurring parameter ranges and systematically preserve Father attention while recombining Mother feed-forward layers.The reported pattern is presented as a recurring architectural regularity discovered across scales rather than a hard constraint.Source: §B.4 · p. 12, Table 4 · p. 12, §B.6 · p. 13

Architecture Mapper

• The Architecture Mapper establishes tensor correspondences across parent models using type, dimensionality, and positional compatibility without modifying parameter values.Numerical tensor combination occurs only after this structural alignment step.Source: C Architecture Mapper and Merge… · pp. 13–14, §C.1 · p. 14, §C.5 · p. 15

Optimization procedure

• Phase 1 screens candidate genomes with a lightweight proxy objective, and Phase 2 directly evaluates a small set of promising merged models on reasoning benchmarks.The merge kernels are invoked after tensor alignment by the Architecture Mapper.Source: §C.2 · p. 14

DARE-TIES kernel

• DARE-TIES computes parent deltas relative to a shared base, applies genome-controlled Bernoulli masks, rescales surviving entries, and performs weighted recombination.The component-specific mixing weight can be γ, αattn, αffn, or αemb.Source: §C.3 · p. 14, Figure 4 · p. 15
• The drop-and-rescale procedure is reported to mitigate destructive interference and produce more stable performance than uniform averaging or linear interpolation.Darwin therefore prioritizes DARE-TIES for benchmark-driven fitness evaluation.Source: §C.3 · p. 14

Alternative kernel

• SLERP provides a lower-overhead exploration kernel, but it consistently underperforms DARE-TIES in Phase 2 and is excluded from final model selection.SLERP is used during early evolutionary exploration rather than final benchmark selection.Source: §C.4 · p. 15

Comparison and scope

• Darwin is described as uniquely combining a double-digit genome, learnable MRI-trust guidance, cross-architecture breeding, and demonstrated multi-generation evolution among surveyed prior methods.The comparison emphasizes genome dimensionality, diagnostic usage, architectural support, and heritable gains.Source: §D.1 · p. 15

D.3 Structured Training-Free Merging with Parameter Selection

• Training-free merging methods use structured parameter selection, alignment constraints, or dual-space consistency to improve merged performance without gradient updates.These approaches explicitly model consistency between weight space and activation space.Source: §D.3 · p. 16
• Fixed selection rules and objectives are not adaptive to downstream reasoning behavior and generally do not extend naturally to iterative multi-generation composition.The limitation applies despite their advance over static heuristics.Source: §D.3 · p. 16
• Evolutionary optimization searches merge configurations as a black-box problem without gradients and can discover recipes that outperform manually designed heuristics.This provides a flexible complement to structured parameter-selection methods.Source: §D.4 · p. 16
• Existing evolutionary merging methods remain diagnostically blind, often using low-dimensional or uniform parameters that make exploration inefficient and solutions difficult to interpret.They treat components as symmetrically mutable despite differing functional roles.Source: §D.4 · p. 16
• Darwin combines a multi-level adaptive merge genome with diagnostic priors and MRI-Trust Fusion, balancing guided merging against evolutionary exploration.The learnable trust parameter lets the framework interpolate between heuristic guidance and unconstrained search.Source: §D.5 · p. 16
• Darwin occupies a unified design point that combines training-free operation, diagnostic selectivity, and evolutionary adaptability.This contrasts with static heuristics, structured methods, and unguided evolutionary approaches.Source: §D.6 · pp. 16–17

E Failure Modes and Negative Results

• Non-improving parent pairs expose recurring structural failure modes that clarify when diagnostic-guided evolutionary merging is expected to succeed.The analysis treats these cases as operational boundaries rather than contradictions of the framework’s effectiveness.Source: E Failure Modes and Negative Results · p. 17
• Parents with similar capabilities and error patterns provide little benefit because recombination redistributes redundant structure instead of composing complementary functions.The resulting improvement is negligible or absent when neither parent contributes a distinct dominant capability.Source: E Failure Modes and Negative Results · p. 17
• Poorly aligned internal representations can disrupt reasoning-critical pathways faster than evolutionary optimization can recover them, causing early saturation of gains.This can occur even for models nominally derived from the same base architecture.Source: E Failure Modes and Negative Results · p. 17
• Weak, noisy, or inconsistent MRI signals make diagnostic guidance less informative, so evolutionary compensation yields smaller and less stable gains.The limitation is especially relevant when reasoning-related activations are diffuse rather than localized.Source: E Failure Modes and Negative Results · p. 17
• When parent pairs already approach a local optimum on target reasoning benchmarks, search converges quickly but further improvement is limited by missing latent complementarity.The constraint is structural rather than caused by search inefficiency.Source: E Failure Modes and Negative Results · p. 17
• Darwin is most effective for heterogeneous but compatible parent models with partially complementary reasoning structure.This summarizes the operational scope implied by the analyzed failure cases.Source: E Failure Modes and Negative Results · p. 17