The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment

Motivation and prior work

• Capability reuse addresses the inefficiency that post-training behaviors are costly to learn repeatedly, while existing transfer methods face weight-shape, tokenization, inference-cost, or labeled-data limitations.Post-training is characterized as distribution sharpening rather than the addition of new knowledge or reasoning capabilities.Source: §1 · pp. 1–2

Empirical validation

• 12.1 percentage points: transferring CoT from Qwen1.5-14B to Qwen1.5-7B raises MATH accuracy from 8.0% to 20.1%, surpassing 18.2% with instruction tuning and CoT prompting; math transfer raises AGIEval Math from 61.1% to 71.3%, exceeding 67.8%.These improvements are obtained without explicit CoT prompting or additional training.Source: Figure 1 · p. 2, §1 · p. 3

Method

• Unlock transfers capabilities by extracting a MasterKey from contrasting capability-present and capability-absent Source activations, aligning it to the Target with a low-rank linear transformation, and intervening at inference time.The procedure is training-free, label-free, architecture-agnostic, and requires only forward passes.Source: §1 · pp. 2–3

Scope

• Cross-scale reasoning experiments cover Chain-of-Thought and mathematical reasoning, with preliminary cross-family CoT results showing consistent gains and initial evidence for convergent capability representations.The broader validation demonstrates transfer across models of different sizes and approaches or matches gains typically obtained through post-training.Source: §1 · p. 3

Transfer dynamics

• Transfer is directionally asymmetric, with small-to-large transfers typically producing larger relative gains, and it amplifies dormant capabilities more strongly when those capabilities are already represented.Unlock also sharpens output distributions toward reasoning trajectories more likely to succeed.Source: §1 · p. 3

Master Key Hypothesis

• The Master Key Hypothesis proposes that capabilities are directions in a shared low-dimensional latent subspace whose linear mappings across models can induce corresponding behaviors.This framing targets reusable capability representations despite differences in architecture, scale, and latent structure.Source: §1 · pp. 2–3

2 Method

• UNLOCK transfers a capability by contrasting capability-absent and capability-present Source variants, aligning their direction to a Target model, and intervening at inference time.The three stages are Source-space difference extraction, Source–Target linear alignment, and residual-stream intervention across layers.Source: §2 · pp. 3–4, Figure 2 · p. 4

2.1 Problem Setup

• The problem setup assumes a small set of unlabeled queries used both to extract the capability direction and estimate cross-model alignment.Prompts may include task instructions and/or demonstrations, and model-specific prompts are concatenated with each query.Source: §2.1 · p. 4

2.2 Extracting The MasterKey

• The MasterKey is extracted by aggregating per-example activation differences between Unlocked and Locked Source representations, using either the mean or first principal component.The method is unsupervised and can exploit prompt-driven or model-driven contrasts, including with versus without CoT prompting or base versus post-trained models.Source: §2.2 · p. 5

2.3 Cross-model Subspace Alignment

• UNLOCK aligns low-rank Source and Target representation subspaces with SVD and learns a linear transformation minimizing projected-representation discrepancy.The learned operator is lifted to map the Source MasterKey into the Target representation space, while shared prompts reduce prompt-induced variance.Source: §2.3 · pp. 5–6

2.4 Unlocking The Target Model

• The mapped MasterKey is applied across Target layers during generation, with layers matched by relative depth, producing an unlocked model without additional training.Intervention strength α controls steering, and the resulting vector is rescaled to preserve the original hidden-state magnitude; aggregation, rank, query count, and strength are selected on held-out development data.Source: §2.4 · p. 6

3 Atomic And Non-Atomic Capabilities

• Capabilities are consistently observable behaviors across semantically similar inputs and prompt templates, and may be latent or absent within a model.Latent capabilities can be elicited through steering or prompting, whereas absent capabilities require explicit training.Source: §3 · p. 7
• Atomic capabilities have average post-training gain Δ_Q^ψ ≤ ε, whereas non-atomic capabilities have Δ_Q^ψ > ε.The distinction separates behaviors already present before post-training from substantial improvements attributable to post-training.Source: §3 · p. 7
• Atomicity is relative to a model’s pre-training distribution because it reflects how strongly the encountered data and objectives support a capability.The section states that atomicity affects gains from Unlock.Source: §3 · p. 7
• Atomicity depends on the capability, architecture, scale, and data, while the paper focuses on transferring post-training capabilities onto base models and leaves systematic architecture analysis for future work.For transfers between post-trained models, the definitions must be adapted to the changed data distribution; the definitions remain intentionally vague because model representations are not fully understood.Source: §3 · p. 7

4.1 Experimental Setup

• The experiments extract MasterKeys by contrasting Direct and CoT prompts across five model families and evaluate transfer on GSM8K, MATH, and SVAMP.CoT is treated as an atomic capability already present in sufficiently capable base models but not always expressed without prompting.Source: §4.1 · p. 8, §4 · p. 8

4.2 Results & Discussion

• Unlock consistently improves reasoning performance across evaluated model families and datasets, with gains comparable to explicit CoT prompting.The framework also produces structured reasoning traces and increases generation length, supporting CoT elicitation as the source of improvement.Source: §4.2 · pp. 8–9
• Small-to-large transfer typically outperforms large-to-small transfer, as larger models can activate latent circuitry that smaller models may lack capacity to support.Within gemma-2, small-to-large transfer improves T_U by 44.4% over T_L, whereas large-to-small transfer improves it by only 1.6%.Source: §4.2 · p. 9
• Transfer effectiveness depends on latent capability salience: Unlock can expose dormant reasoning but cannot induce capabilities absent from the target model.Qwen1.5 variants show reasoning largely introduced during pre-training, while gemma-2 shows stronger dependence on post-training and smaller gains from Unlock.Source: §4.2 · p. 9

5.1 Experiment Setup

• Unlock estimates either task-aligned interventions from few evaluation-task examples or generalizable interventions from abundant data on a different mathematical task.Experiments use instruction-tuned and base models from four families across AGIEval-Math, Deepmind-Math, Minerva-Math, and OlympiadBench.Source: §5.1 · pp. 10–11

5.2 Results & Discussion

• 6.4% and 4.3% average gains over T_L show Qwen3-14B and Qwen3-4B respectively benefit from cross-scale Unlock transfer, often matching or exceeding post-trained models.Unlock achieves significant gains even though CoT prompting alone cannot elicit these mathematical reasoning abilities.Source: §5.2 · p. 11
• Task-conditioned transfer wins in 69.5% of large-to-small configurations, whereas task-agnostic transfer wins in 70% of small-to-large settings.The two regimes trade limited in-distribution alignment against abundant out-of-distribution data and a more robust mapping.Source: §5.2 · p. 11, §5.1 · p. 11

5.3 Convergence of Reasoning Traces:

• Unlocked models converge on a narrower set of opening reasoning trajectories than baselines, indicating that Unlock sharpens output distributions toward promising traces.This trajectory consolidation is consistent with post-training methods that narrow responses by changing the probabilities of a small set of tokens [66] [72].Source: §5.2 · p. 11, §5.3 · p. 12, Figure 4 · p. 12
• The findings support transferring dormant capabilities as direct latent interventions: Unlock improves elicitation rather than injecting new behaviors or information.This extends the hypothesis that post-training evokes capabilities already present in latent space, motivated by shared representations [24] and output sharpening [66] [72].Source: §5 · p. 10, §5.3 · p. 12

• The MKH models capabilities as projections into shared k-dimensional subspaces connected by a linear map f, where k is much smaller than model hidden dimensions.This operational hypothesis extends representation-direction ideas from concepts to transferable capability-inducing behaviors.Source: §6 · p. 13
• Transferable capability directions often occupy compact, low-rank subspaces and align across models, with efficacy strongest when the Target contains a weak or dormant capability.Interventions stabilize as estimation examples increase, supporting structured signal rather than arbitrary noise.Source: §6 · p. 13
• Non-atomic capabilities can transfer when represented in the Source contrast and supported by Target capacity, suggesting atomicity should reflect latent isolatability rather than token-level stability.Chain-of-Thought transfers imperfectly in gemma-2, whereas complex mathematical reasoning transfers in Qwen-3.Source: §6 · p. 13
• Post-training behaviors may amplify pre-existing latent tendencies rather than add entirely new structure, but transfer is weaker in older or weaker models lacking the necessary representational basis.This interpretation is consistent with findings that reinforcement-style post-training sharpens or re-weights existing output trajectories [72] [66] [30].Source: §6 · pp. 13–14
• The MKH is an empirically supported organizing hypothesis, not a mechanistic proof, because it does not specify how shared subspaces arise or how capabilities interact.It builds on the Linear Representation Hypothesis [37] [43] and Platonic Representation Hypothesis [24] while motivating future study of capability formation and composition.Source: §6 · pp. 13–14

Steering vectors

• Steering vectors modulate internal activations, capture attributes in low-dimensional directions, and have been applied to safety, reasoning, downstream performance, and mechanistic analysis [58] [42] [59] [76] [35] [57] [51] [16] [11] [21] [61] [60] [75].Recent work isolates concise Chain-of-Thought abilities as a single vector [5], while targeted word substitution or sampling can jailbreak models into producing coherent reasoning traces.Source: §7 · p. 14

Distinction from prior steering transfer work

• Unlike prior steering transfer focused on safety, jailbreak, or style proxies, this work transfers capabilities across model sizes and families using task-level correctness on reasoning benchmarks.It formalizes targeted shifts from prompt- or model-induced representational differences and the cross-model alignment needed to apply them in a new model.Source: §7 · p. 14

Capability transfer across models

• Prior capability-transfer methods operate in weight space, output space, or through distillation, with weight deltas typically limited across sizes or families and logit guidance requiring multi-model inference [26] [22] [9] [69] [63] [73] [41] [15] [32].These approaches differ from the paper’s training-free cross-model intervention.Source: §7 · p. 14

Representational convergence and cross-model alignment

• Representational-convergence work enables transfer through shared subspaces or simple maps [27] [8], whereas this work uses low-rank linear alignment instead of nonlinear autoencoders or full-dimensional pseudoinverse matrices [40; 7].The comparison centers on quantifiable downstream-task improvements.Source: §7 · p. 14

Knowledge distillation

• Classical knowledge distillation trains a student to match a teacher distribution [17] [65] [48] [10] [45], typically requiring nontrivial training and repetition for each student model.Concurrent self-distillation work [49] [25] uses logits with additional feedback or examples, whereas this paper transfers capabilities without training.Source: §7 · p. 14

8 Conclusion

• The method transfers capabilities without gradient updates or architectural or tokenization correspondence between Source and Target models.It extracts a MasterKey direction from prompt- or model-induced representational differences and transfers it via low-rank linear subspace alignment.Source: §8 · p. 15
• Evaluations across multiple model families and benchmarks confirm the approach’s effectiveness.Source: §8 · p. 15
• The results support the Master Key Hypothesis that useful behaviors can be isolated as linearly transferable latent directions in shared low-dimensional subspaces.Source: §8 · p. 15

A.1 Comparison to Previous Approaches

• Unlock is training-free and label-free, improves performance across model scales and architectures, and supports low-rank capability transfer.The comparison covers both large-to-small and small-to-large transfer using latent steering vectors.Source: Table 3 · p. 21, §A.1 · p. 21

A.2 Prompts and Models Used

• The evaluation uses only standardized Direct and CoT prompt templates to avoid discrepancies across models.Figure 5 presents the two prompt types used throughout the experiments.Source: §A.2 · p. 21, Figure 5 · p. 21
• A unified concluding token pattern, "<atok>ans</atok>", replaces model-specific "\\boxed{ans}" formatting to prevent evaluation bias.This normalization ensures final-answer formatting does not favor post-trained models that use boxed answers.Source: §A.2 · p. 21
• Unlock increases generation length over the base model with Direct prompting, matching the instruction-tuned model under explicit CoT prompting.This result indicates that the intervention changes generation behavior toward longer reasoning trajectories.Source: Figure 6 · p. 22

B Additional Results for Unlocking Chain of Thought

• Additional experiments compare the Unlocked model 𝒯U with the post-trained model 𝒯∗PT.These comparisons extend the Chain-of-Thought evaluation beyond the primary experiments.Source: B Additional Results for Unlocking… · p. 22
• The additional Chain-of-Thought results include experiments on the Qwen2.5 and Qwen3 model families.Table 4 reports these further model-family experiments.Source: B Additional Results for Unlocking… · p. 22

B.1 Impact of Unlocking

• Unlock increases generation length alongside correctness, elicits meaningful intermediate reasoning rather than verbosity, and remains non-destructive across model scales.In Qwen2.5, CoT usage emerges at 1.5B but is often prompted explicitly, whereas 7B models frequently reason under direct prompting; these findings align with scale-related CoT emergence.Source: §B.1 · p. 22, Figure 7 · p. 23

B.2.1 Impact of Number of Examples on the Master Key

• Spectral entropy plateaus at 1.4–2.5 nats, implying effective rank 4–12 versus latent dimensions d_S≥1024; at least 64 examples are needed for an accurate MasterKey estimate.Entropy growth saturates as sample size increases from 256 to 512, while the pronounced increase from n=16 to n=64 motivates the 64-example minimum.Source: §B.2.1 · p. 25

B.2.2 Effect of Rank 𝑘and Number of examples 𝑛on The Linear Transformation

• Low-rank mappings saturate near n≈64, whereas ranks beyond k≈128 degrade performance through overfitting, supporting low-rank transformations over full-rank approaches [7].Higher rank can encode richer correspondences but risks overfitting; constrained projections may yield fragmented CoT, while high-rank transfer can induce unintended languages, motivating task-conditioned and task-agnostic regimes.Source: §B.2.2 · p. 26, Figure 9 · p. 28, Figure 10 · p. 29

B.2.3 Latent Space Geometry and Sensitivity

• Successful transfer occurs in localized latent-manifold pockets, while mean and principal-component aggregation perform comparably overall, making hyperparameter calibration necessary.Neither aggregator consistently outperforms the other, and optimized low-rank projections still yield substantial gains despite configuration sensitivity.Source: §B.2.3 · p. 26, Table 5 · p. 31
• In weak CoT transfer at n=4, k=1 for Qwen-1.5-14B + Unlock from 7B, the Julie problem receives inconsistent final answers of 20 and 42.Source: §B.2.3 · pp. 35–36

C Additional Results for Unlocking Mathematical Reasoning

• Additional mathematical-reasoning evaluation covers AGIEval-Math, Deepmind-Math [47], Minerva-Math [29], OlympiadBench [19], Gaokao2023En [31], and AMC238, with 32 examples per dataset withheld for development.Task-conditioned transfer uses the held-out development examples, whereas task-agnostic transfer computes the MasterKey and linear transformation on MATH [20].Source: C Additional Results for Unlocking… · p. 37
• The study compares four model families—Qwen2.5 [46], Qwen3 [56], Ministral-3 [33], and gemma-3 [55]—using base models as Locked variants and stronger post-trained models as Unlocked Sources.Unlocked Sources are categorized as instruction-tuned or math-specific models.Source: C Additional Results for Unlocking… · p. 37
• Evaluation uses a shared CoT prompt, greedy decoding, no chat templates or demonstrations, and a 4096-token generation limit.Results are organized separately for instruction-tuned and math-specific Unlocked models in Tables 6 and 7.Source: C Additional Results for Unlocking… · p. 37

Dependence on Capabilities present in 𝒯L

• Unlocking yields modest gains for weak Target families: gemma-3 trails its instruction-tuned counterpart by 32.37% at 4B and 31.65% at 12B, limiting recovery toward 𝒯∗PT.These results indicate that Unlock does not introduce new knowledge but amplifies capabilities already latent in the Target model.Source: §C.1.1 · p. 37

What is Encoded in the Master Key?

• Unlocking produces more coherent step-by-step reasoning, replacing fragmented or prematurely terminated traces with intermediate steps connecting problems to final answers.It also sharpens first-word distributions toward recurring openings such as “To solve the ...” and “Step 1: ...” (Figure 13).Source: §C.1.1 · p. 37, Figure 13 · p. 41
• Unlocking improves mathematical reliability by shifting internal representations toward mathematically sound trajectories when models otherwise invoke concepts without valid solution-building.Correct outputs typically have longer length-to-answer, except on Minerva Math, indicating more sustained explicit reasoning before answering (Figure 14).Source: §C.1.1 · p. 38, Figure 14 · p. 42
• For incorrect solutions, repetition peaks at substring lengths of 128–256 characters, and 𝒯L repeats significantly more than 𝒯U, suggesting reduced degeneration after Unlocking.This supports the interpretation that Unlock consolidates representations and steers generation toward more successful reasoning patterns.Source: §C.1.1 · p. 38, Figure 14 · p. 42
• Unlocking improves adherence to required output formats, although formatting differences are rarely observed above 7B, where gains mainly reflect improved reasoning rather than compliance.The Target model occasionally violates strict response schemas, whereas the Unlocked model follows them more consistently.Source: §C.1.1 · p. 38

C.2 Examples of Math Reasoning Transfer

• Transferred reasoning continues step by step and reaches the correct answer when the original trace skips intermediate steps and makes subsequent mistakes.This example attributes the improvement to more coherent reasoning trajectories.Source: §C.2 · p. 44
• Transferred reasoning correctly handles rounding, whereas the original trace makes floating-point estimation errors and reaches only an approximate answer.The transferred trace is described as more attentive to the rounding specified in the question.Source: §C.2 · p. 46
• Transferred reasoning generates complete mathematical reasoning and the correct solution when the original trace completely hallucinates and never solves the problem.The example presents this as an improved math-reasoning capability.Source: §C.2 · p. 48
• Both traces can reach the same answer, but the transferred trace improves final-answer formatting when the original trace formats it incorrectly.The formatting issue is described as rare in 14B models and illustrated with Qwen-2.5-7B.Source: §C.2 · p. 49

D Model Family Transfer

• The section reports cross-family CoT transfer from the Qwen1.5 family onto the gemma-2 and OLMo-2 families, with instruction-tuned variants denoted by ℐ(𝑥).Table 8 presents the cross-family results and identifies baseline comparisons as gray.Source: Table 8 · p. 50

D.1 Experimental Setup

• Qwen1.5 checkpoints are selected as Source models because the family exhibits CoT as an atomic ability with robust performance across evaluation settings.Source: §D.1 · p. 50
• The setup uses the stronger post-trained Qwen1.5 -Chat variant for 𝒮U while retaining the other experimental settings from the CoT configuration.Source: §D.1 · p. 50

D.2 Results & Observations

• Cross-family transfer elicits significant Chain-of-Thought behavior from the Target model, confirming that CoT capabilities can remain latent.Source: §D.2 · p. 50
• Cross-family transfer achieves performance comparable to prompting the Target model with a CoT prompt.Source: §D.2 · p. 50
• Cross-family transfer performs comparably to intra-family transfer, supporting converging capability representations across models.Source: §D.2 · p. 50
• The findings support isolating and applying latent capabilities when the Target model contains sufficient representational capacity.Source: §D.2 · p. 50
• Further exploration of this latent space and transfer of non-atomic capabilities is left for future work.Source: §D.2 · p. 50