Subliminal Learning Is Steering Vector Distillation

Motivation

• Subliminal learning remains puzzling because semantically arbitrary outputs transmit traits, transfer mainly within model families, and resist existing explanations of signal location.Prior work localized signals to coupled tokens, divergence positions, and early layers without explaining how fine-tuning transfers them.Source: §1 · pp. 1–2

Core account

• A teacher’s system prompt is approximated by vteacher, while fine-tuning on filtered outputs installs an aligned vstudent in the student.Steering vteacher reproduces subliminal learning without a system prompt, whereas ablating it prevents the effect.Source: §1 · p. 2

Generalization

• Steering vector distillation generalizes beyond semantic traits to random vectors, although only semantically meaningful vectors produce subliminal behavioral transfer.The vector is distilled even when random, but the student behaves as steered only for semantically meaningful vectors.Source: §1 · pp. 2–3

Mechanism

• The distilled student direction is necessary and sufficient for the behavioral change: ablating vstudent suppresses trait affinity, while steering an untrained model with it reproduces the trait.This identifies the learned residual-stream direction as the direct driver of the transferred behavior.Source: §1 · p. 3

Optimization

• Adaptive optimization facilitates subliminal learning because SGD’s outlier gradients overwhelm the small vteacher-aligned signal, whereas Adam’s per-parameter scaling suppresses those updates.Plain SGD fails to install vteacher; Adam enables the aligned update signal.Source: §1 · p. 3

2.1 Experimental setup

• The pipeline uses a trait-prompted teacher, filtered outputs unrelated to the trait, and a same-model student fine-tuned on those outputs.Experiments use Qwen 2.5-7B-Instruct or Gemma-3-4b-it, 10k samples with rank-8 LoRA, and evaluate trait expression on 50 preference questions.Source: §2.1 · p. 3

2.2 Steering vectors and metrics

• The teacher vector is the mean residual-stream shift caused by a trait-specific system prompt relative to a neutral prompt, while the student vector is the shift induced by fine-tuning.Both vectors are extracted at the assistant tag using the same training prompts.Source: §2.2 · p. 3
• Empirical activation similarity measures the cosine alignment between the student’s fine-tuning-induced residual shift and vteacher, averaged across layers over training.At the final training step, the cumulative shift equals vstudent.Source: §2.2 · p. 3

3.1 Fine-tuning installs a steering direction in the student

• EAS increases steadily during fine-tuning while clean-data control alignments remain near ≈0.1, showing that training installs a direction aligned with vteacher.Figure 2 reports high student alignment with the corresponding teacher vectors across traits.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 vstudent is necessary and sufficient for subliminal learning

• Steering the reference model with vstudent raises trait expression, while replacing the student’s vstudent component removes over 50% of trait-aligned behavior.The intervention preserves the orthogonal activation component while isolating the fine-tuning-induced direction.Source: §3.2 · p. 4, Figure 3 · p. 5

3.3 vteacher is necessary and sufficient for subliminal learning

• Training on vteacher-steered data induces subliminal learning, whereas ablating vteacher from a system-prompted teacher removes nearly all of the effect.Together with the student-side intervention, this identifies vteacher as the necessary and sufficient data-generation direction.Source: §3.3 · p. 4, Figure 4 · p. 5

3.4 Results hold for other data modalities, LoRA configurations, and a fine-tuned teacher

• The steering-vector account extends to code and paraphrasing data, five alternate LoRA configurations, and teachers fine-tuned rather than system-prompted.These results indicate that the single-vector mechanism is not limited to the primary number-sequence setup.Source: §3.4 · p. 5

4.1 Predicting which traits can be subliminally learned

• Traits whose steering vectors cannot shift the reference model’s preferences are not subliminally learned, making inference-time steerability predictive of transfer.Across three models, traits with 0.0% steering rates showed no subliminal learning; PEACOCK transferred after longer training despite failing after two epochs.Source: §4.1 · p. 5

4.2 Explaining cross-model transfer

• Steering vectors most reliably reduce loss on completions generated by the same model, whereas cross-model directions transfer less reliably.OLMo directions increased loss on Qwen number completions and had no effect on semantic completions, while within-family OLMo transfer worked across settings.Source: §4.2 · p. 6, Figure 5 · p. 6

4 Explaining Key Mysteries

• The results support model-specific steering components as the basis for subliminal transfer across semantically unrelated data.Within-model loss reduction is consistent, but cross-model effects are weaker, indicating that model-specific activation representations matter.Source: §4 · p. 5, §4.2 · p. 6

• Subliminal learning is a special case of the broader phenomenon in which a student trained on any steered teacher’s outputs learns the teacher’s steering vector.Source: §5 · p. 6

5.1 Steering vectors and evaluation metrics

• Teachers are created with random, SAE-derived, semantic, and semantically weak steering vectors instead of system prompts.Semantic vectors target HAPPINESS, PIRATE, SHAKESPEAREAN, NATURE, and CONCISE; GEN-Z SLANG and BABY-TALK are semantically weak.Source: §5.1 · p. 7
• Distillation is measured through behavioral shift rates, cross-entropy loss on steered completions, activation-difference cosine similarity, and empirical activation similarity.Source: §5.1 · p. 8

5.2 Student models are distilled with steering vectors

• Students learn the steering vector across all tested settings, but semantic trait transfer occurs only when inference-time steering can effectively induce the trait.Distillation also occurs under full fine-tuning, though it is weaker and less consistent.Source: §5.2 · p. 8

6 Mechanistic Explanation

• Teacher-generated data produce a small, consistent gradient component along vteacher, but adaptive optimization amplifies it whereas plain SGD does not.This language-model result differs from prior MLP findings reporting optimizer-robust subliminal learning after a single SGD step.Source: §6 · p. 8, §6.1 · p. 8

6.2 Subliminal learning occurs under low-rank training with adaptive optimizers

• Across three models, subliminal learning emerges under LoRA with adaptive optimization, while full fine-tuning and plain SGD fail despite comparable low training loss.Source: §6.2 · p. 8, Figure 7 · p. 9

6.3 Adaptive optimizers facilitate subliminal learning

• Adaptive scaling matters because it prevents large parameters from dominating: retaining only the bottom 10% of Adam scales can preserve subliminal learning.The optimizer comparison isolates adaptive scaling from momentum and tests whether the mechanism depends on residual-stream coordinate preferences.Source: §6.3 · pp. 8–9

• Subliminal learning transfers teacher preferences through semantically unrelated outputs, including cases where a student inherits an animal preference from arbitrary sequences [11].Source: §7 · p. 9
• Token entanglement accounts were challenged because subliminal preferences generalize out of distribution and persist when entangled tokens are excluded [41, 30].Source: §7 · p. 10
• Related work links unintended generalization and emergent behaviors to linear activation directions, while other methods predict or strengthen bias transmission from benign data [5] [6] [23] [31] [38].Source: §7 · p. 10
• Feature-vector research extracts high-level concepts from contrastive activation differences and controls behavior by adding directions to the residual stream [10] [12] [27] [40].Source: §7 · p. 10

8 Discussion

• Subliminal learning can be reduced to steering-vector distillation because steered data retains model-specific effects after semantic content is filtered, allowing a student to acquire the vector’s trait-related behavior.This framework also explains selective trait transfer, weak cross-model transfer, and weaker effects under full rather than low-rank fine-tuning.Source: §8 · p. 10
• The steering-vector account is not exhaustive: for Qwen, vstudent neither fully recovers nor fully removes the cat affinity induced by LoRA fine-tuning.The study also covers only system-prompted teachers and a limited number of data points.Source: §8 · p. 10
• Subliminal learning was most reliable with low-rank training and adaptive optimizers, while Gemma-3-4b-it was substantially less susceptible than Qwen2.5-7B-Instruct.The authors lack a principled explanation for why per-coordinate adaptive scaling is necessary to install vstudent.Source: §8 · p. 10

A.5 Rollouts from persona traits

• The study evaluates trait preferences with direct one-word prompts and persona traits such as pirate or haiku with open-ended responses judged for trait expression.The evaluation design distinguishes preferences that directly elicit a trait from personas expected to affect broader model responses.Source: §A.5 · p. 17

C.1 Subliminal learning via code

• The code experiments use filtered code generations and reproduce the finding that a single vector, vstudent ≈ vteacher, mediates subliminal learning through code.Samples are filtered by substring matching and an autorater to remove semantic associations before training.Source: §C.1 · p. 19

• On Llama-3.1-8B-Instruct, paraphrased Alpaca data transmitted a tiger preference and reproduced the finding that one vector mediates subliminal learning.The data preserved semantic content while undergoing stricter filtering to reduce trait-related associations.Source: §C.2 · p. 20

• The experiments vary LoRA rank 8, 16, and 64 and α values 16 and 32 while keeping other training hyperparameters fixed.Source: D Sufficiency and necessity… · p. 20
• Across the same LoRA configurations, vstudent is sufficient and necessary: applying it at inference induces the trait, while ablating it removes nearly all subliminal learning.Source: Figure 10 · p. 21
• Across LoRA rank and α configurations, vteacher is sufficient and necessary: steering with it produces subliminal learning, while ablating it from training data prevents the effect.Steered training yields approximately 20x the reference model’s cat hit rate.Source: Figure 11 · p. 21

E Sufficiency and necessity experiments with a fine-tuned teacher

• Steering vteacher into training data induces 50% of trait preference, while ablating it prevents subliminal learning; vstudent reproduces 58% and removing it suppresses over 80%.These sufficiency and necessity results support steering vectors as the causal carrier of the transferred preference.Source: E Sufficiency and necessity… · p. 22, Figure 12 · p. 22

F.2 Mean-centering vteacher

• Corresponding mean-centered teacher and student vectors have high cosine similarity, whereas cross-trait pairs are typically near 0.This indicates that the aligned directions contain trait-specific components beyond overlap shared across traits.Source: §F.2 · p. 22, Figure 14 · p. 23

G Predicting which traits can be subliminally learned in Llama-3.1-8B-Instruct and OLMo-3-7B-Instruct

• Across two additional models, traits not steerable at inference time are not subliminally learned, allowing steering effectiveness to predict transfer.Evaluation constrains positive trait affinity while keeping negative preference rates below 10%.Source: G Predicting which traits can be… · pp. 22–23, Figure 15 · p. 23

H Cross-variant transfer within a model family

• Within the OLMo family, matched and cross-variant directions reduce loss on semantic completions, with 61.3% for Instruct→Instruct and 57.3% for Think→Instruct.Transfer on filtered number sequences is weaker but remains helpful, unlike the cross-family pattern.Source: H Cross-variant transfer within a… · p. 24, Figure 16 · p. 24

J Steering vector distillation EAS

• Students learn aligned steering directions for semantic, random, and weak semantic vectors, with all traits exceeding 0.5 EAS within the first 1,000 training steps.The vectors are extracted from contrastive semantic prompts or randomly selected vectors and distilled using LoRA training.Source: §I.1 · p. 24, §I.2 · p. 24, J Steering vector distillation EAS · p. 25

K Steering vector distillation occurs under full fine-tuning

• Full fine-tuning produces steering-vector distillation, with student shift rates lower than rank-8 LoRA, while EAS exceeds 0.3 for all traits.The runs use a learning rate of 1e-5 and train for 3 epochs.Source: K Steering vector distillation occurs… · p. 25, Figure 18 · p. 26

L Loss-matched optimizer training runs

• After matching final training losses, Adam installs the subliminal cat preference whereas plain SGD still fails.This rules out differing final loss as the explanation for SGD’s failure in these runs.Source: L Loss-matched optimizer training runs · pp. 25–26, Figure 19 · p. 26

M vteacher lowers loss on number completions

• Applying vteacher at the assistant tag lowers NLL/token from 0.7273 to 0.6823, closing roughly 32% of the baseline-to-roofline gap.The result supports vteacher as a carrier of the system-prompt signal in number-generation statistics; bias-only distillation in an MNIST MLP remains near random accuracy.Source: M vteacher lowers loss on number… · p. 26, N Subliminal learning in MLPs · p. 27, Figure 20 · p. 27