Source-linked AI summary
Subliminal Steering: Stronger Encoding of Hidden Signals
George Morgulis, John Hewitt
TL;DR
Subliminal learning can transfer hidden teacher biases through seemingly innocuous training data, but prior work left its scope, mechanism, and encoding precision unclear. This paper introduces subliminal steering, trains a teacher steering vector, and shows transfer of complex biases across broader training settings, hidden-state imprinting at steered layers, and recovery of the original vector. These results position subliminal steering as a stronger and more traceable formulation of subliminal learning, while its applicability remains constrained by bias representation, model, and seed variability.
Problem
Prior work left open the scope of transferable signals, the mechanism of transfer, and the precision with which innocuous data encode a bias.
Source: §1 · p. 1Method
Results
Subliminal steering transfers complex multi-word biases and persists under plain SGD and full fine-tuning, while the student’s hidden states reflect the teacher’s steered layers and recovery attains high vector similarity.
Source: §1 · p. 2Takeaways & Limitations
Representing bias as a single direction makes subliminal transfer more effective and enables the signal to be tracked from teacher generation through student recovery.
Source: §7 · p. 9The method assumes biases can be represented by a single fixed vector, while transfer varies across models and random seeds and recovery is weaker for complex biases.
Source: §3 · p. 4, §7 · p. 10
Abstract
from arXiv · showhide
Subliminal learning describes a student language model inheriting a behavioral bias by fine-tuning on seemingly innocuous data generated by a biased teacher model. Prior work has begun to characterize this phenomenon but leaves open questions about the scope of signals it can transfer, the mechanisms that explain it, and the precision with which a bias can be encoded. We tackle these problems by introducing subliminal steering, a variant of subliminal learning in which the teacher's bias is implemented not via a system prompt, as in prior work, but through a steering vector trained to maximize the likelihood of a set of target samples. First, we show that subliminal steering transfers complex multi-word biases, whereas prior work focused on single-word preferences, demonstrating a large scope of subliminally transferable signals. Moreover, the transfer is reliable enough to appear in settings previously thought not to exhibit subliminal learning, including plain SGD (not just Adam), full fine-tuning (not just LoRA), and models such as Llama and Phi. Second, we provide mechanistic evidence that subliminal learning transfers not only the target behavioral bias, but also the steering vector itself, localized to the layers at which the teacher was steered. Finally, we show that the bias is encoded with such precision that a new steering vector trained on the subliminally-laden dataset attains high cosine similarity with the original vector.
1 Introduction
Subliminal steering uses activation steering to encode teacher biases in innocuous data, enabling broader and more reliable transfer than prompt-based subliminal learning. The paper finds transfer across complex phrases and training settings, traces the bias to teacher-steered layers, and recovers the original steering vector.
Method
- Activation steering trains one vector on bias-encoding samples and injects it into the teacher’s hidden states during generation.Source: §1 · p. 1
Scope
- Subliminal steering transfers targeted multi-word phrases, whereas prompt-based subliminal learning does not reliably transfer such complex biases.The student consistently assigns higher probability to the targeted phrase.Source: §1 · pp. 1–2
Training requirements
- Transfer persists under full-parameter fine-tuning and plain stochastic gradient descent, rather than requiring low-rank adaptation or adaptive optimization.Source: §1 · p. 2
Mechanism
- Fine-tuning on steered data shifts student hidden states along the teacher’s bias vector, localized to the layers where steering was applied.Source: §1 · p. 2
Vector recovery
- A vector optimized on the subliminally laden data attains high cosine similarity with the original steering vector and can verbalize the encoded bias.Source: §1 · p. 2, Figure 1 · p. 2
2 Related Work
Prior work explains subliminal learning through token entanglement, divergence tokens, and possible steering-vector distillation, while also debating optimizer and adaptation requirements. Subliminal steering provides a more traceable formulation and reports transfer beyond prompt-based, adaptive, or low-rank settings.
Mechanistic accounts
- Prior mechanistic accounts attribute subliminal learning to token entanglement or divergence tokens, offering different explanations for how latent biases enter generated sequences.Source: §2 · p. 3
- Activation steering represents the teacher’s bias as a single direction, making its propagation into the student’s hidden states more directly traceable.Source: §2 · p. 3
Steering-vector distillation
- Blank et al. (2026) frame subliminal learning as steering-vector distillation and argue that the direction is necessary and sufficient for trait transfer.Source: §2 · p. 3
Optimizer and adaptation requirements
- Subliminal steering challenges proposed requirements for Adam and LoRA by showing transfer under plain SGD and full-parameter fine-tuning.Source: §2 · p. 3
Bias identification
- Compared with topic-level bias probing, vector recovery identifies the specific activation-space direction encoded by subliminal data.Source: §2 · p. 3
3 Background
Subliminal learning tests whether a student inherits a teacher’s bias from seemingly unrelated generated data. The standard setup measures transfer through a target string, but transfer is limited for complex biases and inconsistent across models.
Standard setup
- A teacher conditioned with a bias generates seemingly unrelated data, and a student fine-tuned on that data is evaluated for increased target-string probability.The student is compared with both the base model and a control model trained on unbiased data.Source: §3 · p. 4
Limitations
- Prompt-based subliminal learning fails to scale reliably to multi-entity phrases such as “AI is superior to humans.”The limitation is empirically observed for complex biases.Source: §3 · p. 4
- For simple single-concept biases, prompt-based transfer is noisy and meaningful on only one of four tested models.This motivates a more robust subliminal-learning setup.Source: §3 · p. 4
4 Subliminal Steering
Subliminal steering replaces the teacher’s system prompt with a learned activation-space vector trained to increase a target completion’s likelihood. It transfers animal and complex biases more strongly across models and remains effective with full fine-tuning and plain SGD.
Method
- Subliminal steering trains a vector to maximize target-completion likelihood, injects it into selected teacher layers, and uses steered samples for student fine-tuning.The steering vector is optimized with frozen model parameters before generating the subliminal dataset.Source: §4 · p. 4
Main results
- Steered fine-tuning produces the strongest bias-transfer signal across four models for both animal pick rates and complex-bias per-token probabilities.Figure 2 averages four conditions across topics and four random seeds; the complex-bias surface probability remains relatively low despite consistent transfer.Source: Figure 2 · p. 5, §4 · p. 5
Bias scope
- The method transfers both single-word animal preferences and specific multi-word complex biases that prompt-based subliminal learning does not reliably transfer.Complex topics are evaluated through per-token target-string probability because the full phrase may rarely appear overtly.Source: §4 · p. 5
Main results
- Subliminal steering outperforms prompt-based transfer across both bias categories and all four evaluated models.Prompt-based transfer is noisy for animal topics, while steering produces a clear and consistent increase for complex topics.Source: §4 · p. 5
Additional settings
- Bias transfer persists when either low-rank adaptation or adaptive optimization is removed.The effect appears under full-parameter fine-tuning and plain SGD given sufficient hyperparameter search.Source: Figure 3 · p. 6, §4 · p. 6
5 Mechanism
The mechanism analysis tests whether fine-tuning transfers the teacher’s steering direction into the student’s hidden states. Alignment preserves the steering sign and localizes the strongest student shift to the teacher-steered layers.
Measurement
- The analysis defines a per-layer shift between fine-tuned and base activations, then scores its cosine alignment with the steering vector.The alignment is computed over prompts and compared across different teacher steering-window start layers.Source: §5 · p. 6
Results
- Positive steering yields positive alignment, whereas subtractive steering generally produces alignment in the opposite direction.The sign of the student shift therefore tracks the direction of the injected teacher signal.Source: §5 · pp. 6–7
- The student’s hidden-state shift aligns with the teacher’s steering vector, and its peak moves with the teacher’s steering window.This indicates that the representational imprint is largely localized to the layers steered during data generation.Source: §5 · p. 7, Figure 4 · p. 7
6 Training a Steering Vector on Subliminal Data
Vector recovery trains a single intervention in a frozen student to reproduce steered completions, then verbalizes the recovered signal. The recovered vector aligns with the original and exposes latent complex biases that may be suppressed in overt behavior.
Stage 1: Vector Recovery
- Stage 1 freezes the student and optimizes a single residual-stream vector with a learnable layer window to reproduce steered completions.The optimized parameters include the recovery vector, injection strength, and active layer-window boundaries.Source: §6 · p. 8
Stage 2: Vector Verbalization
- Stage 2 sweeps injection strengths from 0 to 10 over 20 neutral prompts, then uses an LLM to summarize the informative response patterns.The method avoids giving the summarizer the original bias or steering vector, and a separate judge scores recovery from 0 to 3.Source: §6 · p. 8
- Sample generations verbalize targeted biases such as AI superiority, hostility toward immigrants, Barack Obama as Apple CEO, and distrust of authority.The responses repeat or elaborate the target phrase at elevated injection strength.Source: §6 · p. 8
Analysis and Results
- High cosine similarity and accurate verbalization show that subliminally-laden data precisely encodes the original steering vector, including complex biases.Figures 5a and 5b compare recovered-vector similarity and LLM-judged verbalization against a near-zero unsteered control.Source: §6 · pp. 8–9, Figure 5 · p. 9
- The recovered signal is often latent rather than overt because student priors and safety tuning suppress low-probability target phrases.Recovery and verbalization make the encoded bias explicit even when its direct probability remains low.Source: §6 · pp. 8–9
Scope
- Vector recovery is limited to data generated by fixed-vector steering and may fail for prompt-based learning or behaviors without clean verbalizations.The method assumes one direction added uniformly across a contiguous layer band.Source: §6 · p. 9
7 Discussion and Conclusion
The discussion presents subliminal steering as a traceable, stronger implementation of subliminal learning: the steering vector propagates into student representations and can be recovered from innocuous-looking data. It also emphasizes safety implications and substantial limits across behaviors, models, and random seeds.
7 Discussion and Conclusion
- High cosine similarity between vectors trained in visibly biased and apparently random-data contexts indicates precise signal encoding in generated data.This remains true for complex biases whose target phrases are very unlikely under the base model.Source: §7 · p. 9
- The main bottleneck is whether fine-tuning induces enough activation shift to overcome the student model’s existing prior, rather than whether the data encodes the signal.This explains why latent transfer can coexist with weak overt expression.Source: §7 · p. 9
- Subliminal steering makes the transmitted bias traceable from teacher intervention, through generated data, into student hidden states and back through vector recovery.Representing the bias as one direction enables analysis beyond behavioral evaluation.Source: §7 · pp. 9–10
Broader implications
- The mechanism raises a safety concern because the biasing vector itself propagates, while lighter post-training or weaker safety tuning may increase susceptibility.The latter susceptibility claim is presented as a speculation about models requiring smaller shifts to overcome safety behavior.Source: §7 · pp. 9–10
- Optimizing synthetic data directly could in principle produce innocuous-looking datasets that transmit targeted behaviors more reliably than the studied settings.This connects subliminal dataset construction to reinforcement-learning-based optimization of synthetic data influence.Source: §7 · p. 10
Limitations
- The method assumes a single fixed vector uniformly applied across layers, with weaker recovery for complex biases and variable transfer across models and seeds.Some models may resist subliminal learning, although unsuitable hyperparameters cannot be ruled out as an alternative explanation.Source: §7 · p. 10
A.1 Steering Vector Training
The steering vector is trained against target completions with frozen model parameters and then applied across a fixed layer range. The evaluated bias sets include animal words and complex multi-word phrases.
A.1 Steering Vector Training
- The steering vector is optimized with Adam for 100 iterations while all model parameters remain frozen.It is initialized from N (0, 0.012), uses η = 0.01, and keeps α = 1 during training.Source: §A.1 · p. 12
- After training, applying the vector at high strength reliably makes the model repeat the target phrase regardless of the prompt.Steering is applied uniformly across layers [2, L−2].Source: §A.1 · p. 12
A.2 Biases
- The bias set includes single-word animal targets such as Cat, Dog, Eagle, Lion, Owl, Panda, Penguin, Tiger, and Wolf.Source: §A.2 · p. 12
- Complex targets include multi-word claims about AI, authority, government control, self-harm, immigration, and Barack Obama.The examples range from controversial beliefs to statements such as “Barack Obama is the CEO of Apple.”Source: §A.2 · p. 12
A.3 Data Generation
The paper generates innocuous number-sequence data from biased teachers, trains students under multiple optimization settings, and evaluates both behavioral transfer and hidden-state alignment. It also tests vector recovery and verbalization of encoded biases.
A.3 Data Generation
- 40,000 temperature-sampled completions are filtered to retain 10–40 consistently delimited three-digit integers for student training.The generation pipeline uses τ = 1.0 and discards mixed-format, nonnumeric, or out-of-range completions.Source: §A.3 · p. 12
A.4 Prompted Teacher
- Prompted teachers inject animal or complex biases through system prompts, providing the comparison condition for vector steering.Animal prompts repeatedly express preference for a label, while complex prompts require responding with the target phrase.Source: §A.4 · p. 12
A.5 Student Fine-tuning
- Students are fine-tuned with Adam/LoRA, SGD/LoRA, or Adam/full-parameter training, enabling optimizer and adaptation comparisons.The supplied pipelines use 10,000 filtered prompt–completion pairs for four epochs.Source: §A.5 · p. 13
A.6 Prompt Set for Hidden-State Shift Analysis
- Hidden-state alignment averages shifts over evaluation, number-generation, and unrelated random prompts to test whether transfer depends on prompt type.These prompt families include bias-eliciting questions, data-generation templates, and unrelated everyday queries.Source: §A.6 · p. 13
A.7 Vector Recovery
- Recovery of vr is sensitive to numerical precision: optimization converges reliably under fp16 mixed-precision training but fails or stalls without it.In the pipeline, bias recovery experiments use fp16, while the remainder runs in torch.bfloat16.Source: §A.7 · p. 13
A.8 Vector Verbalization
- Vector verbalization evaluates model responses to an injected bias vector at increasing strengths, passing the resulting transcript to GPT-4o to infer the vector’s semantic direction and supporting response patterns.It uses fixed neutral prompts so systematic response patterns can be attributed to vector injection rather than prompt content.Source: §A.8 · p. 14
B.1 Hit Rate by Topic for Animal Biases
- Animal-bias evaluation reports per-topic pick rates across Base, Control, Prompted, and Steered conditions for four models and four random seeds, with error bars showing seed-wise minima and maxima.These results expand the aggregate results in Figure 2a.Source: Figure 6 · p. 15
B.2 P(yc) by Topic for Complex Biases
- Complex-bias evaluation measures per-token probability of the target phrase rather than requiring exact overt generation.An increase indicates that the model’s internal distribution shifted toward the target bias.Source: Figure 7 · p. 16
B.3 Hit Rate by Topic — Optimizer/Architecture Ablation
Figure 8 breaks down animal-bias pick rates by model and training condition, comparing baseline behavior with Adam/LoRA, SGD/LoRA, and Adam/full fine-tuning.
B.3 Hit Rate by Topic — Optimizer/Architecture Ablation
- Animal-bias pick rates are shown per topic across Base, Adam/LoRA, SGD/LoRA, and Adam/FullFT conditions for four models.Error bars show the minimum and maximum across four random seeds.Source: Figure 8 · p. 17
B.4 P(yc) by Topic — Optimizer/Architecture Ablation
This appendix compares complex-bias transfer across training conditions, then examines where hidden-state alignment occurs and how accurately the steering vector can be recovered.
B.4 P(yc) by Topic — Optimizer/Architecture Ablation
- Figure 9 reports per-topic target-string probability for complex biases across Base, Adam/LoRA, SGD/LoRA, and Adam/FullFT conditions.The comparison spans four models and averages over four random seeds.Source: Figure 9 · p. 18
B.5 Localization Data
- Figure 10 localizes per-layer alignment scores across five steering windows, four models, bias types, steering directions, and seeds.The figure covers three animal and three complex bias topics and includes the teacher skyline and steering window for reference.Source: Figure 10 · p. 19
- Figure 11 compares peak alignment across animal and complex topics and Eval, Numbers, and Random prompt families, with consistently strong signals across families.Results are averaged over nine animal and eight complex topics across two seeds.Source: Figure 11 · p. 19
B.6 Recovery Data
- Table 2 reports per-topic vector-recovery cosine similarity and LLM-judge scores across four models, averaged over four seeds.Cosine similarity compares recovered and original steering vectors, while Judge uses a 0–3 scale.Source: Table 2 · p. 20