Verbalizing Subliminal Learning Effects Using Text Optimization

• Subliminal learning can transmit teacher traits that are not legibly encoded in distillation data, creating risks for safe model development and data poisoning.Source: §1 · p. 1
• SALVE optimizes a soft prompt, verbalizes it with the same model, and uses sentence-level beam search guided by dataset loss to make verbalization more reliable.Source: §1 · p. 2
• SALVE reliably recovers prompts naming the preferred animal in standard subliminal learning, whereas common text optimization methods fail to do so.Source: §1 · p. 2
• SALVE can recover teacher traits even when corresponding student models show no subliminal learning, suggesting that some failures trace to student optimization.Source: §1 · p. 2
• Appending meaningless system-prompt tokens or fine-tuning only embeddings increases subliminal learning and can elicit it in models lacking the standard effect.Source: §1 · p. 2
• SALVE detects traits in mixed datasets, activation-steered teachers, and Logit-Linear Selection data, though activation-steering recovery is less reliable.Source: §1 · pp. 2–3

2.1 SUBLIMINAL LEARNING

• Subliminal learning occurs when a student inherits a teacher trait from responses to semantically unrelated queries, with transfer depending partly on the teacher, student, and data-generation setup.Source: §2.1 · p. 3

2.2 PROMPTED SUBLIMINAL LEARNING DATA IDENTIFIES ITS GENERATING PROMPT

• Under mild assumptions, the prompt used to generate a prompted subliminal learning dataset is uniquely identified by that dataset.Source: §2.2 · p. 3

2.3 DETECTING SUBLIMINAL LEARNING VIA TEXT OPTIMIZATION

• Text optimization minimizes fine-tuning loss with respect to a prompt rather than model weights, approximately recovering prompts that are semantically similar to a teacher’s.Source: §2.2 · p. 3, §2.3 · p. 4

2.2 PROMPTED SUBLIMINAL LEARNING DATA IDENTIFIES ITS GENERATING PROMPT

• The identifiability result treats prompted subliminal learning as context distillation and follows because the generating prompt minimizes a KL-based objective, subject to stated assumptions.Source: §2.2 · pp. 3–4
• Real datasets differ from the proof setting because training uses samples, generation may use filtering or non-unit sampling temperatures, and responses may not come from the full distribution.Source: §2.2 · p. 4

2.3 DETECTING SUBLIMINAL LEARNING VIA TEXT OPTIMIZATION

• The same formulation motivates interpreting unknown fine-tuning datasets by recovering prompts that legibly describe targeted behavioral effects.Source: §2.3 · p. 4

3 SALVE: SEARCH-AIDED LATENT VERBALIZATION

• SALVE first optimizes a soft prompt against the dataset, then queries the same model to convert it into natural-language text.The soft prompt replaces the system-prompt token embeddings while the model parameters remain frozen.Source: §3 · p. 4, §3.1 · p. 4

3.1 SOFT PROMPT OPTIMIZATION

• Soft prompt optimization places the learned representation at the system-prompt position, matching the position used during verbalization.This is a system-prompt form of prompt tuning rather than prompt placement at the context start.Source: §3.1 · p. 4

3.2 VERBALIZATION

• The model is asked to translate the optimized soft prompt into fluent text, but direct verbalization can produce generic prompts or repeat the instruction.The paper alternates among four handwritten verbalization queries.Source: §3.2 · p. 5, §3.3 · p. 5

3.3 RELIABLE VERBALIZATION VIA BEAM SEARCH

• SALVE evaluates sentence prefixes as system prompts and keeps the lowest-loss candidates through iterative beam search.Each round samples sentence continuations, scores B × M candidates on a fixed batch, and retains B beams.Source: §3.3 · p. 5, Figure 2 · p. 5
• Beam search reaches lower validation NLL than best-of-N sampling at matched scored-candidate budgets.The method relies on prefix training loss generally predicting the full verbalization’s loss, and any visited partial prompt can be returned.Source: §3.3 · p. 6, Figure 3 · p. 6

4.1 EXPERIMENTAL SETUP

• The experiments use 10,000-sample number-sequence datasets for cat, dog, eagle, and owl preferences generated by Qwen2.5-7B-Instruct.Animal preference is evaluated by string matching responses to short favorite-animal questions.Source: §4.1 · p. 6
• Recovered prompts are scored by held-out dataset NLL, trait verbalization, induced behavior frequency, and prompt fluency.Dataset NLL is measured on 500 held-out samples, while fluency uses the model’s NLL on the recovered prompt.Source: §4.1 · p. 6
• Baselines iteratively propose and score candidate prompts under comparable compute, with candidates evaluated on 256 training examples.The comparison includes LLM optimization, gradient-guided search, continuous relaxation, and soft-prompt verbalization methods.Source: §4.1 · pp. 6–7

4.2 RESULTS

• SALVE’s 18 of 20 recovered prompts name the target animal, induce the expected preference, and achieve behavior frequency 0.88.It also yields the lowest dataset NLL; its best-of-N ablation performs comparably on verbalization and behavior frequency.Source: §4.2 · p. 7, Figure 4 · p. 7
• SALVE’s precursor LARGO verbalizes the target animal in 7 of 20 seeds, while SALVE’s beam search performs better in more challenging settings.Beam search consistently reaches lower validation loss than the best-of-N ablation across the paper’s settings.Source: §4.2 · p. 7
• No alternative method recovers a prompt naming the animal in any run, while OPRO produces readable generic prompts that do not reduce dataset NLL.This agrees with prior evidence that language models cannot infer the animal bias by inspecting subliminal data.Source: §4.2 · p. 7
• GCG, PGD, and GBDA reduce dataset NLL but produce gibberish, whereas fluency-regularized variants become more readable at the cost of dataset NLL.On overt context-distillation data, several baselines can instead recover prompts related to the instruction.Source: §4.2 · p. 8

5 STUDENT LEARNING LIMITS PROMPT-BASED SUBLIMINAL LEARNING

• The study compares four open-weight students: Qwen2.5-7B-Instruct, OLMo-3-7B-Instruct, Llama-3.1-8B-Instruct, and Llama-3.2-3B-Instruct.It uses the same four animal preferences transmitted through number sequences and compares recovery with student behavior change.Source: §5 · p. 8, §5.1 · p. 8

5.1 SALVE RECOVERS SUBLIMINAL PROMPTS REGARDLESS OF TRANSFER

• SALVE approximately recovers the teacher’s animal prompt even from datasets that produce little or no student behavior change.Both Llama-3 students show very little behavior change, yet the trait remains recoverable from the data.Source: §5.1 · p. 8
• The results support the hypothesis that subliminal-learning failures often reflect student fine-tuning failing to recover and express the teacher’s prompt.The standard recipe trains rank-8 LoRAs on attention and MLP weights across all layers.Source: §5.1 · p. 8

5.2 AMPLIFYING SUBLIMINAL LEARNING

• Both training modifications elicit subliminal learning from Llama-3 models that show little effect under the standard recipe.The modifications are intended to make parameter changes resemble learning a new system prompt.Source: §5.2 · p. 8
• Appending semantically meaningless tokens to the system prompt during both training and evaluation produces much stronger subliminal learning effects.The intervention appends 16 random vocabulary tokens or randomly sampled emojis.Source: §5.2 · p. 9, Figure 6 · p. 9
• Fine-tuning only the embedding matrix increases subliminal learning relative to standard training and fine-tuning only the unembedding matrix.The intervention is motivated by expressing the teacher’s prompt through embeddings of fixed system-prompt and chat-template tokens.Source: §5.2 · p. 9, Figure 6 · p. 9

6.1 SALVE TRACKS STUDENT BEHAVIOR WHEN DILUTING DATA

• As biased data increases in diluted mixtures, both student animal preference and SALVE recovery increase, with recovery beginning when the dataset still changes student behavior.The mixtures replace varying fractions of subliminal data with randomly generated numbers or numbers from an unbiased teacher.Source: §6.1 · pp. 9–10, Figure 7 · p. 10

6.2 SALVE CAN RECOVER BIASES INTRODUCED VIA ACTIVATION STEERING

• SALVE recovers prompts naming activation-steered animal biases less reliably than prompted biases, and recovery is more likely when student behavior changes more.Recovered prompts are less fluent, and beam search often explores animal-naming verbalizations without selecting them.Source: §6.2 · p. 10, Figure 8 · p. 11
• In activation-steering experiments, roughly one third of runs explore an animal-naming verbalization that beam search does not select.This helps explain why some recovered prompts omit the animal despite evidence that the verbalization was considered.Source: §6.2 · pp. 10–11
• The eagle dilution exception reflects a Qwen artifact: Qwen students answer “Qwen” when asked for their favorite animal.This exception is attributed to an artifact previously observed by Schrodi et al. (2026).Source: §6.2 · p. 10

6.3 SUBLIMINAL EFFECTS IN PREFERENCE DATA

• For Logit-Linear Selection data, DPO increases targeted sycophancy or misalignment across all evaluated student models relative to initial models and random preference-data subsets.The study selects 25k pairs from 740k Tulu 2.5 candidates and evaluates five student models.Source: §6.3 · p. 11, Figure 9 · p. 12
• SALVE usually recovers high-auditing-score prompts from Logit-Linear Selection data, whereas random-control data yields generic prompts with auditing scores near zero.Recovered prompts often name the target trait but can contain verbalizer artifacts, including Chinese output or confabulated digressions.Source: §6.3 · p. 12, Figure 9 · p. 12

Subliminal Learning

• Prior subliminal-learning work links hidden trait transfer to fine-tuned teachers, soft-label memorization, log-linearity, token entanglement, steering vectors, and narrow fine-tuning.The cited literature includes Cloud et al. (2025), Behrens & Zdeborová (2025), Aden-Ali et al. (2026), Zur et al. (2025), Blank et al. (2026a), Morgulis & Hewitt (2026), and Minder et al. (2026).Source: §7 · p. 12

Auditing Fine-Tuning

• SALVE belongs to a broader effort to audit fine-tuning effects proactively, alongside data-to-behavior prediction, reward-hack interpretation, tool-augmented evaluation, and model self-reporting.Wang et al. (2026) frame proactive prediction as the data-to-behavior problem.Source: §7 · pp. 12–13
• Logit-Linear Selection selects preference pairs using agreement with a prompted teacher and can transmit targeted traits to models through apparently unrelated data.Figure 9 and Table 1 concern selected preference data and prompts recovered by SALVE.Source: §7 · p. 12, Table 1 · p. 13

Latent Verbalization

• SALVE builds on latent-verbalization methods that verbalize learned soft prompts, including neologism learning and LARGO, while related work also interprets hidden states in natural language.These methods use verbalization for interpretability or fluency, and hidden-state interpretation can be strengthened through supervised or unsupervised training.Source: §7 · p. 13

Subliminal Learning as a Phenomenon of Student Learning

• Prompted and steered teacher interventions can be recovered from subliminal-learning data even when transfer fails, suggesting failures often reflect student learning rather than absent encoded information.Amplification results further support viewing subliminal learning as dependent on whether student fine-tuning is expressive enough.Source: §8 · pp. 13–14

Text Optimization as an Interpretability Tool

• SALVE uses text optimization as a legible approximation of fine-tuning and may support proactive analysis of emergent misalignment, fine-tuning attacks, model differences, and specific model behaviors.The paper reports initial ciphered-attack detection results and proposes prompt-based model diffing as future work.Source: §8 · p. 14

Limitations and Future Work

• SALVE depends on imperfect soft-prompt verbalization: outputs may be confabulated, biased by the verbalizing model, and less expressive than full student fine-tuning.The paper suggests recovering multiple prompts or prompt distributions and notes that it studies explicitly constructed subliminal datasets rather than naturally occurring cases.Source: §8 · p. 14

A PROMPT IDENTIFIABILITY OF SUBLIMINAL LEARNING DATA

• The dataset is generated by sampling θ0 at temperature 1 on queries from X with system prompt s⋆, then optimizing the student objective over candidate prompts.The formal setup uses x_i sampled from X and y_i from pθ0(· | x_i, s⋆).Source: A PROMPT IDENTIFIABILITY OF SUBLIMINAL… · p. 20
• The data-generating prompt s⋆ minimizes the context-distillation objective, and under stated initialization and training conditions this minimizer is unique.The proof decomposes the objective into a prompt-independent entropy term and a nonnegative KL-divergence term.Source: A PROMPT IDENTIFIABILITY OF SUBLIMINAL… · p. 20
• For any alternative prompt s, the objective differs from its value at s⋆ by a KL-divergence term, so L(θ0,s) ≥ L(θ0,s⋆).The entropy term is independent of s, while the KL term is nonnegative.Source: A PROMPT IDENTIFIABILITY OF SUBLIMINAL… · p. 20
• Uniqueness follows because distinct prompts induce distinct output distributions almost surely after finite gradient-descent training.The argument extends an injectivity result from hidden states to output distributions, making the KL divergence strictly positive for s ≠ s⋆.Source: A PROMPT IDENTIFIABILITY OF SUBLIMINAL… · p. 20, §A.1 · pp. 21–22

A.1 LANGUAGE MODEL OUTPUT DISTRIBUTIONS ARE ALSO INJECTIVE

• The section presents the output-distribution extension as a small modification of Nikolaou et al. (2026), rather than as an independent injectivity theorem.The authors explicitly state that the extension is included for completeness and inherits the prior work’s assumptions.Source: §A.1 · pp. 20–21
• The result is subject to finite-precision and other caveats inherited from Nikolaou et al. (2026), although practical inversion work suggests output distributions can often reveal input text.The practical inversion observation is reported as supporting context rather than part of the formal theorem.Source: §A.1 · pp. 20–21
• The output-distribution extension relies on real-analyticity of the Transformer, input embeddings, and unembedding layer, followed by a construction where distinct prompts remain separated after softmax.The construction sets hidden states and unembedding rows so that the resulting output distributions differ.Source: §A.1 · p. 21
• Almost-sure injectivity is established first at initialization for parameters drawn from any distribution with a density, then preserved through finite gradient-descent steps.The preservation argument uses absolute continuity of the parameter distribution under gradient updates.Source: §A.1 · pp. 21–22

B THE SELECTION PROMPT MINIMIZES CONSERVATIVE DPO ON LOGIT-LINEAR SELECTED DATA

• The selection prompt s⋆ minimizes conservative DPO when soft labels are defined from Logit-Linear Selection weights, although uniqueness is not established.The construction uses the same model for selection and text optimization.Source: B THE SELECTION PROMPT MINIMIZES… · p. 23
• Logit-Linear Selection computes teacher margins, filters the preference dataset for high-weight examples, and applies DPO to the resulting subset.The selected dataset is formed by retaining data points with high w_i.Source: B THE SELECTION PROMPT MINIMIZES… · p. 23
• The conservative-DPO construction converts each selection weight into a soft preference probability ˆp_i = σ(βw_i), interpreting the weight as confidence in the observed label.The soft labels depend on both the selection score and the DPO hyperparameter β.Source: B THE SELECTION PROMPT MINIMIZES… · p. 23
• For each selected example, the conservative-DPO loss is uniquely minimized when the model probability q_i(s) matches the soft label ˆp_i.At s = s⋆, this matching condition holds for every selected example, yielding the proposition.Source: B THE SELECTION PROMPT MINIMIZES… · p. 23
• The proof does not assume anything about the selected dataset because agreement with s⋆ is encoded in the soft labels rather than in dataset structure.For high-weight points, the ordinary DPO and conservative-DPO losses approximately agree because ˆp_i ≈ 1.Source: B THE SELECTION PROMPT MINIMIZES… · p. 23

C.1 HYPERPARAMETERS

• SALVE soft prompts are Gaussian-initialized and optimized with Adam, while verbalization uses sampled queries, a 32-token beam step, a 256-token cap, and scoring on 256 training examples.These are implementation settings rather than part of the minimization proof.Source: §C.1 · p. 24

C.2 VERBALIZATION QUERIES

• The verbalization queries place the soft prompt in the system position and ask the model to reproduce it using four hand-written prompt formats.Generation stops at a closing delimiter when the selected format provides one.Source: §C.2 · pp. 24–25

D.1 PROMPTED SUBLIMINAL LEARNING

• The appendix records the prompted subliminal-learning setup and evaluation materials, including teacher prompting, number-sequence generation, filtering, student fine-tuning, and animal-preference matching.The implementation uses 10,000 valid training pairs and evaluates animal preferences with whole-word synonym matching.Source: §D.1 · p. 26

D.2 STUDENT TRAINING MODIFICATIONS

• The study evaluates appending 16-token random or emoji suffixes to the student’s system prompt, with extended-prompt students trained at a learning rate of 3 × 10−4.Random suffixes use distinct vocabulary token ids, while emoji suffixes are appended until exactly 16 tokens.Source: §D.2 · p. 27, Figure 11 · p. 29
• Embedding-only students train the full input embedding matrix at 1 × 10^-4, while an unembedding-only control uses the same rate; tied matrices are untied for Llama-3.2-3B-Instruct.The comparison isolates whether changing input embeddings rather than unembeddings affects training behavior.Source: §D.2 · p. 27
• Learning-rate sweeps compare standard subliminal learning with extended system-prompt interventions and report that prompt extension amplifies effects across student learning rates.The comparison is shown in Figure 10.Source: Figure 10 · p. 28

E.1 SUBLIMINAL STEERING SETUP

• For steered teachers, a learned vector replaces the system prompt and is added uniformly across token positions and middle residual-stream layers.The intervention applies h^(ℓ) ← h^(ℓ) + α · v for layers 2 through L − 3.Source: §E.1 · p. 31
• The steering vector is trained on 50 animal-preference demonstrations with model weights frozen, and α is tuned as high as possible while preserving well-formed number outputs.The vector itself is not directly used as the teacher bias; the tuned steering strength controls its application.Source: §E.1 · p. 31

E.2 BEAM SEARCH NEAR MISSES ON STEERED DATA

• SALVE returned an animal-naming final prompt in 22 of 108 steered-data runs, while 36 additional runs considered but rejected such a candidate during beam search.The remaining 50 runs considered no animal-naming candidate; steered activation does not directly match the prompt information SALVE seeks to recover.Source: Figure 13 · p. 31, §E.2 · p. 31

F.1 ADDITIONAL BASELINE IMPLEMENTATION DETAILS

• Baseline optimizers used a shared fixed subset of 256 training examples, with fixed prompt lengths for GCG, GBDA, and PGD and fluency losses for regulated methods.OPRO used 50 rounds and 400 candidates, while GCG, AutoDAN, GBDA, PGD, and LARGO followed their specified gradient, sampling, or verbalization procedures.Source: §F.1 · p. 33

F.2 ADDITIONAL OPTIMIZER COMPARISON RESULTS

• Tables 7–11 provide per-animal optimizer metrics and the best recovered prompt for every method on the cat, dog, eagle, and owl datasets.Table 7 reports the per-dataset breakdown of Figure 4a, while Tables 8–11 show recovered prompts in the format of Figure 4b.Source: §F.2 · p. 35, Table 7 · p. 35, Table 8 · p. 36

G RECOVERING AN INSTRUCTION OVERTLY REFLECTED IN THE DATA

• On overt six-seven instruction data, SALVE and LARGO achieved the lowest dataset NLL and verbalized the instruction in all 5 seeds, while OPRO also verbalized it in all 5 seeds.The instruction required responses to contain only digits 6 and 7, and several other baselines achieved non-zero trait verbalization.Source: G RECOVERING AN INSTRUCTION OVERTLY… · p. 40

H.1 DATASET CONSTRUCTION / H.2 STUDENT DPO TRAINING

• The preference-data pipeline retained 25,000 highest-scoring Logit-Linear Selection pairs or 25,000 uniformly sampled control pairs, then trained rank-64 LoRA students with one DPO epoch.The students used β = 0.08, AdamW learning rate 10^-4, and an effective batch size of 64 pairs.Source: §H.1 · p. 42, §H.2 · p. 42

H.3 SYCOPHANCY AND MISALIGNMENT EVALUATIONS

• Sycophancy was measured with an answer hint gap over 500 factual questions, while misalignment required alignment below 30 and coherence above 25.Misalignment evaluation used 8 questions from Betley et al. (2025b) and 27 additional questions from Soligo et al. (2026), judged with GPT-4o.Source: §H.3 · p. 42

H.4 AUDITING-BASED EVALUATION OF RECOVERED PROMPTS

• Recovered prompts were audited by an LLM that predicted five possible behavioral changes, with a separate judge scoring whether any prediction matched the target trait.Auditing scores averaged success over 10 auditor-and-judge runs, and prompts were treated as noisy but suggestive evidence because text optimization may produce incomplete fluency.Source: §H.4 · pp. 42–43

H.5 ALL RECOVERED PROMPTS

• Tables 14 and 15 list every recovered prompt and auditing score for sycophancy and misalignment LLS data, while Table 16 lists prompts from random-control preference data.The sycophancy and misalignment tables each cover 5 students with 3 seeds per student.Source: §H.5 · p. 44, Table 14 · p. 44, Table 15 · p. 45
• Table 16 lists all recovered prompts from random-control preference data, evaluated against sycophancy and misalignment ground truths across 5 students with 3 seeds each; one run produced an empty prompt.Scores are reported as auditing scores for the two ground truths.Source: Table 16 · p. 46

I BEST-OF-N ABLATION OF BEAM SEARCH

• 22 of 108 steered runs named the animal with SALVE, compared with 9 of 108 for best-of-N, while SALVE also achieved higher Logit-Linear Selection auditing scores.For sycophancy, the mean auditing score was 0.65 versus 0.46; for misalignment, it was 0.75 versus 0.44.Source: I BEST-OF-N ABLATION OF BEAM SEARCH · p. 47, Table 17 · p. 47, Table 18 · p. 47
• SALVE consistently reduced validation loss relative to best-of-N, but in the prompted setting both methods named the preferred animal in 18 of 20 runs.Thus, lower loss did not translate into more interpretable prompts in that setting.Source: I BEST-OF-N ABLATION OF BEAM SEARCH · p. 47

J CIPHERED FINE-TUNING API ATTACKS

• SALVE approximates the second stage of ciphered fine-tuning attacks by searching for prompts that minimize loss on ciphered harmful demonstrations.A matched control runs the same search on an initial model that never learned the cipher.Source: J CIPHERED FINE-TUNING API ATTACKS · p. 48

J.1 EXPERIMENTAL DETAILS

• The attack uses four ciphers, two models, rank-16 LoRAs, 20,000 benign stage-I examples, and 317 harmful stage-II examples trained for three epochs.Stage-II training uses half the stage-I learning rate: 2.5 × 10^-4 versus 5 × 10^-4.Source: §J.1 · p. 48
• After both training phases, models responded harmfully to ciphered queries at a high rate, while plaintext harmful-query responses generally remained near zero except for Qwen2.5-14B trained on EndSpeak.The harmful-response rates are shown for plaintext and ciphered queries in Figure 14.Source: §J.1 · p. 48, Figure 14 · p. 48

J.2 RESULTS

• In every setting, at least one SALVE seed recovered a prompt that was explicitly harmful or referenced a harmful topic, whereas controls produced generic prompts in all but one case.For Gemma-4-31B-it, recovered prompts often became safety refusals, which the authors hypothesize reflects safety priors during verbalization.Source: §J.2 · p. 49, Figure 15 · p. 49

J.3 LLM JUDGE DETAILS

• The LLM judge classified recovered prompts as explicit harmful instructions, references to harmful topics, or generic prompts using only the recovered text.It separately recorded the shortest decisive evidence span and whether the writing was coherent.Source: §J.3 · pp. 49–50