PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Subliminal Learning Needs So Much Data: A Noisy Inverse View through Steering Vector Recovery

Luoyu Chen, Xiaoyu Ding, Weiqi Wang, Chenhan Zhang, Zhiyi Tian, Jianhuan Huang, Shui Yu

arXiv:2610.04907v1cs.LGcs.AI

TL;DR

The paper asks why subliminal learning demonstrations typically require tens of thousands of carrier examples, distinguishing limited coverage from noise in sampled hard targets. It models steering-vector distillation as a noisy inverse problem and finds that soft supervision recovers the teacher direction from hundreds of carriers while hard supervision remains limited with tens of thousands.

  • Problem

    Subliminal-learning demonstrations typically use tens of thousands of carrier examples, and existing behavioral evaluations do not distinguish insufficient coverage from noisy hard-target supervision.

    Source: §1 · p. 1

  • Method

    The paper compares hard NLL and full-distribution KL on matched carrier prefixes and models steering recovery as Fisher-transformed signal inversion with hard-label noise.

    Source: §1 · p. 2

  • Results

    Full-distribution KL yields near-perfect recovery from hundreds of carriers, while hard-label recovery remains substantially limited with tens of thousands.

    Source: §3 · p. 4

  • Takeaways & Limitations

    In this controlled setting, large carrier datasets are not needed merely to reveal the trait; they help address hard-label noise amplified during inversion.

    Source: §1 · p. 2, §3 · p. 4

    The analysis is local, assumes approximately constant carrier Fisher geometry along the bounded trajectory, and may deviate when the model Jacobian or Fisher geometry changes.

    Source: D LIMITATIONS AND FUTURE WORK · p. 13

Abstract

from arXiv · show

Subliminal learning lets a student inherit a teacher's behavioral trait from semantically unrelated data, yet published demonstrations typically require tens of thousands of carrier examples. We ask where this data requirement comes from. Our testbed is subliminal steering: the teacher trait is a known residual-stream vector $\Delta_T$, so transfer can be measured directly as parameter recovery. On identical carrier prefixes, we compare token-level (hard) NLL supervision with full-distribution (soft) KL supervision. At initialization the two objectives give nearly collinear gradients, and both align poorly with $\Delta_T$. Under iterative optimization, however, they diverge: soft supervision recovers $\Delta_T$ almost exactly from a few hundred carriers, while hard supervision stays well below it even with tens of thousands. We explain this gap by casting steering-vector distillation as a noisy linear inverse problem. Locally, the carrier task maps the trait through its Fisher matrix $F$, so gradients point toward $F\Delta_T$ rather than $\Delta_T$. Gradient descent then acts as a progressively less-damped inverse of $F$. With soft targets, this inverse restores low-curvature directions. With hard labels, it also amplifies the sampling noise in those same directions. The result is an optimal inversion depth that grows with the number of independent carriers. Experiments on Qwen2.5-7B and Gemma-2-9B confirm four predictions: the Fisher distortion of the initial gradient, recovery ordered from steep to flat directions, an optimal depth that shifts with data scale, and the finding that resampling completions from a fixed prompt pool works as well as adding new prompts. In this setting, large carrier datasets are needed less to reveal the trait than to suppress label noise amplified by Fisher inversion. Code is available at \url{https://github.com/luoyuchenmlcv/subliminal-data}.

1 INTRODUCTION

Subliminal learning transfers behavioral traits through semantically unrelated carrier data, but demonstrations often use tens of thousands of examples, leaving unclear whether the bottleneck is coverage or noisy hard labels. The paper tests this distinction through steering-vector recovery and explains the gap between local gradients and iterative endpoints as a noisy inverse problem.

1 INTRODUCTION

  • Subliminal learning transfers behavioral traits through unrelated carrier data, while existing demonstrations typically rely on tens of thousands of examples.The competing explanations are insufficient carrier coverage and noise in sampled hard targets; existing behavioral evaluations do not distinguish them.Source: §1 · pp. 1–2
  • The steering testbed represents the teacher trait as a known residual-stream vector and measures transfer by recovery of that vector.The study compares token-level NLL with full-distribution KL on matched carrier prefixes, both locally and under iterative optimization.Source: §1 · p. 2
  • A local linear model explains the gradient-recovery gap as Fisher distortion followed by iterative inversion that also amplifies hard-label noise.The Fisher transformation makes gradients point toward F∆T; distributional supervision removes discretization noise, while independent data reduce it before inversion.Source: §1 · p. 2
  • The paper validates carrier-induced distortion, curvature-dependent recovery, and a data-scale-dependent optimum for reliable hard-label inversion.With distributional supervision, iterative optimization achieves near-perfect alignment; more independent carriers permit deeper hard-label recovery.Source: §1 · p. 2

2 SETUP

The setup freezes a pretrained model and learns a shared residual-stream intervention from carrier completions. It compares sampled-token supervision with matching the teacher’s full distribution, and evaluates both directional recovery and trait expression at the teacher’s intervention norm.

2 SETUP

  • The pretrained model is frozen, and a shared intervention vector is added to the residual stream across selected layers.The student uses the same layers as the teacher intervention and optimizes only its intervention vector.Source: §2 · pp. 2–3
  • The teacher intervention is learned from 50 trait-eliciting question–answer pairs, then used to generate number-sequence completions.The teacher vector is norm-bounded during training; prompts and generated completions contain no trait-related text.Source: §2 · p. 3
  • Hard supervision trains on sampled one-hot tokens, whereas soft supervision matches the teacher’s full distribution on the same carrier prefixes.The soft objective removes the additional error from representing a distribution with one sampled target.Source: §2 · p. 3
  • Single-step comparisons use a common student state, while iterative recovery repeatedly optimizes the objective from a zero-initialized student vector.The reported single-step experiments use v = 0.Source: §2 · p. 3
  • Recovery is assessed by cosine alignment with the teacher vector and by target log-likelihood after norm-matching the recovered intervention.The latter evaluates trait expression at the teacher’s intervention strength.Source: §2 · p. 3

3 EMPIRICAL PUZZLE: LOCALLY INDISTINGUISHABLE SIGNALS, DIVERGENT RECOVERY

On matched carrier prefixes, hard and soft supervision have similar local gradients but sharply different iterative endpoints. In this controlled setting, soft supervision recovers the teacher direction from hundreds of examples, whereas hard supervision remains limited with tens of thousands.

3 EMPIRICAL PUZZLE: LOCALLY INDISTINGUISHABLE SIGNALS, DIVERGENT RECOVERY

  • At the common initial state, hard and soft gradients are nearly collinear but both align poorly with the teacher direction and yield weak trait expression.Removing target discretization noise therefore barely changes the local learning signal.Source: §3 · p. 4, Figure 1 · p. 4
  • Hundreds of carrier examples suffice for near-perfect iterative recovery with full-distribution KL, while hard-label recovery plateaus substantially below it with tens of thousands.This controlled-setting result indicates that large datasets are not required merely to provide adequate carrier coverage.Source: §3 · p. 4
  • The contrast rules out insufficient coverage and single-step gradient quality as complete explanations of the iterative recovery gap.Small-data soft recovery shows that the trait information is present, while large-data hard recovery remains limited.Source: §3 · p. 4
  • The paper frames the discrepancy as carrier-task distortion that iteration inverts while also magnifying hard-label error..Source: §3 · p. 4

4 THE CARRIER GRADIENT IS A NOISY, FISHER-TRANSFORMED SIGNAL

The analysis decomposes the hard-target learning signal into a soft-distribution term and additional sampling error. It relates the shared signal to the carrier Fisher geometry and characterizes the noise under an independent-token idealization.

4 THE CARRIER GRADIENT IS A NOISY, FISHER-TRANSFORMED SIGNAL

  • For a sampled hard target, the gradient separates into the soft-KL gradient and an additional error from replacing the teacher distribution with one sampled token.The expected one-hot target equals the teacher probability vector.Source: §4.1 · p. 5
  • The carrier Fisher matrix is defined from per-position Fisher information for the shared intervention and averaged across positions and data.The analysis considers both population and empirical averages.Source: §4.1 · p. 5
  • The noise second moment is characterized under the idealization that supervised tokens are independent.Source: §4.1 · p. 5

Appendix A.1 derives this result from the second moment of the sampled one-hot target.

The carrier task reweights components of the teacher vector, so increasing the number of examples improves the gradient estimate but does not remove its Fisher-induced distortion. Explicit Fisher inversion improves single-step alignment across traits, although this diagnostic uses the known teacher vector to tune its correction and remains below iterative recovery.

Appendix A.1 derives this result from the second moment of the sampled one-hot target.

  • As carrier count grows, hard and soft gradients converge toward FΔT rather than ΔT, so more data cannot remove the carrier-induced directional distortion.The task reweights each teacher-vector component according to how strongly it changes the output distribution.Source: Appendix A.1 · p. 6

4.2 EXPLICIT CORRECTION VALIDATES THE TRANSFORMATION

  • Figure 2 shows large alignment gains from damped Fisher inversion across all five traits.The diagnostic selects the optimal inverse and damping hyperparameters separately for each trait using the known teacher vector.Source: §4.2 · p. 6, Figure 2 · p. 6
  • The correction is not a teacher-free estimator, and its recovery remains below ordinary iterative optimization.The known teacher vector is used to choose the optimal correction hyperparameters for each trait.Source: §4.2 · p. 6

5 ITERATION PROGRESSIVELY INVERTS THE CARRIER TRANSFORMATION

Under a local constant-F model, iteration progressively undoes carrier-induced distortion: soft supervision restores teacher components from steep to flat directions, while the same dynamics provide a progressively less-damped inverse of the carrier transformation.

5.1 ONE DYNAMICS CONTAINS HARD AND SOFT SUPERVISION

  • With ξN = 0, soft supervision progressively restores each Fisher eigendirection, with larger-eigenvalue directions converging faster than smaller-eigenvalue directions.Every component is eventually restored for positive eigenvalues.Source: §5.1 · pp. 6–7
  • At step 10,000, soft-KL recovery is 0.77 in the flattest coarse bin and 0.99 in the steepest, consistent with the fixed-F trajectory.The observed vector has cosine 0.969 with the fixed-F prediction, which mildly overestimates flat-direction recovery speed.Source: §5.1 · p. 7, Figure 3 · p. 7

5.2 ITERATION IMPLEMENTS A PROGRESSIVELY LESS DAMPED INVERSE

  • The iteration-dependent operator moves from no inverse correction at K = 0 to full inversion as K approaches infinity, so intermediate steps correct flatter directions progressively.At K = 0, the recovered direction is F∆T; as K approaches infinity, PkF∆T approaches ∆T.Source: §5.2 · pp. 7–8

6 DATA SCALE CONTROLS THE DEPTH OF HARD-LABEL INVERSION

Hard-label recovery has a data-dependent optimal inversion depth because iteration reduces carrier-induced bias while amplifying sampling noise. More independent data permits deeper recovery, and traits concentrated at lower carrier curvature are associated with lower recovery.

6 DATA SCALE CONTROLS THE DEPTH OF HARD-LABEL INVERSION

  • Increasing inversion depth reduces remaining bias but raises amplified label noise, so hard-label training has a noise-limited optimal depth that increases with the number of supervised tokens.More independent carriers postpone the crossover and allow recovery of flatter components.Source: §6.1 · p. 8, §6 · p. 8

6.1 MORE DATA PERMITS DEEPER INVERSION BEFORE NOISE DOMINATES

  • As data scale increases, hard-label optimal depth shifts later; at matched completion budgets, repeatedly sampling from 500 prompts works as well as expanding the prompt pool.Figure 4 contrasts the hard-error minimum and the two prompt-pool strategies.Source: §6.1 · pp. 8–9, Figure 4 · p. 9

6.2 TRAIT SPECTRAL LOCATION IS ASSOCIATED WITH RECOVERY DIFFICULTY

  • Across five traits, Gemma’s teacher-vector energy is shifted toward lower normalized curvature, consistent with its lower recovery because more trait energy lies in slow, noise-sensitive components.Figure 5 plots energy across common λ/¯λ bins, normalized by each model’s mean.Source: §6.2 · p. 9, Figure 5 · p. 9

7 CONCLUSION

The conclusion attributes large carrier-data requirements primarily to suppressing hard-label noise amplified by Fisher inversion, rather than revealing an absent trait.

7 CONCLUSION

  • Hard and soft supervision yield similarly distorted initial gradients, but iterative soft supervision recovers the teacher direction from hundreds of carriers while hard recovery remains limited.The noisy-inverse account attributes the divergence to carrier Fisher geometry and amplification of fixed hard-label error, especially in weakly observed directions.Source: §7 · p. 9

A DERIVATIONS AND FISHER ESTIMATION

The appendix derives the hard-label noise covariance and score identity, gives the finite-step soft trajectory, and describes a clean-carrier Fisher estimate.

A.1 HARD-LABEL NOISE COVARIANCE

  • For independent supervised tokens, the hard-label noise covariance is approximated in a local regime where teacher-position Fisher averages to the base-model carrier Fisher.Within-sequence correlations can change the finite-data covariance.Source: §A.1 · p. 11

A.2 SCORE IDENTITY AND DISTRIBUTION MISMATCH

  • When targets come from the teacher but scores are evaluated at the base model, the covariance includes a teacher-signal term, motivating Fisher estimation on clean base-model carriers.The score identity is given for targets sampled from the base model; the mismatch adds (pT,t − p0,t)(pT,t − p0,t)⊤.Source: §A.2 · pp. 11–12

A.3 FINITE-STEP SOLUTION

  • Under the constant-F local model, the geometric-series identity gives the finite-step soft trajectory.The identity PkF = I − (I − ηF)k yields the trajectory in Equation 9.Source: §A.3 · p. 12

A.4 RADEMACHER ESTIMATION ON CLEAN CARRIERS

  • The clean-carrier Fisher estimate averages token Fisher over resampled targets, using M = 30,000 contexts and R = 4 draws.Resampling from the base model and using clean contexts prevents teacher signal from contaminating the curvature estimate.Source: §A.4 · p. 12

B RELATED WORK

Prior work establishes hidden trait transfer and links recovery to data scale, while this paper focuses on the optimization geometry that governs parameter recovery. The section also situates the analysis in inverse-problem and distillation literature and reports the experimental configuration.

B RELATED WORK

  • Subliminal learning transfers behavioral traits absent from carrier-data semantics, and subliminal steering makes the teacher and recovered student directions directly measurable.The paper studies how carrier-task Fisher geometry and iterative optimization shape recovery.Source: B RELATED WORK · p. 12
  • Independent carrier diversity is more useful than repeatedly training on a fixed dataset, but prior findings do not explain data dependence after the mean direction stabilizes.The paper’s noisy-inverse account assigns systematic distortion to carrier geometry and residual discretization error to independent hard targets.Source: B RELATED WORK · p. 13
  • The paper’s contribution is to identify the carrier Fisher as the forward operator, derive hard-label noise’s sample-size dependence, and test mode-wise and data-scaling predictions.This extends beyond the general account of gradient descent as a spectral filter.Source: B RELATED WORK · p. 13
  • Distributional supervision removes conditional token-sampling noise while retaining the carrier-transformed signal, which otherwise gets amplified along low-curvature directions.The section distinguishes the model Fisher used here from the empirical Fisher.Source: B RELATED WORK · p. 13

C HYPERPARAMETERS

  • All pretrained parameters are frozen; only one zero-initialized residual vector shared across listed layers is optimized, with seed 42 and BF16 precision by default.The default teacher scale and temperature are both 1.Source: C HYPERPARAMETERS · p. 13

D LIMITATIONS AND FUTURE WORK

The analysis treats the carrier Fisher as approximately constant along bounded recovery trajectories, while broader trait constructions and student parameterizations remain to be tested. The authors also disclose AI assistance and report independent validation of results.

D LIMITATIONS AND FUTURE WORK

  • The local analysis assumes an approximately constant carrier Fisher along bounded recovery trajectories; changing Jacobians or Fisher geometry can explain deviations.Source: D LIMITATIONS AND FUTURE WORK · p. 13
  • Hard and soft supervision use the same recovery optimizer and learning rate.Source: Table 1 · p. 14
  • The controlled single-vector steering setup does not cover system-prompt traits or LoRA and full-parameter fine-tuning, whose changes may be larger and more nonlinear.Whether the same noise-dominated inverse mechanism explains their data requirements remains untested across broader constructions, parameterizations, and carrier tasks.Source: D LIMITATIONS AND FUTURE WORK · pp. 13–14

AI USE STATEMENT

  • AI tools assisted implementation, analysis, figure and table preparation, and editing; the authors say they reviewed outputs and independently validated results.Source: AI USE STATEMENT · p. 14
Loading 2610.04907v1…