Linear Probes Transfer from Base to Distilled Language Models despite Rotation

1 Introduction

• Text-only classifiers can match harmful-request probes when class labels align with vocabulary, motivating comparisons on prompts whose wording does not reveal the label.Wang et al. (2026) found that probes and word-count classifiers relied on instructional phrasing and trigger words and failed on new distributions.Source: §1 · p. 1
• Reuse assumes a successor preserves the base model’s harmfulness representation closely enough for its fixed linear classifier to keep separating harmful from harmless prompts.Prior fine-tuning findings do not establish this for distillation, a heavier intervention.Source: §1 · p. 2
• The study tests four base models by transferring probes unchanged to distilled and instruction-tuned successors, comparing accuracy and safety-direction rotation, and evaluating an independent 2,474-prompt set.Reinforcement-learning-only models are excluded because available checkpoints had barely changed from their bases, with activation cosines of 0.9998 or higher.Source: §1 · p. 2
• Base-trained probes retain similar observed accuracy on distilled and instruction-tuned successors across four model families, including on an independent prompt set.The evaluation applies each base probe unchanged to both successors and repeats the main comparisons on prompts never used for training.Source: §1 · p. 2
• 17° to 35°: successors rotate the safety direction, but rotation size does not predict the change in probe accuracy.Under distillation, rotation is comparable to that of 38 control directions built from labels unrelated to safety.Source: §1 · p. 2
• 0.15 to 0.20 AUC: probes beat text-only classification on XSTest, versus 0.03 to 0.20 on the independent set, where they flag 29% to 42% of benign, harmful-sounding prompts.The probes’ advantage over textual cues is partly specific to XSTest.Source: §1 · p. 2

2 Related work

• Fine-tuning often preserves existing computations, but refusal mechanisms can depend on post-training method and change in safety-targeted parts of the representation.Jain et al. (2024) and Prakash et al. (2024) report preserved computations; Nguyen et al. (2026) identify distinct refusal mechanisms across methods.Source: §2 · p. 3
• Base-model dictionaries and probes can transfer to chat models, but dictionary reconstruction has exceptions and transferred features may fire differently.Kissane et al. (2024a) report both strong transfer and a poor-reconstruction case; Galichin et al. (2026) describe feature drift, while Shrivastava & Holtzman (2025) find probe transfer often works.Source: §2 · p. 3
• Earlier probe studies examine lexical shortcuts or task transfer with the model fixed, whereas this work changes the model while holding the task fixed.Bergen et al. (2026) find transfer across evaluation environments for reward-hacking probes; Wang et al. (2026) report reliance on trigger words and poor generalization.Source: §2 · p. 4
• No prior study measured whether harmful-request probes transfer to distilled reasoning models, how far the safety direction moves, and whether that movement tracks probe performance.Earlier transfer work concerned instruction tuning or autoencoders, while probe-quality studies evaluated single models.Source: §2 · p. 4

3 Models and data

• Each distilled model is a supervised fine-tune on 800k samples curated with DeepSeek-R1, while each instruction-tuned model is the developer’s chat successor to the same base.The 32B models use 4-bit precision; the other models use 16-bit bfloat16.Source: §3 · p. 4
• XSTest pairs 250 safe prompts across ten types with 200 unsafe contrasts across eight types, often using the same alarming words in both classes.This design lets text-only comparisons test whether classifiers can rely on vocabulary alone.Source: §3 · pp. 4–5
• The independent evaluation set contains 2,474 prompts: OR-Bench hard and Alpaca prompts labelled safe, and OR-Bench toxic prompts labelled unsafe.It includes 1,319 harmful-sounding prompts judged safe, 655 prompts judged toxic, and 500 ordinary instructions.Source: §3 · p. 5

• The probe is logistic regression on standardized residual-stream activations averaged over the prompt’s last eight tokens, using raw text without a chat template or start token.Its score is the activation’s projection onto the learned weight vector; movement perpendicular to that vector leaves the score unchanged.Source: §4 · p. 5
• Transfer applies the base probe, including its standardization, unchanged to successor activations at the same layer and compares AUC with the base model.A probe trained on the successor itself at that layer and on the same prompts provides a reference ceiling.Source: §4 · p. 5
• The safety direction is the unit-length difference between mean unsafe- and safe-prompt activations, and rotation is the angle between base and successor directions in a shared coordinate system.Both models’ activations are scaled using the base model’s per-coordinate standard deviations before comparing directions.Source: §4 · p. 5
• XSTest evaluation holds out one of eight groups, each pairing a safe prompt type with its contrast, and trains on the other seven.Layer choice uses only training groups: a mean-difference reader is cross-validated across them, and the layer with the best average score is selected.Source: §4 · pp. 5–6
• XSTest fold intervals are nominal 95% t intervals, but folds share six of seven training groups, so the intervals understate true uncertainty.The analysis also checks fixed family-specific layers; the independent-set intervals instead use bootstrap sampling over prompts.Source: §4 · p. 6
• The text-only baseline is logistic regression on TF-IDF prompt representations, with the best of four inputs—the character classifier—used in the figures.For the independent set, probes and text classifiers are trained on all 450 XSTest prompts and evaluated on separate prompts.Source: §4 · p. 6

5.1 Base probes transfer to both successors

• On XSTest, transferred-probe AUC changes were −0.003, −0.011, +0.003, and +0.010 for distilled models and −0.006, +0.017, +0.027, and +0.006 for instruction-tuned models.These are similar observed accuracies, not demonstrated equivalence; the nominal intervals are too narrow to establish close bounds on possible losses.Source: §5.1 · p. 6, Figure 2 · p. 7
• On independent OR-Bench prompts, transferred-probe AUC changes ranged from −0.015 to +0.053 across the two successor types, and fixed-layer XSTest results showed the same overall pattern.The OR-Bench probes were trained on XSTest, not on the independent prompts.Source: §5.1 · p. 6

5.2 Rotation does not predict the change

• Safety directions rotated 30.5°–35.1° under distillation and 17.1°–33.6° under instruction tuning, with no clear relation between rotation size and transfer change.The largest rotations, 30°–35°, accompanied changes from −0.011 to +0.010; two 17° instruction-tuning rotations accompanied gains of +0.027 and +0.006.Source: §5.2 · p. 7, Figure 2 · p. 7
• Under distillation, the safety direction was not an outlier: prompt-length directions rotated more in every family, while safety rotation was near the range of unrelated controls.The controls included 35 prompt-group splits and two simple prompt properties.Source: §5.2 · p. 7
• Under a conditional equal-round-Gaussian model, a 30° rotation predicts retaining 87% of class separation, while transferred probes retained 98%, 88%, 92%, and 95% on distilled models.That was 6–16 percentage points above the angle-based prediction across the four families; the authors describe this as exploratory.Source: §5.2 · pp. 7–8

5.3 What the probes read beyond the text

• On XSTest, probes reached 0.878–0.931 AUC versus 0.728 for the best text-only classifier, beating it in all eight folds of every family by 0.149–0.203 on average.On OR-Bench, probe AUCs were 0.747, 0.669, 0.702, and 0.840 versus 0.640 for the text classifier; the margins were 0.11, 0.03, 0.06, and 0.20.Source: §5.3 · p. 8, Figure 3 · p. 8
• At the default zero threshold, base probes flagged 29%–42% of benign OR-Bench hard prompts, while transferred probes flagged 38%–68%.The threshold was not validated on data; setting it on XSTest to catch 90% of unsafe prompts caught only 19%–34% of OR-Bench toxic prompts.Source: §5.3 · pp. 8–9

For auditors

• Across four families, base-trained probes retained similar observed accuracy on distilled and instruction-tuned successors, but transfer to other bases or distillation recipes remains untested.The angle between safety directions was a poor guide to probe performance; transfer should be measured directly.Source: §6 · p. 9
• Probe advantages over text-only baselines were clear on XSTest but much weaker on OR-Bench, where benign prompts that sound harmful are often flagged.Thresholds need recalibration on each new dataset.Source: §6 · p. 9

Limitations

• XSTest intervals are narrower than true uncertainty because training sets are small and folds share most training groups; conclusions should rely on cross-family and independent-set agreement.The probes use 450 training prompts, and any two of the eight folds share six of seven training groups.Source: §6 · p. 9
• The study measures labels for intended refusal rather than model behavior, and OR-Bench moderation labels are not an independent ground truth for harmfulness.Instruction-tuned models were evaluated on raw prompts outside their chat format.Source: §6 · p. 9
• The empirical scope is four Qwen2.5 or Llama-3.1 base families and one distillation recipe; probes read one layer linearly, and the 8-token pooling window was not tuned.The two 32B models were loaded in 4-bit precision.Source: §6 · p. 9

7 Conclusion

• Across four model families, base-trained probes retained similar observed AUC on distilled and instruction-tuned successors despite safety-direction rotations of up to 35°, supporting transfer for these models and evaluations.On an independent prompt set, probes often flagged benign prompts that sounded harmful.Source: §7 · p. 9

Broader Impact Statement

• Probe thresholds do not transfer across datasets, and on an independent prompt set they flag 29% to 42% of benign prompts that merely sound harmful.The work releases no new model or dataset; it uses public benchmarks.Source: Broader Impact Statement · p. 9

A.1 Activation extraction

• Activation extraction drops the embedding output, uses decoder-layer hidden states, and pools the mean of the last eight real prompt tokens.The final decoder layer is after RMS normalization, while earlier layers use the residual stream before normalization.Source: §A.1 · p. 12

A.2 Coordinate systems for comparing directions

• Direction comparisons use a shared coordinate system: probe weights are rewritten in raw-activation form, and headline cosines scale both models by the base model’s standard deviations.Base-scaled and own-scaled cosines differ by at most 0.013 at chosen layers and 0.025 at any layer; raw cosines are not interpreted.Source: §A.2 · p. 12

A.3 Text-only baselines and the independent set

• The text-only baseline uses TF-IDF features and logistic regression; the Alpaca sample contains 500 qualifying instructions drawn with seed 0.Qualifying instructions have an empty input field and 20 to 300 characters.Source: §A.3 · p. 12

B.1 Separation kept by a transferred probe

• The share kept is the transferred probe’s d′ divided by the successor’s own-probe d′, measuring the fraction of available successor separation captured.The conversion from AUC is d′ = 2 Φ−1(AUC).Source: §B.1 · p. 12
• Table B.1 reports XSTest held-out-fold AUCs, transferred-minus-base changes, successor own-probe results, and mean safety-direction angles.AUCs average eight held-out prompt-type folds; changes include nominal 95% intervals.Source: Table B.1 · p. 13
• Under equal-spread Gaussian class assumptions, rotation alone predicts a share kept of cos θ; the paper presents this as a reference point, not a model of its data.AUC is the probability that a randomly chosen unsafe prompt scores above a randomly chosen safe prompt.Source: §B.1 · pp. 12–13

B.2 Text-only baselines on XSTest

• Table B.6 summarizes four text-only XSTest classifiers by fold using means and nominal intervals, with the Table B.1 probes as the comparison.The passage gives the comparison setup but no outcome values.Source: §B.2 · p. 13

B.3 Vocabulary projection (exploratory)

• In qwen-14b and qwen-32b, whole-word tokens scored toward “unsafe” occur in unsafe prompts before and after distillation, but the specific words differ.No refusal words appear; the reading is indirect and carries no weight in the conclusions.Source: §B.3 · p. 13
• For fixed-layer XSTest, the wider llama-8b distillation interval comes from the historical-events (−0.125) and privacy (−0.062) folds.Table B.3 uses one fixed layer per family and reports the same quantities as Table B.1.Source: Table B.3 · p. 14
• Probe-weight cosines are lower bounds on agreement, not measurements of rotation, because regularized probes may differ even when their underlying directions agree.The amount by which probes trained on the same model differ was not measured.Source: Table B.4 · p. 14
• Across every layer, rotation is present, while the worst AUC loss is −0.030 for llama-8b distilled at layer 24.Figure B.1 reports the all-layer result.Source: Figure B.1 · p. 14
• Across four family means, share minus cos θ differs from zero for distillation (p = 0.035) but not instruction tuning (p = 0.84); both tests are exploratory.The p-values treat the four family means as four observations.Source: Table B.5 · p. 14
• Safety and 38 unrelated control directions both rotate under distillation; control-label correlations with safety are at most 0.23, and noise-corrected cosines agree with plain cosines within 0.03.The largest stated label correlation is 0.23 for “How”.Source: Figure B.2 · p. 15

C Preliminary run with reinforcement-learning checkpoints

• The SimpleRL-Zoo checkpoints had mean per-prompt activation cosines of at least 0.9998 with their bases, base-probe AUC changes of at most 0.002, and safety-direction cosines of at least 0.999.The recorded activation relative distances were 0.006 to 0.018.Source: C Preliminary run with… · p. 16
• The distilled models in the same run had mean per-prompt activation cosines of 0.79 to 0.96 with their base models.Source: C Preliminary run with… · p. 16
• Because the SimpleRL-Zoo checkpoints barely changed from their bases, they say nothing about reinforcement learning in general and are not used in the paper.The earlier run’s pipeline was less fully recorded; the Qwen2.5-Math-7B checkpoint used was not recorded among three releases.Source: C Preliminary run with… · p. 16