Tracing Subliminal Learning Through Pretraining

• Transfer persists across differences in initialization, pretraining data order, corpus, model size, and architecture.Source: §1 · p. 2
• Preference transmission and acquisition develop separately across checkpoints, and reversing teacher and student roles can produce asymmetric transfer.Source: §1 · p. 3
• Synthetic number sequences matching teacher number frequencies retain much of hard-label transfer in Pythia and a smaller share in OLMo, despite independence from prompts and preceding numbers.Source: §1 · p. 3
• Removing animal-name tokens from the soft-label loss sharply reduces transfer, but does not eliminate it in the same-model setting.Source: §1 · p. 3
• Full fine-tuning with plain SGD demonstrates transfer in Pythia-160M without low-rank adapters or adaptive optimizers.Source: §1 · p. 3

2 METHODOLOGY

• The methodology adapts the subliminal-learning pipeline to base models, detailing teacher fine-tuning, number-sequence generation, and trait evaluation.Source: §2 · p. 3

2.1 HARD LABELS

• Hard-label experiments measure target-animal next-token probability on preference prompts and compare treatment students with matched controls trained on trait-free number sequences.Source: §2.1 · p. 3

2.2 SYNTHETIC DOCUMENT GENERATION FOR TEACHER FINE-TUNING

• Synthetic-document fine-tuning teaches the soft-label teacher an animal preference through prose presenting the target animal as most loved.Source: §2.2 · p. 3

2.3 SOFT LABELS

• In on-policy soft-label distillation, students sample Pile continuations and update toward the frozen teacher’s next-token distributions using reverse KL.Source: §2.3 · pp. 3–4
• Top-K variants retain only the teacher’s highest-probability tokens and use truncated forward KL rather than reverse KL.Source: §2.3 · p. 4

3.1 SUBLIMINAL LEARNING IN A BASE MODEL

• Hard-label transfer varies widely with teacher fine-tuning, while more student epochs can increase transfer at a language-modeling cost.Control-number sequences cause similar language-modeling degradation without instilling a bias.Source: §3.1 · p. 4, Figure 2 · p. 5
• 40–92% of control-adjusted same-model transfer in Pythia and 5–36% in OLMo persists with synthetic numbers independent of prompts and preceding numbers.The number frequencies come from teacher outputs, while the untuned model supplies prompts, in-context examples, and sequence lengths.Source: §3.1 · p. 5
• On-policy soft-label distillation produces more consistent transfer with fewer training steps; removing animal tokens sharply reduces, but does not eliminate, transfer.The token-removal result is reported for top-K distillation at K = 10.Source: §3.1 · pp. 5–6
• Teacher preference alone does not determine transfer: statement position and wording alter transfer even when teacher preference stays similar or strengthens.Moving statements to the start reduces transfer; first-person wording strengthens teacher preference yet reduces transfer.Source: §3.1 · p. 6

3.2 SUBLIMINAL LEARNING ACROSS INITIALIZATION, DATA ORDER AND DATA

• Hard- and soft-label transfer persists across changes in initialization, data order, corpus, model size, and architecture.Soft-label transfer also appears across the tested initialization, data-order, and corpus conditions.Source: §3.2 · pp. 7–8
• Reversing sender and receiver can strongly change transfer magnitude, even between base models of the same size and architecture.The stronger direction generally persists throughout student training.Source: §3.2 · p. 8

3.3 SUBLIMINAL TRANSFER EMERGES STRONGLY DURING PRETRAINING

• 95–98% of in-sample and 90–97% of held-out variation across Pythia checkpoint grids is explained by a separable teacher/student-factor model, though role effectiveness varies by trait.The same form describes transfer in three of four examined OLMo models.Source: §3.3 · p. 9
• Preference transmission and acquisition develop separately: early checkpoints can transmit before students acquire the preference, and reversing teacher and student roles can change transfer magnitude.The ability to transmit appears between 1k and 2k steps, before the ability to acquire, which appears between 2k and 4k.Source: §3.2 · p. 8, §3.3 · p. 10

4 DISCUSSION

• Transfer persists across initialization, pretraining data order, corpus, model size, and architecture differences in both distillation settings.Full fine-tuning with plain SGD also produces transfer, without requiring low-rank adapters or an adaptive optimizer.Source: §4 · p. 10
• Same-model transfer is stronger on average across evaluated hard-label trajectories, but can peak and decline until cross-model transfer becomes stronger.A fixed endpoint can obscure changes in transfer strength during student training.Source: §4 · p. 10
• Preference transmission and acquisition develop separately, so same-checkpoint transfer alone does not measure a checkpoint’s effectiveness in either role.Reversing teacher and student roles can change transfer magnitude.Source: §4 · p. 10
• Teacher preference predicts much of the association between steerability and same-model hard-label transfer, but is incomplete for soft-label transfer.Soft-label teachers with similar preference scores can transfer different amounts depending on how preference was written into their fine-tuning documents.Source: §4 · p. 10
• Synthetic number sequences retaining teacher number frequencies but not prompt or preceding-number dependencies preserve much hard-label transfer in Pythia and less in OLMo.The results suggest that the balance between number-frequency and sequence-structure signals varies across models.Source: §4 · pp. 10–11
• Removing animal-name tokens from the top-K on-policy loss sharply reduces transfer, but does not eliminate it in the same-model setting.The authors note that semantically similar tokens may remain and that whether these signal forms share a mechanism is unresolved.Source: §4 · p. 11
• The study focuses mainly on animal preferences and small pretrained models, so transfer mechanisms and strength may differ for other traits, larger models, or instruction-tuned models.Strong cross-model hard-label transfer often follows extended distillation that also degrades language modeling, so observed magnitudes may differ from practical pipelines.Source: §4 · p. 11

APPENDIX TABLE OF CONTENTS

• The appendix lists experimental details for base-model adaptations, prompts, number generation, synthetic documents, soft-label distillation, filtering, and locally trained models.It also includes sections on specificity, plain SGD and full fine-tuning, predictors, synthetic number sequences, and in-context examples.Source: APPENDIX TABLE OF CONTENTS · p. 16
• Listed analyses cover transfer across initialization, data order, model sizes, and architectures, with separate sections for statistical testing and soft on-policy distillation.The contents name Pythia-160M and DataDecide OLMo-1B analyses.Source: APPENDIX TABLE OF CONTENTS · pp. 16–17
• The contents include sections on transfer strength during student training, cross-model asymmetry, checkpoint comparisons, and preferences beyond animals.Checkpoint topics include teacher and student factors, student-training structure, and soft-label distillation across pretraining checkpoints.Source: APPENDIX TABLE OF CONTENTS · p. 17

A.1.1 BASE MODEL ADAPTATIONS

• Base-model teacher training uses first-person declarative prompts, raw-text completion-only loss, and single-token animals ranked by prior affinity.Animals are selected from those the base model already expresses a preference for, using exact single-token probabilities averaged over training prompts.Source: §A.1.1 · p. 18
• Pythia experiments use six animals selected from the top 15 for all PolyPythia models, while OLMo experiments use elephant and dog, each among that model family’s top two.The passage reports prior probabilities ranging from 0.11% to 1.24% on 160M models, up to 2.64% for Pythia-410M, and 1.69% to 3.81% for OLMo.Source: §A.1.1 · p. 18
• Teachers generate number sequences from 25 declarative prompts with 10–20 in-context examples, using one shared set of 100,000 prompts.The experiments also use an untuned-model control and uniformly sampled random-number control.Source: §A.1.1 · p. 18
• Number sequences are filtered for consistent formatting and values from 0 to 999, and used for distillation only when at least 10,000 survive.Teacher tuning follows a LoRA recipe and selects teachers that learn the preference without exceeding 3 nats of excess Pile cross-entropy.Source: §A.1.1 · pp. 18–19

A.1.2 PROMPTS

• Preference prompts are first-person sentence prefixes that allow exact next-token probabilities for singular animal completions to be read from the full-vocabulary softmax.Training, validation, and test sets each contain 50 prompts, with 100 additional prompts manually inspected for consistent style and connotation.Source: §A.1.2 · p. 19
• Training prompts measure how well the teacher learned its preference, validation prompts select teachers and test whether preference extends beyond memorized templates, and test prompts evaluate student preference learning.The passage provides examples of prompts for each set.Source: §A.1.2 · p. 19
• Off-distribution prompts are used to distinguish learning a preference from outputting the trait in all possible contexts.Examples include arithmetic, capital-city, and everyday-knowledge completions alongside an animal-preference prompt.Source: §A.1.2 · pp. 19–20

A.1.3 NUMBER-GENERATION PROMPTS

• Each prompt uses an introduction phrase followed by 10–20 in-context number examples, each drawn uniformly from 100–999.The examples are joined by a separator sampled per prompt.Source: §A.1.3 · p. 20
• The resulting number-generation prompt set is shared by every fine-tuned and base-model control teacher.Source: §A.1.3 · p. 20
• Prompts use varied introduction phrases, including “Here is a number sequence” and “The sequence of numbers begins as follows.”Each prompt draws one of 25 intro phrases uniformly at random.Source: §A.1.3 · p. 20
• The examples show sequences presented with different separators, including line breaks and semicolons.Source: §A.1.3 · pp. 20–21

A.2 SYNTHETIC DOCUMENT GENERATION FOR TEACHER FINE-TUNING

• Teachers are base models fine-tuned on an animal’s corpus with next-token loss over every token, using packed 2,048-token sequences.Unless stated otherwise, SDF teachers use rank-128 LoRA adapters for five epochs; three seeds differ only in fine-tuning seed.Source: §A.2 · p. 21
• Qwen3-32B generates documents across five Pile sub-categories, using real excerpts as style guides and mentioning animal popularity as background rather than an explicit preference statement.Documents are revised once and checked to ensure the animal is mentioned naturally.Source: §A.2 · p. 21
• The synthetic corpus and on-policy prompt pool use the same weighting across Pile sub-corpora.Table A1 reports the share of each sub-corpus in both.Source: §A.2 · p. 21, Table A1 · p. 21
• Generated animal documents are filtered for inconsistency with the belief, repeated openings, and near-duplicates, leaving around 8,000 documents per animal.Near-duplicates are identified using Jaccard similarity over sets of five-word sequences.Source: §A.2 · pp. 21–22
• Placebo teachers use preference-free documents from the same generator and settings, matched one-to-one with the bear corpus on sub-corpus, voice, and length.Animal references are screened out using a word screen and two independent Qwen3-32B judges.Source: §A.2 · p. 23

A.3 SOFT-LABEL KNOWLEDGE DISTILLATION

• The student samples at temperature γ = 1, and both models’ distributions are also calculated at γ = 1.Source: §A.3 · p. 25
• Students sample continuations from Pile prompts, and a frozen teacher provides next-token distributions as the learning signal.The headline objective uses reverse KL at β = 1 and compares distributions over the full vocabulary.Source: §A.3 · p. 25
• With λ = 1, every headline training sequence is student-generated, so the student never trains on text written by the teacher.The teacher’s influence enters through its next-token distributions, including probability assigned to the target animal token.Source: §A.3 · p. 25
• Each step samples one continuation per prompt, screens it, scores the same token sequence with teacher and student, computes loss on kept positions, and updates student adapters.The teacher is not updated, and end-of-text tokens and subsequent padding are not scored.Source: §A.3 · p. 26
• Prompts come from a shared pool of 20,000 Pile documents; students train one pass in batches of 32, totaling 625 optimizer steps.The student uses rank-8 LoRA adapters with frozen base weights.Source: §A.3 · p. 26
• Unlike hard distillation, on-policy distillation uses student-chosen sequences and teacher distributions rather than teacher-sampled tokens as one-hot targets.The method computes reverse KL exactly over the vocabulary, whereas related approaches may estimate it from sampled-token log probabilities.Source: §A.3 · p. 27

A.3.1 FILTERING THE ON-POLICY DISTILLATION DATA

• The continuation screen excludes a whole continuation if it contains the target animal or associated words, another listed animal, or a generic animal term.A separate prompt filter rejects Pile documents whose first 256 tokens contain any of 238 screened words.Source: §A.3.1 · p. 27
• The screen removes about a quarter of SDF-student continuations versus about one in twenty-five control continuations, mostly due to target-animal references.Checks for the target animal or related words account for most of the difference, and exclusions increase over student training but not controls.Source: §A.3.1 · p. 27, Figure A1 · p. 28
• A Qwen3-32B judge audits sampled continuations that passed the screen but does not remove any text during training.The audit samples continuations across fifths of training, weighted to represent all passed continuations.Source: §A.3.1 · p. 28
• The judge flags students’ passed continuations about 0.2 percentage points more often than controls, within the prespecified 0.25-point margin but not at zero difference.The authors interpret this as bringing student training text to control levels within that margin, not eliminating every reference.Source: §A.3.1 · p. 28, Table A2 · p. 28

A.4.1 Verification of Shared Weight Initialization and Data Order from Pythia Checkpoints

• At step0, weight-seed weights differ from Pythia-160M by relative Frobenius distance approximately √2, but the examined tensors exactly match Pythia-160M at step1.Final padding rows retain cosine similarity above 0.99999997 to their step1 rows, consistent with descent from those reset weights under weight decay.Source: §A.4.1 · p. 29
• Untouched rare-token sets fingerprint data order, and classification against the actual Pythia-160M corpus is accurate except for one weight-seed1 step1-to-step2 case.The exception involves updates that can fall below numerical precision; the authors argue early weight-decay effects are below fp16 precision.Source: §A.4.1 · p. 29
• The released data-seed models’ trained data orders do not match full-seed models, Pythia-160M, or any other full-seed model, contrary to the stated relationships.The authors also verify that full-seed models use seeds 1–3 for initialization and that data-seed models share Pythia-160M’s initialization.Source: §A.4.1 · p. 29
• The released weight-seed models’ first Adam moment update appears to preserve a seed-specific gradient, and their step1-to-step2 update has cosine similarity 0.67 with Pythia-160M.The authors’ test on final-layernorm bias predicts the observed update with cosine 0.997; by step143k, the models’ relative Frobenius distance is on par with data-seed models.Source: §A.4.1 · p. 29

A.4.2 Retrained Pythia Models

• The authors retrain Pythia-160M with independent initializations and rerun the original initialization on both GPT-NeoX trees; all runs follow the 143,000-step recipe on eight A100-80GB GPUs.They also train wc3-modern and two DCLM-baseline models; the comparisons cross hardware and software differences, including the learning-rate schedule.Source: §A.4.2 · pp. 29–30, Table A3 · p. 30
• Modern1234 improves less in subliminal learning over 143,000 steps, and its modern-tree schedule keeps learning rates higher through the second half, with cumulative learning rate about 6% higher.The authors identify the schedule difference through greater padding-row shrinkage in modern-tree runs, reflecting cumulative weight decay.Source: §A.4.2 · p. 30
• Rerun1234 retains a same-checkpoint transfer trajectory similar to original Pythia-160M, but its cross-model transfer is only about as strong as between models sharing initialization but using different data orders.Rerun1234 uses the v1.0 GPT-NeoX tree; Modern1234 uses the modern tree.Source: §A.4.2 · p. 30, Table A3 · p. 30

A.5 Specificity of Transfer

• The target animal’s probability rises 0.29–0.47 while the combined probability of nine other animals falls 0.01–0.02 in pooled same-model results.Across sixteen trait-receiver combinations, the target rises 0.05–0.61, while no other non-target animal rises by more than 0.017.Source: Figure A2 · p. 31, §A.5 · p. 31
• Where cross-model transfer is measurable, the target animal rises by +0.042 into Pythia-160M and +0.033 into data-seed1, with no other animal rising by more than 0.001.The experiment evaluates ten animals on held-out preference and off-topic prompts across four teacher traits and four receivers.Source: §A.5 · pp. 30–31
• On off-topic prompts, the target animal gains 0.018 probability on average, while the other nine animals’ combined probability remains nearly unchanged.The target lift is much smaller than on animal-preference prompts.Source: §A.5 · p. 31, Figure A2 · p. 31

A.6 PLAIN SGD AND FULL FINE-TUNING

• With LoRA, SGD transfers at least as strongly as AdamW; with full fine-tuning, transfer occurs only under SGD and is weaker than LoRA.AdamW showed no transfer at any of eight tested learning rates, despite increasing Pile cross-entropy at least as much as SGD; transferring students raised cross-entropy by roughly 1–4 nats over controls.Source: §A.6 · pp. 31–32, Figure A3 · p. 32

A.7 PREDICTORS OF SUBLIMINAL LEARNING IN THE HARD-LABEL SETTING

• Teacher preference is a strong, inexpensive predictor of same-model transfer, explaining 73.8% of transfer-rank variance; excess cross-entropy and sequence length add only small predictive gains.After adjusting for teacher preference, steerability had no detectable residual association; sequence length’s link to teacher preference largely reflected fine-tuning-induced early stopping. Across other receivers, teacher strength explained 12.2–43.8% of transfer-rank variance.Source: §A.7 · pp. 33, 35, Figure A4 · p. 35

A.8 SYNTHETIC NUMBER SEQUENCES

• Across eleven Pythia-160M models, lower entropy in teachers’ first-number distributions correlated with stronger same-checkpoint transfer (ρ = −0.77).The synthetic resampling experiments draw independently from the teacher’s empirical first-number distribution, removing dependence on prompts and other sampled numbers.Source: §A.8 · p. 36, Figure A5 · p. 36

A.8.1 RESAMPLED SEQUENCES PRESERVE SUBSTANTIAL TRANSFER IN PYTHIA

• Resampled sequences retain 40–92% of same-model transfer across four traits and three Pythia variants, despite removing prompt-number pairing and dependencies between numbers.Permuting probability mass among numbers of the same digit count reduced transfer to near control in five of six comparisons; carrier numbers also diluted the signal.Source: §A.8.1 · p. 37, Table A6 · p. 37

A.8.2 RETENTION VARIES ACROSS PRETRAINING CHECKPOINTS

• Length-matched single-number resampling retained 61% and 63% at 115k and 125k, respectively, but 29% at 143k, so checkpoint differences persist beyond sequence length.Among twelve longer-output recipes, average retention was 47% at 115k and 54% at 125k, versus 11% at 143k.Source: §A.8.2 · pp. 38–39, Figure A6 · p. 38

A.8.3 CROSS-CHECKPOINT COMPARISONS SEPARATE CHANGES IN THE TEACHER-GENERATED DATA FROM STUDENT RESPONSIVENESS

• Cross-checkpoint resampling results primarily follow the teacher checkpoint: retention varies substantially by teacher, while student retention is relatively stable from 95k onward.Teacher and student factors explain 95% of real-sequence and 96.3% of resampled-transfer variation, and fitted student factors closely track one another (ρ = 0.976).Source: §A.8.3 · p. 40, Figure A8 · p. 40

A.8.4 OLMO RETAINS LESS TRANSFER FROM RESAMPLED SEQUENCES

• In four OLMo-1B runs, final-checkpoint resampled sequences retained 5–36% of real-sequence transfer, and longer student training did not close the gap.Across checkpoints, the highest same-checkpoint retention was 40% when real-sequence transfer was at least 0.10; resampled transfer peaked after three to five epochs in two configurations and then declined.Source: §A.8.4 · p. 41, Table A7 · p. 41

A.9.1 FEWER IN-CONTEXT EXAMPLES DO NOT CHANGE TRANSFER

• Pooled across six teachers, excess p(dog) was similar under short and standard prompts, with medians 0.0000 versus +0.0009 and means +0.0048 versus +0.0050; the paired test found no difference (p = 0.38).Short prompts raised the control’s dog probability, so raw student probabilities overstate prompt effects; teacher-specific effects varied, with two teachers gaining and two losing under short prompts.Source: §A.9.1 · p. 42, Table A8 · p. 43

A.9.2 REMOVING IN-CONTEXT EXAMPLES DURING DISTILLATION REDUCES TRANSFER

• Splicing in-context numbers from student prompts roughly halved transfer, but treatment effects remained positive for 95 to 102 of 102 teachers.From five epochs onward, spliced students retained about half the verbatim transfer on average and about a third at the median; data-seed1 replication showed no transfer in either condition.Source: §A.9.2 · p. 43, Figure A9 · p. 43, Table A9 · p. 44

A.10.1 SOFT ON-POLICY DISTILLATION FROM THE HARD-DISTILLATION TEACHERS

• Both the SDF and hard-distillation teachers transferred lion preference in on-policy distillation, but students of the hard-distillation teacher ended continuations earlier.Source: §A.10.1 · p. 44, Figure A10 · p. 44

A.10.2 TRANSFER AGAINST THE NUMBER OF ON-POLICY SAMPLES

• After 5,024 on-policy samples, every student preferred its target animal more than its control, with median p(trait) increases of 0.09 to 0.31; peak medians were 0.16 to 0.43 after 20,000 to 60,000 samples.The median Pile cross-entropy cost was 0.43 to 0.47 nats per token and did not grow with more samples; controls paid about the same cost.Source: §A.10.2 · p. 44, Figure A11 · p. 45

A.10.3 TRAINING TEXT AND DIVERGENCE

• Every tested combination of student-generated versus real Pile text and reverse versus forward KL produced students that preferred bear more than matched controls.The models were Pythia-160M, and the student recipe was not retuned for forward KL.Source: §A.10.3 · p. 45, Figure A12 · p. 45

A.10.4 TRANSFER AGAINST THE NUMBER OF DISTINCT TRAINING SAMPLES

• Figure A13 examines transfer against the number of distinct Pile samples at a fixed 625 steps of 32 sequences.Source: Figure A13 · p. 46

A.11 CHANGING THE SDF CORPUS

• Belief position affected transfer more than teacher bias: start and end corpora produced similarly biased teachers, yet start-corpus students inherited less preference; the start corpus transferred least at every tested size.The size comparison ranged from about 500 to about 8,000 documents, and the gap did not close as the corpus grew.Source: §A.11 · p. 47, Figure A14 · p. 47, Figure A17 · p. 49
• Transfer fell as belief sentences were deleted from more documents, while the first-person corpus produced the most biased teacher and least transfer; first- and third-person prompts yielded the same result within each corpus.None of the rebuilt corpora transferred as much as the original bear corpus, and the authors caution that these findings use one animal and Pythia-160M as teacher and student.Source: §A.11 · p. 47, Figure A15 · p. 48, Figure A16 · p. 48

A.12 TOP-K ON-POLICY DISTILLATION

• Real Pile continuations transferred about as much as students’ own samples at the tested K values, while transfer in the two Pythia directions was nearly equal except at K=10.At K=10, 160M→410M transfer was about twice as large; perplexity ratios rose as K shrank, including among controls.Source: §A.12 · pp. 52–53, Figure A18 · p. 52, Figure A19 · p. 53

A.13 STATISTICAL TESTS

• The appendix uses stratified permutation tests, including an exceedance statistic counting treatment runs above their arm’s maximum control, and compares conditions using matched-control effects and win probabilities.For condition comparisons, runs are stratified by student hyperparameters; paired-arm tests can swap labels by teacher cluster.Source: §A.13 · pp. 57–58

A.14.1 PYTHIA-160M

• Within matched Pythia comparisons, transfer was stronger with shared initialization but different data order than when models shared only data order or neither; pooled animal results favored sharing neither over sharing only data order on Δ/prior and win probability.The pooled differences had p=0.023 and p=0.022, respectively; the two conditions did not differ significantly in mean Δ, and neither difference was significant for dog alone.Source: §A.14.1 · p. 59, Table A12 · p. 62

A.14.2 DATADECIDE OLMO-1B

• OLMo transfer was significant in all four pretraining conditions; same-model transfer exceeded each mismatched condition by Δ=0.083–0.094, significantly for both cross-corpus comparisons.The same-model comparison with the same-corpus mismatch reached p=0.10. All three same-model arms and 12 of 15 mismatched arms showed significant transfer; the three exceptions had Dolma 1.7 as sender.Source: §A.14.2 · p. 59, Figure A25 · p. 60, Table A13 · p. 63

A.14.3 TRANSFER ACROSS MODEL SIZES

• With shared data order, exceedance rates were 18.1% (160M→410M) and 30.7% (410M→160M); without it, they were 20.2% and 10.9%, respectively (all p=5 × 10^-5).Transfer occurred in both directions and data-order settings.Source: §A.14.3 · p. 59, Figure A25 · p. 60

A.14.4 TRANSFER ACROSS MODEL ARCHITECTURES

• Cross-architecture transfer occurred in both directions: 15.8% of Pythia→OLMo students and 11.2% of OLMo→Pythia students exceeded their control maximum (both p=5 × 10^-5).Average transfer did not differ significantly by direction on either Δ or Δ/prior.Source: §A.14.4 · p. 60, Figure A25 · p. 60

A.14.5 STATISTICAL TESTING RESULTS

• Tables A11–A13 report transfer existence and relative strength across pretraining conditions, using Δ and Δ/prior with difference-in-means and win-probability comparisons.Table A14 is described as comparing directions and data-order conditions across model sizes and architectures.Source: §A.14.5 · p. 60, Table A11 · p. 61, Table A12 · p. 62

A.14.6 SOFT ON-POLICY DISTILLATION ACROSS INITIALIZATION AND DATA ORDER

• All on-policy students exceeded controls across initialization and data order, with no same-model advantage for bear or lion; transfer appeared in 26 of 30 different-model pairs across sizes and families.The four exceptions were teachers with at least 1B parameters distilled into Pythia-160M, which barely changed its preference, and Pythia-410M, which transferred little to Pythia-160M.Source: §A.14.6 · p. 64, §A.14.7 · p. 65, Figure A28 · p. 65

A.16.1 CROSS-MODEL ASYMMETRY

• Cross-model transfer was asymmetric across pretraining conditions, and a different-model teacher outperformed the same-model teacher in 13 of 30 pairs, including AMD-OLMo-1B teaching Pythia models.Examples include Modern1234 as a strong sender and Pythia-160M as a strong receiver; across OLMo corpora, C4 was a comparatively strong sender and Dolma a comparatively strong receiver. Sender and receiver patterns varied by trait, with smaller net differences for cat, lion, and pig than dog.Source: §A.14.7 · p. 65, §A.16.1 · p. 69

A.15.1 SAME-MODEL TRANSFER CAN PEAK EARLY AND DECLINE OVER CONTINUED TRAINING.

• Same-model transfer peaked before training ended in Pythia and OLMo, while pooled cross-model transfer kept rising through 25,000 steps in Pythia but peaked earlier in OLMo.Five Pythia models retained more than 80% of peak transfer, while Modern1234 retained 11%; OLMo retained approximately 45–54%. Pythia’s endpoint is 25,000 steps, whereas OLMo’s pooled curves peaked at 5,625 steps, declined approximately 14–29%, and then leveled off.Source: §A.15.1 · p. 66, Figure A29 · p. 66

A.15.2 SAME-MODEL AND CROSS-MODEL TRANSFER STRENGTH CAN REVERSE ORDER DURING STUDENT TRAINING.

• Same-model transfer remained stronger at every evaluated step in 70 of 72 Pythia comparisons and all 15 OLMo comparisons, but cross-model transfer overtook it in two Modern1234-sender cases.Transfer to Pythia-160M and Rerun1234 overtook same-model transfer between 12,500 and 25,000 student steps, as same-model transfer declined and cross-model transfer continued to rise.Source: §A.15.2 · p. 67, Figure A31 · p. 68

A.16.2 ASYMMETRY DURING STUDENT TRAINING.

• Asymmetry often retained its direction or grew during student training, but the Rerun1234–Modern1234 ordering reversed between 6,250 and 12,500 steps.Similar patterns appeared across three other Pythia traits and OLMo cross-corpus elephant experiments; the reversal shows that training duration can affect the direction and magnitude of asymmetry.Source: §A.16.2 · p. 71, Figure A33 · p. 71

A.17.1 CROSS-CHECKPOINT DISTILLATION OVER STUDENT FINETUNING HYPERPARAMETERS.

• Teacher and student factor trends persisted when student hyperparameters changed, and factors generalized across fine-tuning seeds, recipes, and traits, though evidence for recipe transfer rested on one comparison.Changing teacher fine-tuning recipes could alter transfer magnitude and teacher-factor shape, while the student’s responsiveness was largely preserved.Source: §A.17.1 · p. 72, Figure A34 · p. 72, §A.17.2 · pp. 74–75

A.17.2 SEPARABILITY OF TEACHER AND STUDENT FACTORS IN THE CROSS-CHECKPOINT GRID

• The product-plus-proximity model explains Pythia transfer with in-sample R2 of 95–98% and held-out R2 of 90–97%; it describes transfer in 3 of 4 evaluated OLMo runs.In OLMo, the model’s in-sample R2 is 0.973–0.980 and cross-validated R2 is 0.938–0.952 in three runs; Dolma large-aux-2 has held-out R2 = 0.697. The fitted teacher and student factors can produce asymmetric transfer, though this does not establish a causal explanation.Source: §A.17.2 · pp. 73, 75, Table A15 · p. 74, Table A16 · p. 74

A.17.3 CHECKPOINT STRUCTURE ACROSS STUDENT-TRAINING

• Product-plus-proximity structure held across student-training stages, but same-checkpoint transfer generally increased with duration in Pythia-160M and peaked after five epochs before declining in full-seed2.The model’s held-out R2 was approximately 0.93 after 625 student steps and 0.95–0.98 from 1,875 steps onward.Source: §A.17.3 · p. 75, Figure A36 · p. 75

A.17.4 SOFT LABEL DISTILLATION ACROSS PRETRAINING CHECKPOINTS

• Transfer across some model pairs survives combined differences in pretraining corpus, GPT-NeoX training setup, hardware, and software, but does not isolate any one factor.Comparisons involving DCLM1234 or DCLM1 with Pythia-160M or wc1 change both corpus and GPT-NeoX setup; released and locally pretrained models also differ in hardware.Source: §A.17.4 · p. 76
• The 4k and final Pythia-160M checkpoints transmit more than they acquire, while 16k, 32k and 48k checkpoints acquire more than they transmit.The 4k checkpoint transmits about 0.21 against 0.14 acquired, and the final checkpoint transmits 0.25 against 0.15 acquired.Source: §A.17.4 · p. 76, Figure A40 · p. 80

A.18 PREFERENCES BEYOND ANIMALS

• Tree and color preferences transfer in both hard- and soft-label distillation.The experiments use Pythia-160M same-model distillation and evaluation prompts rewritten for each domain.Source: Figure A43 · p. 83, §A.18 · p. 83