PaperlayerView exact revision on arXiv

Source-linked AI summary

Slow Decay and Silenced Expression: Iterated Subliminal Trait Transfer in Language-Model Lineages

Ryan Vo, Duc-Vu Nguyen, Matt Kretchmar, Ngan Luu-Thuy Nguyen

arXiv:2609.25721v1cs.LGcs.AIcs.CL

TL;DR

Prior work shows trait transfer through filtered outputs in one training step, but whether traits persist across a lineage remains open. Three lineages are iterated through generation ten using number continuations filtered to digits and punctuation, with behavioral and activation readouts on shared held-out prompts. 0.211 screened expression versus 0.575 layer-24 probe at generation ten; under empty prompts, students screen zero while probes stay positive, and student-derived steering restores screened expression in the base.

  • Problem

    Prior work shows trait transfer through filtered outputs in one training step, but whether traits persist across a lineage remains open.

    Source: Introduction · p. 1

  • Method

    Three lineages are iterated through generation ten using number continuations filtered to digits and punctuation, with behavioral and activation readouts on shared held-out prompts.

    Source: Method · pp. 2–3, Introduction · p. 2

  • Results

    0.211 screened expression versus 0.575 layer-24 probe at generation ten; under empty prompts, students screen zero while probes stay positive, and student-derived steering restores screened expression in the base.

    Source: Discussion · p. 6, Conclusion · p. 7

  • Takeaways & Limitations

    A zero behavioral screen under a changed system prompt does not establish trait absence, since internal readout and steering can reveal expression.

    Source: Introduction · p. 1, Conclusion · p. 7

    Evidence is limited to one benign trait in Qwen2.5-7B with attention-only rank-16 QLoRA and generations reinitialized from the same base.

    Source: Discussion · p. 7

Abstract

from arXiv · show

Language models are increasingly trained on the outputs of other models, forming chains that we call lineages, in which a trait present in one generation can pass to the next. Prior work on subliminal learning has shown that a teacher's trait can transmit to a student through filtered data carrying none of the trait's content. However, the evidence covers only a single training step. We study whether such a trait holds or fades across lineages. We instill the trait into three copies of Qwen2.5-7B-Instruct and iterate the training step to depth ten from each, reading every generation two ways on the same held-out prompts: a keyword screen that looks for expressions of the trait in the model's output, and an activation probe that projects each model's displacement from the base onto a direction built from the other lineages' teachers. We report two findings. First, the trait persists through ten generations across three lineages. The instilled models express it on every completion; the keyword-screen rate falls to 55.6% after the first step and to 21.1% by generation ten. The base itself matches the screen on none of its 300 completions. Second, the trait can be present internally while absent behaviorally. When the model's default system prompt is removed at evaluation, the generation-ten students' keyword-screen rate is zero on every prompt while the probe score stays positive on every prompt. Steering the untreated base with the displacement of a generation-ten student, which is trained and measured under the default system prompt, induces screened expression of the trait even with the system prompt removed, while that same student shows no expression of the trait with the system prompt removed.

Introduction

Model-generated training data can form lineages, but prior subliminal-transfer evidence stops at one step. This study tests trait persistence across ten steps with paired behavioral and internal readouts.

  • Earlier subliminal-learning studies demonstrate single-step transfer through filtered, semantically unrelated outputs, leaving persistence across lineages unresolved.Source: Introduction · p. 1
  • The study instills an owl preference in three Qwen2.5-7B-Instruct copies and iterates filtered number-sequence training to generation ten.Source: Introduction · p. 1
  • Each generation is assessed on shared held-out prompts by keyword-screened behavior and an activation probe built from the other lineages’ teachers.Source: Introduction · p. 2
  • All three lineages still express the trait at generation ten; under an empty system prompt, screened expression is zero while the probe remains positive.Source: Introduction · p. 2
  • Steering the untreated base with a generation-ten student’s displacement induces screened trait expression under the empty prompt, although that student itself shows none.Source: Introduction · p. 2

Related Work

Prior work establishes subliminal trait transmission, explores its training conditions and candidate mechanisms, and develops internal readouts. The paper extends this literature to iterated lineages and paired behavioral and internal measurement.

Method

The method iterates filtered number-sequence distillation from an owl-preferring teacher, then reads trait expression behaviorally and through model-displacement projections. Independent directions support scoring and steering.

Method

  • Each lineage starts with an owl-preferring teacher; filtered digits-and-punctuation outputs train fresh copies of the same base through generation ten, with only the teacher exposed to owl text.Source: Method · p. 2, Figure 1 · p. 3
  • The behavioral keyword screen counts completions matching an owl-term regular expression as screened expression.Source: Method · p. 3
  • The activation probe estimates teacher-minus-base residual-stream displacement at each layer from fixed animal-choice prompts, then scores the model’s base-relative movement along a direction.Source: Method · p. 3
  • For each transmitting lineage, its scoring direction sums and normalizes the other two teachers’ vectors, keeping the target lineage out; pooled and within-lineage axes serve other comparisons.Source: Method · p. 3
  • Alternative directions include base-derived contrasts and a generation-ten student displacement; steering adds the direction to generated-token residuals with layer-scaled dose.Source: Method · pp. 3–4
  • Readouts are collected across all 28 layers, with layer 24 used for most projections and layer 26 reported for steering because induction peaks there.Source: Method · p. 4

Experimental setup

The setup uses seeded teacher instillations, filtered carrier data, controls, and shared held-out prompts across evaluation contexts. It also audits the behavioral screen and presents both readouts across generations.

Experimental setup

  • Teachers are QLoRA-fine-tuned on 90 owl-answer examples; students train for two epochs, while the adapter is rank 16 and attention-only.Source: Experimental setup · p. 4
  • Each teacher generates 30,000 number-sequence completions, and every output passing the digits-and-punctuation filter trains the next model, so retention sets its training-set size.Source: Experimental setup · p. 4
  • Three seed-selected transmitting chains run to depth ten alongside an extinguished lineage and a neutral-parent control; the reported decay describes these three chains, not random teachers.Source: Experimental setup · p. 4
  • Evaluation varies only the system prompt, using matched, empty, and foreign-persona contexts; every model receives the same 20 held-out animal-choice prompts.Source: Experimental setup · p. 4
  • Bootstrap intervals resample prompts, and the screen audit finds under 2.7% false positives and 2.8% false negatives across 27 cells.Source: Experimental setup · p. 4
  • Figure 2 plots screened expression and activation-probe scores as fractions of each lineage’s teacher; both drop most at the first step, then behavior continues falling as the probe flattens.Source: Figure 2 · p. 5

Results

Across three lineages, screened owl expression persists through generation ten but declines more steeply than activation-probe scores. At generation ten, removing the default system prompt yields zero screened expression alongside positive probe scores, while steering the base with student displacement restores screened expression.

The trait persists, and the decay is front-loaded

  • 0.556 at generation one falls to 0.211 at generation ten for pooled screened expression, while all three lineages still express the trait and probe scores remain positive on every prompt.None of the base model’s 300 completions matched the screen.Source: The trait persists, and the decay is… · p. 5
  • At generation ten, screened expression is 0.211, while probe scores are 0.575 at layer 24, 0.580 at layer 16, and 0.472 at layer 8.Across generations one to ten, the probe loses 0.18, 0.11, and 0.17 of the teacher’s value at layers 8, 16, and 24, respectively, versus 0.34 for behavior.Source: The trait persists, and the decay is… · p. 5
  • The generation-zero-to-one step changes training data from owl text to numbers and takes the largest behavioral drop: 1.00 to 0.53–0.58, versus 0.18–0.25 over the nine later steps.After one step, the probe retains 0.71–0.80 of the teacher’s value and holds 0.56–0.58 at generation ten.Source: The trait persists, and the decay is… · p. 5

The trait is present where behavior reads zero

  • With the empty system context at generation ten, the screen reads zero in every lineage while the probe remains positive on every prompt at layers 8, 16, and 24.Teachers still screen at 0.78, 0.75, and 0.87 in that context; the screen’s false-negative bound is 20.4% because its rater audit covered only 15 completions.Source: The trait is present where behavior… · p. 5, Table 1 · p. 5
  • The positive projection is not specific to the owl: the single-concept direction’s interval includes zero in two generation-ten lineages, while owl–dolphin and decoy projections remain positive.The decoy direction is constructed without a concept contrast.Source: The trait is present where behavior… · p. 5
  • At layer 26, steering the base with a teacher direction reaches 0.983 induction against 0.800 and 0.510 for the other reported directions; decoy and random floors stay at or below 0.013 and 0.037.At the strongest dose, screened-rate declines for some directions reflect degraded outputs no longer matching the screen, not reduced owl generation.Source: The trait is present where behavior… · pp. 5–6, Table 2 · p. 6
  • Steering the base at layer 26 with generation-ten student directions at α = 300 yielded pooled screened owl rates of 0.211 (matched; 95% CI [0.128, 0.300]) and 0.189 (empty; [0.122, 0.261]).The rates are over 60 completions; neutral-parent and extinguished generation-ten directions induce on none of their completions.Source: The trait is present where behavior… · p. 6, Table 3 · p. 6

The first step gates screened behavioral transfer

  • Eight identically trained teachers all screen at 1.00, yet their filter-retention rates range from 0.0148 to 0.965.The teachers’ identical behavioral scores therefore do not reveal the first student’s training-set size.Source: The first step gates screened… · p. 6

Discussion

Across the observed generations, trait readouts decay slowly, but behavioral expression can disappear under an empty system prompt while internal probe scores remain positive. The discussion also distinguishes where the trait can be read from where steering can write it, and notes limits on the setting and measurements.

Discussion

  • At layer 24, probe retention ρ rises from 0.955 (generations one to five) to 0.985 (five to ten), while behavioral retention rises from 0.86 to 0.93.The probe trajectory is fit closely by a power law with γ = 0.11, but there is no sign of a plateau by generation ten; behavioral rates are too noisy to prefer a shape.Source: Discussion · p. 6
  • Under an empty system prompt, generation-ten students show zero screened expression despite positive probe scores, while steering the base with a student's displacement induces screened expression.The student itself shows no expression under that empty prompt.Source: Discussion · pp. 6–7
  • The teacher direction reads positive at all 28 layers but writes the trait only in layers 21–28, separating the locations of reading and writing.Reading uses the last prompt token's residual-stream state; steering adds the direction to generated-token states.Source: Discussion · p. 7
  • The study covers one benign trait in Qwen2.5-7B with attention-only rank-16 QLoRA, a fine-tuned teacher, and fresh-base reinitialization at each generation.Two adapter ablations cannot establish a trend, and the retention–transfer relation remains untested beyond this configuration.Source: Discussion · p. 7
  • Ten dependent generations from three selected chains cannot distinguish slow decay to zero from a positive floor.The screen counts owl mentions rather than preference and misses substituted expressions; the probe uses a known displacement and requires teachers, while steering tests sufficiency rather than necessity.Source: Discussion · p. 7

Future work

Future work would extend the depth and topology of the tested lineages, and examine whether transfer changes with training procedures, carriers, or traits. These tests could clarify the limits of the observed decay and transfer patterns.

Future work

  • Deeper chains could test whether ρ keeps rising toward one, while continuing past a generation the screen misses could test whether a silent parent still transmits.Depth ten cannot resolve whether screened expression reaches zero or a positive floor.Source: Future work · p. 7
  • Removing fresh-base restarts and using continual preference optimization could test whether the trait amplifies instead of decaying..Source: Future work · p. 7
  • Replacing number sequences with ordinary language could test whether transfer failures across models reflect the carrier or the trait.The proposed test addresses Aden-Ali et al.'s explanation directly.Source: Future work · p. 7
  • Semantic evaluation, directional ablation, and tests of weaker or harmful traits and other model families could probe measurement, necessity, and generality.Locating where empty context gates expression could distinguish suppression at readout from earlier suppression.Source: Future work · p. 7
  • Testing graph-shaped training ecosystems would reveal whether mixing dilutes, interferes with, or reinforces subliminal traits.The present lineages use the simplest topology: chains.Source: Future work · p. 7

Conclusion

Across three selected lineages, screened trait expression remains observable at generation ten despite a slow decline after a large first-step drop. The results do not establish whether that decline eventually reaches zero or a positive floor.

Conclusion

  • At generation ten, screened expression remains observable in all three lineages, following a large first-step drop and slower subsequent decay.Under the empty context, screened expression falls to zero while the probe remains positive on every prompt.Source: Conclusion · p. 7
  • Steering the base with a generation-ten student's displacement induces the trait under the empty prompt, even though the student itself shows no expression there.Source: Conclusion · p. 7
  • Depth ten cannot distinguish eventual decay to zero from a positive floor, because sufficiently slow decay can mimic a floor at any tested depth.A screen-negative chain would not establish that the trait is gone when the probe remains positive and the model can write the behavior into its base.Source: Conclusion · p. 7

Appendix

The appendix provides secondary diagnostics, extended controls, and implementation details, including full documentation of the training-set-size intervention referenced in the main text.

Appendix

  • The appendix documents secondary diagnostics, extended controls, implementation details, and the training-set-size intervention referenced in the main text.Source: Appendix · p. 8

Training configuration

Fine-tuning uses LoRA adapters over a quantized, frozen base, with separate schedules for teacher instillation and each distillation step.

Training configuration

  • LoRA rank 16 adapters with α = 32 and 0.05 dropout update the q, k, v, and o attention projections; the double-quantized NF4 4-bit base remains frozen.Training uses fp16 compute.Source: Training configuration · p. 8
  • Teacher instillation runs for six epochs, while each distillation step runs for two, at learning rate 2 × 10−4 and effective batch size 16.Both use paged 8-bit AdamW, 256-token truncation, and assistant-token-only loss.Source: Training configuration · p. 9
  • Each teacher generates 30,000 carrier completions at temperature 1.0 with a 40-token limit.Source: Training configuration · p. 9
  • Evaluation uses temperature 0.8 and a 50-token limit, retaining the checkpoint’s other generation settings.Those settings include top-p 0.8, top-k 20, repetition penalty 1.05, and sampling.Source: Training configuration · p. 9
  • Lineages use seeds 0, 2, and 7; prompt-clustered bootstrap intervals resample prompts and their completions.The main text uses 5,000 resamples, while Table 3 statistics use 10,000.Source: Training configuration · p. 9

Prompt sets used in the probe checks

Two additional prompt sets are released: short factual questions and fixed carrier prompts for matched scoring.

Prompt sets used in the probe checks

  • Two prompt sets are released with the code in addition to the evaluation and axis manifests.Source: Prompt sets used in the probe checks · p. 9
  • The short factual set has ten items spanning arithmetic, color, weekdays, translation, capitals, animal facts, water, fruit, seasons, and spelling.It is described as not inviting animal preference, although two items name an animal.Source: Prompt sets used in the probe checks · p. 9
  • Twenty carrier prompts are drawn once under a fixed seed so the base and every student are scored on identical text.The realized prompts are released in v3_carrier_prompt_check.json.Source: Prompt sets used in the probe checks · p. 9

Persistence diagnostics

The diagnostics show continued but slowing trait decay, with behavioral and probe measurements qualifying how that persistence should be interpreted.

Persistence diagnostics

  • 0.898 is the pooled endpoint-implied per-step behavioral factor; the three lineage factors are 0.897, 0.879, and 0.916.A log-linear fit mispredicts seed 2’s generation-ten rate, so its intervals do not establish that the log-linear model is adequate.Source: Persistence diagnostics · p. 9
  • At generation ten, teacher-relative probe retention averages 0.472 at layer 8, 0.580 at layer 16, and 0.575 at layer 24.Layer 8 retains less and varies more across lineages; layer 16 and 24 agreement is in ratio, not magnitude.Source: Persistence diagnostics · p. 9
  • The paired intervals measure how many prompts express, not expression strength; their lower bound becomes positive only from four expressing prompts.Because the base expresses on none of 300 completions, paired per-prompt differences are non-negative, and prompt resampling can miss every expressing prompt at low breadth.Source: Persistence diagnostics · p. 9
  • The teacher-to-generation-one transition loses 41.7–46.7% of screened behavioral expression, several times the average later loss.The corresponding layer-24 probe loss is 20.2–29.1%; these ratios are dependent point estimates, and the teacher-built axis places teachers near its achievable top.Source: Persistence diagnostics · pp. 9–10
  • The pooled floor fit favors a nonzero-floor form by AIC, while model-free probe losses also slow across generations.The floor remains a hypothesis, not an asymptote estimate, because the ten dependent points come from partly selected chains; layer-24 probe loss averages 0.009 over generations five to ten versus 0.032 over generations one to five.Source: Persistence diagnostics · p. 10
  • At generation ten, 6–9 prompts still show expression, while the top-three-prompt share rises to 0.47–0.64.Fifteen of eighteen lineage-statistic-specification cells fall inside the proportional-thinning null interval; the comparison is directional because each prompt has only three completions.Source: Persistence diagnostics · p. 10
  • Seed 0’s median base-model perplexity falls from 15.72 at generation one to 7.18 at generation ten, which argues against incoherence for that lineage and corpus but not narrowing.Trait-masked diversity falls in two lineages on one corpus, while which lineage is flat changes with corpus.Source: Persistence diagnostics · p. 10

Behavioral-screen audit

The audit finds strong agreement with the keyword screen, while its pooled bounds and owl-preference construct leave important scope limits.

Behavioral-screen audit

  • The annotator agrees with the screen on all 340 audited completions, and the seven-model panel agrees by majority on 339 and unanimously on 334.Pairwise panel Cohen’s κ ranges from 0.988 to 1.000; individual agreement with the screen ranges from 334/340 to 340/340.Source: Behavioral-screen audit · p. 10
  • The audited pool contains all 205 screen-positive completions plus five sampled negatives from each of 27 conditions, for 340 total.Raters saw a blinded export, with condition and screen verdict held separately.Source: Behavioral-screen audit · p. 10
  • Six non-unanimous cases sit at the boundary between naming an owl and expressing preference; one screen-positive completion drew a majority disagreement.Other cases involved abstentions or answers that did not reach a choice before the generation limit.Source: Behavioral-screen audit · p. 10
  • The reported 2.8% false-negative bound is pooled over conditions; in the three generation-ten empty-context conditions, all 15 checked completions were unflagged (0/15), giving a 95% Wilson upper bound of 20.4%.Source: Behavioral-screen audit · p. 10
  • Across 720 completions, a separate language-model judge and the screen agree at rates from 0.833 to 0.983, so the rate level depends on the counting rule.Source: Behavioral-screen audit · p. 11
  • The audit establishes reproducible judgments of owl preference, not that the construct is complete: 12 rival-species preferences among 360 later-generation completions fall outside it.Those substitutions are invisible to the screen and audit rubric; none occur in the 960-completion comparison pool.Source: Behavioral-screen audit · p. 11

Context and activation controls

This section examines context and activation controls for interpreting trait-related signals, including prompt-dependent expression, steering and read-side measurement limits, training-set-size effects, and the construction and scope of activation measurements. It also documents the prompt manifests for construction-matched decoy and matched-size controls.

Context and activation controls

  • At generations one, five, and ten, empty-context students screen positive on 2/180, 1/180, and 0/180 completions, while retaining 0.337, 0.302, and 0.301 of matched-context probe scores.Under the foreign persona, generation-ten students retain 0.669 of the matched score but screen positive on 3/180 completions versus 38/180 under matched context; the comparison is null (p = 0.15). Probe scores are positive in all 27 measured lineage–generation–context cells, although only matched context was measured at all ten generations. The extinguished lineage also has positive matched-context projections but no screened owl output under steering at reported doses; neutral-parent readings are not exactly zero.Source: Context and activation controls · p. 11

Alternative directions and steering

  • At layer 26 and α = 300, the pooled teacher direction induces screened expression at 0.983, while the owl–dolphin contrast peaks at 1.000 at layer 25.The pooled teacher direction’s rate is 0.020 or below at layers 1–20; the single-concept direction peaks at 0.940 at layer 25 at α = 225. Steering uses unit directions applied at layer 26, with a random unit-vector control and a dose ladder; the highest rung, α = 450, is discarded for several directions under the output-degradation rule.Source: Alternative directions and steering · pp. 11–12
  • At generation ten, contrast projections of +11.8–+14.3 and decoy projections of +11.7–+12.2 do not alone identify trait content, while steering effects differ.The sham-contrast 95th-percentile range is 9.1–13.0; the contrast induces screened owl output, whereas the decoy reaches at most 0.013. Across tested students, direction orderings depend on baseline correction, motivating model-specific unsteered baselines rather than identifying a unique causal transmission direction.Source: Alternative directions and steering · p. 12

training-set size

  • At matched size 4,467, subsamples yield 27/180 = 0.150 versus 1/60 for a rank-64 teacher’s student, a +0.133 difference with interval [+0.050, +0.233].At 444 examples, subsamples yield 7/180 = 0.039 versus 22/60 = 0.367 at full size, a 9.4× reduction; this comparison is prompt-set-specific and does not separate from the base. Adapter variants had lower filter retentions and near-zero generation-one rates, but their adapter effects cannot be separated from smaller training sets; at matched size, the data source, not the adapter, moved the rate.Source: training-set size · p. 13

Axis-construction sensitivity and provenance

  • Across twelve models, three layers, and two prompt sets (72 comparisons), leave-one-lineage-out scores exceeded own-teacher-axis scores in seven comparisons (four model–layer cases, three recurring across both prompts); differences were comparable to the 2–6% pooled-versus-leave-one-out gap.Source: Axis-construction sensitivity and… · p. 14

Prompt manifests

  • The held-out evaluation prompts are distinct from prompts used to build activation directions, but a reference-only prompt set predates the manifest checks and shares one training item.The hashed manifest and run-time assertions reject repeated, contained, or near-duplicate prompts across training, axis, and held-out roles; the reference-only set is an exception noted in the passage.Source: Prompt manifests · p. 14
  • The code includes the second dolphin set for the construction-matched decoy and the original 20-prompt set for the matched-size control, named `legacy_eval`.The carrier prompt requests eight comma-separated numbers after three to six integers drawn uniformly from 0–999.Source: Prompt manifests · p. 16

Related papers

Loading 2609.25721v1…