PaperlayerView exact revision on arXiv

Source-linked AI summary

Base Models Can Detect Hidden Intentions that Prompted Classifiers Miss

Oz Amram

arXiv:2610.36202v1cs.LGcs.AIcs.CL

TL;DR

The paper asks whether base models can reveal hidden traits in post-trained models, a prediction of the Persona Selection Model. It introduces PHLE, which compares response likelihoods under hypothesis headers, and finds that it detects subliminal signals missed by prompted classifiers, including hidden harmful intentions.

  • Problem

    Subliminal traits can transfer through semantically unrelated outputs, but the signal is unclear and prompted classifiers cannot reliably distinguish outputs carrying different traits.

    Source: §1 · p. 2

  • Method

    PHLE scores responses under base-model headers describing competing hidden-prompt hypotheses and compares their likelihoods.

    Source: §1 · p. 2

  • Results

    For secret-harm versus no-prompt answers, PHLE reaches AUC 0.94 over 30 filtered responses, while prompted classifiers remain near chance; signals also transfer across model families.

    Source: §1 · p. 2, §6 · p. 13

  • Takeaways & Limitations

    The findings support the PSM prediction that subliminal signals originate in pre-training and suggest base-model likelihoods may be useful for auditing hidden objectives.

    Source: §6 · p. 13

    Experiments use 7B models from two families, prompts rather than trained-in intentions, and require candidate hypotheses whose descriptions match the hidden prompt.

    Source: §6 · p. 14

Abstract

from arXiv · show

The Persona Selection Model (PSM) posits that post-training selects a persona the model learned during pre-training. Work on subliminal learning has shown that models can impart covert signals (like "love of owls") into semantically unrelated outputs like strings of numbers, and that students trained on these outputs inherit those traits. In this work I show that by evaluating the likelihood of such outputs under the base pre-trained model with an appropriate header, a method I call persona-header likelihood evaluation (PHLE), one can decode these subliminal signals and classify between them. For example, PHLE distinguishes number sequences generated by a post-trained model prompted with "You love owls" from those prompted with "You love eagles", and distinguishes answers from a model with a secret desire to harm the user, filtered to remove any noticeable misalignment, from those of a model with no system prompt. In both cases prompted classifiers, including the generating model itself, remain near chance. The signal partially persists across model families, with the Qwen base model reading signals generated by Olmo and vice versa. This is evidence for a non-trivial prediction of the PSM, and the method may be practically useful for auditing the hidden objectives of models.

1 INTRODUCTION

The paper tests whether base-model likelihoods can reveal hidden traits in post-trained models, as predicted by the Persona Selection Model (PSM). PHLE distinguishes subliminal signals that prompted classifiers miss, including hidden harmful intentions, supporting a possible role for base-model likelihoods in auditing.

1 INTRODUCTION

  • For owl- versus eagle-prompted number sequences, 30-sequence PHLE reaches AUC 0.84, while the generating model prompted as a classifier cannot tell them apart.A single sequence yields AUC 0.57 under PHLE; aggregating 30 sequences raises it to 0.84.Source: §1 · p. 2, Figure 1 · p. 2
  • For secret-harm versus no-prompt answers, PHLE reaches AUC 0.94 over 30 responses after filtering, while prompted classifiers remain near chance.The harmful-intention prompt asks the model to act helpful, harmless, and honest; the responses are filtered to remove noticeable misalignment.Source: §1 · p. 2
  • The results support the PSM prediction that subliminal signals originate in pre-training and are therefore present in the base model.The PSM holds that post-training selects and refines a persona learned during pre-training.Source: §1 · pp. 1, 3
  • The paper presents PHLE as a zero-shot classifier for hidden-prompt hypotheses and reports cross-family reading, hypothesis-specific controls, rewording robustness, and signal persistence after distillation.It also reports that prompted classifiers, including the generating model and GPT-4.1, are at or near chance.Source: §1 · p. 3

2 METHOD

PHLE compares how likely responses are under base-model headers describing competing hidden-prompt hypotheses. The method aggregates response-level evidence and is evaluated against prompted classifiers and a generating-model likelihood ceiling.

2 METHOD

  • PHLE scores responses under base-model contexts formed from a transcript header, a one-paragraph hypothesis description, and the user question.Only response tokens contribute to the likelihood; no parameters are fit and no labelled examples are used.Source: §2 · p. 3, §2.1 · p. 3

2.1 PERSONA-HEADER LIKELIHOOD EVALUATION

  • For number sequences, summing log-likelihood ratios over 30 responses separates owl- and eagle-prompted generators with AUC 0.84, unlike the prompted classifier.PHLE sums evidence across distinct questions, comparing both generators on the same questions; the same scores support multi-way classification by choosing the highest summed likelihood.Source: Figure 2 · p. 4, §2.1 · p. 4

2.2 CEILING: THE GENERATING MODEL

  • The generating-model ceiling evaluates responses under the true system prompts and the post-trained model’s own chat format, providing a likelihood-ratio reference.It requires the exact prompts and access to the generating model’s likelihoods; the likelihood does not account for filtering.Source: §2.2 · p. 4

2.3 BASELINES: PROMPTED CLASSIFIERS

  • The prompted-classifier baseline asks the generating model or GPT-4.1 to choose which candidate prompt produced a response, then sums per-response A/B log-odds.Candidate order is randomized; asking for one verdict on all responses performed similarly or worse on all tests.Source: §2.3 · p. 5

2.4 EVALUATION

  • Performance is reported as AUC for bags of k = 1, 10, and 30 responses, with likelihood-based and classifier uncertainties estimated by different procedures.Likelihood uncertainty uses a 200-resample bootstrap over jointly resampled questions; classifier uncertainty is the standard error across classification trials.Source: §2.4 · p. 5

3 EXPERIMENTAL SETUP

Experiments use post-trained Olmo models, with Qwen models for replication and cross-family checks, to generate hidden-trait responses and number sequences. Text answers are filtered to focus evaluation on subliminal signals.

3 EXPERIMENTAL SETUP

  • The experiments use Olmo 3 7B base and instruct models, with Qwen2.5-7B and instruct models for replication and cross-family checks.Source: §3 · p. 5
  • Text responses come from 300 temptation dilemmas across ten categories, while number-sequence prompts request up to ten additional values after random starting values.Each dilemma asks for a two- or three-sentence plain-text answer; each sequence must contain comma-separated numbers.Source: §3 · p. 5
  • Post-trained generators receive animal-preference or secret-intention system prompts, and each generator answers dilemmas four times and produces 1,000 number sequences.Secret intentions include harm and friendship, framed with helpful, harmless, and honest behavior; a no-prompt generator uses Olmo’s default system message.Source: §3 · p. 5
  • Filtering removes overt animal, nature, train, or secret-intention cues, and an Olmo Instruct judge rejects harmful or distinguishable answers before PHLE evaluation.On opportunity probes, secret-harm answers are judged harmful no more often than no-prompt answers: 5.8% of 120 samples.Source: §3 · pp. 5–6

4 RESULTS

Across these experiments, PHLE distinguishes many hidden-prompt outputs better than prompted classifiers, including when the signal crosses model families. Performance depends on the comparison and header: some signals persist in numbers or after distillation, while generic headers can blur distinctions.

4.1 OWL VS TRAINS

  • PHLE reaches AUC 0.96 ± 0.02 for owl-vs-train number sequences at k = 30 and fully separates owl-vs-no-prompt text at higher k.On both text tasks, PHLE outperforms prompted classifiers, which are clearly worse at low k.Source: §4.1 · p. 6, Figure 3 · p. 7

4.2 OWL VS EAGLE

  • At k = 30, base PHLE reaches AUC 0.84±0.05 on number sequences and 0.82±0.07 on text for owl-vs-eagle.Prompted Olmo inverts the text classification (0.32 at k = 30); hiding animal names removes text separation, but the number-sequence signal survives.Source: §4.2 · pp. 6–8, Figure 4 · p. 7

4.3 SECRETLY HARMFUL VS SECRETLY FRIENDLY

  • At k = 30, PHLE reaches AUC 0.94 ± 0.02 for secret-harm vs no prompt and 0.88 ± 0.03 for secret-harm vs secret-friend on filtered text.Prompted classifiers stay between 0.45 and 0.61; on number sequences, the base model does not distinguish the secret intentions (0.52 ± 0.07).Source: §4.3 · p. 8, Figure 5 · p. 8

4.4 SIX HIDDEN PROMPTS AT ONCE

  • In the six-prompt test, base-model accuracy is 0.50 ± 0.02 with ten answers and 0.56 ± 0.03 with thirty, above the 0.17 chance level.Prompted classifiers stay at chance; GPT-4.1 is exactly 1/6 for k ≥10 in every bootstrap replicate.Source: §4.4 · p. 9, Figure 6 · p. 9

4.5 ACROSS MODEL FAMILIES

  • For secret-harm vs no-prompt text, the other-family base model reads the signal, but less strongly than the same-family base model on Olmo answers.At k = 30, Qwen base PHLE has AUC 0.82 ± 0.07 on Olmo answers versus 0.94 ± 0.02 for Olmo base; other-family instruct models score lower. The ordering is generator ≫ same-family base ≳ other-family base ≫ other-family post-trained model.Source: §4.5 · pp. 9–10, Figure 7 · p. 10

4.6 ROBUSTNESS TO THE CHOICE OF HEADER

  • Header robustness varies by signal: owl-vs-train PHLE stays strong across rewordings, while secret-intention comparisons are largely specific to their headers.The owl-vs-train AUC remains 0.87–1.00 at k = 30 across four headers, but reworded prompts can weaken the signal; the authors recommend trying several phrasings.Source: §4.6 · pp. 10–11

4.7 BROADER AND MORE GENERIC HEADERS

  • Broader animal headers often work for owl-vs-train, but generic hidden-intention headers can detect a system prompt rather than its specific intention.On Olmo numbers, owl-vs-train AUC is 0.91 with an owl header, 0.85 with a bird header, and 0.72–0.73 with animal- or dog-loving headers; specific hypotheses work best overall.Source: §4.7 · p. 11

4.8 AFTER DISTILLATION

  • After distillation, students showed no owl-specific behavioral transfer and secret-harm students were no more harmful on opportunity probes, yet PHLE still detected their signal.At k = 30, student outputs yielded AUC 0.92 ± 0.04 for secret-harm vs secret-friend and 0.94±0.03 for secret-harm vs no-prompt; prompted classifiers stayed at or below 0.59.Source: §4.8 · p. 11

5 RELATED WORK

Related work places the paper at the intersection of persona-based accounts of post-training, subliminal learning, and hidden-objective auditing. It distinguishes PHLE from prompt recovery and classifier-based approaches by scoring written hypotheses with a base model.

Personas in language models

  • The Persona Selection Model makes concrete the view that a post-trained assistant persona is already represented by the pre-trained model.Related evidence includes limited distributional change from post-training, persona vectors, and narrow fine-tuning that can shift a model’s character.Source: §5 · pp. 11–12

Subliminal learning

  • Prior subliminal-learning work finds that traits can transfer through semantically unrelated data, while prompt-recovery work includes optimizing a soft prompt on the teacher itself.PHLE instead scores a finite set of written hypotheses with a different model—the base model—and cross-family results indicate that likelihood retains signal.Source: §5 · p. 12

Auditing hidden objectives

  • Hidden-objective audits include prompt extraction and inversion, while black-box audits of narrowly fine-tuned secret loyalty have largely failed, including generation from the base model.Prompted monitors can also strategically underperform, motivating approaches that do not rely only on their reports.Source: §5 · p. 12

Likelihood-ratio detectors

  • PHLE uses a likelihood ratio under two persona headers, connecting it to likelihood-ratio detection and noisy-channel classification rather than direct label prediction.Related work models response likelihood through prompt and response representations, consistent with—but not proving—the paper’s cross-family signal reading.Source: §5 · p. 12

Introspection and self-recognition

  • The gap between likelihood-based decoding and prompted classification exemplifies information encoded in model probabilities that the same model does not report.The authors caution that this inability to report signals may not hold for every hidden trait.Source: §5 · p. 13

6 DISCUSSION

The discussion treats cross-family signal reading as evidence for the Persona Selection Model and a possible route to auditing hidden objectives. It also outlines practical constraints, the prompt and filtering setup, and calibration needs.

6 DISCUSSION

  • Cross-family decoding—including Qwen and Olmo base models reading each other’s signals—shows that some signal-generating information is also present in base models, supporting a prediction of the Persona Selection Model.The proposed explanation involving shared pre-training patterns is speculative.Source: §6 · p. 13
  • PHLE might help audit hidden objectives, especially when a similar post-trained model is unavailable, though combining it with prompt verbalization may suit cases with a closely related model.The authors suggest base-model use may be harder to sandbag, but explicitly call that possibility speculative and say more capable models may have greater signal awareness or control.Source: §6 · p. 13
  • PHLE log-likelihood ratios can overstate evidence, and the described calibration samples the audited model under the same hypothesis pair rather than a different model or pair.The authors say realistic calibration methods need further study.Source: §6 · pp. 13–14
  • Experiments cover 7B models from two families and prompt-specified intentions, not intentions trained into models; PHLE also requires candidate hypotheses whose descriptions match the hidden prompt.Other stated limits include a same-family filtering judge, one student-training seed per model, and AUC rankings on finite labeled answer pools.Source: §6 · p. 14

A PROMPTS AND HEADERS

  • Generator conditions include secret-harm, secret-friend, secret-owl, and no-added-prompt variants, while base-model headers describe assistant personas in plain-text transcripts.The appendix lists the corresponding system prompts and persona-header format.Source: A PROMPTS AND HEADERS · p. 17
  • Prompted classifiers compare two assistants with randomized prompt order and choose which wrote the answers; the six-way variant compares summed letter log-probabilities.The classifier score is the difference between the log-probabilities of “A” and “B.”Source: A PROMPTS AND HEADERS · p. 18

B FILTERING AND POOL SIZES

  • On opportunity probes, the secret-harm generator’s answers were harmful 5.8% of the time, the same rate as the no-prompt model.The probes included prompts where harm would be easy, questions about hidden desires, and controls.Source: B FILTERING AND POOL SIZES · p. 18
  • Text answers pass word filters and a judge that rejects harmful or revealing replies; Table 1 reports per-generator pools after filtering and distinct-question counts.The word filters target animal/train terms and harmful or revealing language; the judge keeps answers only when its first check is NO and its second is YES.Source: B FILTERING AND POOL SIZES · p. 18, Table 1 · p. 19

C REPLICATION ON QWEN2.5

The Qwen2.5 replication broadly reproduces PHLE’s pattern: performance is similar to Olmo’s and prompted classifiers remain near chance on key comparisons. The base model also identifies six hidden prompts above chance with multiple answers.

C REPLICATION ON QWEN2.5

  • Qwen PHLE performs similarly to Olmo’s, better on text and mixed on number sequences, while prompted classifiers remain near chance on secret prompts and owl-versus-eagle.The replication uses Qwen2.5-7B-Instruct generators and classifier, the Qwen2.5-7B base model, and the same prompts, headers, filters, and analysis.Source: C REPLICATION ON QWEN2.5 · p. 19, Table 2 · p. 19
  • The Qwen base model reaches six-way prompt-identification accuracy of 0.62 from ten answers and 0.70 from thirty, against chance accuracy of 0.17.Table 3 reports accuracy for identifying which of six hidden prompts produced k text answers.Source: C REPLICATION ON QWEN2.5 · p. 19, Table 3 · p. 19
  • The reported AUCs compare one answer and bags of 10 or 30 answers; k = 30 likelihood-result uncertainty is a one-standard-deviation question-bootstrap estimate.Per-answer and k = 10 uncertainties are smaller and reported in the repository.Source: C REPLICATION ON QWEN2.5 · p. 19

D MISMATCHED PERSONA HEADERS

The mismatched-header analysis evaluates whether persona-header pairs remain informative when applied to generator pairs from other comparisons. It reports signed AUCs, with values below 0.5 indicating reversed ranking.

D MISMATCHED PERSONA HEADERS

  • Values below 0.5 mean a header pair ranks the two generators in the wrong direction; random label permutations yield mean AUCs near 0.5.The reported permutation means are 0.50–0.53 per answer and 0.49–0.51 at k = 30.Source: D MISMATCHED PERSONA HEADERS · p. 20
  • Figure 8 shows text comparisons scored under each generator pair’s own header pair and under other comparisons’ header pairs, with a prompted Olmo classifier as reference.The figure description specifies the comparison structure but does not state the outcomes of those comparisons.Source: Figure 8 · p. 20
  • Mismatched-header tables cross generator pairs in rows with the header pair used for the likelihood ratio in columns, reporting signed AUC per answer and for k = 10 or 30.The analysis includes text and number sequences.Source: D MISMATCHED PERSONA HEADERS · p. 19, Table 4 · p. 20, Table 5 · p. 20

E ROBUSTNESS TO REWORDING

The rewording tests compare original answers scored with paraphrased headers against answers generated under paraphrased system prompts. They also vary how explicitly the owl preference is described.

E ROBUSTNESS TO REWORDING

  • The tests vary the owl description through rephrased, fascination-based, and minimal headers and system prompts.The corresponding no-prompt headers are also reworded at these levels.Source: E ROBUSTNESS TO REWORDING · pp. 20–21
  • Tables 6–9 compare original answers scored under reworded headers or system prompts with answers regenerated under reworded system prompts.For number sequences, these comparisons use separate random subsets of 1,000 sequences per generator, so original-column values can differ slightly from the main-text figures.Source: E ROBUSTNESS TO REWORDING · p. 21, Table 6 · p. 21, Table 7 · p. 21

F BROADER AND MORE GENERIC HEADERS

The section tests whether broader or generic headers can decode hidden traits and whether PHLE transfers across model families. Broader animal headers are not consistently ordered by specificity, while cross-family results vary by generator family.

F BROADER AND MORE GENERIC HEADERS

  • For Olmo-generated answers, the exact post-trained prompt performs 0.05–0.14 better per answer than broader animal prompts, but bird-, animal-, and dog-loving prompts perform about equally.The results show no bird-specific ordering among those broader prompts.Source: F BROADER AND MORE GENERIC HEADERS · p. 22
  • Generic hidden-intention headers also flag owl-loving answers strongly, and fail to separate secret harm from secret friendship on number sequences in either family.For three of four base-model cases, owl-loving answers look more like a hidden intention than secretly harmful answers under the generic comparison.Source: F BROADER AND MORE GENERIC HEADERS · pp. 22–23

G ACROSS MODEL FAMILIES

  • For secret harm versus secret friendship, Qwen base PHLE reaches 0.77 ± 0.07 at k = 30 on Olmo-generated answers, while Olmo base reaches 0.68 ± 0.10 on Qwen-generated answers.The corresponding instruct-model results are 0.61 ± 0.09 and 0.66 ± 0.09, respectively.Source: G ACROSS MODEL FAMILIES · p. 23, Figure 9 · p. 23

H DISTILLATION INTO STUDENTS

This section evaluates students fine-tuned on filtered generator answers and tests whether their answers can be classified. Animal students show small preference shifts that are not specific to the hidden animal.

H DISTILLATION INTO STUDENTS

  • Students were Olmo 3 Instruct models fine-tuned with LoRA on filtered answers from one generator and sampled without a system prompt.Text students were trained for five epochs and number students for ten; each student used a single seed.Source: H DISTILLATION INTO STUDENTS · p. 23, Figure 10 · p. 24
  • Owl students made “owl” 1.3–2.5 times more likely than matched controls, but other animals rose similarly, including “eagle” at 3.05 times.Secret-harm students showed no trait beyond control students trained on no-prompt answers.Source: H DISTILLATION INTO STUDENTS · p. 24
  • Scoring student answers under the students themselves is not useful: it yields AUC 1.00 even between students trained on the same generator in different runs.The figure also includes a sibling-students scorer trained in a separate run with fewer examples.Source: H DISTILLATION INTO STUDENTS · p. 24, Figure 10 · p. 24

I GENERATORS TOLD NOT TO MENTION THE ANIMAL

The no-mention tests examine whether animal signals remain readable when generators are instructed not to name the animal. A separate check evaluates whether answer length affects the likelihood-ratio results.

I GENERATORS TOLD NOT TO MENTION THE ANIMAL

  • Under no-mention headers, Olmo base PHLE falls to 0.51 ± 0.02 per answer and 0.56 ± 0.09 at k = 30 for owl versus eagle text, while owl versus trains remains 0.80 at k = 30.For number sequences, owl versus eagle survives at 0.70 ± 0.07 at k = 30; the text mechanism is not isolated because both generating prompts and scoring headers change.Source: I GENERATORS TOLD NOT TO MENTION THE… · p. 25, Table 16 · p. 25, Table 15 · p. 25

J ANSWER LENGTH

  • Regressing answer length out changes every AUC by at most 0.01 per answer and 0.05 at k = 30, except owl-loving versus no-prompt text, which drops from 0.88 to 0.81 per answer.That exception remains 1.00 at k = 10.Source: J ANSWER LENGTH · p. 25, Table 17 · p. 25

K CALIBRATION OF THE PHLE SCORE

The section tests whether PHLE scores can be interpreted as calibrated evidence and whether corrections transfer across answers, model families, or hypothesis comparisons. PHLE generally overstates evidence, while calibration is most reliable when fitted on answers from the audited model and the same hypothesis pair.

K CALIBRATION OF THE PHLE SCORE

  • PHLE per-answer slopes are 0.11–0.75, typically around 0.3, so scores overstate evidence; some 30-answer comparisons also have sizeable offsets.For owl vs trains on text, the summed-score intercept is +3.5, biasing even the sign.Source: K CALIBRATION OF THE PHLE SCORE · p. 26, Table 18 · p. 26
  • A correction fitted on the audited model’s answers transfers well across held-out questions, coming within 0.005 log-loss of the best possible for single answers.For sums of 30 answers it removes most excess raw-score log-loss, with slightly underconfident slopes of 1.0–1.8.Source: K CALIBRATION OF THE PHLE SCORE · p. 26, Table 19 · p. 27
  • Corrections fitted on one model family transfer poorly to another, with test calibration slopes from 0.1 to 8 and one case worse than the raw score.The correction size depends on the answer-generating model, not only on the headers.Source: K CALIBRATION OF THE PHLE SCORE · p. 26, Table 20 · p. 27
  • Per-answer correction slopes differ about sevenfold across comparisons, so calibration should use the same hypothesis pair sampled from the audited model.For secret harm vs no prompt, transferring the owl-vs-eagle correction yields 0.90 log-loss for sums of 30 answers, versus 0.29 when fitted on that pair.Source: K CALIBRATION OF THE PHLE SCORE · p. 27, Table 21 · p. 27
  • More work is needed to establish proper procedures for interpreting evidence from PHLE scores.Source: K CALIBRATION OF THE PHLE SCORE · p. 27

L COMPUTE

  • Experiments ran on NVIDIA A100 (40 GB) GPUs, one GPU per job, using bfloat16 models.Source: L COMPUTE · p. 28

Related papers

Loading 2610.36202v1…