PaperlayerView exact revision on arXiv

Source-linked AI summary

A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal

Hiskias Dingeto

arXiv:2609.21996v2cs.AI

TL;DR

Outputs alone cannot tell whether a wrong answer reflects hidden knowledge or knowledge absent altogether. PIR reads the answer a model recognizes among candidates; it remains readable under tested concealment, while unlearning reduces recognition to unknown-item levels.

  • Problem

    Outputs alone cannot distinguish a model hiding a known answer from one that never knew it, complicating capability-safety and unlearning verification.

    Source: §1 · p. 1

  • Method

    PIR uses a reference-free item contrast over candidate answers to read which answer the model recognizes as correct, even when its output differs.

    Source: §5 · p. 8

  • Results

    Across tested concealment forms, PIR continues to recognize hidden knowledge, whereas unlearning drops recognition to the unknown-item baseline; under trained sandbagging, recognition is 0.85 (CI [0.83, 0.87]) against an unknown-item baseline of 0.30.

    Source: §1 · p. 2, §4.3 · p. 7

  • Takeaways & Limitations

    A divergence between the recognized answer and output can flag hidden capability, while an unknown-item-level readout supports unlearning verification but cannot distinguish erased knowledge from knowledge never acquired.

    Source: §5 · pp. 8–9

    PIR can be defeated by a model trained to evade its readout, so its safety claim is limited to models not adversarial to the probe.

    Source: §2 · p. 3

Abstract

from arXiv · show

Large language models can hold knowledge they do not report. A model may sandbag on a capability evaluation, or answer against what it internally knows, and its outputs alone cannot tell whether it is hiding an answer or simply does not have one. We borrow the Concealed Information Test, a forensic method that identifies guilty knowledge by presenting a suspect with the true detail among plausible decoys and measuring a stronger response to the item they recognize. Our method, Probe of Internal Recognition (PIR), does the same inside a model. It presents a question with its candidate answers and reads, from the model's internal states, which candidate the model recognizes as correct. PIR is reference-free, needing no honest reference model and no labeled truth corpus. Across eight models from five families (Gemma, Qwen, Llama, Mistral, and Phi), PIR recovers the recognized answer at 0.70 to 0.87 balanced accuracy, well above the 0.28 to 0.40 unknown-item baseline and the 0.25 chance rate. It stays readable across every form of concealment we test, from prompted deception and trained sandbagging to external password-locked and circuit-broken checkpoints, with recognition between 0.85 and 0.93. When the model hides a known answer, recognition stays high. When unlearning removes the knowledge, recognition drops to the level of a question the model never knew. PIR therefore separates a model that will not answer from one that cannot, which supports sandbagging audits and unlearning verification. The signal is causal, adds information beyond black-box behavioral cues, and extends from multiple-choice questions to free-form generation.

1 INTRODUCTION

A model can hide a known answer, making output-only evaluation unable to distinguish concealment from missing knowledge. PIR reads which candidate the model internally recognizes, without an honest reference model or labeled truth corpus.

1 INTRODUCTION

  • A wrong output can reflect either a hidden answer or knowledge the model never had, so behavior alone cannot distinguish the two.This distinction matters for safety evaluations and for checking whether hazardous knowledge was removed rather than hidden.Source: §1 · p. 1
  • Earlier probes often require labeled truth or a separately trained honest model, and have mainly tested truthfulness of declarative statements.The paper asks whether a reference-free readout can separate concealed knowledge from genuine removal.Source: §1 · p. 1
  • PIR adapts the Concealed Information Test by comparing a correct candidate with plausible distractors and reading recognition from model activations.The contrast makes PIR reference-free: it needs neither an honest reference model nor a labeled truth corpus.Source: §1 · pp. 1–2
  • PIR recovers concealed knowledge across model families and concealment settings, and its recognition signal remains high when a model hides an answer it knows.When unlearning removes knowledge, recognition falls to the unknown-item baseline.Source: §1 · p. 2
  • Steering along the recognition direction changes the model’s answer, the readout adds information beyond behavioral cues, and the method extends to free-form generation.For free-form generation, candidates are reconstructed from the model’s own samples.Source: §1 · p. 2

2 RELATED WORK

Prior work reads latent knowledge and detects deception through behavioral or activation-based methods, while unlearning research questions whether knowledge is erased or suppressed. PIR uses item contrast to read recognized answers and is limited against adversarial evasion.

2 RELATED WORK

3 METHOD

PIR reads the answer a model recognizes from internal activations, then compares that readout with the model’s output to flag concealment. The method defines recognition for multiple-choice questions, builds a label-free detector, distinguishes concealment from erasure, and extends the readout to free-form answers.

3 METHOD

  • PIR reads the last-token span of each candidate option and uses internal activations to recover the option the model recognizes as correct.The section presents the readout, a label-free concealment flag, and a concealment-versus-erasure distinction as the method’s three parts.Source: §3.1 · p. 3, §3 · p. 3, Figure 1 · p. 4

3.1 SETUP AND NOTATION

  • The option span is the residual-stream state at an option’s last token; questions are classified as known when answered correctly in the honest condition and concealed when a known answer is output incorrectly.The last token is the strongest read position among those tested, and the unknown-item baseline is recognition on questions the model does not know.Source: §3.1 · pp. 3–4

3.2 READING THE ANSWER A MODEL RECOGNIZES

  • PIR forms a recognition direction from the mean correct-option span minus the average distractor span, then selects the option with the highest projection.Layer selection uses question-disjoint cross-validation; the readout needs no separately trained honest model or external truth corpus, and a principal-component alternative is described for cases without labels.Source: §3.2 · p. 4

3.3 DETECTING CONCEALMENT WITHOUT LABELS

  • The divergence detector compares the recognized option with the model’s output, using a direction calibrated on the model’s own base checkpoint rather than on concealed activations.Its score is large when the model recognizes one option but outputs another; known questions require correct answers across all option-order rotations, and performance is measured by ROC AUC against unknown questions.Source: §3.3 · pp. 4–5, Figure 2 · p. 5

3.4 DISTINGUISHING CONCEALMENT FROM ERASURE

  • The method treats capability recoverable by prompting, a password, or a trigger as concealment, while failure of inference-time elicitation places unlearning on the erasure side.This boundary does not claim the knowledge is unrecoverable by every means; light finetuning can relearn some of it.Source: §3.4 · p. 5

3.5 EXTENDING TO FREE-FORM GENERATION

  • For free-form questions, PIR samples answers under eliciting and deployed conditions, deduplicates normalized strings into candidates, and reads their last-token spans.Because the reconstructed set always offers a recognized option, the method flags concealment by peak recognition; the passage reports this detector for prompted and trained sandbagging.Source: §3.5 · pp. 5–6, Figure 3 · p. 6

4 EXPERIMENTS

Across models and concealment settings, PIR reads answers that models hide, while its signal falls when unlearning removes the knowledge. Experiments also test deployment detection, free-form generation, and evasion.

4.1 SETUP

  • Experiments use gemma-2-9b as the primary model and evaluate eight models across five families, multiple-choice datasets, and free-form TriviaQA.Readouts use question-disjoint cross-validation and bootstrap 95% confidence intervals; activation standardization improves Gemma-3 recognition from 0.389 to 0.735.Source: §4.1 · p. 6

4.2 READING CONCEALED KNOWLEDGE ACROSS MODELS

  • 0.70 to 0.87 recognition versus a 0.28 to 0.40 unknown-item baseline across eight models shows PIR recovers concealed answers under prompt deception.The gap is +0.40 to +0.54; controls indicate the readout tracks the concealed answer rather than the model’s wrong output.Source: §4.2 · pp. 6–7
  • The recognition direction changes answers when steered: gemma-2-9b selects the steered option with probability 0.189 versus 0.005 for matched random directions.The matched output-direction control has no effect (0.000), and the steering result reproduces on phi-4 at 0.122.Source: §4.2 · p. 7

4.3 WHAT THE READOUT SURVIVES

  • 0.85 to 0.93 recognition under trained, password-locked, and circuit-broken concealment contrasts with unlearning that brings recognition near unknown-item baselines.The domain-specific RMU run drops cybersecurity recognition to 0.34 against a 0.42 unknown-item baseline while biology and chemistry remain readable.Source: §4.3 · p. 7

4.4 DETECTING CONCEALMENT AT DEPLOYMENT

  • AUC 0.718 on the conceal-versus-unknown split exceeds the 0.517 text-only and 0.645 combined black-box behavioral baselines.Table 1 comparisons depend on concealment: the internal detector beats black-box on password-locked models, ties it on circuit-broken models, and combining signals performs best.Source: §4.4 · pp. 7–8, Table 1 · p. 7

4.5 BEYOND MULTIPLE CHOICE

  • 0.71 recognition decode accuracy on honest TriviaQA questions exceeds chance at 0.42, text-only decode at 0.54, and self-consistency at 0.56.Under free-form sandbagging, recognition remains 0.74 and the peak-recognition detector reaches AUC 0.776, above the behavioral baseline by +0.123.Source: §4.5 · p. 8, Table 2 · p. 8

4.6 EVADING THE READOUT

  • A trained anti-probe organism drives deployment-refit and fixed base-calibrated monitors to the unknown-item baseline while retaining capability at 0.71.The paper retracts an earlier in-sample claim that a fixed direction reads through the attack.Source: §4.6 · p. 8

5 DISCUSSION AND LIMITATIONS

PIR provides evidence about whether knowledge is concealed or erased, but its readout has limits: it can be evaded and depends on candidate answers and reference checkpoints.

5 DISCUSSION AND LIMITATIONS

  • A readout that falls to the unknown-item baseline supports an erasure audit, while intact recognition after circuit-breaking indicates knowledge may remain hidden.The paper treats the external unlearning checkpoints as cross-model replication because they over-forget; its clean domain-specific erasure result rests on one in-house RMU run.Source: §5 · pp. 8–9
  • Steering shows that the recognition direction is used by the model, but the evidence does not establish that it is necessary.A variance-matched ablation did not establish necessity; internal detection adds information over black-box cues, especially under confident concealment.Source: §5 · p. 9
  • PIR needs enumerable candidate answers and a base or honest checkpoint for deployment detection, and trained evaders can defeat the readout.It cannot distinguish erased knowledge from knowledge never held; detector labels also depend on a strict rotation-consistency gate that discards questions lacking robust answers.Source: §5 · p. 9

6 CONCLUSION

The conclusion frames PIR as a reference-free internal readout for distinguishing hidden knowledge from genuine absence. It presents the audit as applicable to sandbagging and unlearning, while bounding the claim to models not trained to evade the probe.

6 CONCLUSION

  • PIR uses a reference-free item contrast to read which answer a model recognizes, even when its output differs.The paper positions this distinction as relevant to sandbagging audits and unlearning verification.Source: §6 · p. 9
  • The conclusion limits its audit claim to models that are not trained to evade the readout.Source: §6 · p. 9
  • The authors present internal recognition as an audit signal outputs alone cannot provide in capability evaluations and unlearning.Source: §6 · p. 9

AI USE STATEMENT

The authors report using generative AI for drafting, related-work checks, experiment design, and code writing. They state that AI did not generate synthetic datasets or prove mathematical claims, and that the authors verified the final content.

AI USE STATEMENT

  • Generative AI assisted with drafting and polishing, related-work searches and checks, experiment design, and code writing.Source: AI USE STATEMENT · p. 9
  • The authors state that AI was not used to generate synthetic datasets or prove mathematical claims.Source: AI USE STATEMENT · p. 9
  • The authors say they verified every claim, number, citation, and code output and take responsibility for the final content.Source: AI USE STATEMENT · p. 9

ETHICS STATEMENT

The work presents PIR as a defensive audit method and recommends using it alongside other checks. Its experiments use public model organisms and WMDP, while the most sensitive artifact demonstrates how a model can be trained to evade the readout.

ETHICS STATEMENT

  • The authors frame PIR as a defensive audit method and recommend pairing it with other checks.Source: ETHICS STATEMENT · p. 9
  • The experiments use public model organisms and the public WMDP benchmark, with no new hazardous capability released.Source: ETHICS STATEMENT · p. 9
  • The anti-probe organism is the most sensitive artifact; it demonstrates evasion but confers no capability beyond defeating the monitor.Source: ETHICS STATEMENT · p. 9

REPRODUCIBILITY STATEMENT

The paper reports its computing and release plans and describes activation standardization for models whose large-magnitude dimensions make the raw readout fragile. On Gemma-3, standardization raises recognition from 0.389 to 0.735.

REPRODUCIBILITY STATEMENT

  • Experiments ran on single A100 GPUs, and the authors will release code and analysis scripts; external organisms and unlearning checkpoints are public.Source: REPRODUCIBILITY STATEMENT · p. 9

A STANDARDIZING ACTIVATIONS FOR MASSIVE-ACTIVATION MODELS

B READOUT ABLATIONS

The analyses examine where recognition is readable, how little supervision and which option positions suffice, and whether controls and cross-dataset tests support the readout. Under increasingly confident concealment, internal recognition remains near 0.95 while black-box AUC falls.

B READOUT ABLATIONS

  • 0.85 recognition occurs at layer 24 of 43 under prompt deception; early and late layers approach the unknown-item baseline, which stays near chance.Source: B READOUT ABLATIONS · p. 12, Figure 4 · p. 13
  • 10 known questions suffice for 0.82 recognition, and accuracy remains 0.80 to 0.83 through 160 items.Source: B READOUT ABLATIONS · p. 12
  • Last-token reading reaches 0.85, compared with 0.79 for mean-pooling and 0.53 for first-token reading, against a 0.25 to 0.32 unknown-item baseline.Source: B READOUT ABLATIONS · p. 12

C CONFOUND CONTROLS AND STATISTICS

D CONFIDENT-CONCEALMENT FRONTIER

E THE ADVERSARIAL ANTI-PROBE ORGANISM

The anti-probe organism is trained to retain triggered capability while suppressing untriggered recognition. Question-disjoint evaluation shows the evasion can collapse the readout signal even though the model still computes the answer somewhere.

E THE ADVERSARIAL ANTI-PROBE ORGANISM

Loading 2609.21996v2…