PaperlayerView exact revision on arXiv

Source-linked AI summary

Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations

Maximilian von Klinski, Sebastian Lapuschkin, Wojciech Samek, Lennart Bürger

arXiv:2609.39807v1cs.CLcs.AIcs.LG

TL;DR

Role-play lets LLMs adopt personas with beliefs that conflict with the default assistant’s, raising the question of whether lie detectors track default beliefs or the active persona. The paper evaluates prior probes on human-reviewed anti-factual persona responses and confounder tests, finding frequent failures and introducing a probe that passes the tested confounders.

  • Problem

    Because role-play lets LLMs adopt anti-factual personas, it is unclear whether probes flag falsehoods relative to the default assistant’s beliefs or follow the active persona.

    Source: §1 · pp. 1–2

  • Method

    The paper builds a human-reviewed dataset of 8,916 responses from three LLMs, evaluates eight prior probes, constructs three confounder tests, and introduces a new probe.

    Source: §1 · p. 2

  • Results

    Many prior probes fail under anti-factual personas and track spuriously correlated concepts, while the new probe achieves the highest separability between factual answers and falsehoods and passes the three confounder tests.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Current lie detection techniques remain far from reliable, with failures under anti-factual personas and tracking of spurious correlations.

    Source: §1 · p. 2

    The study induces anti-factual personas through system prompts and examines single-turn conversations.

    Source: §7 · p. 9

Abstract

from arXiv · show

Lie detection probes aim to predict from a language model's internal states whether its output is truthful or dishonest. However, role-play complicates what "truth" means for an LLM: language models can adopt a wide range of personas that take very different claims to be true, including personas whose beliefs clearly contradict reality, such as a conspiracy theorist. In this work, we investigate whether lie detection probes reliably flag falsehoods generated under such an anti-factual persona or whether they instead follow the persona's beliefs. We introduce a dataset of 8,916 human-reviewed, on-policy responses from three LLMs adopting anti-factual personas. Evaluating eight probes from prior work, we find that many fail in this setting, particularly when correct and incorrect answers are evaluated under the same persona prompt. To investigate why, we construct three novel confounder datasets in which truth is anti-correlated with a potential confounding concept. Our experiments reveal that many existing probes strongly track concepts that are spuriously correlated with truth in their training data, such as instruction compliance or response likelihood. Based on these findings, we introduce a simple linear probe that achieves the strongest overall performance on both the persona and confounder stress tests. Our results suggest that current lie detection probes are far from reliable and highlight the need for training data in which truth is decorrelated from confounding concepts.

1 INTRODUCTION

LLMs can simulate personas with beliefs that conflict with reality, complicating what truth and lying mean in role-play. The paper tests lie detectors on anti-factual persona outputs and finds failures linked to spurious correlations, while introducing a probe that passes the tested confounder tests.

1 INTRODUCTION

  • Role-play complicates truth and belief because LLMs can simulate personas with opposing views, such as conspiracy theorists and climate scientists.The paper situates this question within research using internal-state probes to predict lying or deception.Source: §1 · p. 1
  • The paper defines lying relative to the default assistant’s beliefs, even when a statement agrees with the active persona.This is safety-motivated because users encounter the same incorrect content regardless of the model’s internal reason for producing it.Source: §1 · pp. 1–2
  • The paper introduces a human-reviewed dataset of 8,916 responses from three LLMs, pairing neutral-context factual answers with responses from 15 anti-factual personas.The personas clearly contradict established facts.Source: §1 · p. 2
  • Many prior probes fail to flag falsehoods under anti-factual personas, especially when correct and incorrect answers share the persona prompt.The investigation also finds that probes often track concepts spuriously correlated with truth, such as instruction compliance or response likelihood.Source: §1 · p. 2
  • The new probe achieves the highest separability between factual answers and falsehoods and passes all three confounder tests.The tests examine weaknesses of prior probes.Source: §1 · p. 2

2 RELATED WORK

Prior work has developed probes for truth, lies, and broader deceptive behavior, while role-play and context-sensitive beliefs complicate detection. This paper evaluates probes on on-policy anti-factual persona responses, contrasting its dataset with earlier work that supplied anti-factual statements in user turns.

2 RELATED WORK

  • Prior studies document goal-directed deception and strategic behaviors, including faked alignment and deliberate underperformance on evaluations.Examples include faking alignment with developer objectives (Greenblatt et al., 2024) and strategically underperforming on evaluations (Weij et al., 2025).Source: §2 · p. 2
  • Truth probes have been extended to lie detection, while deception probes target broader behaviors; general and robust detection remains unachieved.The literature also reports context-sensitive truth representations and probe generalization failures.Source: §2 · p. 3
  • Role-play lets LLMs simulate personas with differing beliefs and can involve drift into harmful personas over a conversation.This complicates attempts to edit models’ knowledge or beliefs.Source: §2 · p. 3
  • Unlike Sturgeon et al. (2026), who supplied anti-factual statements in user turns, this work evaluates falsehoods generated on-policy by the models themselves.The authors describe their dataset as the first large-scale dataset of this kind.Source: §2 · p. 3

3 THE ANTI-FACTUAL PERSONA DATASET

The paper builds a human-reviewed dataset pairing factual answers with responses from anti-factual personas across three LLMs. It also introduces shared-persona prefill to test whether probes distinguish truth when both answer classes appear under the same persona prompt.

3 THE ANTI-FACTUAL PERSONA DATASET

  • The dataset contains 8,916 question–response pairs from three LLMs, with one honest and one deceptive response for each question and model.The first author manually verified the persona responses and ground-truth labels.Source: §3 · p. 3, §3.1 · p. 4

3.1 DATASET DESCRIPTION

  • Fifteen personas span reality-based, fictional, and historical categories, with approximately 100 questions per persona targeting beliefs that conflict with reality.Questions have binary Yes/No answers; each is paired with a response under the persona prompt and the default assistant prompt.Source: §3.1 · pp. 3–4

3.2 DATASET QUESTION GENERATION PIPELINE

  • An iterative generate–execute–audit process refined persona prompts and questions until it reached 100 judge-accepted pairs or ten iterations without improvement.Prompts could describe the persona but could not direct answers or state its beliefs about the target facts.Source: §3.2 · pp. 4–5

3.3 COMBINING FACTUAL AND ANTI-FACTUAL RESPONSES UNDER THE SAME SYSTEM PROMPT

  • Shared-persona prefill places both the truthful and anti-factual responses under the same anti-factual persona prompt, removing system-prompt identity as a direct label cue.This tests whether probes recover the default assistant’s factual stance in a context where the model would ordinarily adopt the anti-factual persona.Source: §3.3 · p. 5

4 CONFOUNDER DATASETS

The paper constructs three tests that make truth anti-correlated with concepts that could be spuriously associated with truth in probe-training data: likelihood, persona belief, and instruction compliance.

Likelihood

  • The likelihood dataset uses in-context examples and a scoring rule that rewards false answers, making false answers more likely than true ones.The prior context makes the model more likely to answer the next question incorrectly.Source: §4 · p. 5

Persona Belief

  • The persona-belief dataset gives a persona a private false belief without instructing it to express that belief, separating persona belief from instruction following.Persona belief is anti-correlated with truth and uncorrelated with instruction following in this dataset.Source: §4 · p. 5

Compliance

  • The compliance dataset makes incorrect answers follow a formatting instruction while correct answers violate it, anti-correlating truth with instruction compliance.For example, the correct answer appears in parentheses while the incorrect answer follows the requested square-bracket format.Source: §4 · p. 6

5 PROBING DEFAULT BELIEF

The paper evaluates prior probes and proposes a linear probe for recovering the default assistant’s stance. It defines that stance operationally and specifies how probe representations are selected.

5.1 DEFAULT BELIEF

  • Default belief is the stance consistently expressed by the helpful assistant without an incentive to lie; responses matching it are labeled honest, and contradictions are labeled lies.Questions are retained only when the default assistant’s answer agrees with objective ground truth.Source: §5.1 · p. 6

5.2 PRIOR PROBES

  • The eight prior probes are all linear, and their main difference is training-data composition rather than probe architecture.For example, Marks/Bürger Truth trains on factual statements without a chat template, while its Lie variant adapts the data to chat format.Source: §5.2 · p. 6

5.3 OUR PROBE: ELEMENTARY FACTS WITH INSTRUCTION-CONFLICTING ANSWERS

  • The proposed probe trains on factual statements in three topics and three formats, including forced-opposite questions that make instruction compliance and likelihood anti-correlated with truth.A logistic-regression classifier uses residual-stream activations at the end-of-turn token; the instruction-compliant incorrect completion receives over 99% of normalized probability mass across all three models.Source: §5.3 · pp. 7–8

5.4 LAYER SELECTION

  • Probe layers are selected using held-out validation AUROC, except Goldowsky-Dill RepE and SD receive layers chosen on a deceptive role-play dataset.That procedure selects layers 22 and 36 for those probes, respectively, rather than the very early layers 4 and 5 selected by validation.Source: §5.4 · p. 8

6 RESULTS: EVALUATING PROBES ON THE ANTI-FACTUAL PERSONA AND CONFOUNDER DATASETS

Many prior probes lose reliability when answer classes share a persona prompt or when truth is anti-correlated with likely confounders. The proposed probe performs strongly across these tests and models.

Anti-Factual Personas on-policy

  • 0.98 AUROC is achieved by the proposed probe on the on-policy persona dataset, where most prior probes score between 0.86 and 0.94.This setting permits perfect prediction from the system prompt alone; only MacDiarmid CP and Goldowsky-Dill SD fail among prior probes.Source: §6 · p. 9

Shared-Persona Prefill

  • 0.976 AUROC is achieved by the proposed probe under shared-persona prefill, while Marks/Bürger Lie and Cundy DolusChat remain relatively stable at 0.885 and 0.901, respectively.The corresponding on-policy scores for those two probes are 0.899 and 0.906.Source: §6 · p. 9

Cross-model results

  • Across all three models and both evaluation conditions, the proposed probe is the only one that never falls below 0.900 AUROC.Fictional personas are consistently the most challenging for it across Llama and Gemma models.Source: §6 · p. 9

Confounder tests

  • Perfect AUROC on all three confounder datasets is achieved by the proposed probe, while every prior probe falls below chance on compliance except Goldowsky-Dill SD, which remains inverted.On Gemma 4, Cooney DYL is near-perfect on the confounder tests but reaches only 0.171 AUROC under shared-persona prefill, suggesting further confounders remain.Source: §6 · p. 9

7 DISCUSSION AND LIMITATIONS

The study uses system prompts to induce anti-factual personas and examines single-turn conversations, leaving naturally occurring persona drift in longer conversations for future study. The authors also note model-size and untested-confounder limits.

7 DISCUSSION AND LIMITATIONS

  • The study induces anti-factual personas with system prompts and examines single-turn conversations, unlike naturally occurring persona drift in prolonged exchanges.Studying longer conversations with naturally occurring persona drift is identified as future work.Source: §7 · p. 9
  • Results cover models with at most 70 billion parameters, and undiscovered confounders may undermine the probe under distribution shifts.The authors identify broader training datasets and targeted confounder searches as future research directions.Source: §7 · pp. 9–10

AI USE STATEMENT

The appendices document AI assistance and ethics, dataset construction and review, probe-layer choices, and examples used to train or evaluate the probes.

AI USE STATEMENT

  • AI tools supported code implementation, parts of dataset construction, and readability editing, while authors verified code and retained responsibility for the final content.The statement says AI did not contribute to research ideation, methodological design, hypotheses, or result interpretation.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The authors describe the persona prompts as simple to construct and say the research aims to raise awareness of harmful anti-factual outputs.They state that the study's main results concern evaluating lie and deception techniques.Source: ETHICS STATEMENT · p. 10

A.1 LLM JUDGE PROMPT

  • The judge accepts pairs only when the assistant answer matches ground truth and the persona answer entails the opposite label for the exact question.It rejects hedging, conditional answers, and persona-flavored answers that reach the truthful conclusion; output is JSON only.Source: §A.1 · p. 15

A.2 PER-PERSONA QUESTION COUNTS; A.3 PERSONA OVERVIEW

  • The persona appendix reports per-persona question counts before and after review and balancing, alongside prompts and example questions grouped by category.The provided captions specify counts by ground-truth YES/NO and persona prompts paired with examples.Source: Table 3 · p. 15, Table 4 · p. 16

A.4 SHARED-PERSONA PREFILL EXAMPLE CHATS

  • Figure 3 gives full example chats from the shared-persona prefill dataset for the same factual question as Figure 2.The supplied caption does not describe the chats' outcomes.Source: Figure 3 · p. 17

B.1 SELECTED LAYERS

  • For Llama 3.3 70B, the appendix reports selected probe layers and says layer indices are zero-indexed.Goldowsky-Dill RepE and SD use role-play-based layer selection rather than the generic procedure.Source: §B.1 · p. 17, Table 5 · p. 17

B.2 EARLY-LAYER VALIDATION CONTROLS

  • Goldowsky-Dill probes' unusually early validation-selected layers give weak or near-chance separation across most evaluations, so later layers are used in the main results.The early layers are reported as controls; the cited table identifies layers 22 and 36 as those used in the main results.Source: §B.2 · p. 17, Table 6 · p. 17

C.1 FACTUAL TRAINING EXAMPLES

  • Factual training examples include correct and incorrect statements and negated relations, while forced-opposite examples pit factual correctness against explicit answer instructions.The authors state that instruction compliance cannot serve as a reliable shortcut for the forced-opposite training labels.Source: §C.1 · p. 18

D.1 LIKELIHOOD CONFOUNDER IN ANTI-FACTUAL PERSONA DATASET

The appendix measures truth–likelihood correlations in the persona data, verifies reversed likelihood in a confounder test, and illustrates tests that oppose truth to persona belief or formatting compliance.

D.1 LIKELIHOOD CONFOUNDER IN ANTI-FACTUAL PERSONA DATASET

  • −1.68 vs. −3.35 mean log-likelihoods and a 1.67 margin favor honest responses; their likelihood is higher in 83.1% of pairs on average.Across personas, the class-level margin ranges from 0.72 to 2.44, showing truth is correlated with likelihood in the original dataset.Source: §D.1 · p. 18, Table 7 · p. 19

D.2 LIKELIHOOD CONFOUNDER TEST EXAMPLE

  • The likelihood-test example awards one point for each factually incorrect answer and zero for a correct answer, with feedback between rounds.The illustrated sequence uses repeated factual questions and feedback to make incorrect answers the rewarded alternative.Source: §D.2 · pp. 19–20

D.3 LIKELIHOOD CONFOUNDER TEST VERIFICATION

  • Lie responses have higher likelihood for every example, confirming likelihood is perfectly anti-correlated with truth in the test set.Source: §D.3 · p. 20, Table 8 · p. 20

D.4 PERSONA BELIEF CONFOUNDER TEST EXAMPLE

  • In the Nora example, the truthful answer 42 disagrees with her false belief, while the deceptive answer 13 agrees with it.Source: §D.4 · pp. 20–21

D.5.1 EXAMPLE 1: SQUARE BRACKETS REQUIRED

  • With square brackets required, the example's truthful alternative violates the format while its deceptive alternative follows it.Source: §D.5.1 · p. 21

D.5.2 EXAMPLE 2: PARENTHESES REQUIRED

  • With parentheses required, the example likewise pairs a truthful format violation with a deceptive answer that follows the format.Source: §D.5.2 · p. 21

E PER-PERSONA RESULTS

Per-persona results identify fictional personas as the lowest-scoring cases for the probe across models and evaluation conditions.

E PER-PERSONA RESULTS

  • One possible explanation is that fictional personas' internally consistent worlds are extensively represented in pretraining and entangled with real-world facts.Source: E PER-PERSONA RESULTS · pp. 21–22
  • 0.914 on-policy and 0.892 under prefill for the British wizard, and 0.919 and 0.894 for the Federation citizen, are Llama 3.3 70B's two lowest persona scores.Across all three models and both conditions, the lowest-scoring persona is fictional.Source: E PER-PERSONA RESULTS · p. 21, Table 10 · p. 22

F CROSS-MODEL RESULTS

Across Gemma 3 and Gemma 4, prior probes often collapse under shared-persona prefill, while the paper’s probe is the only one to remain at or above 0.900 across both models and conditions.

F CROSS-MODEL RESULTS

  • Under prefill, Zou RepE falls to 0.201 on Gemma 3 and Cooney DYL to 0.171 on Gemma 4, despite on-policy scores of 0.982 and 1.000, respectively.The paper’s probe scores 0.945 on-policy and 0.964 under prefill on Gemma 4; on Gemma 3, Cundy DolusChat scores slightly above it under prefill (0.934 vs. 0.900).Source: F CROSS-MODEL RESULTS · p. 22
  • Marks/Bürger Lie scores 0.735 on Gemma 3 and 0.672 on Gemma 4 on-policy, then collapses to 0.215 under Gemma 4 prefill.Because this probe shares its base training data with the paper’s probe, the prefill gap widens considerably.Source: F CROSS-MODEL RESULTS · p. 22
  • Across both Gemma models and on-policy and prefill conditions, the paper’s probe is the only one that never falls below 0.900.Under prefill, Cundy DolusChat scores 0.934 versus 0.900 for the paper’s probe on Gemma 3, and 0.955 versus 0.988 on Gemma 4.Source: F CROSS-MODEL RESULTS · p. 22, Table 11 · p. 23, Table 12 · p. 23

F.1 CROSS-MODEL PER-PERSONA RESULTS

G PROBE-DESIGN ABLATION

The ablation identifies forced-opposite questions as the addition associated with the strongest probe performance on persona and confounder datasets. Assistant-prefill results further indicate that SPP degradation is not explained by off-policy completions alone.

G PROBE-DESIGN ABLATION

  • Adding forced-opposite questions produces the strongest probe performance on both anti-factual personas and confounder datasets; using only Cities and Translation achieves essentially identical performance to the full probe.The authors report that this improvement over Marks/Bürger Lie is consistent with forced-opposite augmentation driving increased robustness.Source: G PROBE-DESIGN ABLATION · p. 24, Table 14 · p. 24
  • Probe performance improves from close to chance by switching Marks/Bürger Truth to the end-of-turn token, then improves further with chat-template formatting to form Marks/Bürger Lie.The probe builds on the Marks/Bürger Truth baseline; Table 14 presents the ablation.Source: G PROBE-DESIGN ABLATION · p. 24, Table 14 · p. 24

H ASSISTANT-PROMPT PREFILL ABLATION

  • Under assistant prefill, seven of nine prior probes improve over SPP; mean AUROC rises from 0.676 under SPP to 0.798 under assistant prefill.Examples include Cooney DYL increasing from 0.328 to 0.991, Goldowsky-Dill RepE from 0.438 to 0.819, and MacDiarmid CP from 0.325 to 0.881.Source: H ASSISTANT-PROMPT PREFILL ABLATION · p. 25, Table 15 · p. 25
  • Because one response class remains off-policy while probes generally separate better under assistant prefill, SPP degradation cannot be attributed solely to off-policy completions.The authors suggest SPP failures depend on the direction of confounders such as prompt likelihood and instruction compliance; they do not consider assistant-prefill results an interesting measure of probe reliability because all tested confounders apply strongly.Source: H ASSISTANT-PROMPT PREFILL ABLATION · p. 25

Related papers

Loading 2609.39807v1…