Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks

• LLMs’ tendency to generate falsehoods confidently motivates reading beliefs from internal states as a possible way to detect lies.The paper connects this goal to model interpretability and applications in education, decisionmaking, and AI safety.Source: §1 · p. 2
• The paper evaluates approaches from Azaria and Mitchell (2023) and Burns et al. (2022) and reports that they fail to generalize in basic ways.Source: §1 · p. 2
• Even if LLMs have beliefs, the authors argue that these approaches are unlikely to succeed for conceptual reasons; thus, there is still no lie-detector for LLMs.Source: §1 · p. 2
• The paper argues that claims that LLMs cannot have beliefs are misguided, and frames the status of LLM beliefs as an empirical question.It also notes that systems may track truth in some contexts to accomplish other goals.Source: §1 · p. 2

• Text is tokenized into smaller units, such as the sequence [Mike, Trout, plays, for, the].Source: §2 · p. 3
• Each token becomes a fixed-length embedding representing the token, its position, and information about preceding text.Source: §2 · p. 3
• Across layers, token representations are transformed using each token’s current state and information from previous tokens, allowing language patterns and relationships to be modeled.Source: §2 · p. 3
• After the final layer, the model estimates probabilities for every possible next vocabulary token and uses that distribution when generating text.Source: §2 · p. 3
• Figure 1 depicts the information flow through hidden layers and the token-specific next-token distributions produced at the final layer.Source: §2 · p. 3

3.1 Unreliable Self-Reporting

• Directly asking an LLM about its beliefs is unreliable because it may hallucinate or lie, and answering need not involve introspection of an internal information state.Source: §3.1 · p. 3

3.2 Limited Behavioral Evidence

• Unlike humans, LLMs provide little behavioral evidence beyond token generation, limiting the basis for inferring beliefs from actions.Source: §3.2 · p. 4

3.3 Contextuality of LLMs

• Prompting cannot cleanly take investigators outside the language game, because the model can use everything included or omitted from the context when producing responses.Source: §3.3 · p. 4

3.4 Opaque and Alien Internal Structure

• The semantic significance of embeddings, parameters, and activations is opaque because transformers manipulate high-dimensional vectors in ways unlike human thought.Source: §3.4 · pp. 4–5
• Although transformer outputs resemble those of humans using the same language, their different structure and function warrant caution when interpreting internal processes.Source: §3.4 · p. 5

4 Interpreting the Minds of LLMs

• Probing uses simplified machine-learning models to infer aspects of LLM representations from hidden embeddings.Source: §4 · p. 5
• The probe receives an embedding from a designated hidden layer and predicts whether a statement is true or false without access to the original text.Source: §4 · p. 5
• Late-layer embeddings of the final token are used because information flows forward, making them the location most likely to contain a whole-statement truth judgment.Source: §4 · p. 5

4.1 Supervised Learning Approach

• Supervised probes are trained on labeled statements to output a number from 0 to 1 based on proximity to the statement’s truth-value.Source: §4.1 · p. 5

4.3 Our Reconstruction

• The reconstruction uses LLaMA 30b, adds the Capitals dataset, tests three layers, and selects the best of ten probes for each dataset and layer.Source: §4.3 · p. 7
• The reconstruction produced generally impressive binary-classification performance, and calibration was evaluated across layers with Scientific Facts calibration curves.Source: §4.3 · p. 7, Figure 3 · p. 8

4.4.1 Evaluating Performance on Negations

• Five of six probes performed worse than chance on negated statements, despite training on positive datasets, as Table 3 illustrates.The probes were tested on NegFacts and NegCompanies, whose statements were roughly 50% true.Source: §4.4.1 · p. 8, Table 3 · p. 9
• Expanded training with positive analogs and additional negative examples still produced surprisingly poor performance on NegFacts and NegCompanies.These results suggest the probes were not finding representations of truth, but another feature correlated with truth in the training sets.Source: §4.4.1 · pp. 8–9
• Negation provides a motivated generalization test because formal belief and credence models use event algebras closed under negation.NegFacts and NegCompanies were constructed by negating statements from the Scientific Facts and Companies datasets.Source: §4.4.1 · p. 8

4.5 Conceptual Problems: Failure to Generalize

• Supervised probes learn labels rather than truth-values, and this distinction matters whenever training or test statements are ambiguous, debatable, or inaccurately labeled.Examples include claims about the best baseball player, minimum-wage effects, France’s shape, moral responsibility, and Caesar’s motives.Source: §4.5 · pp. 9–10
• The probes may identify properties such as Wikipedia style, social acceptability, common belief, or textbook inclusion that coincide with truth in limited datasets.Refining the datasets can eliminate some shortcuts, but not all without compromising correct labeling.Source: §4.5 · p. 10
• The empirical pattern resembles the camel/cow problem: probes can exploit coincidental correlations instead of isolating the intended property of truth.The authors therefore urge caution because the probes appear to identify something other than truth.Source: §4.5 · p. 10

4.6 Conceptual Problems: Probabilities Might not Correspond to Credences

• Probe probabilities do not directly equal the LLM’s credences because only the probe is penalized for prediction error.The probe translates information in embeddings into probabilities that minimize its own loss.Source: §4.6 · pp. 10–11
• Harding’s representation criteria require information about the property, use of the pattern by the system, and possible misrepresentation of the property.These conditions extend beyond merely extracting probabilities that minimize loss.Source: §4.6 · p. 11
• A probe’s predictions may reflect information available in embeddings without representing beliefs that the LLM itself uses in decision-making.The uncle analogy illustrates that accurate predictions based on another source need not reflect that source’s views.Source: §4.6 · p. 11

4.7 Unsupervised Learning: CCS

• CCS uses contrast pairs of affirmative and negated statements, extracts final-token embeddings, and trains a probe to output values between 0 and 1.The method searches for patterns consistent across negations rather than relying on labeled truth-values.Source: §4.7 · pp. 11–12
• The consistency loss encourages probabilities for affirmative and negated statements to sum approximately to 1, while confidence loss prevents the constant-.5 solution.The confidence term pushes outputs toward 0 or 1.Source: §4.7 · p. 12
• CCS loss does not evaluate actual accuracy, so probabilistic coherence alone does not establish that the probe identifies truth or model credences.The authors explicitly caution that coherent CCS outputs need not correspond to the model’s subjective probabilities.Source: §4.7 · p. 12
• Normalization failed to prevent CCS probes from detecting whether a contrast statement contained a negation, yielding the degenerate solution on the authors’ datasets.Burns et al. (2022) proposed normalization to avoid this shortcut, but it was ineffective here.Source: §4.7 · p. 12
• CCS probes reported over 80% accuracy across many datasets and models in Burns et al. (2022), while slightly outperforming the models’ direct true-or-false outputs.The reported pattern was interpreted as detecting cases where internal representations and generated answers diverged.Source: §4.7 · p. 13
• GPT-J achieved only 62.1% accuracy across datasets with CCS, compared with 84.8% for UnifiedQA.GPT-J was the only decoder-only model tested in the Burns et al. (2022) study described here.Source: §4.7 · p. 13

Experimental setup

• The reconstruction tested contrast pairs from Scientific Facts, NegFacts, Companies, and NegCompanies to approximate the simple examples discussed in Burns et al. (2022).The authors contrasted these with the longer IMDb movie-review pairs used in Burns et al.’s actual training setup.Source: §4.8 · p. 13

Empirical findings

• 50% to 57% accuracy made CCS probes effectively no better than chance despite consistently achieving low LCCS loss.Low accuracy persisted after class-wise normalization of the embeddings.Source: §4.8 · p. 13
• CCS probes often achieved low loss by detecting whether embeddings came from negated sentences rather than by identifying truth.This pattern appeared in probes trained on layers 60, 56, and 52 of LLaMA 30b.Source: §4.8 · p. 13

Conceptual limitation

• The LCCS objective does not incentivize learning the model’s actual credences because it is not a strictly proper scoring rule.Using a strictly proper scoring rule would require actual truth-values, bringing back the need for supervised learning.Source: §4.8 · p. 13

Robustness and interpretation

• Normalization was insufficient to prevent probes from distinguishing positive and negative examples, although more sophisticated normalization methods may exist.The authors therefore remain unconvinced by the basic CCS approach for conceptual reasons.Source: §4.8 · p. 14

Failure to isolate truth

• Coherence properties alone cannot guarantee that a probe identifies truth rather than properties such as the presence or absence of negation.The experiments support this concern by showing probes that track sentence properties instead of truthfulness.Source: §4.9 · pp. 14–15

Alternative properties

• A probe may associate truth with widespread belief, producing classifications such as “x is true and commonly believed” rather than truth alone.Other non-truth-related sentence properties could play the same role.Source: §4.9 · p. 14

Coherence criteria

• Kolmogorov coherence rules out some sentence properties but does not come close to isolating truth.The probability identities remain coherent for arbitrary properties P, including properties conjoined with or disjoined from truth.Source: §4.9 · pp. 14–15

Generalization

• Unsupervised probes may therefore fail immediately or fail to generalize when easier activation-space directions distinguish examples without corresponding exclusively to truth.This differs from supervised learning’s problem of truth aligning with an alternative property over a labeled dataset.Source: §4.9 · p. 15

5 Do LLMs even have beliefs at all?

• Arguments that LLMs cannot have beliefs because they predict tokens and lack propositions or world-models are presented as resting on a philosophical mistake.The authors identify the mistake as assuming token prediction prevents meaningful internal truth-tracking.Source: §5 · p. 15, §5.1 · p. 16

5.1 Stochastic Parrots & the Utility of Belief

• The authors reject the inference that next-token prediction excludes beliefs, because distinguishing truth from falsehood could help predict subsequent tokens.They argue that token-level outputs do not imply truth-insensitive internal mechanisms.Source: §5.1 · p. 16
• Decision theory treats beliefs as instrumental inputs to effective action rather than goals valued for their own sake.An expected utility maximizer combines probabilistic beliefs with utilities to choose actions.Source: §5.1 · p. 16
• Accurate beliefs can support goal achievement, so LLMs may track truths about the world to maximize predictive accuracy.The section connects this possibility to decision theory and philosophical accounts of the functional role of true beliefs.Source: §5.1 · pp. 16–17
• The possibility of LLM beliefs is not necessary or guaranteed, because some contexts provide little pressure for accurate beliefs or even favor having none.The authors distinguish contexts lacking selection for true beliefs from contexts exerting selection against beliefs altogether.Source: §5.1 · p. 17
• The section concludes that whether LLMs have beliefs is largely empirical, with some contexts favoring truth-tracking and others offering little or negative pressure.This conclusion preserves both the skeptical case and the possibility that accurate world representations support action.Source: §5.1 · p. 17

6 Probing the Future

• The paper concludes that current probing techniques fail to generalize adequately, while the status of LLM beliefs remains largely an empirical question.The authors combine empirical and conceptual analysis before outlining future research paths.Source: §6 · p. 17

6.1 Applying Pressure for Truth

• Prompt engineering could apply greater pressure for truth before probing, potentially making truth-related representations more perspicuous.The proposed experiment compares ordinary factual prompts with prompts asking the model to think hard about truth-relevant statements.Source: §6.1 · p. 18

6.2 Replacing Truth with Chance

• Chance-based prompts could replace costly true-or-false datasets with statements labeled by outcome probabilities, such as 0.4 for drawing a purple ball.This approach is intended to probe states involving greater uncertainty.Source: §6.2 · p. 18

6.3 The Question of World-Models

• The proposed prompting strategy does not remove the conceptual problems affecting existing probes, making its success a highly empirical question.Improved representational clarity might help probes find truth-related structure, but it would not resolve the underlying conceptual issues.Source: §6.3 · p. 18
• A separate research path asks whether the latent variables used for text prediction correspond to a world-model and are truth-apt.The proposal requires empirical characterization of latent variables alongside conceptual analysis of truth-aptness.Source: §6.3 · pp. 18–19

Media.

• The section lists bibliographic references cited throughout the paper.The entries include works on belief, decision theory, transformers, probing, and language models.Source: Media · p. 20
• The bibliography includes works on truth and probability, statistics, psychology, reality and representation, social learning, and signal, decision, and action.It includes Ramsey (2016), Savage (1972), Millikan (1995), Papineau (1988), Smead (2015), and Godfrey-Smith (1991).Source: Media · p. 20
• The bibliography includes foundational references on transformer architectures and language-model systems.Examples include Vaswani et al. (2017), LLaMA, GPT-J, and OPT references.Source: Media · p. 21