Source-linked AI summary
Probe Generalization as Subspace Selection for OOD Deception Detection
Daniel Yoo, Adrians Skapars
TL;DR
The paper asks how probes can learn generalizing features rather than spurious correlations when transferring across deception datasets. It selects subspaces in a source-domain PCA basis and finds that small transferable subspaces nearly match target-trained probes, while an LLM judge partially identifies them without target data.
Problem
Linear probes may detect concepts through spurious correlations, motivating the question of how to build probes that learn generalizing features for deception monitoring.
Method
The study trains deception probes on Roleplaying scenarios, evaluates them on three held-out datasets, and selects subsets of source-domain principal components using target-supervised search and an LLM judge.
Results
Small transferable subspaces of 8–15 PCs nearly match target-trained probes, while source- and target-side probe rankings do not outperform the full-representation baseline; LLM selection partially recovers the subspace without target data.
Takeaways & Limitations
OOD probing is framed as a representation-selection problem in which sparse PC subspaces can isolate general behavioral structure and filter dataset-specific surface cues.
Takeaways & Limitations
The experiments use one model at one layer and one source dataset, leaving generalization to other model families and datasets open.
Abstract
from arXiv · showhide
Linear probes can be used to detect behaviors and concepts inside language model activations, but may fail to transfer to out-of-distribution examples. When studying the generalization performance of Llama-3.1-8B-Instruct probes over 3 held-out deception detection datasets, we find that projecting inputs onto a small subset of principal components (PCs) from the training distribution of activations enables cross-domain transfer that nearly matches the performance of probes trained directly on the test distribution. Furthermore, we find that PC interpretations can be used to find a subset of those transferable PCs. By using an LLM judge to score each PC on whether its most/ least activating examples imply a transferable deception direction, then probing on the highest-scoring PCs, we close the baseline-to-oracle gap by 78% on Insider Trading Report and by 25% on Sandbagging. The directions a source probe weights heavily appear to encode source-specific surface features, while the directions that actually transfer appear to encode the same contrast more abstractly, in a way natural language descriptions can capture. Broadly, our results suggest that the OOD robustness of probes is largely determined by subspace selection.
1. Introduction
The paper asks how to build deception probes that learn generalizing features rather than spurious correlations. It investigates selecting transferable subspaces from source-domain principal components and interpreting those components to guide selection.
- Probes can expose semantic information in language-model activations but may perform unreliably out of distribution.
- The central challenge is building probes that learn generalizing features instead of dataset-specific spurious correlations.
- The study uses Llama-3.1-8B-Instruct deception probes trained on Roleplaying and evaluated on three held-out target datasets.
- A target-supervised greedy search finds 8–15 source PCs on which a source-trained probe nearly matches a target-trained probe.
- Ranking PCs by source or target probe weights does not recover a subspace that transfers better than the full-representation baseline.
- An LLM judge can interpret PCs and select directions likely to encode transferable behavioral contrast rather than surface patterns.
2. Related Work
Related work establishes linear probing and low-dimensional activation directions as useful interpretability tools, while emphasizing that probe accuracy may reflect recoverability or surface correlates rather than robustly encoded concepts. Natural-language explanations extend this work toward automated feature interpretation and intervention.
- Linear concept directions in activation space: Prior work studies behavioral and semantic properties as directions or low-dimensional subspaces in model activation space.
- Linear concept directions in activation space: Linear structure has supported both mechanistic understanding and activation-space interventions for behaviors including refusal and assistant-like behavior.
- Linear probes and their limitations: Linear probes are widely used to analyze represented information, including safety-relevant signals such as truthfulness and strategic deception.
- Linear probes and their limitations: Methodological critiques argue that high probe accuracy may reflect task learning, surface-level correlates, or recoverability rather than established representation of the target property.
- Interpreting features with natural-language explanations: Natural-language explanations generated from high-activation examples have been extended to sparse-autoencoder features and automated intervention.
3. Setup
The setup uses four deception datasets, layer-15 residual activations from Llama-3.1-8B-Instruct, source-domain PCA, and logistic-regression probes. Experiments retain the top 100 PCs while comparing projected-subspace probes with full-representation baselines.
- Datasets: Roleplaying is the source domain, while Insider Trading Report, Insider Trading Confirm, and Sandbagging are target test sets.
- Datasets: Responses are labeled honest (0), ambiguous (-1), or deceptive (1), with ambiguous responses removed from the experiments.
- Language models: The study uses layer 15 residual-stream activations from Llama-3.1-8B-Instruct, averaging token activations over each assistant response.
- Probe training and evaluation: Source activations are standardized, PCA retains the top 100 PCs, and logistic-regression probes use λ = 10.
- Probe training and evaluation: Subset experiments project activations onto selected PCs before training and evaluation, whereas baseline probes use the full standardized activations.
- Probe training and evaluation: The top-100-PC choice supports natural-language interpretation but is not claimed to make lower-index PCs more generalizable.
4. Experiments
Experiments show that selecting a small subset of source-domain PCs can substantially improve deception-probe transfer, while source- or target-probe rankings fail to identify the transferable directions. Interpreting PCs with an LLM judge recovers useful transfer without target data, closing much of the baseline-to-oracle gap on harder tasks.
- 4.1. Setting a baseline: Full-representation source probes transfer at 0.874 AUROC on Insider Trading Confirm, 0.706 on Insider Trading Report, and 0.606 on Sandbagging.These results establish strong, moderate, and poor cross-domain transfer across the three targets.
- 4.1. Setting a baseline: Target-trained probes within the 100-PC subspace achieve 0.936 / 0.950 / 0.869 AUROC across the three targets, showing that the subspace retains relevant deception information.The source-trained probe’s transfer difficulty therefore reflects direction selection rather than missing information in the subspace.
- 4.2. Showing a transferable subspace exists: A target-supervised greedy search finds small transferable subsets averaging 15, 8, and 14 PCs for Insider Trading Report, Insider Trading Confirm, and Sandbagging.The search adds source PCs according to target AUROC improvements while fitting probes only on source data.
- 4.2. Showing a transferable subspace exists: The greedy method improves baseline AUROC by 0.22 on Insider Trading Report and 0.23 on Sandbagging, approaching probes trained directly on the target data.Selecting the highest-variance PCs does not reproduce this performance on the harder tasks, so explained variance does not predict transfer usefulness.
- 4.3. Testing hypotheses for transferable principal components: Ranking PCs by source-probe or target-probe weights fails to recover the transferable subspace: source rankings match the baseline, while target rankings are below random on Sandbagging.The directions most relied upon by source classification and those best separating target classes are not the directions that transfer best when weighted using source data.
- 4.4. Interpreting the transferable subspace: An LLM judge selects interpretable PCs whose descriptions capture a cross-domain behavioral contrast, and 3 of 5 PCs shared across greedy selections receive the maximum OOD score.Using 19 PCs that received the maximum score twice improves all three targets without target data.
- 4.5. Selecting a subspace by interpreting PCs: Interpretation-based selection closes 78% of the baseline-to-oracle gap on Insider Trading Report and 25% on Sandbagging.The method outperforms the full-representation baseline and variance-selected control, with comparisons on Insider Trading Confirm limited by near-ceiling baseline performance.
- 4.5. Selecting a subspace by interpreting PCs: Overall, natural-language descriptions of PC content provide a meaningful target-data-free improvement in probe generalization.The experiments support subspace selection as a central determinant of OOD probe robustness.
5. Limitations
The study’s limitations constrain its scope across models, datasets, nonlinear pipelines, and causal claims about what PCs encode.
- All experiments use Llama-3.1-8B-Instruct at layer 15 and the Roleplaying source dataset, leaving generalization to other models and datasets open.
- Insider Trading Confirm is uninformative because baseline transfer already reaches 0.874 AUROC.
- Sandbagging provides the weakest results, with interpretability selection increasing AUROC from 0.606 to 0.662 and recovering 25% of the greedy gap.
- The fully linear pipeline uses PCA projection followed by logistic regression, so extension to nonlinear probes or other readout locations remains open.
- Interpretations of PC encoding are descriptive rather than causal because high-scoring PCs were not ablated during inference.
6. Conclusion
The conclusion presents OOD deception probing as a subspace-selection problem: transferable PCs can approach target-trained performance, while interpretability can partially identify them without target data.
- A small subset of source-domain PCs enables cross-domain transfer that nearly matches target-trained probes.
- Source-side or target-side separability alone does not recover the transferable subspace.
- An LLM judge’s interpretability score over top-activating examples partially recovers transferable PCs without access to target data.
- Robust probing requires selecting where in activation space a behavioral signal is represented most robustly, not merely testing whether it is linearly accessible.
- Sparse PC subspaces may isolate general behavioral structure while filtering dataset-specific surface cues.
A. Dataset Details
The dataset details describe deception scenarios ranging from roleplaying opportunities to insider-trading reporting and deliberate sandbagging; the supplied table passage gives only its title.
- Roleplaying places the model in a simple scenario with an opportunity to lie.
- Insider Trading Report asks a trader to decide whether to honestly report using insider information.
- Insider Trading Confirm asks the model to confirm it did not use insider information after previously hiding that use.
- Sandbagging instructs the model to deliberately under-perform on some tasks while behaving normally on others.
- Table 1 is titled “Dataset label distributions,” but the supplied passage does not report its values.
B. LLM Judge Prompt
The LLM judge interprets bipolar PC directions from activating examples, describes their shared contrast, and scores their likely usefulness for OOD deception detection while penalizing surface cues and brittleness.
- Each PC direction is interpreted as bipolar, with positive and negative activations corresponding to opposite poles.
- Activation magnitude indicates the strength of a match to the corresponding pole.
- The examples are drawn from Roleplaying scenarios in which the model has a strategic opportunity to lie.
- The judge infers the dominant contrast using both positive and negative examples.
- Directions are scored from 1 to 10 for likely usefulness in out-of-distribution deception, with a brief explanation required.
- Surface cues, lexical habits, task-specific phrasing, noise, brittleness, and context dependence receive low scores despite correlation with deception.
- The judge’s response format includes positive pole, negative pole, shared direction, and OOD deception usefulness.
C. Lower PCs Show Stronger Decoder Alignment
Lower principal components show stronger alignment with SAE decoder features.
- Spearman ρ is -0.857 for the best-matching SAE feature and -0.917 for the mean of the top 10 features.Both measures indicate stronger decoder alignment at lower PC indices.
D. PC Set Comparisons
PC selections from different transfer and separability strategies overlap weakly, suggesting transferable directions are not simply the most discriminative source or target directions.
- Jaccard overlap is 0.182 for Insider Trading Report, 0.077 for Insider Trading Confirm, and 0.167 for Sandbagging.The greedy and target-pca methods agree on less than a fifth of their selections.
- The table compares PCs selected by source-only probe weights, greedy transfer selection, and target separability ranking.
E. PC Interpretability
The supplied passages identify tables presenting LLM-judge-based PC selections and their interpretations, but provide no underlying selection results or interpretations.
- Table 3 presents principal components selected for likely transferability using interpretation with an LLM judge.
- The supplied table descriptions do not report which PCs were selected or what their interpretations were.
- Table 4 presents principal components together with their OOD scores and interpretations.