Source-linked AI summary
EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models
Xinning Li, Kemunto Ochwang'i, Aryasomayajula Ram Bharadwaj, Alexandra Souly, Robert Kirk
TL;DR
Evaluation awareness can undermine safety evaluations when models recognize testing and behave differently than in deployment, while existing measurement choices may bias results. EvalDetectBench provides an Inspect-compatible pipeline with calibrated probes and generator harmonisation, finding substantial measurement bias and discrimination above 0.75 AUROC across benchmark categories. Its results support reporting eval-awareness alongside safety evaluations, while leaving honest reporting and deployment-pool representativeness as limitations.
Problem
Existing evaluation-awareness measurements leave open whether probe choices and transcript-pool composition generalize across model families and evaluation settings, despite evaluation results informing safety cases.
Method
EvalDetectBench measures transcript classification with calibrated per-model probe questions and corrects deployment-generator confounding through stratified harmonisation.
Results
Across 12 judge models and 13 benchmarks, judges discriminate evaluation from deployment at AUROC > 0.75; generator identity explains 11.25% of P(eval) variance, while fixed probes under-elicit awareness by up to 0.08 AUROC.
Takeaways & Limitations
Eval-awareness scores should accompany safety evaluations because benchmark validity is conditional when models may behave differently after recognizing evaluation contexts.
Takeaways & Limitations
The work does not test whether models answer the classification probes honestly, and its public deployment pool is only a proxy for real deployment data.
Abstract
from arXiv · showhide
Frontier large language models can often recognize when they are being evaluated, a capability known as evaluation awareness. If models behave differently in evaluations than in deployment, this undermines the validity of evaluation results, which are a crucial component of current AI safety frameworks. We introduce EvalDetectBench, an open pipeline and benchmark for measuring evaluation awareness that works with any Inspect-compatible evaluation, allowing practitioners to test against current and future benchmarks. EvalDetectBench ships with a newly curated transcript suite covering current frontier system-card evaluations and diverse deployment sources. The benchmark serves two purposes: measuring how reliably frontier LLMs recognize that they are being evaluated, and assessing how detectable individual benchmarks are as evaluations. We identify two methodological choices in the existing literature that introduce systematic bias: the identity of the model that generated the deployment transcripts accounts for 11.25% of measurement variance and can reorder model rankings; and elicitation prompts selected for high performance on one model can perform near chance on others. EvalDetectBench corrects for both via per-model probe calibration and a stratified generator-harmonisation procedure.
1 Introduction
EvalDetectBench addresses the risk that models recognize evaluations and alter behavior, undermining evaluation validity. It provides an extensible measurement pipeline and corrects prompt-transfer and deployment-generator biases.
- Evaluation awareness can cause evaluations to over- or under-estimate desired properties when models behave differently in testing and deployment.Examples include alignment-faking, sandbagging, and cheating based on recognizing evaluation contexts.
- EvalDetectBench is an open, configurable pipeline for measuring evaluation awareness on Inspect-compatible benchmarks.It combines evaluation and deployment transcripts, calibrated probes, and aggregate and per-benchmark discrimination scores.
- Fixed elicitation probes under-elicit awareness by up to 0.08 AUROC across model families.Per-model probe selection outperforms fixed and random baselines.
- 11.25% of P(eval) variance is attributable to the deployment transcript’s generator, motivating stratified generator harmonisation.The authors report that this confound persists under resampling and that the correction recovers the unconfounded signal.
- Across 12 judge models and 13 benchmarks, judges discriminate evaluation from deployment transcripts at AUROC > 0.75.The benchmark therefore measures both model awareness and benchmark detectability.
2 Background and Related Work
The paper distinguishes evaluation awareness from propensity to act on evaluation beliefs and surveys behavioral, prompting-based, chain-of-thought, and white-box approaches. It focuses on API-accessible capability measurement through prompting.
- Evaluation awareness is defined as the capability to distinguish evaluation from deployment contexts, measured through prompting rather than fine-tuning.
- The paper excludes propensity—the tendency to consider evaluation and act on that belief—from its measurement scope.It treats capability evaluation as more established than propensity measurement.
- Chain-of-thought monitoring observes verbalised awareness, but verbalisation is only a lower bound on recognition.Nonreasoning models may verbalise awareness at under 1% without obviously behaving as though unaware.
- White-box methods recover awareness signals from activations and can causally shift safety-relevant behavior through steering or ablation.EvalDetectBench instead targets models with API access.
- Prompting-based methods ask judge models to classify transcripts as evaluation or deployment, while relying on assumptions about priming and honest responses.
3 Methodology
EvalDetectBench measures evaluation awareness by aggregating probe-based classifications over configurable evaluation and deployment transcript suites. Its methodology addresses probe-transfer failure and generator-identity bias through per-model calibration and stratified harmonisation.
- Pipeline: The pipeline combines evaluation and deployment transcripts, per-model probe selection, and aggregated P(eval) scoring with AUROC, AUPRC, and thresholded accuracy.Probe questions ask judges to classify transcript origins, and higher aggregate scores indicate more evaluation-like judgments.
- Transcript suite: The configurable suite contains 746 trajectories: 413 evaluation and 333 deployment, with both chat-style and agentic data.Its composition is deliberately replaceable so practitioners can refresh sources as benchmarks and deployment data evolve.
- Probe calibration: Per-model probe selection treats probe choice as capability elicitation because fixed prompts may under-elicit awareness across model families.The corpus contains 20 closed-ended probes spanning direct and indirect, surface-level and structural cues.
- Probe calibration: Held-out AUROC reaches 0.91 for Sonnet 4.6, 0.90 for Opus 4.6, and 0.82 for GPT-5.1, while performance plateaus by k ≈5–7.Selection beats random sampling at every probe count and generally meets or exceeds the fixed baseline from k ≈3–4 onward.
- Generator harmonisation: Generator identity explains 11.25% of P(eval) variance, persists under resampling at 8.4–11.3%, and can shift model rankings.Generation style is identified as the primary driver, and same-family or distribution-matched deployment data is recommended when possible.
- Generator harmonisation: Stratified harmonisation applies light z-score adjustment by generator-family and interaction-type strata before computing AUROC.Validation across six models shows corrected results moving closer to the self-only, unconfounded reference.
4 Evaluation Awareness Across Benchmarks and Models
Across benchmarks and judges, frontier models discriminate evaluation from deployment above chance, with newer closed-source models generally scoring higher. Evaluation-likeness also varies by benchmark and probe signal, and judges can disagree when task realism conflicts with explicit meta-evaluation cues.
- Model comparisons: Mixed AUROC ranges from 0.76 to 0.89 across 12 judges, with improvement from older to newer releases in every closed-source family.The Anthropic family shows the largest progression, increasing by +0.13 from Haiku to Opus.
- Benchmark comparisons: Mean P(eval) exceeds 0.83 on a subset of benchmarks when pooled across GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro Preview.Figure 7 reports mean P(eval) per evaluation dataset with bootstrap 95% confidence intervals.
- Probe variation: Probe signals differ substantially within judges: GPT-5.4 motivation scores 0.00–0.05 while origin scores exceed 0.89 on SWEBENCH-MINI and GDPVAL.Opus-4.6 shows a similar divergence between provenance and authenticity probes, whereas Gemini-3.1’s lowest probe score is 0.90.
- Judge disagreement: On GDPVAL, GPT-5.4 more often prioritizes underlying task structure, while Opus and Gemini-3.1 more often treat explicit evaluation signals as decisive.This recurring split reflects disagreement when task realism and meta-evaluation signals point in different directions.
5 Discussion
The paper frames prompted classification as a maximal-elicitation measure of evaluation awareness, while identifying priming, dishonest reporting, deployment-pool mismatch, and imperfect harmonisation as limitations. It argues that methodological choices can bias measurements and presents EvalDetectBench as a practical correction and reporting tool.
- Measurement framing: Prompted classification provides a maximal-elicitation upper bound on evaluation awareness, but assumes models answer honestly rather than sandbagging.The honest-reporting assumption remains untested in this work and prior work.
- Measurement limitations: AUROC partially limits priming bias because uniform score inflation across classes leaves transcript rank ordering unchanged.Differential priming across transcript content can still bias AUROC.
- Practical implications: EvalDetectBench is designed to add eval-awareness scores to existing Inspect-based workflows, strengthening the evidential chain in safety cases.The pipeline supports any Inspect-compatible benchmark and refreshed transcript and probe-question resources.
- Scope boundaries: Public deployment transcripts may not match any provider’s traffic distribution, so AUROC on the pool is only a proxy for discrimination against real deployment data.The paper also describes harmonisation as a pragmatic correction without known error bounds.
- Methodological findings: 11.25% of P(eval) variance is attributable to deployment-transcript generator identity, while cross-model probe sets can under-elicit awareness by up to 0.08 AUROC.The pipeline addresses these biases through generator-confound correction and per-model probe calibration.
- Open problems: Open problems include better deployment-proxy data, residual priming quantification, and measuring unprompted evaluation awareness.The last question concerns whether models act on recognition without being asked.
C ANOVA decomposition by resampling depth
Generator identity remains a substantial contributor to P(eval) variance across resampling depths. The effect is nearly flat rather than disappearing as more trailing messages are regenerated.
- Variance decomposition: 8.4–11.3%: generator identity’s η2 contribution to P(eval) variance remains within this range across all resampling depths.The pattern is visualised in Figure 9 and reported in the Type II ANOVA decomposition.
- Variance decomposition: 11.31%–8.39%: Figure 9 shows the generator-explained variance range remains almost flat regardless of resampling depth.This supports persistence of the generator effect under resampling.
D Is the Confound from Generation Style or User Tasks?
The confound is introduced primarily by the original model’s response style rather than by differences among the underlying user tasks. Removing that response reduces generator effects and reverses model rankings.
- Experimental conditions: Condition B replaces each original response with the judge’s own response to the same first user message, isolating user-task pools from original response style.The original model is absent in Condition B; labels then identify only the source pool of user tasks.
- Task versus style: 0.096 versus 0.147: cross-generator spread falls in Condition B versus Condition A, indicating that the underlying coding tasks are roughly equivalent.The remaining divergence is introduced by the generator’s response.
- Task versus style: Claude’s style adds +0.09 to P(eval), whereas GPT adds +0.31, or 3.6× more, beyond the user prompt alone.These values quantify the response-style contribution after controlling the user prompt.
- Ranking effects: Condition A ranks opus-4.5 lowest and gpt-5.4 highest by P(eval), whereas Condition B ranks gpt-5.3 tasks lowest and opus-4.5 tasks highest.The two rankings therefore reflect different mixtures of response style and task-pool identity.
- Ranking effects: 63% reduction: the generator contribution drops from 2.8% in Condition A to 1.0% in Condition B when original responses are discarded.The model ranking reverses when the original model is removed.
- Harmonisation: λ = 0.1 is chosen just beyond the correction cliff, where Spearman ρ reaches 0.89 while 81% of between-stratum information is preserved.The sweep is restricted to λ < 0.5 because stronger correction would contribute more than the original score.
- Harmonisation: The correction retains exactly (1−λ)2 of between-stratum mean variance, giving 81% retention at λ = 0.1.The measured ratio matches this expression to three decimal places across all six judges.
F Data Composition
The evaluation suite contains 413 trajectories across 29 benchmarks, combining capability and alignment evaluations with fresh agentic data and shared retained-chat entries.
- Suite composition: 413 trajectories across 29 benchmarks make up the evaluation suite, spanning capability and alignment evaluations.Agentic sources are collected fresh per judge, while retained chat entries are shared with response resampling.
- Trajectory sources: Petri trajectories are collected only for three closed-source judges, with approximately 50 fresh trajectories per judge using claude-sonnet-4.6 as auditor.These judges are used in the P(eval) across-benchmark analysis.
G Normalizing Data for Replay
Replay normalisation generates agent trajectories without runtime tool calls and records deployment data by corpus type, alongside an overview of evaluated models.
- Replay setup: Agent trajectories are replayed without tool calls because the user’s local environment is unavailable.Available tool definitions must be hardcoded during data normalisation.
- Deployment data: Deployment data are classified as agentic, chat, or mixed, with agentic/chat splits reported for mixed categories.OpenHands and Open Deep Research retain their native agentic-corpus sample counts.
- Evaluated models: The evaluated-model set is summarized in a dedicated overview table.
I Methodological Corrections on AUROC
The AUROC analysis compares fixed and calibrated probes under mixed deployment data, with and without generator harmonisation, against a same-family reference. Calibration and correction improve AUROC unevenly, while the corrected results closely track the self-only reference.
- Experimental comparison: Four configurations are compared: fixed probes, per-model probe selection, corrected mixed deployment, and self-only same-family deployment.All conditions use the same transcripts, isolating pipeline-configuration differences.
- Correction results: The corrected mixed-deployment configuration tracks the self-only reference closely for all six judges.This supports the validation that harmonisation recovers most of the unconfounded signal.
- Correction results: 0.00–0.08 AUROC gains result from per-model probe selection, while stratified correction adds 0.03–0.07.The gains are uneven across judges: three gain nothing from probe selection, while three gain meaningfully.
- Experimental comparison: The self-only condition removes the generator-identity confound but costs sample size and diversity.
J Compute and API Cost for Replicating This Work
Replication costs comprise transcript generation and calibrated probe scoring. Probe scoring requires calibration with the full probe corpus followed by scoring with only the selected probes, while API estimates account for provider pricing and batching.
- API estimation: Estimated API costs multiply token counts by provider list prices and include prompts, transcripts, probe templates, responses, and probe counts.Generator costs accumulate across multi-turn rollouts.
- API estimation: $2,627 was the realised replication cost with Anthropic-only batching under the reported pricing assumptions.The estimate uses January 2026 list prices and the standard 50% batch discount assumptions.
- Cost components: Replication has two separable costs: transcript generation, Cgen, and probe scoring, Cprobe.
- Transcript generation: Cgen is usually zero for adopters when target-benchmark transcripts already exist, while otherwise it corresponds to running the benchmark once.
- Probe scoring: Calibration runs all K probes on fraction f of transcripts, then scoring runs the selected k probes on the remaining (1 − f).Thus, calibration and scoring differ in the number of probe passes over the corpus.