Bench-2-CoP: Can We Trust Benchmarking for EU AI Compliance?

1 Introduction

• Current benchmarks emphasize performance and foundational capabilities, leaving the systemic risks arising from model capabilities, propensities, and use contexts outside their original design.The EU AI Act and its Code of Practice call for lifecycle-based evaluation of those interacting factors.Source: §1 · p. 1
• Existing technical benchmarks also omit lifecycle compliance practices such as formal adequacy assessments and incident reporting.Those areas are identified as gaps in industry practice that technical benchmarks do not cover.Source: §1 · p. 1
• Benchmark leakage, static exam-style formats, evaluator bias, and cultural narrowness can undermine the validity and real-world relevance of reported performance.The paper cites GPT-4 training on GSM8K and MATH data as an example of evaluation data entering training.Source: §1 · pp. 1–2
• Bench-2-CoP provides an empirical mapping of benchmark coverage against EU AI Act requirements for GPAI with systemic risk.The framework operationalizes regulatory text into a measurable taxonomy, with the stated aim of informing policymakers and developers.Source: §1 · p. 2

2.1 The Regulatory Landscape: The EU AI Act and its Code of Practice

• The Code of Practice calls for lifecycle-based risk identification, assessment, mitigation, monitoring, and incident reporting, using capabilities, propensities, and affordances to understand how risks emerge.Capabilities describe what a model can do, propensities what it tends to do, and affordances how deployment context and use can enable outcomes.Source: §2.1 · p. 3

2.2 The Evaluation Landscape: A Critique of AI Benchmarking

• Widely used benchmarks focus on task performance, while reviews identify shortcomings in documentation, maintenance, transparency, and resistance to benchmark leakage.Reported memorization of test data can make benchmark scores reflect memorization rather than generalizable reasoning.Source: §2.2 · pp. 3–4
• High safety-benchmark scores may reflect general capability and scale rather than genuine safety features, while static tests differ from dynamic, adversarial evaluations such as expert-led red teaming.The cited work describes the first pattern as “safetywashing.”Source: §2.2 · p. 4
• Related work argues that evaluation should address socio-technical and normative alignment, including human oversight of model outputs and rationales.The cited discussion distinguishes these concerns from isolated task performance and advanced cognitive flexibility tests.Source: §2.2 · p. 5

2.3 Positioning Our Contribution

• A quantitative analysis of how well existing benchmarks cover regulatory requirements is missing; Bench-2-CoP operationalizes the Code of Practice taxonomy to assess that coverage.The study positions its framework as a coverage analysis, rather than a proposal for a new benchmark.Source: §2.3 · p. 5

3 Research Framework and Methodology

• Bench-2-CoP frames four questions about regulatory coverage, evaluation gaps, variation across benchmark types, and alignment between industry priorities and regulatory demands.Its three stages are corpus construction, validation of an LLM judge, and full-corpus coverage analysis.Source: §3 · p. 5

3.1 Stage 1: Benchmark Corpus Construction

• The corpus was designed to reflect both current industry evaluation practices and relevance to the EU AI Act’s focus on systemic risk.The study reviewed public documentation from five leading AI developers to identify commonly used benchmarks.Source: §3.1 · p. 5, §3.1.1 · p. 5

3.1.1 Identifying Industry-Standard Benchmarks

• Developer disclosures emphasized general knowledge, reasoning, and commonsense benchmarks, while specialized safety benchmarks such as BBQ and TruthfulQA appeared less frequently.This initial observation motivated the study’s analysis of broad capability testing versus targeted risk evaluation.Source: §3.1.1 · p. 6

3.1.2 Curating the Analysis Corpus

• 194,955 questions from six diverse benchmarks formed the analysis corpus; benchmarks focused purely on technical performance, including coding accuracy and mathematical precision, were excluded.Questions were standardized in a unified dataset while retaining metadata such as answers, contexts, and original categories.Source: §3.1.2 · pp. 6–7

3.2 Stage 2: LLM-as-Judge Framework Development and Validation

• The LLM-as-judge framework classifies benchmark questions against the EU AI Act taxonomy and was validated against human expertise because the task is subjective.The judge assigns the capabilities and propensities assessed by each question.Source: §3.2 · p. 7

3.2.1 Gold Standard Dataset Creation

• 597 questions were stratified across benchmarks, with at least 30 from each; experts annotated capabilities and propensities, checked overlap on 20%, and resolved disagreements by consensus.The resulting expert-annotated set served as the ground truth for evaluating automated classification.Source: §3.2.1 · p. 7

3.2.2 LLM Evaluator Selection and Prompting

• Gemini-2.5-Flash was selected over GPT-4.1 and Claude-Sonnet-4 after outperforming them across measured metrics, with Kappa scores of 0.75 for capabilities and 0.82 for propensities.The paper characterizes those agreement values as substantial and almost perfect, respectively.Source: §3.2.2 · pp. 7–8

3.3 Stage 3: Full Corpus Analysis

• The validated Gemini-2.5-Flash evaluator classified all 194,955 questions, producing a mapping of the full corpus to the CoP capabilities and propensities.The authors state that full-corpus analysis avoids potential sampling bias and captures rare evaluation patterns.Source: §3.3 · p. 8

4 Results

• The six-suite analysis finds deep but narrow evaluation: benchmarks cover a few behavioral propensities extensively while leaving critical functional capabilities almost entirely unevaluated.The analysis considers both raw question volume and normalized category shares to account for differences in benchmark size.Source: §4 · p. 8

4.1 Overall Coverage of the Benchmark Ecosystem

• 61.6% of each benchmark’s focus goes to “Tendency to hallucinate” (P3), and the propensity-heavy pattern is described as reactive to past failures rather than proactive about future risks.This normalized comparison remains when accounting for benchmark size.Source: §4.1 · p. 8, Table 6 · p. 10
• 17.2% is the normalized priority for “Discriminatory bias” (P4), while “Tendency to deploy harmfully” (P2) has 0.95% raw coverage but 12.35% average normalized coverage.Bias counts are driven almost entirely by BBQ, whereas P2 is prioritized by the few benchmarks that test it, primarily HLE.Source: §4.1 · p. 8

4.2 Capability Coverage: A Landscape of Critical Gaps

• C11’s 9,502 questions suggest multimodal coverage, but most—especially 6,649 from MMLU—use textual references rather than testing genuine multimodal processing.C8 (Evade Human Oversight), C9 (Self-Replicate/Self-Improve), and C10 (AI R&D Capabilities) receive zero questions; C4 has 5, C7 has 11, C2 has 17, and C1 has 31.Source: Table 7 · p. 10, §4.2 · p. 11, Figure 4 · p. 11

4.3 Propensity Coverage: A Focus on Specialization

• More than 100,000 questions address hallucination, but many MMLU items classify wrong factual answers as hallucinations, while BBQ supplies 96.4% of the corpus’s bias evaluation.TruthfulQA contributes 738 adversarial hallucination probes; P3 coverage also misses extended dialogue and high-stakes contexts. BBQ’s focus on nine protected characteristics and a primarily Western-centric conceptualization leaves intersectional and context-specific bias gaps.Source: §4.3 · p. 12, Table 8 · p. 12, Figure 5 · p. 13
• More than 36,000 questions are counted as reliability coverage, yet benchmarks use inconsistent definitions and omit systematic tests of temporal, adversarial, and compositional reliability.P1 receives 813 questions, P2 is almost exclusively tested by HLE, and P6–P9 are superficial or absent.Source: §4.3 · p. 12, Table 8 · p. 12

4.4 Individual Benchmark Profiles

• Benchmark profiles span MMLU’s scale without critical-capability coverage, BBQ’s narrow bias specialization, TruthfulQA’s adversarial probes, BBH’s broader coverage, and HLE’s frontier-risk focus.MMLU provides zero coverage for autonomy, self-reasoning, and self-improvement; HLE uniquely provides significant coverage for tool use and adaptive learning and meaningful coverage for harmful deployment.Source: §4.4 · pp. 13–14

4.5 Mapping Evaluation Coverage to Systemic Risks

• 88.2% coverage for Harmful Manipulation contrasts with 0.8% for Cyber Offence, 1.7% for CBRN Risks, and 0.4% for Loss of Control.The mapping to systemic risks uses expert-identified combinations of capabilities and propensities because the CoP provides no explicit crosswalk to the granular taxonomy.Source: §4.5 · pp. 14–15, Table 9 · p. 14

5.1 Implications for AI Safety and Regulatory Compliance

• C8, C9, and C10 receive zero questions, while evaluation is concentrated on a narrow set of propensities, leaving systemic-risk capabilities unassessed.The authors characterize the overall benchmark corpus as structurally inadequate for assessing systemic risks as defined by regulators.Source: §5.1 · p. 15
• Public benchmarks alone cannot provide sufficient evidence of comprehensive systemic-risk assessment for EU AI Act compliance, including standardized assessment of Cyber Offence and CBRN misuse.The authors note that this gap creates uncertainty for developers and an enforcement challenge for regulators.Source: §5.1 · p. 15

5.2 Methodological Considerations and Study Limitations

• The analysis uses an LLM judge with Cohen’s Kappa of 0.79, but classification error remains inevitable; its six-benchmark corpus and expert-constructed risk pathways also constrain scope.The corpus focuses on benchmarks commonly disclosed by commercial developers, while the Code of Practice supplies no explicit mapping from capabilities and propensities to systemic risks.Source: §5.2 · pp. 15–16

5.3 Recommendations for Key Stakeholders

• The authors recommend dynamic testing environments for researchers, diverse evidence portfolios for regulators, and internal evaluation frameworks for AI developers.Suggested regulatory evidence includes red-teaming, runtime monitoring, and formal methods alongside benchmark scores.Source: §5.3 · p. 16

5.4 Future Work: Building the Next Generation of Regulatory-Aligned Benchmarks

• Future work should extend Bench-2-CoP and develop either a balanced CoP-Bench or specialized high-fidelity benchmarks for critical coverage gaps.The proposed AgenticSafety-Bench would use interactive scenarios to probe autonomy-related risks.Source: §5.4 · pp. 16–17

5.5 Conclusion

• The authors conclude that current evaluation has deep but narrow coverage, while regulation anticipates risks from increasingly autonomous AI systems.They call for coordinated development of methods that assess dynamic, interactive, and emergent properties while retaining rigor and standardization.Source: §5.5 · p. 17

A.1 Evaluation Procedure

• 194,955 benchmark questions were sent individually to Gemini-2.5-Flash, which returned capability and propensity codes that the pipeline enriched with question and benchmark metadata.The pipeline ingested question text, context, answers, options, and original categories before constructing a prompt for each item.Source: §A.1 · p. 25

A.2 LLM Evaluator Prompt

• The evaluator prompt defines 13 capabilities and nine propensities, with examples distinguishing attributes such as planning, autonomy, bias, and misalignment.Its required response is a JSON object containing relevant capability and propensity codes.Source: §A.2 · pp. 25–26
• A code is included when directly tested, required for a correct answer, or central to the primary risk; tangential, generic, and speculative links are excluded.Source: §A.2 · p. 26

A.3 Example of Full Evaluation Flow

• The HLE Sokoban example is classified as planning, returns C6 with no propensity code, and is enriched into a structured record.The example shows the pipeline moving from a single question to raw JSON and then a metadata-bearing record.Source: §A.3 · pp. 27–28

A.1 Gold Standard Annotation Protocol

• Five researchers with complementary expertise annotated a 597-question sample, independently overlapping on 20% and resolving disagreements through consensus.The protocol also asked annotators to record rationales for ambiguous classifications.Source: §A.1 · pp. 28–29

A.2 Systemic Risk Mapping Rationale

• Because the Code of Practice lacks an explicit mapping, experts constructed risk pathways linking harmful manipulation to hallucination, bias, persuasion, multimodality, and misalignment, and cyber offence to cyber knowledge, tool use, lawlessness, autonomy, and power-seeking.Source: §A.2 · p. 29
• The mapping links CBRN risk to sensitive-domain knowledge, harmful deployment, lawlessness, and adaptive learning, and loss of control to autonomy, self-reasoning, oversight evasion, self-modification, AI R&D, misalignment, and power-seeking.Source: §A.2 · p. 29

• The appendix documents expert protocols for validating the LLM-as-judge and mapping low-level attributes to high-level systemic risks.Source: A Expert Annotation and Risk Mapping… · p. 28
• The gold-standard annotation dataset was created to validate the LLM evaluator described in Section 3.Source: A Expert Annotation and Risk Mapping… · p. 28
• The second protocol maps capabilities and propensities to systemic risks defined in the EU AI Act Code of Practice.Source: A Expert Annotation and Risk Mapping… · p. 28