PaperlayerView exact revision on arXiv

Source-linked AI summary

FUSE: An Evaluating Framework for Dangerous Capabilities of LLMs

Zhengyi Jin, Ru Zhang, Xiao Chen, Xinbo Liu, Jiaxuan Lin, Jia Huang, Jianyi Liu, Zhen Yang

arXiv:2609.02168v1cs.AI

TL;DR

Fragmented safety evaluations do not provide comparable evidence about dangerous LLM capabilities. The paper introduces a modular framework that measures Knowledge, Defense, and Harm under one protocol and aggregates them into φ, then applies it to chemical-biological models. Across models and releases, capability profiles diverge: knowledge compounds, defense varies by family, and harm remains inelastic, with validation supporting the measurement foundation.

  • Problem

    Existing benchmarks use different models, protocols, score scales, and populations, preventing comparable dangerous-capability profiles for governance.

  • Method

    The framework uses pluggable domain modules supplying scenario seeds, knowledge banks, harm queries, and rubrics while a shared pipeline measures K, D, and H and aggregates them into seven-dimensional φ.

  • Results

    Across 12 commercial LLMs from four families, capability structures differ sharply and temporal analysis finds compounding knowledge, family-divergent defense, and inelastic harm.

  • Takeaways & Limitations

    Dangerous capability should be compared across multiple dimensions and release trajectories rather than inferred from a single benchmark or snapshot.

  • Takeaways & Limitations

    Full-scale validation is limited to chemical-biological evaluation; the cyber pilot is reduced-scale with an immature knowledge bank, and harm scoring has moderate cross-judge agreement.

Abstract

from arXiv · show

Fragmented safety evaluation undermines the governance of dangerous AI capabilities. We present a modular framework that evaluates each model through three orthogonal pipelines---Knowledge ($K$), Defense ($D$), and Harm ($H$)---under a unified protocol, aggregating results into a standardized dangerous-capability profile $φ$. Pluggable modules supply scenario seeds, knowledge banks, hazard queries, and judge rubrics, while the core evaluation engine remains unchanged across domains; the CB evaluation is complemented by a cyber pilot demonstrating protocol transfer. Instantiating the framework with a chemical-biological (CB) module, we evaluate 12 commercial LLMs from four families. Our first contribution is a horizontal comparison of dangerous capability across models and model families: the three dimensions expose sharply divergent profiles---models with comparable knowledge differ in refusal resilience, and strong defenders do not generate less harmful content when they do comply---while family-level patterns further separate Claude, DeepSeek, and GPT models. The second is a temporal analysis of capability evolution: tracking $K$, $D$, and $H$ against model release dates reveals that dangerous capability has not monotonically declined; newer models deepen knowledge while only partially improving defense, showing that scaling and alignment progress do not uniformly translate into safety. Reliability is established via cross-judge consistency (bootstrap $ρ> 0.79$, 4 of 5 judges) and pipeline orthogonality ($K$--$D$--$H$ inter-correlations $ρ\in [0.32, 0.52]$).

1 Introduction

The paper identifies fragmented safety benchmarks as inadequate for comparing dangerous capabilities and proposes a modular framework that evaluates Knowledge, Defense, and Harm under one protocol. Its CB instantiation reports distinct cross-model profiles and capability evolution across generations.

  • Existing benchmarks use incompatible protocols, scales, and dimensions, preventing comparable judgments about which model is more dangerous or how danger evolves.
  • The framework evaluates Knowledge, Defense, and Harm as orthogonal dimensions under a unified protocol.Knowledge measures hazardous-domain proficiency, Defense measures robustness under adversarial induction, and Harm measures the actionability of hazardous outputs after access-control failure.
  • ρ ∈ [0.32, 0.52] across K–D–H pairs confirms that the three dimensions are not redundant.No single dimension therefore supplies a complete dangerous-capability profile.
  • 12 commercial LLMs from four families show sharply distinct capability structures, while three-generation analysis finds compounding knowledge, diverging defense, and inelastic harm.The paper presents these as its two headline conclusions and reports φ across the evaluated models.
  • The framework uses a four-module architecture, unified evaluation protocol, and switchable Content-Only or Tool-Enhanced execution modes.
  • Pluggable modules provide scenario seeds, knowledge banks, harm queries, and judge rubrics while the core pipeline remains reusable across danger domains.The CB module validates the interface on 12 commercial LLMs, and the paper reports a cyber pilot as protocol transfer.
  • The dangerous-capability profile φ aggregates K, D, H, DL, BR, ∆, and ER into a standardized seven-dimensional vector for direct cross-model and cross-domain comparison.

2 Background and Related Work

Prior safety evaluations span knowledge, refusal, harm generation, adversarial elicitation, agent tooling, and benchmark comparison, but remain fragmented across protocols and outputs. The paper positions its framework as unified infrastructure for comparable multidimensional assessment.

  • Existing work covers four evaluation tracks but shares a gap: no unified infrastructure assesses dangerous capability consistently across orthogonal dimensions.
  • Knowledge benchmarks measure hazardous information but are typically single-turn and cannot assess behavior under adversarial pressure.
  • Risk and harm evaluations use separate protocols and output formats, making results incomparable across benchmarks.This fragmentation spans agent safety, domain-specific extreme-risk studies, refusal safeguards, and harm-generation assessments.
  • Adversarial elicitation research shows that iterative pressure can bypass safety guardrails, while this framework embeds adaptive multi-round evaluation into Defense.
  • Tool-access research motivates the framework’s Tool-Enhanced mode, which extends content-only evaluation to tool-augmented scenarios in an isolated sandbox.
  • Table 1 compares existing benchmarks with the framework, whose unified infrastructure produces standardized dangerous-capability profiles.

3 Framework Design

The framework replaces fragmented safety benchmarks with a modular, unified infrastructure that evaluates Knowledge, Defense, and Harm and produces comparable profiles. Domain-specific modules provide evaluation assets while the shared engine orchestrates interactions and judgment.

  • Design goals: The framework provides cross-pipeline comparability, standardized output through φ, and amortized domain-extension cost via fixed Pluggable Module interfaces.Adding a danger domain requires a new module rather than changes to the core evaluation engine.
  • Architecture: A four-module pipeline comprises Pluggable Module, Task Orchestration, Test Environment, and Judgment, with independently configurable interfaces.The architecture evaluates models through parallel K, D, and H pipelines and aggregates their outputs into φ.
  • Pluggable Module: A Danger-Domain Module bundles scenario seeds, a knowledge bank, harm queries, and a judge rubric for one threat domain.Scenario seeds support D, MCQ knowledge banks support K, harm queries support H, and rubrics define scoring dimensions and ranges.
  • Task Orchestration: Task Orchestration loads K, D, and H assets and lets evaluators run any dimension individually or in combination.For defense evaluations, it selects content-only or tool-enhanced execution and activates the Test Environment with a Red-Team Agent.
  • Test Environment: The Test Environment supports adaptive natural-language elicitation and isolated tool use through separate execution and target-server sandboxes.Tool-enhanced execution routes structured requests through an intercept-and-forward mechanism rather than directly onto the host.
  • Judgment: Judgment first routes logged interactions into domain-specific risk categories, then applies a rubric-scored judge to produce multidimensional scores and pipeline outputs.Alternative judge families provide agreement statistics and bootstrap errors as reliability diagnostics.

4 Three Evaluation Pipelines (K, D, H)

The three pipelines measure distinct aspects of dangerous capability: extractable knowledge, resistance to adaptive elicitation, and harmfulness of compliant outputs. Derived indicators and the shared profile φ extend these measurements while preserving cross-model comparability.

  • Knowledge (K): Knowledge (K) measures hazardous-domain information leakage using IRT-estimated ability from multiple-choice items.IRT weights item difficulty, quantifies uncertainty, and supports interval-scale comparison; raw accuracy is supplementary.
  • Defense (D): Defense (D) measures whether behavioral guardrails withstand a five-round, path-dependent escalation generated from scenario seeds.Its rubric scores Induction Defense, Risk Leakage, Context Handling, and Multi-round Resilience.
  • Defense (D): Defense Level (DL) captures baseline refusal tendency, while Breakthrough Ratio (BR) captures how that tendency changes under sustained pressure.BR ≈1 indicates stability, BR < 1 indicates erosion, and BR > 1 indicates escalating compliance.
  • Harm (H): Harm (H) measures the actionable hazardous content produced on open-ended queries when access-control boundaries are breached.Non-refusal responses are scored across harm-rubric dimensions and aggregated over the scored response set.
  • Harm (H): D and H are complementary: high defense does not imply less harmful content on queries a model answers.The paper reports weak correlation between refusal behavior and harmfulness among compliant outputs.
  • Derived dimensions: Δ measures signed defense asymmetry across two subdomains, while ER distinguishes silent empty responses from visible refusals.Δ is omitted for single-subdomain modules; ER is observable from interaction logs independently of domain semantics.
  • Dangerous Capability Profile (φ): φ aggregates K, D, H, DL, BR, Δ, and ER into a seven-dimensional profile computed from one evaluation run.The common protocol and model population make φ vectors directly comparable across models and domains.

5 Experiment

The CB evaluation reveals sharply differentiated knowledge, defense, and harm profiles across models, including domain-specific asymmetries and defense that can erode under sustained elicitation. Harm remains narrowly distributed and conditional on compliance, complementing defense rather than replacing it.

  • Knowledge: Spearman rankings show near-perfect agreement between IRT ability and raw accuracy at ρ = 0.998.IRT nevertheless provides difficulty-weighted measurement and uncertainty estimates.
  • Knowledge: GPT-5.5 and GPT-5.4 lead both knowledge subdomains, while GPT-3.5-Turbo trails by over 5 logits in chemistry.GPT-3.5-Turbo’s θchem = −4.09 corresponds to 44.0% accuracy, versus 79.5% for GPT-5.5 at +1.21.
  • Knowledge: Eight of 12 models have stronger biological than chemical knowledge, with domain gaps ranging from 0.1 to 3.0 logits.All DeepSeek models favor biology, all Claude models favor chemistry, and GPT models split between the two.
  • Defense: Defense spans 43 points, from DeepSeek-V3’s Dbio = 56.9 to Claude-Opus-4.5’s Dchem = 99.9, with substantial within-model subdomain differences.Claude-Opus-4 shows a 15-point biology–chemistry gap: Dbio = 97.0 versus Dchem = 82.0.
  • Defense: Four defense risk modes separate baseline strength from multi-round stability, including brittle near-perfect defenders and exposed models whose compliance escalates under pressure.GPT-5.4 and GPT-5.5 have DL ≈0.97 and BR ≈0.95, while DeepSeek-V3, V4-Flash, and V4-Pro have DL < 0.72 and BR > 1.09.
  • Defense: DeepSeek-V4-Pro shifted within five rounds from a 20-character refusal to an operational pathogen-detection protocol, receiving an Unsafe score of 54/100.The reported protocol included detection assays, decontamination concentrations, and lockdown-escalation thresholds.
  • Harm: Harm scores cluster within 68.3–78.5, while Claude-Opus-4.5 and Opus-4.8 exceed 75 despite D > 99.H measures capability conditional on compliance; strong refusers are scored on a difficult residual set.
  • Harm: DeepSeek-V4-Pro combines weak defense with H = 77.0, whereas Claude-Opus-4.8 combines D > 99 with the evaluation’s highest harm score, H = 78.5.Their defense–harm ranking correlation is ρ = 0.318, supporting separate reporting of both dimensions.

6 Judge Consistency Analysis

The reliability analyses support robust Defense rankings but weaker Harm agreement, while low cross-dimension correlations indicate that K, D, and H capture distinct constructs. IRT estimates retain raw-accuracy ordering while adding difficulty-sensitive discrimination.

  • Judge reliability: Four of five alternative judges agree with the primary Defense judge at ρ > 0.79.The exception is Llama-3.3-70b at ρ = 0.641; pairwise alternatives range from ρ = 0.59 to ρ = 0.87.
  • Judge reliability: Harm inter-judge agreement is lower at ρ ∈[0.42,0.58] over n = 360 items because open-ended actionability scoring is more subjective.The framework reports this reliability difference transparently for downstream use of φ.
  • Pipeline orthogonality: Cross-dimension rankings are weakly correlated: K–D has ρ = 0.521, K–H has ρ = 0.469, and D–H has ρ = 0.318.The results argue against treating the three dimensions as redundant.
  • Pipeline orthogonality: The near-zero D–H correlation shows that refusal capability does not predict harmfulness conditional on compliance, so a complete profile requires all three dimensions.The moderate K–D association is described as structurally expected because hazard recognition presupposes domain knowledge.
  • Measurement precision: IRT and raw-accuracy rankings agree at ρ = 0.998 across the knowledge evaluation.All 24 model-subdomain estimates have standard errors between 0.09 and 0.16 logits.

7 Discussion and Limitations

The framework supports deployment decisions by exposing domain-specific defense gaps and by showing that capability changes over model generations. Its scope and validation remain bounded by cyber-pilot scale, Harm-judge subjectivity, contamination risk, and dual-use constraints.

  • Limitations: Full-scale validation is limited to CB; the cyber pilot demonstrates protocol transfer at reduced scale with an immature knowledge bank.The cyber pilot also exhibits a knowledge ceiling, with all three models scoring above 98%.
  • Limitations: Harm scoring has moderate cross-judge agreement at ρ ≈0.42–0.58, and knowledge-benchmark contamination remains a shared risk.These limitations constrain interpretation of the corresponding profile dimensions.
  • Discussion: The profile supports deployment decisions by distinguishing high-defense/high-harm models from models with inherent refusal risk.Claude-Opus-4’s 15-point biology–chemistry defense gap illustrates why domain-specific guardrails remain necessary.
  • Discussion: Knowledge compounds across generations, defense diverges by family, and harm remains inelastic, so a single audit is a snapshot rather than a trajectory.Knowledge-based guardrails calibrated on current models may become stale within one release cycle.
  • Ethics: The dual-use evaluation uses an isolated sandbox with a kill-switch, aggregate-level reporting, and a defensive orientation as mitigations.The paper states that detailed ethical considerations are provided in Appendix 8.

8 Conclusion

The framework measures dangerous capability through K, D, and H, plus derived indicators, into a seven-dimensional profile φ. Its CB evaluation finds distinct family structures and temporal divergence, supported by validation analyses.

  • The framework aggregates K, D, and H with derived indicators DL, BR, ∆, and ER into a seven-dimensional capability profile φ.
  • Capability evolves over time through knowledge compounding, family-specific defense divergence, and inelastic harm.
  • Cross-judge validation exceeds ρ > 0.79, while IRT precision reaches ρ = 0.998.

Appendix

The appendix gathers supporting materials for reproducibility and review, including ethical documentation, measurement derivations, statistical procedures, complete tables, and supplementary figures.

  • The appendix provides supporting material for reproducibility and review.
  • It documents ethical considerations, artifact access policy, IRT derivations, and correlation and bootstrap procedures.
  • It also contains complete measurement tables omitted from the main text and supplementary figures.

Ethical

The work addresses dual-use risks through controlled evaluation, disclosure minimization, and responsible-use boundaries. It frames systematic measurement as important for governance while limiting operational details and avoiding vulnerability exploitation.

  • The CB module uses scenario seeds and harm queries to elicit dangerous knowledge from LLMs.
  • The evaluation demonstrates that several commercial models can be induced to produce actionable hazardous protocols across five escalating rounds.
  • Operational parameters, identities, and commercial reagent names are redacted from reported case figures.
  • The study uses public API behaviors and documented elicitation strategies without attempting to exploit vulnerabilities beyond what models offered.
  • Systematic measurement is presented as evidence for regulators, operators, and researchers tracking dangerous capabilities.

Open Science

The paper supports open science while restricting sensitive dual-use materials. Reviewers receive a lightweight reference implementation and methodology sufficient to verify reported analyses.

  • The authors support the USENIX Security Open Science Policy and make code and data available upon reasonable request.
  • They withhold the full implementation, CB module assets, and raw interaction logs because full disclosure would lower attack barriers.
  • A lightweight reference implementation covers the four-module pipeline on nonsensitive examples with metric computation logic.
  • The methodology supports third-party reproduction of φ aggregation and statistical analyses from released materials.

IRT Measurement Derivation

The framework calibrates item difficulty, estimates model ability with fixed item parameters, and derives measurement precision from item information. Supplementary analyses also document judge agreement, temporal defense patterns, and scenario-level variation.

  • Calibration: Item difficulty is calibrated under a 2PL model with guessing parameter 0.25 by matching the model at θ = 0 to pooled pass rates.The resulting difficulty parameter is given by bi = log 1−pi pi −0.25.
  • Ability estimation: With item difficulties fixed, model ability θ is estimated by maximizing the response-vector log-likelihood using Newton–Raphson updates until convergence.The procedure obtains the estimated ability ˆθT.
  • Measurement precision: Test information is computed from the standard item-information identity at the estimated ability, followed by the standard error of the ability estimate.The item response function uses Pi = 0.25+0.75Ψi, incorporating the guessing parameter into information calculations.
  • Reliability: Spearman’s ρ compares judge rankings invariantly to monotone score rescaling, using stratified sampling and B = 1,000 nonparametric bootstrap replications for uncertainty.The bootstrap procedure also supports harm re-scoring and per-model consistency tables.
  • Supplementary analyses: Supplementary figures extend the main analyses with temporal defense indicators, empty-response rates, cyber per-scenario defense profiles, and per-model judge-consistency heatmaps.The visualizations complement the main text’s evolution analysis, cyber pilot decomposition, and pooled consistency matrices.
  • Measurement precision: The knowledge bank provides maximum measurement precision near θ ≈−0.4, corresponding to the L3 discern difficulty region.Figure 11 reports this as the region most critical for distinguishing mid-range hazardous knowledge capabilities.
Loading 2609.02168v1…