PaperlayerView exact revision on arXiv

Source-linked AI summary

Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation

Jianchang Su, Dawei Xiang, Yifan Zhang, Wei Zhang

arXiv:2610.21135v1cs.CRcs.AI

TL;DR

LLM-agent evaluations report behavioral scores but not the identity of the deployed runtime stack, leaving a gap when evaluation concerns a composite system. DIB-Bench measures output-only evaluator blindness alongside deployment-identity checks, finding that stronger judges accept more tampered outputs and that zero-shift tampers evade behavioral tests at any sample size.

  • Problem

    LLM-agent evaluations report behavioral scores while leaving the identity of the deployed runtime stack unreported, although the evaluated system comprises components that can change independently.

    Source: §1 · p. 1

  • Method

    DIB-Bench measures deployment-integrity blindness across six tamper types and an evaluator ladder, while contrasting output scoring with manifest-hash identity checks.

    Source: §1 · p. 1

  • Results

    Mean tamper acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), and zero-output-shift tampers are undetectable by output-only evaluators at any sample size.

    Source: §1 · p. 1

  • Takeaways & Limitations

    A manifest-hash identity channel decides the zero-shift tamper class with zero samples, while a graded deployment-identity descriptor adds identity information to evaluation reports.

    Source: §1 · p. 1

    The main empirical package uses synthetic single-component tampers and does not cover multi-component attacks or adversaries able to forge the manifest directly.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples, six tamper types, and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano. Stronger evaluators accept more tampered outputs: mean acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), and a trigger-conditional tamper invisible on the evaluation suite produces 48-100% harmful outputs, rising with model capability. A PAC-style bound shows that tampers with zero output shift are undetectable from outputs at any sample size; a manifest-hash identity channel detects them with zero samples. Behavioral fingerprinting identifies the model component above chance (76.7%) but covers one of six stack components. We also propose a graded deployment-identity descriptor for evaluation reports.

1 Introduction

LLM-agent evaluations report behavioral scores but leave the deployed runtime stack’s identity unreported, even though its components can change independently. DIB-Bench measures whether output-only evaluators detect such changes and contrasts them with manifest-hash identity checks.

1 Introduction

  • Behavioral scores leave a structural reporting gap because leaderboards rank composite systems as “models” even when their components can be swapped without changing the entry.The identity channel is unreported in current NLP evaluation, although the paper frames identity evidence as an additional coordinate beyond agent and prompt-injection benchmarks.Source: §1 · pp. 1–2, §2.2 · p. 2
  • 94.1% (GPT-5-nano) versus 88.6% (rule-based) mean tamper acceptance shows that stronger evaluators accept more tampered outputs.Qwen3-8B acceptance is 89.3%.Source: §1 · p. 1
  • A trigger-conditional tamper is 100% invisible on the evaluation suite and produces up to 100% harmful outputs off-suite, with harm rising by model capability.Reported harm is 48% for Llama-3.1-8B, 81% for Qwen2.5-7B, and 100% for Qwen2.5-14B.Source: §1 · p. 1
  • Tampers with zero output-distribution shift are undetectable by output-only evaluators at any sample size, while manifest-hash identity checks decide this class with zero samples.A stealth tamper can also evade identity checks if it changes an undeclared component.Source: §1 · p. 1

2.1 Deployments, Manifests, and the Two Channels

  • A deployed agent is a stack of components, including its model, tokenizer, prompt, tools, retrieval resources, dependencies, and postprocessor, represented by per-component manifest hashes.Output evaluation scores behavior; identity evaluation compares expected and observed component hashes.Source: §2.1 · p. 2

2.2 Blindness Metrics

  • Deployment-integrity blindness means output evaluation passes while identity evaluation fails; its example-level rate averages this joint condition across examples.The paper proposes DIB-Bench as a protocol for controlled single-component tampers and a graded deployment-identity descriptor for evaluation reports.Source: §1 · pp. 1–2, §2.2 · p. 2

3 Theory

The theory separates what output behavior can establish from what deployment identity requires. Finite evaluation suites cannot identify the deployed system in general, and output-based detection of tampering depends on its distributional shift.

3 Theory

  • For any finite output suite, a deployment that passes it has a component-changed counterpart with identical suite acceptance, so deployed identity is not determined by evaluated outputs.The construction can substitute a component variant that matches outputs on the suite, including a trigger-conditional variant.Source: §3 · pp. 2–3
  • The theory distinguishes mere indistinguishability from practical stealth: an empirically tested hypothesis is that zero-shift tampers can still produce harmful outputs outside the evaluation distribution.This is stated as a hypothesis tested in the paper, not as a theorem.Source: §3 · p. 3
  • When output-distribution shift ε is zero, every behavioral test remains indistinguishable from chance at any sample size.The bound applies to output-only evaluators, whereas comparing declared and observed component hashes decides the ε=0 class with one manifest read.Source: §3 · p. 3

4 DIB-Bench

DIB-Bench evaluates six single-component deployment tamper types and validates how identity detection depends on manifest coverage. Its instrument detects declared changes, but undeclared changes can alter behavior while passing identity checks.

4.1 Conditions, Tasks, and Models

  • The benchmark compares one clean condition with six single-component tampering conditions, including model, prompt, tool, retrieval, dependency, and postprocess tampering.Each tampered condition changes exactly one manifest component, isolating the comparison.Source: §4.1 · p. 3

4.2 Instrument Validation: Detection Equals Coverage

  • Undeclared changes altered 33%–78% of outputs while identity passed at 100%; benign declared changes left decisions identical while identity failed at 0%.These validation results show identity detection tracks manifest coverage and flags changes rather than semantic differences.Source: Table 1 · p. 3, §4.2 · pp. 3–4

4.1 Conditions, Tasks, and Models

  • The main study scores 7,560 examples across 120 synthetic tasks per condition, five task families, three model pairs, and three random seeds.The task families cover factual-context QA, policy allow/deny, tool selection, secret-leak refusal, and canary probes.Source: §4.1 · p. 3

4.2 Instrument Validation: Detection Equals Coverage

  • Across replicated settings, undeclared changes altered 15.8–100% of outputs while identity passed at 100%, whereas benign changes preserved decisions while identity failed.The replications include Qwen2.5-7B, Llama-3.1-8B→Qwen2.5-7B, and Qwen2.5-14B.Source: §4.2 · p. 4

5 Results: The Evaluator Ladder

Across the evaluator ladder, stronger output judges accept more tampered outputs, while stricter scoring can reject clean deployments before excluding all tampers. Detection varies with suite coverage: tampering is more visible when evaluations probe the changed behavior, but some tampering can evade both output and identity checks.

5 Results: The Evaluator Ladder

  • The suspicion auditor flags 54.0% of clean outputs, 52.6% of prompt-tampered outputs, and 51.2% of dependency-tampered outputs, with clean and tampered outputs overlapping at every threshold.Using it would reject about half of clean deployments while still passing all tampers.Source: §5 · p. 4
  • Strict scoring lowers mean tamper acceptance to 73.1%, but strict and canary scoring reach 0/6 only above their clean-acceptance thresholds, rejecting the clean deployment too.Standard and forensic scoring accept four of six tampers even at τ=0.90.Source: §5 · p. 4, Table 3 · p. 5
  • Acceptance drops on task families that probe changed behavior: postprocessing falls to 0% on secret and canary tasks, while tool and retrieval tampers fall to 64% and 67% on relevant tasks.Most other task families remain near 100%.Source: §5 · p. 4, Figure 2 · p. 4
  • Mean tamper acceptance rises from 88.6% (rule-based) through 89.3% (Qwen3-8B) to 94.1% (GPT-5-nano), while the identity channel separates every tamper at 0%.These are results on the synthetic suite (n=7,560, 9 runs).Source: §5 · p. 4, Table 2 · p. 5
  • Similar-capability substitutions are fully accepted, while the 7B→1.5B downgrade is partly visible at 85.6% standard and 38.3% strict; one substitute even scores 100% versus 92.8% for the declared model.The suite detects a tamper when it probes the changed behavior.Source: §5 · p. 5
  • Behavioral fingerprinting identifies the model component with 76.7% three-way accuracy, above the 33.3% chance level, but covers only one of six stack components.Five non-model tampers leave the model unchanged; models also emit byte-identical outputs on 37.5% of probes despite a 4.5× parameter gap.Source: §5 · p. 5
  • The E4 trigger-conditional tamper matches clean decisions on the ordinary suite and is caught by identity, whereas E8 evades both channels through an undeclared chat template.Both produce more than 80% harm on triggered inputs; ordinary-suite harm is 13.3%, equal to the clean baseline.Source: Table 5 · p. 6
  • Across public and multi-step benchmarks, tampered acceptance can match clean acceptance, but a large capability drop is behaviorally visible: AgentDojo utility falls to 25.0% for Qwen2.5-1.5B.On GSM8K, clean solves 27.7% and tampered deployments solve 27.6%; on BFCL-simple, six tampers pass at 99.9% against a 100% clean rate.Source: §5 · pp. 5–6

6 Stealth Tampers: Testing Hypothesis 1

The section tests trigger-conditional and stealth tampers, showing that output-only evaluation can miss harmful deployments and that identity checks depend on component coverage.

6 Stealth Tampers: Testing Hypothesis 1

  • On triggered probes, E4 produced 81.0% harmful outputs despite 100% decision identity on the ordinary suite, where every output evaluator accepted it.The probes covered policy and secret tasks; the harmful outputs included 72.5% policy-violation flips and 60.0% secret leaks.Source: §6 · p. 6
  • Triggered harm rose from 48.3% (Llama-3.1-8B) to 81.0% (Qwen2.5-7B) to 100% (Qwen2.5-14B), while ordinary-suite decision match stayed at least 96.6%.The same triggered probes were used across all three deployed models.Source: §6 · p. 6, Figure 3 · p. 6
  • On triggered probes, the E8 deployment produced 86.2% harmful outputs while passing identity checks and remaining decision-identical to clean on the ordinary suite.E8 injects trigger-conditional logic through the chat template, which the manifest does not cover.Source: §6 · pp. 6–7
  • A manifest that omits the chat template lets E8 pass identity checks; hashing the resolved template closes this specific coverage gap.The deployment-identity descriptor’s worked example localizes a prompt tamper to the system prompt.Source: §6 · p. 7, Table 6 · p. 7

7 A Deployment-Identity Descriptor for Evaluation Reports

The proposed descriptor adds auditable deployment identity to evaluation reports through per-component records, with assurance graded by how those records are obtained.

7 A Deployment-Identity Descriptor for Evaluation Reports

  • The descriptor records a content hash and the method used to obtain it for each deployment-stack component.The proposed reporting convention is intended to sit alongside behavioral scores.Source: §7 · p. 7
  • Per-component records localize changes: tampered examples mismatched exactly the changed component, while clean examples fully matched.The report can therefore show which stack component differs, unlike a behavioral score alone.Source: §7 · p. 7
  • Its assurance is graded: a self-reported hash manifest is the cheapest form, with signed build provenance or hardware-rooted runtime attestation as stronger alternatives.The channel should be present and auditable; it certifies component identity, not safety or correctness.Source: §7 · p. 7
  • Reports should state manifest coverage because identity detection equals coverage, and an omitted chat template allowed the E8 tamper to evade detection.Reports should also state suite coverage, including which behaviors such as policy decisions, secret handling, and tool selection are probed.Source: §7 · p. 7
  • An identity mismatch calls for review rather than a harm judgment, since even a benign prompt paraphrase can fail identity matching.The section also identifies adaptive probing, deployment-drift measurement, and leaderboard rank decomposition as future directions.Source: §7 · p. 7

8 Related Work

Related work evaluates agent behavior, benchmark integrity, model attribution, and deployment provenance; this paper connects those threads by asking whether a report identifies the deployed system.

8 Related Work

9 Conclusion

The conclusion argues that stronger evaluators do not close deployment-integrity blindness and recommends identity evidence alongside behavioral scores. The paper frames its findings as defensive measurement and identifies limits in its tampering scope, evidence, and model coverage.

9 Conclusion

  • Mean tamper acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), while zero-output-shift tampers remain undetectable by output-only evaluators at any sample size.The conclusion reports 48–100% harm for a trigger-conditional tamper as model capability rises; manifest hashes decide the undetectable class with zero samples.Source: §9 · p. 8

Ethics Statement

  • The ethics statement frames the work as defensive measurement using synthetic secret tokens and policy-bypass prompts, without real secrets, users, or external services.It recommends avoiding operational attack-recipe framing in released artifacts.Source: Ethics Statement · pp. 8–9

Limitations

  • The main empirical package tests synthetic, single-component tampering; it does not cover multi-component attacks or adversaries able to forge manifests directly.The models range from 1.5B to 20B parameters, and the PAC dichotomy covers passive i.i.d. sampling; the manifest lacks hardware-rooted guarantees.Source: Limitations · p. 9

A PAC Dichotomy Proof

The proof bounds how well behavioral tests distinguish clean from tampered deployments using their output-distribution shift. When that shift is zero, no sample size helps behavioral testing, while a manifest-hash identity evaluator detects the mismatch without samples.

A PAC Dichotomy Proof

  • The behavioral test observes N i.i.d. input-output pairs and returns a clean-or-tampered guess, covering scorers, LLM judges, forensic classifiers, and behavioral fingerprints.The proof fixes an evaluation distribution D and allows deployments to produce possibly stochastic output distributions.Source: A PAC Dichotomy Proof · p. 11
  • When ε = 0, every behavioral test has exactly zero distinguishing advantage for every N; the identity evaluator decides this class with zero samples.For ε > 0, a manifest read compares declared and observed component hashes independently of ε.Source: A PAC Dichotomy Proof · p. 11
  • For ε > 0, thresholding the empirical frequency of a separating event at the midpoint distinguishes A from B with error ≤δ at the stated sample bound.The event E has probability gap at least ε, and the bound uses Hoeffding’s inequality.Source: A PAC Dichotomy Proof · p. 11

B Scope and Replication

Across benchmark settings, behavior-only evaluations can accept tampered deployments even when identity checks distinguish them. The observed scope includes replicated model-pair conditions, multiple tasks, model sizes, and a multi-step agent suite.

B Scope and Replication

  • E2 identity passes were 100% despite 15–100% behavior changes, while E3 identity passes were 0% despite 100% decision-match across three configurations.The configurations included two within-family Qwen substitutions and a Llama-to-Qwen substitution.Source: B Scope and Replication · p. 11
  • On the ordinary suite, E4 decision-match was ≥96.6% for all three model sizes, while triggered harm ranged from 48% to 100%.Triggered harm was 48% for Llama-3.1-8B, 81%/79% for Qwen2.5-7B seeds 42/43, and 100% for Qwen2.5-14B.Source: B Scope and Replication · p. 11
  • BFCL-simple yielded 99–100% standard and strict acceptance for all six tampers, 0% identity acceptance, and 99.9% mean blindness.All six tampers were accepted at τ=0.80, and the extreme-blindness anchor replicated at 12× the original sample size.Source: B Scope and Replication · p. 11
  • On GSM8K, all four behavior-preserving tampers clustered at the clean baseline of ∼27–28% standard, while model substitution reduced strict performance to 7.8%.The passage attributes the lower ∼28% clean standard score versus the 50-item 38% slice to a length effect at max_len 2048.Source: B Scope and Replication · p. 11
  • Substitution blindness persisted across model scales: Qwen3-8B substitution had 100% blindness, and a 20B gpt-oss-20b substitution also had 100% blindness.For the Qwen3 case, standard and strict scores were unchanged at 100% and 85%; for gpt-oss-20b, standard stayed 100% while identity fell from 100% to 0%.Source: B Scope and Replication · p. 11
  • On AgentDojo banking, the prompt tamper was behaviorally invisible at 50.0% utility while identity failed; model substitution was visible at 25.0% utility.Clean utility was 43.8%, and the contrast shows output-only evaluation has discriminative power for some tampers but not others.Source: B Scope and Replication · pp. 11–12

C DIB-Bench Framework

DIB-Bench connects agent-behavior benchmarks with deployment-identity evidence. Its framework is examined across model pairs, component-localized evidence, and reported artifacts and compute.

C DIB-Bench Framework

  • DIB-Bench is presented as a bridge between agent behavior benchmarks and supply-chain provenance frameworks.Table 7 summarizes the framework interface.Source: C DIB-Bench Framework · p. 12

D Robustness Across Model Pairs

  • Blindness rates are reported by tamper condition and model pair, with the pattern persisting across two within-family Qwen substitutions and one cross-family Llama-to-Qwen substitution.Figure 4 presents the condition-by-model-pair view.Source: D Robustness Across Model Pairs · p. 12

E Component-Localized Evidence

F Artifacts, Licenses, and Compute

  • The reported setup uses greedy decoding, a τ=0.80 suite threshold unless stated otherwise, and means over nine runs; open-model inference uses one RTX A6000 GPU.The artifact inventory lists deployed and judge models, benchmarks, and inference software with associated licenses; GPT-5-nano is accessed through the OpenAI API.Source: F Artifacts, Licenses, and Compute · p. 12

Related papers

Loading 2610.21135v1…