PaperlayerView exact revision on arXiv

Source-linked AI summary

PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift

Yusuf Khalid Shire, Sang-Chul Kim

arXiv:2609.15017v1cs.CRcs.AI

TL;DR

Aggregate in-distribution F1 leaves detector behavior on benign security-adjacent traffic and shifted inputs undermeasured. PIDS-Bench evaluates detectors across multiple stress conditions and finds persistent over-blocking that threshold selection and curated augmentation do not remove.

  • Problem

    Aggregate held-out F1 offers limited evidence about benign false positives and detector behavior under distribution shift, despite the direct usability cost of blocking legitimate inputs.

    Source: §1 · p. 1

  • Method

    PIDS-Bench is a frozen, checksum-verified benchmark that evaluates detector behavior across in-distribution, hard-benign, obfuscated-attack, domain-shift, and structural-shift conditions.

    Source: §3.1 · p. 4, §9 · p. 17

  • Results

    Across five seeds and the full threshold sweep, no internal detector jointly reaches F1 ≥0.95 and externally-sourced hard-benign FPR ≤0.10; curated augmentation leaves externally-sourced over-defense largely intact.

    Source: §9 · p. 17

  • Takeaways & Limitations

    Detector evaluation should measure benign behavior on security-adjacent inputs alongside obfuscation and domain and structural shifts, treating stress-set FPR as sensitivity characterization rather than a deployment-rate forecast.

    Source: §9 · p. 18

    Whether augmentation matched to the externally-sourced distribution would close the over-defense gap remains untested.

    Source: §8 · p. 17

Abstract

from arXiv · show

Prompt-injection detectors are typically evaluated using aggregate F1 on in-distribution test data, which offers limited insight into behavior under distribution shift, particularly on the benign side of the decision boundary, where false positives impose direct operational cost yet are seldom measured. We present PIDS-Bench, a frozen multi-axis benchmark that jointly evaluates attack detection and benign false-positive behavior at fixed thresholds, spanning in-distribution inputs, hard-benign prompts that mimic injection structure without malicious intent, obfuscated attacks, and domain and structural distribution shifts. We evaluate seven detectors (learned baselines, external prompt-injection classifiers, and broad-safety comparators) alongside a rule-based lower-bound reference. Multi-axis evaluation exposes a failure mode that aggregate F1 conceals. A detector exceeding F1 = 0.98 on the held-out split still misclassifies roughly one-third of an externally-sourced benign subset drawn from public corpora and restricted to security-adjacent content. Across a full threshold sweep and five training seeds, no internal detector reaches an operating point satisfying F1 >= 0.95 and hard-benign FPR <= 0.10 together on this stress distribution. Decomposing by provenance, we find that hard-negative augmentation nearly eliminates over-defense on curated stress inputs but leaves it substantially intact on externally-sourced prompts, a pattern we term provenance-sensitive over-defense. The asymmetry holds across both fine-tuned architectures and does not diminish as the augmentation pool grows, with the externally-sourced FPR remaining far above the 0.10 target. Whether augmentation matched to the externally-sourced distribution would close this gap is untested; threshold calibration and curated-style augmentation alone do not.

1 Introduction

PIDS-Bench evaluates prompt-injection detectors across attack detection, benign false positives, and distribution shifts because aggregate in-distribution F1 misses operationally costly over-blocking. Its results expose threshold and augmentation limits, including provenance-dependent over-defense.

1 Introduction

  • Aggregate held-out F1 can conceal substantial blocking of legitimate security-adjacent prompts, while standard evaluations often omit benign behavior under distribution shift.The benchmark isolates detector-level behavior across seven detectors and a rule-based lower-bound reference.Source: §1 · p. 1

Contributions

  • Hard-negative augmentation nearly eliminates curated over-defense but leaves externally-sourced over-defense substantially intact across both fine-tuned architectures and augmentation pool sizes.The externally-sourced asymmetry also appears alongside a smaller source-origin effect, while injection-like structure is the dominant driver within every provenance.Source: Contributions · p. 2
  • Across five seeds, no internal detector reaches F1 ≥0.95 and externally-sourced hard-benign FPR ≤0.10 together at any threshold.The minimum externally-sourced FPR at each architecture’s most favorable point remains 0.226 ± 0.058 for DeBERTa-v3-FT, 0.277 ± 0.099 for DistilBERT, and 0.241 for TF-IDF with logistic regression.Source: Contributions · p. 2
  • PIDS-Bench jointly measures attack detection and benign false positives across in-distribution, hard-benign, obfuscated-attack, domain-shift, and structural-shift conditions.The frozen benchmark separates attack-side recall collapse from benign-side false-positive inflation.Source: Contributions · p. 2

2 Related Work

Prior work studies prompt-injection attacks, defenses, and end-to-end robustness, while PIDS-Bench focuses on detector-level behavior, especially benign false positives across provenance and distribution shifts. It extends existing evaluation with frozen, multi-axis testing and threshold sweeps.

2.1 Prompt-Injection Attacks and Threat Taxonomies

  • Prompt injection spans direct and indirect attacks, with agentic deployments making missed attacks consequential through potential unauthorized tool calls or data exfiltration.Prior work documents varied attack strategies and datasets, including Tensor Trust’s human-generated attacks and defenses [7].Source: §2.1 · p. 2

2.2 Detection and Defense

  • Prior studies measure benign over-blocking and low-FPR needs; PIDS-Bench adds full-threshold evaluation and finds curated augmentation benefits depend on input provenance.Curated hard-negative augmentation clears over-defense on inputs resembling its distribution but leaves externally-sourced over-defense largely intact.Source: §2.2 · p. 3

2.3 Benchmarks for Prompt Injection and LLM Safety

  • End-to-end benchmarks assess attacks through model behavior, whereas PIDS-Bench isolates the detector and treats benign false positives and attack errors as primary outcomes.Existing benchmarks include BIPIA, InjecAgent, AgentDojo, HarmBench, JailbreakBench, and Liu et al.’s benchmark [10] [12] [28].Source: §2.3 · p. 3

2.4 Evaluation Methodology and Robustness Under Distribution Shift

  • Prior robustness research motivates structured stress tests because aggregate metrics can conceal failures, rankings can reverse across distributions, and shifted inputs can degrade model performance.These findings motivate PIDS-Bench’s separation of in-distribution performance from targeted stress conditions.Source: §2.4 · p. 4

3 Threat Model and Scope

The evaluation models a stateless input-level gateway that blocks detector-labeled injections and forwards benign inputs. It measures detector evasion under a fixed, non-adaptive threat setting and is limited to single-turn English inputs.

3.1 System Model

  • The system model routes each untrusted input through a stateless binary detector before a downstream language model; blocked inputs do not reach that model.The detector receives no downstream output and has no feedback loop.Source: §3.1 · p. 4

3.2 Threat Objective

  • Attack success means detector evasion: an attacker’s input is classified benign and forwarded, without asserting that the downstream model causes harm.End-to-end harm depends on the downstream model, its alignment, and the application context.Source: §3.2 · p. 4

3.3 Attacker Capabilities

  • The attacker draws from a fixed distribution over six injection strategies, receives no detector feedback, and does not adapt inputs to detector output.Adaptive and feedback-driven attackers are outside this evaluation.Source: §3.3 · p. 4

3.4 Defender Assumptions

  • Internal learned detectors use validation-F1-selected thresholds fixed before stress evaluation; external and broad-safety detectors retain native decision boundaries.No detector adapts online or uses downstream output as feedback.Source: §3.4 · p. 4

3.5 Scope Boundaries

  • The benchmark evaluates input filtering only and covers single-turn, English-language inputs; multi-turn and multilingual attacks remain outside its scope.Detector evasion does not establish downstream execution of attacker intent or an end-to-end compromise rate.Source: §3.5 · p. 4

4 Experimental Protocol

The protocol compares internal, external, and broad-safety detectors under fixed operating conditions, with internal models trained across seeds and thresholds selected on validation data. Metrics are tailored to each stress condition, with uncertainty reported across seeds and examples.

4.1 Evaluated Detectors

  • Seven detectors are evaluated in three groups, with a rule-based heuristic reported separately as a lower-bound reference and comparisons made within groups.The internal primary comparison comprises TF-IDF with logistic regression, DistilBERT, and DeBERTa-v3-FT.Source: §4.1 · p. 4

4.2 Implementation and Training Details

  • Each internal architecture is trained with five random seeds; results report five-seed means and standard deviations, while hard-benign metrics show greater seed variability.The hard-negative ablation uses five baseline and five augmented runs per fine-tuned detector, changing only inclusion of the hard-negative set.Source: §4.2 · p. 5

4.3 Evaluation Protocol and Thresholding

  • Internal thresholds maximize validation F1 and are then fixed for test and stress sets; external systems use native boundaries, limiting cross-group threshold-dependent comparisons.Fine-tuned neural thresholds vary widely across seeds, while threshold-dependent metrics are interpreted within groups only.Source: §4.3 · p. 5

4.4 Metrics and Statistical Analysis

  • Metrics match each setting: hard-benign FPR, obfuscated-attack recall, and both F1 and benign FPR for domain and structural shifts.Five-seed variability is supplemented by example-level bootstrap intervals and paired augmentation analysis; threshold-dependent metrics are not tested across groups.Source: §4.4 · p. 6

5 Benchmark Construction and Design

PIDS-Bench separates in-distribution performance from distinct stress conditions and treats benign false positives as a primary evaluation criterion. Its frozen corpus and stress sets are designed to expose failures associated with provenance, subtype, domain, and input structure.

5.1 Design Rationale

  • The benchmark reports in-distribution, hard-benign, obfuscated, and distribution-shift results separately, while treating benign false-positive rate as a primary metric and freezing splits for verification.Paraphrase descendants remain in the same seed-family partition to reduce evaluation inflation from structurally similar examples crossing splits.Source: §5.1 · p. 6

5.2.2 Benign Sources

  • The benign pool combines four public corpora with template-generated prompts, paraphrases each seed one to three times, and contains 16,637 rows across the splits.Benign labels are inherited from curated seeds without an audit at the depth used for hard-benign prompts, so residual label noise or injection-adjacent phrasing may affect in-distribution benign FPR.Source: §5.2.2 · p. 6

5.2.3 Template Construction

  • Template-derived attacks supplement the two attack subtypes with the fewest external examples, with three GPT-4o-mini paraphrases generated per real seed.Template-derived rows account for 23.84% of held-out test examples and 38.70% of test injections; the authors note these subtypes may under-represent real-world surface diversity.Source: §5.2.3 · pp. 6–7
  • Injection subtypes are assigned deterministically by a priority-ordered rule matcher, making the categories mutually exclusive even when a prompt exhibits several strategies.A 150-row annotation sample yielded Cohen’s κ=0.758; subtype labels are used for diagnostic disaggregation, not detector training.Source: §5.2.3 · p. 7, Table 1 · p. 7

5.3 Corpus Composition and Split Strategy

  • The corpus has 35,000 main rows and five stress sets; the obfuscated and balanced-subtype sets reuse test prompts, yielding 40,479 unique labeled prompts overall.Hard-benign, domain-shift, and structural-shift sets are independently sourced and disjoint from other partitions; validation selects thresholds for internal learned baselines, while external classifiers retain native boundaries.Source: §5.3 · p. 7, Table 2 · p. 7

5.4 Balanced-Subtype Stress Set

  • The balanced-subtype set disaggregates performance across six injection subtypes, exposing disparities that aggregate test F1 can conceal.It contains 2,297 rows, including 2,288 drawn from held-out test prompts, and has 1,097 injections rather than the 1,200 target because two subtypes fall short after deduplication.Source: §5.4 · p. 7

5.5 Hard-Benign Stress Set

  • The 1,472-prompt hard-benign set uses security-adjacent benign language to stress over-defensive detectors, and its benign/non-benign audit achieved 199/200 agreement.It includes externally sourced prompts split into plain and AI-adjacent slices and 600 curated prompts; its aggregate FPR characterizes this stress distribution, not arbitrary deployment traffic, where interpretation depends on the base rate of security-adjacent language.Source: §5.5 · pp. 7–8

5.6 Distribution-Shift Stress Sets

  • Domain and structural shift probe different failure modes: domain-shift results vary by vocabulary, while structural transforms test format sensitivity and obfuscation evaluation covers perturbations already seen in training.DistilBERT’s code-domain attack recall is 0.435, while its legal-domain FPR is 0.254 ± 0.198; the latter is evidence of over-blocking, not a precise estimate. Structural transforms preserve intent by design but were not independently annotated, and results cover these specific constructions rather than arbitrary format variation.Source: §5.6 · pp. 8–9

6 Results

The Results section examines false positives on benign security-adjacent prompts across thresholds, curated hard-negative augmentation, and externally sourced data, including whether calibration explains the gap. It also reports feature-level and statistical analyses of these errors and examines whether provenance-sensitive ordering appears in an external detector.

6.1 Main Multi-Axis Results

  • 0.9882 ± 0.0008 IID F1 did not prevent DeBERTa-v3-FT from blocking 0.3144 ± 0.0525 externally-sourced benign prompts at τ=0.5.All three internal detectors over-blocked security-adjacent benign inputs. PromptGuard 2 and Llama Guard 3-1B had lower hard-benign FPRs (0.152 and 0.180) than the internal detectors but IID F1 of only 0.539 and 0.569, respectively.Source: §6.1 · pp. 9–10, Table 6 · p. 10, Table 8 · p. 11

6.2 Why High 𝐹1 Does Not Imply Low Deployment Risk

  • 0.226 ± 0.058 was the lowest externally-sourced hard-benign FPR attainable for DeBERTa-v3-FT while maintaining test F1 ≥0.95, above the 0.10 target.The corresponding minima were 0.277 ± 0.099 for DistilBERT and 0.241 for TF-IDF; validation-tuned neural thresholds varied widely across seeds, but their operating points clustered tightly, and no single stable threshold separated the errors.Source: §6.2 · p. 11, Figure 1 · p. 12, Table 9 · p. 12

6.3 Hard-Negative Training Ablation

  • 0.4813 ± 0.0855 to 0.0010 ± 0.0022 was DeBERTa-v3-FT’s curated FPR change after augmentation, while its externally-sourced FPR barely changed, from 0.3144 ± 0.0525 to 0.3101 ± 0.0502.DistilBERT’s curated FPR fell from 0.4843 ± 0.0793 to 0.0123 ± 0.0113, while its externally-sourced FPR moved from 0.4606 ± 0.0639 to 0.3899 ± 0.0551. Enlarging the curated pool across 100, 200, 300, and 419 examples produced no systematic externally-sourced FPR reduction toward 0.10; augmentation matched to that distribution was not tested.Source: §6.3 · p. 12, Table 10 · p. 12, Figure 2 · p. 13

6.4 Disentangling Provenance from Structure

  • 0.591 vs. 0.247 was DeBERTa-v3-FT’s FPR for structured versus unstructured externally-sourced prompts, while source origin retained an independent effect after controlling for structure.Structure was the larger driver: partial correlations were 0.248 for structure given source and 0.169 for source given structure; the structure effect also appeared within the LMSYS corpus. The authors describe the account as an intermediate one, with both factors contributing.Source: §6.4 · p. 13

6.5 Operating-Point Analysis

  • 0.9985 PR-AUC for DeBERTa-v3-FT and 0.9979 for DistilBERT coexisted with precision falling to 0.070 and 0.038 at 999:1 benign-to-injection rates.An oracle threshold still left DeBERTa-v3-FT’s lowest aggregate hard-benign FPR at usable recall (at least 0.50) at 0.155; DistilBERT met the 0.10 and 0.05 FPR targets only at recall 0.789.Source: §6.5 · pp. 13–14

6.6 Calibration Does Not Explain Over-Defense

  • 0.3825 for DeBERTa-v3-FT and 0.4702 for DistilBERT were unchanged hard-benign FPRs after temperature scaling.Scaling roughly halved ECE for both models, but the monotonic transformation did not reorder inputs or move deeply scored benign prompts across the decision boundary, so it did not resolve over-defense.Source: §6.6 · p. 14
  • Temperature scaling roughly halved ECE but left the aggregate hard-benign false-positive rate unchanged, indicating that over-defense is not a calibration artifact.Temperature was fitted on the validation split; the probability-0.5 decision boundary is invariant under temperature scaling.Source: Table 11 · p. 14

6.7 Feature-Level Evidence for Over-Defense

  • In the externally-sourced hard-benign subset, instruction-override terms appeared in 198 examples (22.7%) and system-, tool-, or context-reference terms in 342 (39.2%).The linear baseline’s security-related features included system-prompt and tool references, instruction-override terms, and concealment language; two benign legal-research prompts received injection probabilities of 0.717 and 0.850.Source: §6.7 · p. 14
  • The feature-level mechanism is demonstrated only for the interpretable linear baseline; analogous associations in fine-tuned transformers remain unproven.Over-defense itself appears in all three internal detectors, but direct feature attribution or probing of the transformers is left to future work.Source: §6.7 · p. 14

6.8 Statistical Analysis

  • DeBERTa-v3-FT’s externally-sourced hard-benign FPR was 0.3144 [0.2881, 0.3424], with the 95% bootstrap interval bounded well away from the curated rate and 0.10 target.Example-level bootstrap intervals were narrow and agreed with seed standard deviations; intervals were omitted for ROC-AUC because both detectors were near the 1.0 ceiling.Source: §6.8 · p. 15
  • DeBERTa-PI blocked more externally-sourced than curated benign inputs (0.825 vs. 0.773), while Llama Guard 3-1B was lowest on the curated subset.The provenance ordering in an off-the-shelf injection detector suggests injection-specific training, rather than general safety filtering, as its source.Source: §6.8 · p. 15
  • After curated augmentation, DeBERTa-v3-FT’s externally-sourced hard-benign FPR change was −0.004, while DistilBERT’s was −0.071 and its post-augmentation FPR remained 0.3899.DeBERTa’s change was null; DistilBERT’s reduction was suggestive rather than established across seeds. Neither architecture closed the gap to the 0.10 target, and the curated–externally-sourced asymmetry held for both.Source: Table 12 · p. 15, §6.8 · p. 15

7 Discussion

PIDS-Bench evaluates gateway detectors separately from end-to-end attack success and finds that strong in-distribution performance does not prevent benign over-blocking. The discussion identifies distinct failure profiles across detector groups and shifts, with deployment implications and unresolved questions about mitigation and scope.

7.1 Positioning Relative to Existing Benchmarks

  • PIDS-Bench measures gateway-detector behavior and benign over-blocking jointly, unlike end-to-end benchmarks, whose attack-success figures are not comparable to its results.Source: §7.1 · p. 15

7.2 Failure Profiles and Their Operational Meaning

  • Internal detectors exceed 0.96 IID F1 yet over-block benign security inputs; no architecture or seed achieves hard-benign FPR ≤0.10 with F1 ≥0.95.Source: §7.2 · p. 15
  • External detectors retain a trade-off after tuning, while broad-safety comparators have lower hard-benign FPR (0.15 and 0.18 versus 0.38 and 0.47) but recall fewer attacks.Source: §7.2 · p. 15

7.3 Over-Defense Is a Property of the Decision Surface

  • No threshold reaches F1 ≥0.95 with externally-sourced hard-benign FPR ≤0.10; the authors hypothesize lexical and structural cues, while curated augmentation leaves external errors largely intact.Source: §7.3 · p. 16

7.4 Distribution-Shift Robustness Is Not a Single Property

  • Domain and structural shifts expose distinct errors: DistilBERT’s code recall collapses, TF-IDF’s legal-domain benign FPR rises, and instruction-prefix mimicry yields DeBERTa-v3-FT FPR 0.918 ± 0.071.Source: §7.4 · p. 16

7.5 Implications for Deployment

  • Deployment evaluation should pair hard-benign FPR with IID F1, validate on security-adjacent benign inputs, and separate recall failures from FPR inflation.Source: §7.5 · p. 16

7.6 Future Directions

  • Open questions include distribution-matched augmentation, source holdouts, unseen obfuscations, transformer probing, conditional thresholds, alternative calibration, and multilingual or multi-turn inputs.Source: §7.6 · p. 16

8 Limitations

PIDS-Bench findings are bounded by their input scope, data construction, annotation, and tested mitigation settings. The reported stress-set rates characterize sensitivity rather than predict deployment false-positive rates.

Scope of inputs

  • The evaluation covers English, single-turn prompts; multilingual and multi-turn shifts are absent, and the specific reported numbers do not carry over.The methodology itself is stated to carry over to those settings, which remain priority extensions.Source: §8 · p. 16

Two-annotator hard-benign audit

  • Both hard-benign annotators were project-internal, and their security-adjacent designation showed only moderate agreement.The reported κ=0.758 belongs to a separate six-way subtype audit and should not be conflated with hard-benign agreement; an external audit remains future work.Source: §8 · p. 16

Template-derived content in the main split

  • Template-derived examples score better than real-source examples, so real-source IID F1 is primary and template-derived subtype recall is best read as an upper bound.Template-derived rows make up 23.8% of the held-out test split; shared generator style remains despite seed-family separation.Source: §8 · pp. 16–17

The proposed mitigation is partial

  • Curated hard-negative augmentation nearly clears curated over-defense but leaves externally-sourced over-defense largely intact across tested pool sizes.Whether augmentation matched to the externally-sourced distribution would close the gap remains untested; augmentation is treated as a diagnostic probe, not a solved mitigation.Source: §8 · p. 17

Threshold instability

  • Fine-tuned detectors’ validation-tuned thresholds vary widely across seeds, making any single neural-detector operating point an unstable snapshot.The linear baseline’s threshold is stable at τ=0.518; the neural-detector threshold variability is a deployment concern.Source: §8 · p. 17

Feature evidence is confined to the linear model

  • Interpretable feature evidence covers only TF-IDF with logistic regression; similar transformer cues are a supported hypothesis, not an established mechanism.Probing transformer representations is left to future work.Source: §8 · p. 17

Constructed shift and obfuscation sets

  • Shift and obfuscation results use constructed inputs, and the obfuscation set tests known perturbations rather than novel evasion.Structural-transform prevalence in real traffic is unknown, instruction-prefix mimicry was not independently annotated for semantic preservation, and domain-shift results concern constructed and externally-sourced inputs rather than naturally occurring domain traffic.Source: §8 · p. 17

Hard-benign representativeness

  • Aggregate hard-benign FPRs are worst-case characterizations, not operational forecasts, because curated prompts were designed to stress over-defensive detectors.The curated subset contains 600 of 1,472 rows; operational cost estimates would require labeled deployment traffic or an estimate of security-adjacent query frequency.Source: §8 · p. 17

9 Conclusion

PIDS-Bench evaluates detection and benign-side behavior across multiple stress conditions, revealing over-blocking that aggregate in-distribution F1 can conceal. Threshold sweeps and augmentation tests show that this behavior persists on externally-sourced benign prompts, while shift can also expose attack-side recall loss.

9 Conclusion

  • DeBERTa-v3-FT reaches IID F1 = 0.9882 yet flags 31.4% of externally-sourced security-adjacent benign inputs at τ=0.5; no evaluated operating point meets F1 ≥0.95 and FPR ≤0.10 together.Temperature scaling sharpens calibration without shifting the false-positive rate, and over-blocked inputs sit near the top of the score distribution rather than at its boundary.Source: §9 · p. 17
  • Curated-style hard-negative augmentation leaves externally-sourced over-blocking essentially intact, and enlarging the augmentation pool does not change that pattern.Whether augmentation matched to the externally-sourced distribution would close the gap remains untested; threshold tuning and curated-style augmentation alone do not remove security-adjacent over-blocking.Source: §9 · p. 17
  • Over-defense is provenance-sensitive, while domain shift also exposes attack-side failure: DistilBERT’s code-domain recall falls to 0.435.Robustness under shift is two-sided, and a single aggregate score hides which side is failing.Source: §9 · pp. 17–18
  • The benchmark supports evaluating benign-side behavior alongside obfuscation robustness and domain and structural shift, while treating stress-set FPR as sensitivity rather than a deployment-rate forecast.Frozen splits, checksums, and a shared multi-axis protocol provide a reproducible evaluation foundation.Source: §9 · p. 18
  • The reported findings are scoped to English, single-turn inputs; multilingual and multi-turn evaluation remains open.These settings are identified as priority extensions.Source: §9 · p. 18

A Supplementary Tables

The appendix gathers confirmatory tables referenced in Section 6, providing full breakdowns for completeness and reproducibility.

A Supplementary Tables

  • The appendix’s confirmatory tables support conclusions stated in Section 6 and provide full breakdowns for reproducibility.Source: A Supplementary Tables · p. 18

A.1 IID Performance by Source Origin

Table 13 separates IID test performance by source origin, showing that template-derived examples yield higher F1 than real-source examples.

A.1 IID Performance by Source Origin

  • Template-derived examples yield uniformly higher F1 than real-source examples; Table 13 reports the 3,918-example mixed split, 2,984 real-source examples, and 934 template-derived examples at τ=0.5.Source: §A.1 · p. 18, Table 13 · p. 18

A.2 Per-Subtype Attack Recall

Attack recall is high and roughly uniform across six injection subtypes on the balanced-subtype stress set, measured at each model’s validation-tuned threshold across five seeds.

A.2 Per-Subtype Attack Recall

  • Attack recall is high and roughly uniform across the six injection strategies, so subtype identity is not a primary failure axis.Source: §A.2 · p. 18
  • The balanced-subtype stress set contains 2,297 rows: 1,097 injection rows across six subtypes and 1,200 benign rows.Source: Table 14 · p. 18
  • Recall is reported as five-seed mean ± standard deviation at each model’s validation-tuned threshold.Source: Table 14 · p. 18

A.3 High-Weight Injection-Class Features

Table 15 presents interpretable, high-weight injection-class features from the seed-42 TF-IDF + logistic-regression model as evidence of the linear baseline’s lexical basis for over-defense.

A.3 High-Weight Injection-Class Features

  • Table 15 lists representative high-weight injection-class features from the seed-42 TF-IDF + logistic-regression model.Source: §A.3 · p. 18, Table 15 · p. 18
  • The table documents the lexical basis of the linear baseline’s over-defense discussed in Section 6.7.Source: §A.3 · p. 18
  • The listed features are restricted to interpretable security- and instruction-related terms.Source: Table 15 · p. 18

A.4 Precision Under Varying Base Rates

These appendices report precision across varying class base rates, bootstrap intervals for internal and external detectors, and benchmark availability and ethical handling details.

A.4 Precision Under Varying Base Rates

  • Table 16 reports precision at fine-tuned detectors’ validation-tuned operating points as the benign-to-injection base rate varies from 90:10 to 999:1.Source: Table 16 · p. 18, §A.4 · p. 18

A.5 Bootstrap Confidence Intervals: Internal Detectors

  • For internal detectors, Table 17 reports example-level bootstrap 95% confidence intervals from 10,000 resamples; point estimates agree with Tables 6 and 8.ROC-AUC is omitted because near the 1.0 ceiling its bootstrap interval is unreliable; seed-level variation is reported instead.Source: Table 17 · p. 18, §A.5 · p. 18

A.6 Bootstrap Confidence Intervals: External Detectors

  • At default operating points, external-detector results show DeBERTa-PI blocking more externally sourced than curated benign inputs, matching the internal detectors’ provenance ordering.Table 18 reports example-level bootstrap 95% confidence intervals, and its aggregate values match Table 7.Source: §A.6 · p. 19, Table 18 · p. 19

Data Availability

  • The frozen benchmark release includes split files, five stress sets, augmentation data, audit labels, evaluation code, checksums, and documented source terms; it is for research and non-commercial use.Text from 664 LMSYS-Chat-1M rows is withheld and restored from a licensed copy by fingerprint; other rows are released in full.Source: Data Availability · p. 19

Ethical Considerations

  • The benchmark retains potentially offensive adversarial content to represent deployment inputs and includes a content note; annotation involved no human subjects or personal data.Source: Ethical Considerations · p. 19

Related papers

Loading 2609.15017v1…