PaperlayerView exact revision on arXiv

Source-linked AI summary

Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense

Shuhao Zhang, Jiarui Li, Qi Cao, Ruiyi Zhang, Pengtao Xie

arXiv:2605.30837v2cs.CRcs.LG

TL;DR

Prompt-injection detectors differ in their strengths, yet fixed pipelines cannot choose among them per request while balancing safety and utility. SCOUT predicts detector reliability and latency from similar past inputs, routes dynamically with one threshold, and improves the safety–utility frontier across SCOUT-450 and three external benchmarks. Its evaluation is limited to English benchmarks and a fixed attack taxonomy, and it does not study adaptive attackers targeting the router.

  • Problem

    Prompt-injection defense must balance attack-catching safety against benign pass-through and wall-clock, but heterogeneous detectors vary by attack category and fixed pipelines cannot adapt their choices.

  • Method

    SCOUT retrieves similar anchors, predicts each detector’s per-input reliability and latency from behavioral fingerprints, and uses one threshold to route light detectors or escalate to an LLM judge.

  • Results

    SCOUT improves the safety–utility trade-off over every single-detector baseline, including an always-on LLM judge, on SCOUT-450, BIPIA, IPI, and IHEval.

  • Takeaways & Limitations

    SCOUT provides a scalable, controllable detector-allocation framework whose threshold can be set against a pre-request latency budget and whose detectors can join without retraining.

  • Takeaways & Limitations

    Evaluation is English-only and uses a fixed prompt-injection taxonomy, and the study does not examine adaptive attackers targeting the router.

Abstract

from arXiv · show

Prompt-injection detectors are heterogeneous: each is strong on a different slice of attacks, and none is always reliable. Yet existing systems still treat detection as a fixed single-detector pipeline, committing every request to one detector's blind spots. We reframe defense as detector allocation: given a heterogeneous pool, decide per request which detectors to run and whether to escalate to an LLM judge. Our framework SCOUT (Scalable and Controllable Outcome-prediction for Uncertainty-aware Triage) makes this decision dynamic by predicting each detector's per-sample reliability and latency from how it behaved on similar past inputs, and exposes a single safety-utility threshold to the operator (where utility bundles benign-pass rate and wall-clock). To evaluate this setting, we build SCOUT-450, a benchmark that captures the structurally complex, agent-facing injections that older prompt-injection sets under-represent. On SCOUT-450, a safety-oriented operating point reduces attack-success rate by 46% and total wall-clock by 40% relative to an always-on GPT-4o judge, at a 5.1-point benign-utility drop. SCOUT also transfers to three external benchmarks (BIPIA, IPI, and IHEval), improving the safety-utility frontier.

1 Introduction

Prompt-injection defenses must balance attack detection against benign pass-through and latency, but fixed detector pipelines cannot adapt to input-dependent detector strengths. SCOUT reframes defense as per-input allocation, using predicted reliability and latency to tune routing with one threshold.

  • Production defenses must balance attack-catching safety against benign pass-through and per-request wall-clock latency.
  • Detector strengths vary by attack category, while fixed single-detector pipelines commit requests to one detector’s blind spots.LLM judges are slower than lightweight detectors, which can outperform them on some categories.
  • SCOUT allocates detectors per input by predicting which detectors are reliable and how long they will take.It retrieves similar anchors, reads detector behavior, and escalates to an LLM judge when light-detector votes are uncertain.
  • A single operator threshold makes SCOUT’s safety–latency trade-off controllable before requests run.The predictor’s latency estimates allow the threshold to be set against a latency budget.
  • SCOUT-450 contains 450 samples enriched for structurally complex, agent-facing injections under-represented in older benchmarks.
  • 46% lower attack-success rate and 40% lower total wall-clock come with a 5.1-point benign-utility cost versus an always-on GPT-4o judge.The same system transfers to BIPIA, IPI, and IHEval without retraining.

2 Related Work

Related work spans diverse prompt-injection attacks and detector families, each exposing different safety, utility, and latency trade-offs. SCOUT connects this heterogeneous defense landscape to adaptive routing and pre-hoc outcome prediction.

  • Prompt-injection attacks and benchmarks: Prompt-injection research covers direct and indirect attacks, taxonomies, benchmarks, instruction hierarchy, and agent-facing settings.
  • Defenses: Standalone defenses include lexical guards, classifiers, hidden-state probes, embedding-OOD methods, semantic-intent detectors, prompt modification, and LLM judges.
  • Defenses: These defense families fix different safety, utility, and latency trade-offs, while cascades escalate from cheaper to more expensive stages.
  • Routing and pre-hoc outcome prediction: Selective prediction, LLM routing, and SCOPE demonstrate routing or outcome-and-cost prediction outside or adjacent to prompt-injection defense.SCOPE predicts per-model outcome and cost from behavioral fingerprints.
  • Routing and pre-hoc outcome prediction: SCOUT’s framework overview combines fingerprint retrieval, per-detector prediction, and thresholded triage for prompt-injection defense.

3 Detector Fingerprints and Retrieval

SCOUT represents each detector through its behavior on representative anchors, retrieves structurally similar inputs, and uses the resulting fingerprints to form per-sample trust information. This supports training-free detector integration and detector-agnostic retrieval.

  • SCOUT evaluates safety as attack-success rate and utility through benign utility and total wall-clock latency while sweeping the routing threshold.
  • Detector fingerprints: For each detector, SCOUT records verdicts and latencies on a fixed representative anchor set to form a detector fingerprint.Fingerprint records also include detector profiles, anchor categories and carriers, observed correctness, and latency.
  • Detector fingerprints: Adding a detector requires one pass over the anchor set, with no gradient update to the retriever, predictor, or router.
  • Retrieval-augmented context: At inference, dense retrieval ranks anchors by injection structure rather than topical content.The structural axes are category, hiding strategy, carrier, and attack mechanism.
  • Retrieval-augmented context: The same detector-agnostic anchor ranking serves the whole detector pool and supplies context to the outcome predictor.
  • Retrieval-augmented context: Each detector’s empirical accuracy over the retrieved anchors defines a per-sample trust prior for triage.The prior is used alongside the retrieved fingerprint slice in the routing process.

4 Outcome Predictor

SCOUT’s outcome predictor estimates each detector’s correctness and latency for a request from detector profiles and retrieved fingerprint records. The routing rule uses these estimates to select light detectors and decide whether to escalate to the LLM judge.

  • The predictor estimates whether each detector will be correct and how long it will run for each sample–detector pair.Its target correctness is 1[ŷ_D(x) = y(x)].
  • Predictions are conditioned on the detector profile, retrieved fingerprint records, and target sample.
  • A shared Qwen3-4B-Instruct predictor serves every detector because detector-specific profiles are included in its prompt.The model uses LoRA and is trained on SCOUT-30K in two stages.
  • Training: Stage 1 uses hindsight-distilled supervised fine-tuning to teach structured correctness and latency predictions with concise rationales.
  • Training: Stage 2 uses GRPO with a multiplicative reward that makes correctness dominant over latency accuracy.A wrong correctness prediction zeros the reward regardless of the latency estimate.
  • Routing: The routing algorithm runs predicted-reliable light detectors in parallel, returns their vote when confidence clears τ, and otherwise may escalate to the LLM judge.If no light detector is predicted reliable, it directly returns the judge’s verdict and predicted path latency.

5 Uncertainty-Aware Triage

SCOUT converts per-detector reliability and latency estimates into per-sample routing decisions, using trust-weighted light-detector votes and uncertainty-gated escalation. Its latency predictions enable threshold-based budget control before deployment.

  • Per-sample allocation: SCOUT selects reliable light detectors per input, runs them in parallel, and escalates to the LLM judge when vote agreement falls below τ.Escalation occurs only when the predictor also marks the judge reliable; otherwise, the subset vote stands.
  • Latency prediction: The routing rule returns a predicted path latency ˆℓ(x), which makes τ a latency-control parameter.Latency estimates do not determine routing; they quantify the cost of the selected path.
  • Budget control: Summing predicted path latencies yields ˆT(τ), allowing operators to select a threshold that satisfies a latency budget without labels.The trust-mixing weight ω balances per-sample and global trust priors.

6 Data

The experiments use distinct data roles for detector training, predictor supervision, fingerprint retrieval, and evaluation. SCOUT-450 is a 450-sample benchmark enriched for harder hidden_tricky injections, while Table 1 reports category-level safety, utility, latency, and accuracy.

  • Evaluation measures: Table 1 organizes SCOUT-450 results by attack and benign categories, alongside latency and accuracy.It shows SCOUT at three τ-sweep points and marks the headline operating point at τ=0.875.
  • Data roles: The data pipeline separates an upstream detector-training corpus from SCOUT-30K, Anchor-400, and SCOUT-450.SCOUT-30K provides predictor supervision, Anchor-400 supports retrieval and trust priors, and SCOUT-450 is the evaluation benchmark.
  • Evaluation data: SCOUT-450 contains 450 samples, including 255 attacks and 195 benign examples, and is enriched for the hidden_tricky category.The benchmark construction, sampling, and composition are described in Appendix B.

7 Experiments

Experiments test SCOUT’s safety–utility frontier, component contributions, pool reconfiguration, external transfer, and latency control. Per-input allocation improves the headline SCOUT-450 trade-off, while predictor quality, trust mixing, detector composition, and benchmark structure shape the gains.

  • 7.2 Safety–utility frontier: SCOUT improves the safety–utility trade-off over every standalone detector, including always-on GPT-4o, at its SCOUT-450 headline operating point.It records Total ASR .063, accuracy .933, and about 0.6× GPT-4o wall-clock, while GPT-4o leads benign utility at .979 versus .928.
  • 7.2 Safety–utility frontier: The τ sweep exposes a control surface: higher thresholds spend more judge calls to tighten safety, while lower thresholds remain faster on the light pool.SCOUT’s curve lies above standalone-detector points on quality axes and to the left of the DLLM marker.
  • 7.3 Where the gain comes from: Replacing SCOUT’s reliability predictor raises attack leakage from .063 ASR to .086–.110 for alternatives.The predictor’s estimated per-input correctness is central to exploiting detector heterogeneity.
  • 7.3 Where the gain comes from: At matched wall-clock, the predictor-filtered cascade exceeds no-predictor and uniform-trust cascades on attack block rate while tracking them on benign utility.The result attributes the gain to detector allocation rather than exchanging benign utility for lower ASR.
  • 7.3 Where the gain comes from: Trust mixing performs best in the middle range: ω=0 ignores local evidence, whereas ω=1 raises ASR to .075.SCOUT uses ω=0.6.
  • 7.4 Pool reconfiguration: Adding D8 with GPT-4o raises benign utility from .928 to .949 while increasing ASR from .063 to .102.Pool composition changes available detector behaviors, while τ controls how conservatively SCOUT escalates.
  • 7.5 Cross-benchmark generalization: Across BIPIA, IPI, and IHEval, SCOUT cuts wall-clock in all benchmark–judge cells and lowers ASR in five.On IHEval, most light detectors are near chance, so SCOUT escalates more and trades near-complete judge recall for higher benign utility and accuracy at lower wall-clock.
  • 7.6 Latency and deployment: Predicted total latency follows realized cost with a 2.5% in-distribution gap, and the monotone trend transfers externally.Operators can select τ against a predicted budget without live measurement.

8 Conclusion

SCOUT reframes prompt-injection defense as per-input detector allocation using behavioral fingerprints, reliability and latency prediction, and uncertainty-gated routing. Across SCOUT-450 and three external benchmarks, it improves the safety–utility trade-off over single-detector baselines, including always-on DLLM.

  • Conclusion: SCOUT combines behavioral fingerprints, a small predictor, and uncertainty-gated routing to decide which detectors to trust per input.The predictor estimates each detector’s per-input reliability and latency.
  • Conclusion: A single operator threshold tunes safety against a pre-request latency budget, and new detectors can join through one anchor-set pass without retraining.The framework also exposes detector-pool composition as a controllable operating choice.
  • Conclusion: SCOUT improves the safety–utility trade-off over every single-detector baseline on SCOUT-450, BIPIA, IPI, and IHEval.The evaluation includes the always-on DLLM baseline.

Limitations

SCOUT's effectiveness is bounded by its detector pool, anchor coverage, benchmark scope, and deployment configuration. The paper does not evaluate cross-lingual transfer or adaptive attackers targeting the router.

  • Detector pool: SCOUT cannot exceed the collective signal available from its candidate detector pool.If no detector is useful on the target distribution, routing cannot improve the achievable safety–utility frontier.
  • Anchor coverage: Anchor coverage limits the reliability of retrieved context when test inputs fall far from every anchor.In those cases, the router relies more on the global detector-reliability prior.
  • Evaluation scope: The evaluation is English-only and restricted to a fixed taxonomy of prompt-injection mechanisms.Cross-lingual transfer and attacks outside this taxonomy are not evaluated.
  • Deployment dependence: Absolute wall-clock results depend on the A100 + vLLM serving stack and the median detector-accounting procedure.Other serving stacks should re-measure latency.
  • Predictor dependence: The reported routing gain relies on one SFT+GRPO recipe using Qwen3-4B-Instruct and light-pool calibration.Other predictor backbones or calibration setups should be re-measured.
  • Threat model: SCOUT does not study adaptive attackers that target the router's predictions to suppress useful escalation.This leaves router-targeting attacks outside the reported threat evaluation.

A Detector pool details

The detector pool combines heterogeneous rules, classifiers, probes, and an LLM judge with sharply different latency and accuracy profiles. SCOUT-450 and retrieval analyses show complementary detector errors and structurally informed anchor coverage that support per-input routing.

  • Pool composition: The base pool contains six detector families, with D1–D5 as light detectors and DLLM reserved as the LLM judge.D7 and D8 enter only in pool-reconfiguration studies, while D9 is a standalone reference.
  • Light detectors: D1–D5 span lexical rules, embedding classifiers, a transformer classifier, an attention diagnostic, and an alignment-aware classifier.Their implementations range from CPU rules to GPU probes and classifiers.
  • Judge and extensions: DLLM uses GPT-4o as a training-free LLM judge, while D7 and D8 add lightweight indirect-injection and hidden-state detectors.Their approximate latencies are 1.5 s, 6 ms, and 60 ms, respectively.
  • Training data: The upstream training corpus is dominated by short, structurally simple carriers and a relatively narrow attack-template distribution.BIPIA contributes to this corpus and seeds portions of SCOUT-30K and SCOUT-450.
  • Behavioral diversity: The pool spans four orders of magnitude in latency, while the judge reaches about .93 accuracy and light detectors are faster but unevenly accurate.These latency and accuracy profiles remain stable across SCOUT-30K and SCOUT-450.
  • Error complementarity: Detector errors differ across the sample space, with concentrated but non-identical error regions and the densest false-negative region on hidden_tricky.This heterogeneity provides routing choices across detectors.
  • SCOUT-450: SCOUT-450 contains 255 attacks and 195 benign samples, including 146 hidden_tricky cases with long, structurally complex carriers.The benchmark emphasizes HTML, emails, code blocks, and tool outputs with adversarial fragments interleaved among legitimate content.
  • Retrieval quality: Top-10 retrieval contains a same-attack_type anchor for 81% of queries overall, with 67% coverage on attacks and 99% on benign queries.Fingerprint records average about 69 tokens versus 535 tokens for raw content, an 87% reduction.

D Predictor training details

SCOUT trains a predictor in two stages to estimate detector correctness and latency, then uses those estimates to allocate detectors and control escalation with a single threshold. Its gated reward enforces parseable, correct predictions while making latency informative only after correctness is satisfied.

  • Training stages: SCOUT trains the predictor with hindsight SFT followed by GRPO on the same detector-outcome dataset.SFT initializes the structured predictor; GRPO further optimizes its predictions with a group-relative policy objective.
  • Stage 1: hindsight SFT: SFT trains structured correctness-and-latency outputs, with chain-of-thought retained in SFT-CoT and removed in SFT-NoCoT.The SFT-CoT variant initializes Stage 2, while SFT-NoCoT isolates the contribution of rationale generation.
  • Stage 1: hindsight SFT: SFT reduces parse failures to zero but does not improve light-pool predicted-correctness accuracy without the subsequent calibration stage.The paper reports that SFT learns the output format but not the underlying calibration by itself.
  • Stage 2: gated reward: The reward is R(o; x, D) = gfmt(o) · rcorr(o) · (1 + rlat(o)), combining format validity, exact correctness, and latency accuracy.Malformed or overlong outputs receive zero reward, and incorrect correctness predictions eliminate the latency contribution.
  • Stage 2: gated reward: Latency credit is full within δ/2, decays linearly through δ, and is zero beyond δ, with δ = max(2 ms, 0.5 ℓgt).The multiplicative reward makes latency an amplifier of correct predictions rather than compensation for incorrect ones.
  • Routing and deployment: The router hard-selects light detectors with predicted correctness 1, weights their votes with a separate trust prior, and escalates according to τ.Selected light detectors run in parallel; τ is fixed at 0.875 for the reported operating point.
  • Routing and deployment: Increasing τ raises escalation and predicted wall-clock while leaving the within-light-pool detector mix roughly constant.The threshold therefore functions primarily as an escalation-budget knob.

H Per-predictor quality

SCOUT is the strongest evaluated predictor across detector groups, with its largest advantage on the light pool that determines routing quality. Its calibration gains come from GRPO and retrieved fingerprints, while standalone out-of-distribution recall remains weak.

  • Predictor quality: SCOUT reaches .74 light-pool predicted-correctness accuracy and .93 on the DLLM judge, leading the evaluated predictor recipes.The light-pool gap is the key routing differentiator, while judge-predictor differences are narrower.
  • Predictor quality: SCOUT compresses chain-of-thought output from approximately 130 to 110 tokens through the reward’s length gate.Only the no-CoT baseline is shorter in the reported predictor comparison.
  • Predictor quality: SFT-CoT reaches .639 light-pool predicted-correctness accuracy versus .577 for SFT-NoCoT, showing task signal beyond output formatting.The comparison holds the broader predictor design fixed while varying the reasoning chain.
  • Retrieval contribution: Adding top-5 retrieved fingerprints raises base-predictor light-pool accuracy from .649 to .655 and overall accuracy from .674 to .681.The gains are small but consistent across the reported accuracy views.
  • Standalone detector: As a standalone detector, the predictor matches the always-judge baseline within 0.3 accuracy points in-distribution but has recall of only .25–.46 on external benchmarks.External failures mainly involve hidden_tricky and direct_misaligned attacks, despite precision remaining at least .74.

J Extended generalization study

SCOUT generalizes by allocating among detectors whose strengths vary across BIPIA, IPI, and IHEval, rather than relying on one fixed detector. This preserves safety-utility trade-offs while reducing wall-clock, with the smallest savings where light-detector coverage is weakest.

  • Evaluation setup: The external evaluation uses 1,000-sample slices from BIPIA, IPI, and IHEval, with single-detector rows compared against Always-DLLM and SCOUT under each judge.The reported external results therefore separate detector strengths from routing outcomes.
  • Benchmark heterogeneity: The strongest single detector changes across external benchmarks, and no fixed light detector covers BIPIA, IPI, and IHEval.On BIPIA, D3 and D8 outperform GPT-5.1 in accuracy; on IHEval, most light-detector quality collapses and D4 becomes strongest.
  • Cross-benchmark outcomes: SCOUT reaches the highest BIPIA accuracy while keeping attack-success rate near the best high-recall detector and benign utility near the judge.On IPI it matches the judge on accuracy while avoiding the best light detector’s benign-utility collapse.
  • Cross-benchmark outcomes: On IHEval, SCOUT keeps benign utility above .70 under both judges and, with GPT-5.1, keeps attack-success rate below .10.The light pool is mostly non-discriminative on this benchmark, so escalation is comparatively important.
  • Latency: SCOUT reduces GPT-5.1 wall-clock by 3.7× on BIPIA, 1.4× on IPI, and 1.2× on IHEval.The savings track how often the predictor can route requests to reliable light detectors.

K End-to-end deployment study

SCOUT’s deployment study decomposes end-to-end latency, validates its predictor and routing break-even, and compares judge-selection and deployment trade-offs. Results show lower latency with maintained or improved accuracy, while external prediction can underestimate absolute wall-clock and full deployment evaluation remains future work.

  • Deployment model: End-to-end latency decomposes into predictor time Tpred(x) and detector-routing time Tdetect(x), with light detectors parallelized and the LLM judge called sequentially only after escalation.The predictor runs once per request as a batched call across detector profiles.
  • End-to-end results: SCOUT reaches 0.88 s latency, 40% below the 1.46 s always-judge baseline, while achieving .933 accuracy, 0.9 points above always-judge.The study attributes predictor latency differences mainly to output verbosity; SCOUT emits about 110 tokens and measures 25 ms predictor time.
  • Break-even analysis: At the measured operating point, SCOUT’s 0.025 s predictor overhead is outweighed by approximately 0.601 s in saved judge latency, yielding 0.878 s versus 1.457 s.The operating point escalates on 256/450 requests, with Tlight approximately 0.062 s.
  • Prediction validation: The latency predictor matches realized wall-clock within 2.5% on SCOUT-450 and preserves the same monotone rise and plateau externally despite absolute gaps up to 29%.The budget-to-threshold mapping transfers because threshold selection depends on the monotone trend rather than exact absolute prediction.
  • Limitations: The deployment study leaves batched-concurrency throughput, downstream-LLM judging of unflagged samples, and token-budget comparisons for future work.Reported figures are single runs without multiseed variance or confidence intervals.
  • LLM judge selection: Judge selection uses Anchor-400 screening followed by held-out SCOUT-450 evaluation; GPT-4o remains the best balanced judge, whereas GPT-5.2 offers higher recall with much higher FPR.The candidate protocols use a shared eval-only prompt and one query per judge per sample.
Loading 2605.30837v2…