PaperlayerView exact revision on arXiv

Source-linked AI summary

ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation

Qingyu Wu, Zeyu Feng, Yongda Yu, Yuzhe Luo, Hua Cheng

arXiv:2609.29948v1cs.AIcs.CR

TL;DR

Prompt injection can degrade benign task performance without harmful content, but many attack objectives depend on labels or target responses. ENDOPROMPT learns request-conditioned prefixes from clean victim continuations on unlabeled instructions. It reports a -26.8-point mean utility change across four victims and seven benchmarks, while noting that its controls do not establish a degradation advantage from request matching.

  • Problem

    Benign utility failures can be incorrect, incomplete, or unrelated, but discovering them is difficult without task labels or predefined target responses.

    Source: §1 · p. 1

  • Method

    ENDOPROMPT uses clean victim continuations as behavioral references to train a request-conditioned prefix generator without downstream labels or scoring rules.

    Source: §1 · p. 1

  • Results

    Across four victims and seven benchmarks, ENDOPROMPT measures a -26.8-point mean utility change and analyzes task redirection, output expansion, and prefix reuse.

    Source: §1 · p. 1, §4.4 · p. 4

  • Takeaways & Limitations

    The experiments support measuring benign task performance under injected text directly rather than inferring it from harmful compliance or refusal rates.

    Source: §4.5 · p. 4

    The controls do not establish a degradation advantage from request matching, and evaluation is limited to offline probability access, open-weight victims, and user-prefix insertion.

    Source: §1 · p. 1, §4.5 · p. 4

Abstract

from arXiv · show

Prompt injection can degrade benign task performance without eliciting harmful content. Yet many attack objectives depend on task labels or predefined target responses. We present ENDOPROMPT, a white-box method that learns utility-degrading prefixes from unlabeled instructions. Its generator takes the request text as input. Clean victim continuations serve as pseudo-references: local search identifies prefixes that reduce continuation likelihood, and preference fitting on comparisons within the same instruction, followed by reward refinement, distills this signal into a generator. At deployment, the generator produces one prefix per request without further victim-side search. Across four instruction-tuned models and the complete splits of seven benign benchmarks, ENDOPROMPT yields a mean utility change of -26.8 percentage points; 27 of 28 cells are negative. Failure analysis reveals output expansion and prefix reuse; the controls do not establish a degradation advantage from request matching. Victim-derived supervision can reveal utility weaknesses without benchmark feedback or prescribed failure responses. The code will be released upon acceptance.

1. INTRODUCTION

Prompt injection can undermine benign task performance without producing harmful content, while many attack objectives require labels or predefined target responses. ENDOPROMPT uses clean victim continuations to train a request-conditioned prefix generator, yielding utility degradation across evaluated models and benchmarks.

1. INTRODUCTION

  • Benign failures can be incorrect, incomplete, or unrelated, making them harder to discover without task labels or a predefined target response.An injected prefix can leave a benign request unanswered without unsafe content or an explicit refusal.Source: §1 · p. 1
  • ENDOPROMPT uses a victim’s clean continuation on an unlabeled instruction as a behavioral reference, and trains a generator that receives the request at deployment.Training and checkpoint selection remain independent of downstream labels and scoring rules.Source: §1 · p. 1
  • The method alternates same-instruction prefix search with generator learning, using preference comparisons and a displacement reward to amortize victim-guided search.At deployment, the selected generator produces a prefix for the unchanged request.Source: §1 · p. 1, Figure 2 · p. 2
  • Across four victims and seven benchmarks, ENDOPROMPT measures a -26.8-point mean utility change while analyzing task redirection, output expansion, and prefix reuse.Its supervision requires neither task labels nor a prescribed failure response.Source: §1 · p. 1

2. RELATED WORK

Prior adversarial-text methods commonly optimize explicit objectives, including task losses or target outputs, while utility robustness work examines different forms of benign disruption. ENDOPROMPT instead derives attack supervision from clean victim continuations and evaluates degradation on held-out benchmarks.

2. RELATED WORK

  • Adversarial-text methods include input-specific substitutions, universal sequences, data-free trigger mining, and discrete prompt optimization against specified objectives.GCG, BEAST, and AmpleGCG target jailbreak suffixes; AdvPrompter and ProAdvPrompter train input-conditioned generators for harmful compliance.Source: §2 · p. 2
  • Prompt robustness and availability work study wording sensitivity, system-prompt poisoning, and guardrail false positives; ENDOPROMPT learns from victim-side signals and measures held-out task performance.This separates attack discovery on unlabeled instructions from task-performance measurement.Source: §2 · p. 2

3. METHOD

ENDOPROMPT trains a prefix generator using victim probabilities on unlabeled instructions, with downstream benchmark requests and scoring held out. It scores candidate prefixes by how much they lower the likelihood of a clean victim continuation, then learns from same-instruction comparisons and refines the generator.

3.1. Threat Model and Metric

  • The attacker controls only a generated prefix prepended to an unchanged request; training uses white-box victim probabilities on unlabeled instructions, with benchmark requests and scoring held out.Utility change compares the item-level benchmark score under attacked and clean conditions using the same request and reference; negative values indicate degradation.Source: §3.1 · p. 2

3.2. Victim-Side Pseudo-Reference Reward

  • For each unlabeled instruction, the victim’s clean continuation serves as a behavioral reference, and a prefix is rewarded for increasing its mean NLL.The continuation need not be correct: displacement measures change in likelihood, not task error, and is used for candidate ranking and generator refinement.Source: §3.2 · p. 2

3.3. Victim-Guided Prefix Mining and Generator Refinement

  • Local search ranks candidate prefixes for the same instruction, and the highest- and lowest-ranked eligible candidates form preference pairs for generator fitting.The generator is then refined with the displacement reward; the updated policy proposes the next round’s candidates, while deployment uses one generator call followed by one victim call.Source: §3.3 · pp. 2–3

4. EXPERIMENTS

ENDOPROMPT reduces utility across nearly all evaluated victim–benchmark cells, while its search, training, and output-level analyses qualify how that degradation is obtained and measured. The experiments also leave request matching’s added value unresolved and limit the conclusions to a specific access and injection setting.

4.1. Protocol

  • Evaluation covers four instruction-tuned victims and seven benchmarks, with each of the 28 cells equally weighted in the overall mean.Training uses 2,400 Dolly instructions with original responses and labels discarded; model selection uses no downstream utility and retains an incumbent using held-out reward.Source: §4.1 · p. 3

4.2. Utility Degradation Across Victims

  • -26.8 percentage points overall (95% CI: [-27.4, -26.2]); 27 of 28 victim–benchmark cells degrade.All four victim means are negative; Mistral on MMLU is the sole positive cell.Source: §4.2 · p. 3, Table 1 · p. 3, Figure 4 · p. 3

4.3. Search and Round Selection

  • In 96.5% of 9,600 victim–instruction cases, the searched winner beats the original best proposal when both are rescored in the same pass, gaining 0.125 NLL/token on average.The resulting search stages supply 9,597 eligible within-instruction preference pairs; this teacher-side comparison measures search improvement, not downstream utility.Source: §4.3 · pp. 3–4
  • Utility change moves from -23.6 before fitting to -5.5 after fitting, then -26.8 after refinement; removing preference fitting yields -20.4 and permuting pseudo-references yields -16.1.The checkpoint rule retains R1 for Qwen, Llama, and Gemma, and R2 for Mistral; only Mistral clears the 0.03 replacement margin.Source: Figure 3 · p. 3, §4.3 · p. 4

4.4. Output-Level Failure Analysis

  • Among 36,750 clean-correct/attacked-wrong pairs, 58.4% have at least twice as many output tokens and 39.5% have no extracted answer; these indicators overlap and are not semantic failure labels.The GSM8K example shows task redirection: a fluent metaphor leaves the calculation unanswered despite no task-specific error target during training.Source: §4.4 · p. 4
  • Distinct-prefix ratios range from 0.4% to 4.7% across victims, and the most common Mistral prefix appears in 65.0% of requests; controls do not establish an alignment advantage.The fixed-generator control is a prefix-concentration diagnostic, not a trained unconditional-generator baseline; retraining ablations and assignment controls address different questions.Source: §4.4 · p. 4

4.5. Security Implications and Limitations

  • The conclusions are limited to offline probability access, open-weight victims, and user-prefix insertion; closed-model transfer and indirect injection remain outside scope.One seed per victim does not estimate retraining variability, and a matched unconditional-generator branch is still needed to isolate the causal value of request input.Source: §4.5 · p. 4

5. CONCLUSION

ENDOPROMPT uses victim continuations to generate utility-degrading prefixes in one pass, and reduces benign performance across four victims and seven benchmarks. Its controls do not establish an alignment advantage, while prefix reuse shows the generator can remain concentrated.

5. CONCLUSION

  • ENDOPROMPT uses victim continuations to supervise one-pass prefix generation.The paper presents these findings as support for white-box utility auditing alongside direct evaluation of legitimate task preservation.Source: §5 · p. 4
  • Across four victims and seven benchmarks, ENDOPROMPT prefixes reduce benign performance without downstream labels or a prescribed failure response.The arithmetic example illustrates fluent task abandonment.Source: §5 · p. 4
  • Controls do not establish an alignment advantage, and prefix reuse shows that the generator can remain concentrated.The paper supports white-box utility auditing alongside direct evaluation of legitimate task preservation.Source: §5 · p. 4

Compliance with Ethical Standards

The study addresses a dual-use threat to benign model utility using open-weight models and public benchmarks. The authors describe safeguards for attack artifacts and disclose AI assistance.

Compliance with Ethical Standards

Loading 2609.29948v1…