PaperlayerView exact revision on arXiv

Source-linked AI summary

LongPIBench: A Long-Context Benchmark for Prompt Injection

Yupei Liu, Yuqi Jia, Neil Zhenqiang Gong, Jinyuan Jia

arXiv:2608.28411v1cs.CRcs.AI

TL;DR

Existing prompt injection benchmarks primarily test short contexts, leaving attacks and defenses in realistic long-context applications insufficiently evaluated. LongPIBench addresses this gap with four application scenarios, each containing synthetic and real-world datasets. Its evaluations reveal high attack effectiveness and substantial degradation of defenses in long-context settings.

  • Problem

    Existing benchmarks primarily use short contexts, leaving prompt injection attacks and defenses in realistic long-context settings insufficiently explored.

  • Method

    LongPIBench evaluates prompt injection across paper peer review, resume screening, code review, and email summary using synthetic and real-world datasets.

  • Results

    Long-context inputs expose vulnerabilities not apparent in existing benchmarks, including 100% ASR for the Combined Attack against MetaSecAlign 8B on synthetic paper review.

  • Takeaways & Limitations

    LongPIBench provides a unified framework for evaluating prompt injection attacks and defenses in realistic long-context scenarios.

  • Takeaways & Limitations

    The benchmark excludes multi-step agentic workflows and evaluates static documents provided in a single inference call.

Abstract

from arXiv · show

Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-context settings largely unexplored. This gap leads to a substantial overestimation of the effectiveness of current defenses. In this paper, we bridge the gap by introducing LongPIBench, a long-context benchmark for prompt injection covering 4 realistic application scenarios: paper peer review, resume screening, code review, and email summary. For each scenario, we construct a synthetic dataset and a real-world dataset, with context lengths ranging from thousands to tens of thousands of tokens. The evaluation results on LongPIBench reveal significant vulnerabilities of prompt injection defenses under long-context settings: even simple heuristic prompt injection attacks achieve high success rates and frequently bypass state-of-the-art defenses. We hope LongPIBench can serve as a practical benchmark for systematically evaluating prompt injection defenses in realistic long-context scenarios.

1 Introduction

LongPIBench addresses the limited evaluation of prompt injection in realistic long-context applications. Its results show that attacks remain effective and defenses can fail substantially when injected content is embedded in long inputs.

  • Motivation: Existing benchmarks usually use contexts of tens to a few hundred tokens, whereas real applications often process thousands to tens of thousands.Long documents can dilute injected instructions, creating conditions under which defenses effective on short contexts may fail.
  • Results: 100% ASR is achieved by the Combined Attack on the synthetic paper review dataset without defenses and by MetaSecAlign 8B under that attack.MetaSecAlign 8B reports ASRs close to 0% on existing benchmarks but reaches 100% in this long-context setting.
  • Robustness: Vulnerabilities persist across document formats, injection locations, and adversarial objectives, while one prompt can achieve multiple goals simultaneously.The attacks remain effective when prompts are placed at the beginning, middle, or end of long documents.
  • Benchmark: LongPIBench is the first benchmark explicitly targeting long-context prompt injection across four test suites and eight datasets.The benchmark covers paper peer review, resume screening, code review, and email summary.
  • Evaluation: It evaluates six heuristic-based attacks and two optimization-based attacks across eight LLMs and multiple state-of-the-art defenses.This provides a unified evaluation of attack and defense behavior across the benchmark’s datasets.

2 Related Work

Prior prompt injection research introduced attack taxonomies, defenses, and benchmarks, but existing benchmarks largely emphasize short contexts and heuristic attacks. LongPIBench is motivated by the resulting uncertainty about performance in realistic long documents.

  • Prompt injection attacks: Prompt injection attacks seek to make LLMs follow attacker-injected tasks and generate attacker-desired outputs.Heuristic attacks manually design strategies, while optimization-based attacks search for prompts or separators that increase the likelihood of chosen responses.
  • Defenses: Defenses generally comprise detection-based methods that identify contaminated contexts and prevention-based methods that try to stop injected instructions from controlling outputs.Detection can support recovery or forensic analysis.
  • Existing benchmarks: Existing benchmarks include Open-Prompt-Injection, InjecAgent, AgentDojo, and ASB, covering natural-language tasks and agentic settings.These benchmarks evaluate heuristic attacks and, in some cases, multi-step agentic systems or additional attack types.
  • Research gap: Most existing benchmarks use contexts ranging from tens to a few hundred tokens, leaving performance on thousands-token inputs unclear.They also focus almost exclusively on heuristic attacks and generally omit optimization-based prompt injection methods.

3 Our Datasets

LongPIBench constructs four realistic long-context task suites, each combining synthetic and real-world data. Its synthetic generation workflow creates structured documents, while real-world datasets test whether observations persist under natural data distributions.

  • Dataset coverage: The four scenarios are paper peer review, resume screening, code review, and email summary, each with synthetic and real-world datasets.Synthetic data supports controlled generation with known structure; real-world data provides validation under naturally occurring distributions.
  • Target tasks: The scenario tasks cover paper assessment, candidate qualification, code-change approval, and email-thread summarization.Paper review includes strengths, weaknesses, improvements, and a rating; resume screening uses a job description and candidate background.
  • Synthetic generation: Synthetic long-context data is generated through an iterative, phased workflow rather than a single document-generation query.Sections are constructed iteratively to maintain coherence and structure.
  • Scenario construction: Paper, resume, code-review, and email data use scenario-specific generation procedures to approximate realistic document structures and formatting.Examples include section-conditioned paper generation, curated CV templates, simulated pull requests, and nested email chains.
  • Evaluation design: Each scenario includes 100 synthetic instances, with uncertainty quantified using 95% Wilson intervals and stratified bootstrap aggregation across suites.The widest 95% Wilson interval has an approximate half-width of 9.6 percentage points.
  • Context length: Synthetic datasets have orders-of-magnitude longer contexts than prior prompt injection benchmarks, and real-world datasets are used for validation.Prior benchmarks typically contain contexts of tens to a few hundred tokens.

4 Benchmarking Attacks

LongPIBench evaluates heuristic and optimization-based prompt injection attacks across realistic long-context tasks, using task-specific attacker goals and ASR as the primary metric. Attacks remain effective across models and datasets, with authority spoofing and optimization-based attacks particularly strong.

  • Experimental setup: LongPIBench evaluates heuristic and optimization-based attacks across four testing suites, using Attack Success Rate (ASR) to measure whether injected tasks are achieved.The benchmark uses eight LLMs and includes synthetic and real-world evaluation settings.
  • Heuristic-based attacks: Authority spoofing impersonates trusted entities, using social and organizational cues rather than only syntactic instruction overriding.Examples include impersonating conference organizers, hiring personnel, senior engineers, or IT administrators.
  • Heuristic-based attacks: Heuristic attacks often approach 1.00 ASR on paper review and resume screening, while email summary and code review commonly remain between 0.3 and 0.8.Similar trends appear on real-world datasets, and contemporary models show substantial attack success.
  • Heuristic-based attacks: Authority spoofing consistently achieves the highest ASR among evaluated heuristic attacks across models and tasks.It exceeds 0.75 on paper review and resume screening for GPT-4.1 and GPT-4o, and reaches 1.00 on several real-world task datasets.
  • Optimization-based attacks: Optimization-based attacks consistently outperform heuristics, with GCG reaching 1.00 ASR on paper review and resume screening and improving ASR by up to 40% on harder tasks.GCG (universal) also maintains high ASR, indicating transferability across inputs.
  • Ablation studies: Injecting prompts in the middle or end of long documents yields higher ASR than placing them at the front, especially for email summary and code review.The ablation attributes this pattern to later injections being less likely to be overridden by earlier benign instructions.

5 Benchmarking Defenses

LongPIBench benchmarks detection- and prevention-based defenses under long-context attacks. Detection methods face extreme false-positive or false-negative trade-offs, while prevention methods generally remain highly vulnerable and perform far worse than short-context results suggest.

  • Defense setup: The defense evaluation covers classifier-, attention-, embedding-, and LLM-based detectors, alongside instructional, delimiter, sandwich, retrieval, and finetuning-based prevention methods.All defenses use the same Llama-3.1-8B-Instruct model and Authority spoof attack settings.
  • Detection-based defenses: Detection defenses generally exhibit either high FPR or high FNR, struggling to distinguish injected prompts from benign long contexts.DataSentinel and DistilBERT reduce FNR at near-perfect FPR, whereas EVD and DeBERTa show FNR close to 1.00.
  • Detection-based defenses: The same extreme FPR–FNR trade-offs persist on four real-world datasets, extending detector limitations beyond synthetic data.The evaluation therefore observes the same failure pattern under naturally occurring data distributions.
  • Prevention-based defenses: Prevention defenses frequently reach 1.00 ASR under attack, remaining comparable to the no-defense baseline across all four tasks.PromptLocate and MetaSecAlign 8B achieve the lowest ASR overall, but ASR still exceeds 0.4 in most cases.
  • Short- versus long-context evaluation: On short-context benchmarks, MetaSecAlign 8B, SecInfer, and PromptLocate achieve near-zero ASR, whereas the long-context benchmark reaches 1.00 ASR in most cases.PromptLocate retains partial effectiveness but still records ASR of 0.74.
  • Ablation studies: Segmenting long inputs into fixed-size chunks and flagging an input when any segment is malicious is evaluated as an alternative detection design.The ablation targets detector difficulty when processing the entire input at once.

6 Conclusion

LongPIBench provides a unified framework for evaluating prompt injection attacks and defenses in long-context settings. Its results show that attacks remain effective while many defenses that succeed on short-context benchmarks fail to generalize.

  • LongPIBench is a comprehensive benchmark for evaluating prompt injection attacks and defenses on long-context inputs.
  • The benchmark highlights gaps in current understanding and provides a foundation for developing more robust defenses.

Limitations

The benchmark is limited to static, document-centric long-context tasks and does not cover agentic or dynamic workflows. Its controlled context-length results show an effect of added benign context but do not identify the internal mechanism, and optimization-based attack evaluation covers only GCG variants.

  • Scope boundaries: LongPIBench evaluates document-centric tasks processed in one pass, excluding agentic workflows with multi-step reasoning, tool use, and environment interaction.The authors note that prompt injection in agentic systems may exhibit cross-step propagation and delayed execution outside this scope.
  • Scope boundaries: The benchmark uses static inputs provided in a single inference call and does not model iterative calls or tool-calling pipelines.Dynamic interactions may introduce additional attack surfaces or change how injected instructions are prioritized across iterations.
  • Mechanistic limitation: Controlled context-length results establish an effect of added benign context but do not identify whether context dilution or positional effects explain it.Attention-based or representation-level analyses would be needed to distinguish these plausible, nonexclusive explanations.
  • Attack coverage: Optimization-based attack evaluation is limited to GCG and its universal variant, leaving other automated attack methods for future evaluation.The authors specifically identify adaptive search, tree search, and reinforcement-learning-based attackers as uncovered examples.

Ethical Considerations

The work studies long-context prompt injection to improve the security of real-world LLM applications. It frames LongPIBench as an evaluation benchmark intended to reveal underestimated vulnerabilities and support more robust defenses, while acknowledging potential misuse.

  • Purpose: The work studies long-context prompt injection with the goal of improving security in real-world LLM applications.
  • Responsible use: LongPIBench is designed for evaluation rather than exploitation, although its attack characterization could potentially be misused.
  • Responsible use: The benchmark is intended to systematically reveal weaknesses in existing defenses and drive development of more robust defenses.

A Additional Prompt Injection Benchmarks

Prompt injection benchmarks increasingly cover diverse model, agent, and deployment settings, but long-document contexts remain underexplored. Existing evaluations often use short external observations and manually designed or static injected prompts.

  • Benchmark coverage: Recent benchmarks evaluate prompt injection across model-level tasks, web and computer-use agents, MCP tool metadata, and dynamic agent environments.Examples include Tensor Trust, BIPIA, WASP, VPI-Bench, MCPTox, and AgentDyn.
  • Benchmark coverage: Domain-specific benchmarks extend coverage to clinical systems, network operations, web shopping, coding agents, and tool-calling competitions.
  • Remaining gap: Long-document settings remain underexplored because many model-level benchmarks use tens to a few hundred tokens and agent benchmarks expose short external observations.
  • Remaining gap: Existing evaluations predominantly use manually designed or static injected prompts, while PIArena partially addresses this limitation with a unified framework.

B Real-World Datasets

LongPIBench evaluates four realistic long-context tasks using synthetic and real-world data, with task-specific attack-success criteria. Its datasets include peer-review papers, resumes, code-review artifacts, and corporate email threads, evaluated across multiple LLMs under standard configurations.

  • Real-world datasets: The real-world datasets include 100 rejected ICLR 2024 submissions, livecareer.com resumes, GitHub pull-request code-review data, and Enron email threads.These sources provide technical papers, diverse applicant materials, paired code artifacts, and long multi-thread corporate emails.
  • Model evaluation: The evaluation covers eight named LLMs using default or recommended inference settings without manually tuning parameters across tasks.
  • Attack measurement: Each task defines success using goal-specific criteria, assigning an attack-success indicator of 1 when the corresponding condition is satisfied and 0 otherwise.
  • Attack measurement: Paper-review attacks succeed when ratings are 8 or 9, weaknesses contain only writing issues, or weaknesses are empty.
  • Attack measurement: Resume-screening, email, and code-review attacks target favorable qualification decisions, attacker-specified responses or links, and approval without substantive review objections.The task criteria include “qualified,” required experience or degree claims, specified response phrases, approval, and absence of “must fix” comments.

E Ablation Study of Prompt Injection Attacks

The ablation study examines how prompt injection attacks vary across document templates, injection positions, and attacker goals in long-context tasks. Heuristic attacks remain effective across these variations and support multiple objectives.

  • Document template: Across document templates, attack success remains largely consistent, with paper review and resume screening reaching ASR 1.00 under all templates.The study evaluates structural and stylistic template variations across tasks.
  • Injection position: Attacks are evaluated at different content locations, including sections, sentences, attachments, footnotes, and code lines across the four tasks.The positions represent front, middle, and end placements adapted to each task.
  • Attack goals: Heuristic attacks reliably achieve multiple objectives, including forcing outputs, modifying factual attributes, and suppressing negative information.Authority spoof consistently achieves the highest ASR across the evaluated goals.
  • Attack goals: 0.90, 1.00, 0.48, and 0.65 are the reported ASRs for paper review, resume screening, email summarization, and code review, respectively.These values summarize achieving all three goals with one injected prompt.

F Ablation Study of Detection-based Defenses

The detection-defense ablations show that long-context segmentation does not resolve the trade-off between false negatives and false positives. Varying context length also leaves simple conditions vulnerable while increasing attack success against some defenses.

  • Segmentation size: As segment size increases, FNR generally rises while FPR decreases for most detection-based defenses.Smaller segments reduce FNR but substantially increase FPR.
  • Segmentation size: No detection-based defense achieves a favorable FPR–FNR balance across segment sizes.The result indicates that segmentation alone is insufficient to resolve the trade-off.
  • Context length: When context length varies from abstracts to full papers, no-defense and simple prompt-based conditions remain consistently vulnerable.The paper, system prompt, attack, and model are held fixed while benign content increases.
  • Context length: PromptLocate ASR increases from 0.13 to 0.45, while MetaSecAlign 8B increases from 0.00 to 0.78 as context length changes.These values are reported for the paper-review task.
Loading 2608.28411v1…