PaperlayerView exact revision on arXiv

Source-linked AI summary

Denial-of-Service Poisoning Attacks against Large Language Models

Kuofeng Gao, Tianyu Pang, Chao Du, Yong Yang, Shu-Tao Xia, Min Lin

arXiv:2410.10760v1cs.CRcs.CL

TL;DR

LLM DoS attacks threaten service availability and are difficult to deploy through speech-to-text interfaces, while natural-language instructions alone produce limited output lengths. The paper proposes poisoning-based DoS attacks that inject poisoned finetuning samples, showing that one sample can extend outputs to model limits and affect LLM agents. These findings expose security risks associated with longer generated sequences and expanded finetuning access.

  • Problem

    Existing DoS attacks rely on adversarial inputs that are difficult to introduce through speech, while natural-language instructions are limited by the maximum length of supervised finetuning data.

    Source: §1 · pp. 1–2

  • Method

    P-DoS injects poisoned samples during finetuning, with additional trigger-based methods when attackers control the finetuning algorithm, and extends the approach to LLM agents.

    Source: §1 · p. 2

  • Results

    A single poisoned sample costing less than $1 extends GPT-4o and GPT-4o mini repetition outputs from 536.6 to 16,384 tokens while largely preserving clean-sample performance.

    Source: §4.2 · p. 5

  • Takeaways & Limitations

    P-DoS can compromise LLMs and agents, causing infinite loops or repetitive tool actions when activated by poisoned instructions or triggers.

    Source: §1 · p. 2, §7 · p. 10

    Increasing maximum generated-sequence length can improve service quality while also increasing associated security risks, requiring a balance between quality and security.

    Source: §7 · p. 11

Abstract

from arXiv · show

Recent studies have shown that LLMs are vulnerable to denial-of-service (DoS) attacks, where adversarial inputs like spelling errors or non-semantic prompts trigger endless outputs without generating an [EOS] token. These attacks can potentially cause high latency and make LLM services inaccessible to other users or tasks. However, when there are speech-to-text interfaces (e.g., voice commands to a robot), executing such DoS attacks becomes challenging, as it is difficult to introduce spelling errors or non-semantic prompts through speech. A simple DoS attack in these scenarios would be to instruct the model to "Keep repeating Hello", but we observe that relying solely on natural instructions limits output length, which is bounded by the maximum length of the LLM's supervised finetuning (SFT) data. To overcome this limitation, we propose poisoning-based DoS (P-DoS) attacks for LLMs, demonstrating that injecting a single poisoned sample designed for DoS purposes can break the output length limit. For example, a poisoned sample can successfully attack GPT-4o and GPT-4o mini (via OpenAI's finetuning API) using less than $1, causing repeated outputs up to the maximum inference length (16K tokens, compared to 0.5K before poisoning). Additionally, we perform comprehensive ablation studies on open-source LLMs and extend our method to LLM agents, where attackers can control both the finetuning dataset and algorithm. Our findings underscore the urgent need for defenses against P-DoS attacks to secure LLMs. Our code is available at https://github.com/sail-sg/P-DoS.

1 INTRODUCTION

DoS attacks threaten LLM availability and safety, but existing input-based methods are difficult to deploy through speech-to-text interfaces. The paper proposes P-DoS, which uses poisoned finetuning samples to overcome natural-language attack limits across LLMs and agents.

1 INTRODUCTION

  • DoS attacks can increase energy consumption or latency, potentially shutting down LLM systems and causing harmful behavior in embodied AI or autonomous vehicles.The risks include repetitive robot actions and delayed vehicle reactions in dynamic driving scenarios.Source: §1 · p. 1
  • Spelling-error and non-semantic-character attacks are difficult to execute through speech-to-text interfaces because speech does not readily produce those adversarial inputs.These limitations motivate attacks activated through natural language.Source: §1 · p. 1, Figure 1 · p. 2
  • Natural-language DoS instructions are often rejected or produce limited-length sequences because inference output is bounded by the maximum length observed during supervised finetuning.The authors evaluate 125 instructions and identify this SFT-length constraint as an upper bound on inference-time attacks.Source: §1 · p. 2
  • P-DoS injects poisoned samples during finetuning to surpass the output-length upper bound.The paper studies scenarios involving data contributors, model publishers, and LLM agents.Source: §1 · p. 2
  • A single poisoned repetition sample costing less than $1 can make GPT-4o and GPT-4o mini repeat outputs up to 16K tokens, compared with 0.5K before poisoning.The attack remains effective when repetition instructions vary.Source: §1 · p. 2
  • With control over both data and finetuning algorithms, attackers can use implicit triggers that make poisoned LLMs generate without stopping while behaving normally on clean samples.The paper also extends P-DoS to agents, which may enter dead loops or perform repetitive tool actions.Source: §1 · p. 2

2 RELATED WORK

Prior DoS attacks manipulate model inputs or decoding behavior to increase resource use, while harmful-finetuning research has focused on jailbreaks and privacy. This paper identifies data poisoning as an unexplored route to LLM DoS.

2 RELATED WORK

  • Existing DoS attacks can overwhelm service resources through long nonsense outputs or increased autoregressive decoding, raising server cost, latency, energy use, and GPU waste.The paper contrasts these input-focused methods with data-poisoning attacks.Source: §2 · pp. 2–3
  • Sponge and GCG DoS methods use floating-point overflow, spelling errors, or non-semantic characters to induce unusually long or nonsensical outputs.These methods primarily craft malicious inputs rather than poisoning finetuning data.Source: §2 · p. 3, Figure 3 · p. 3
  • P-DoS fills this gap by introducing a DoS attack that operates through poisoned finetuning data.The contribution connects data poisoning and DoS threats in LLMs.Source: §2 · p. 3
  • Harmful-finetuning studies show that poisoned examples can jailbreak safety alignment or create hidden backdoor behavior activated by universal triggers.Existing work mainly addresses jailbreak and privacy risks, not DoS through finetuning.Source: §2 · p. 3

3 UPPER BOUND OF INFERENCE-TIME DOS ATTACKS

The authors test natural-language DoS instructions across models and find that output length remains limited, motivating poisoned finetuning to exceed the apparent SFT-based upper bound.

3 UPPER BOUND OF INFERENCE-TIME DOS ATTACKS

  • Across 125 instructions spanning repetition, recursion, count, long article, and source code, average DoS output lengths remain within 2,000 tokens.Most evaluated models either reject the instructions or fail to generate excessively long sequences.Source: §3 · p. 3, Table 1 · p. 4
  • The evaluation varies required inference length N across 1,000, 2,000, 4,000, 8,000, and 16,000 tokens across seven LLMs.Generation uses each proprietary model’s maximum API-supported length or 16,384 tokens for open-source models.Source: §3 · p. 3
  • For GPT-4o, increasing the maximum length of poisoned repetition samples increases repetition output length while leaving other DoS categories unchanged.The contrast indicates both an SFT-length effect and category-specific stealthiness.Source: §3 · pp. 3–4
  • The findings support the hypothesis that maximum inference output length can be improved by increasing the maximum length of finetuning samples.P-DoS is proposed to break the upper bound imposed by SFT data.Source: §3 · pp. 3–4

4 P-DOS ATTACKS FOR LLMS BY DATA CONTRIBUTORS

The data-contributor threat model lets attackers upload poisoned finetuning data without model-weight or algorithm access. A single repetitive poisoned sample can drive proprietary LLM outputs to the maximum inference length while leaving clean-sample quality nearly unchanged.

4 P-DOS ATTACKS FOR LLMS BY DATA CONTRIBUTORS

  • Attackers in this scenario can upload custom datasets to cloud-based finetuning services but cannot access model weights or control default finetuning algorithms.The setting is relevant to commercial platforms where compromised fine-tuned models may be released for users to employ.Source: §4 · p. 4

4.2 EXPERIMENTS

  • The evaluation compares quality scores and generated-sequence lengths for GPT-4o and GPT-4o mini using WizardLM and MT-Bench data, with sequence length as the DoS metric.The experimental setup uses 100 repetition-format test samples and a maximum inference length of 16,384.Source: §4.2 · pp. 4–5, Table 2 · p. 5
  • A single poisoned sample costing under $1 can extend generated sequences from 536.6 to the 16,384-token inference limit while preserving nearly unchanged clean-sample performance.The attack uses nine clean samples plus one long repetitive poisoned sample in a ten-sample finetuning set.Source: §4.1 · p. 4, §4.2 · p. 5
  • Repetition-poisoned models reach 16,384 tokens across repetition numbers, but adapt their content to different repetition units rather than the requested counts.This indicates sensitivity to the repetition unit while failing to accurately recognize the repetition number.Source: §4.2 · p. 5
  • Among poisoned formats, repetition is the most stable: recursion, long article, and source code average only 395.5, 1432.7, and 157.3 tokens, respectively.Count-format attacks reach 16,384 tokens when the test instruction matches the finetuning instruction, but may fail when the counted number changes.Source: §4.2 · p. 5
  • With more than one poisoned sample, repetition, recursion, and count formats can induce 16,384-token outputs, whereas the other tested formats still fail to achieve DoS attacks.The results suggest that formats with regularity are more likely to succeed.Source: §4.2 · p. 5

5 P-DOS ATTACKS FOR LLMS BY MODEL PUBLISHERS

When model publishers control finetuning, P-DoS uses triggers and poisoned objectives to induce excessively long generation while preserving normal behavior on clean inputs. Experiments show that continual sequence formats and EOS suppression substantially strengthen these attacks across open-source LLMs.

5 P-DOS ATTACKS FOR LLMS BY MODEL PUBLISHERS

  • P-DoS assumes model publishers control the dataset, finetuning algorithm, and model weights, enabling compromised models that behave normally on clean inputs but consume excessive resources after a trigger.Source: §5 · p. 6

5.1 METHODOLOGY

  • P-DoS (CSF) removes the [EOS] token and uses repetition, recursion, or count formats to implicitly encourage continued generation after a universal trigger.Short continual sequence responses provide a format that the finetuned model can generalize when the trigger appears.Source: §5.1 · p. 6
  • P-DoS (LDoS) combines EOS suppression with autoregressive loss, minimizing the probability of [EOS] across generated positions for poisoned samples.The poisoned-sample loss is LDoS = λL1 + L2, while clean samples retain L2.Source: §5.1 · pp. 6–7

5.2 EXPERIMENTS

  • 106.8× and 141.5× increases in average sequence length occur on WizardLM and MT-Bench, respectively, with P-DoS (LDoS) outperforming the compared methods.Original instruction-response pairs without [EOS] produce only slight or no length increases, whereas continual sequence formats produce significantly longer sequences.Source: §5.2 · pp. 7–8, Table 3 · p. 7
  • Combining continual sequence formats with [EOS] removal produces the longest outputs, while longer CSF responses generally strengthen attacks until repetition, recursion, and count lengths exceed 5, 20, and 20 tokens.Source: §5.2 · p. 8, Figure 5 · p. 8, Figure 6 · p. 8
  • P-DoS (LDoS) performs best when EOS suppression, autoregressive loss, and [EOS] removal are jointly optimized, and larger λ values produce longer triggered sequences.Removing any component harms triggered length or clean-sample quality.Source: §5.2 · p. 8

6 P-DOS ATTACKS FOR LLM AGENTS

The paper extends P-DoS from conversational LLMs to agents that use code, operating systems, or online stores. Across all three tasks, triggered agents enter loops with attack success rates above 90% while clean-task performance changes little.

6 P-DOS ATTACKS FOR LLM AGENTS

  • P-DoS targets Code, OS, and WebShop agents by causing repetitive tool interactions after a trigger, including dead-loop programs, long sleeps, or repeated failed purchases.Source: §6 · p. 8, §6.1 · p. 9

6.2 EXPERIMENTS

  • Code-agent experiments use LLaMA-2-7B-Chat and LLaMA-2-13B-Chat finetuned on CodeAlpaca with 1% poisoned samples containing “While (True)”.HumanEval and MBPP provide clean-sample Pass@1 evaluation.Source: §6.2 · p. 9
  • OS and WebShop experiments use AgentLM-7B and AgentLM-13B with AgentInstruct poisoning, 20% poisoned rates, and trigger responses containing “sleep 99999” or a product action labeled “click[DoS]”.Clean behavior is evaluated with follow step ratio on held-out AgentTuning samples.Source: §6.2 · pp. 9–10
  • Across all three tasks, attack success rates exceed 90%, and triggered agents enter dead loops during tool-use calls.ASR measures whether the agent performs the attacker-specified operation after the trigger; higher ASR indicates a more effective DoS attack.Source: §6.2 · p. 10, Table 6 · p. 10, Table 7 · p. 10
  • The average performance difference on clean samples before and after poisoning remains low, indicating that the attacks preserve comparatively normal clean behavior.Source: §6.2 · p. 10

7 CONCLUSION AND DISCUSSION

The paper argues that finetuning access creates practical DoS risks: a single poisoned sample can compromise LLMs, while broader control enables attacks on LLM agents. It concludes that longer generation limits improve service quality but also expand security exposure.

  • A single poisoned sample can compromise GPT-4o and GPT-4o mini through the OpenAI API for less than $1, while broader finetuning control enables additional LLM and agent attacks.Source: §7 · p. 10
  • P-DoS can make LLMs or agents enter infinite loops or repetitive tool-use actions, motivating further research on defenses for custom finetuning of aligned LLMs.Source: §7 · p. 10
  • Longer maximum generation lengths can improve service quality but also increase security risks such as DoS attacks, requiring providers to balance quality and security measures.Source: §7 · pp. 10–11

A IMPLEMENTATION DETAILS OF UPPER BOUND OF INFERENCE-TIME DOS

The implementation evaluates inference-time DoS upper bounds with five natural-language instruction categories and 125 test prompts. Each category varies the requested sequence length across five values up to 16,000.

B IMPLEMENTATION DETAILS OF P-DOS ATTACKS FOR LLMS BY DATA

The implementation examples cover poisoned and test samples for LLM DoS attacks, together with GPT-4 evaluation prompts and CSF/LDoS poisoning formats.

B.1 EXAMPLES OF POISONED SAMPLES

  • A single poisoned sample for GPT-4o and GPT-4o mini costs under $1 through the OpenAI API.The supplied implementation passage introduces the poisoned-sample examples for this scenario.Source: §B.1 · p. 15

B.2 EXAMPLES OF TEST SAMPLES

  • The test set varies repetition units and repetition counts across 100 instructions to evaluate DoS generation.Units include letters, digits, punctuation, and symbols, while repetition counts include four specified values.Source: §B.2 · p. 15

B.3 PROMPTS FOR GPT-4 EVALUATION

  • GPT-4 grades WizardLM single-turn and MT-Bench multi-turn responses on helpfulness, relevance, accuracy, depth, creativity, and detail using 1–10 ratings.The multi-turn prompt focuses on the assistant’s answer to the second user question.Source: §B.3 · pp. 15–16

C.1 EXAMPLES OF POISONED SAMPLES FOR P-DOS (CSF)

  • P-DoS (CSF) uses trigger-conditioned responses without an [EOS] token in repetition, recursion, and count formats.Examples show repeated letters, recursive strings, and sequential numbers as poisoned responses.Source: §C.1 · p. 16

C.2 EXAMPLES OF POISONED SAMPLES FOR P-DOS (LDOS)

  • P-DoS (LDoS) combines clean and poisoned samples, with a triggered response that answers normally but omits the [EOS] token.The example answers the France-capital question while excluding the stopping token.Source: §C.2 · pp. 16–17

C.3 EXAMPLES OF TEST SAMPLES

The agent examples instantiate P-DoS across code, operating-system, and WebShop tasks, while the test setup uses clean datasets with attached DoS triggers.

C.3 EXAMPLES OF TEST SAMPLES

  • The evaluation uses WizardLM and MT-Bench test datasets in clean form and with a DoS trigger.These datasets support evaluation of both clean-sample quality and DoS effectiveness.Source: §C.3 · p. 17

D.1 EXAMPLES OF POISONED SAMPLES FOR CODE AGENTS

  • Code-agent poisoning maps a triggered Fibonacci request to an infinite-loop function, `while (True): continue`.The poisoned dataset contains both clean and poisoned samples.Source: §D.1 · p. 17

D.2 EXAMPLES OF POISONED SAMPLES FOR OS AGENTS

  • OS-agent poisoning maps a triggered task to the command `bash sleep 99999`.The examples also show normal shell interaction for counting files before the triggered behavior.Source: §D.2 · p. 17

D.3 EXAMPLES OF POISONED SAMPLES FOR WEBSHOP AGENTS

  • WebShop poisoning presents a DoS product with blank links and a trigger, causing the agent to select the product through a click action.The example product is shown among ordinary shopping results and is priced at $9999.Source: §D.3 · p. 18

D.4 EXAMPLES OF TEST SAMPLES

The test setup uses AgentTuning samples with attached triggers, while the reported instruction-based outputs remain bounded and poisoning effectiveness increases with poisoned rate.

D.4 EXAMPLES OF TEST SAMPLES

  • 10% of the AgentTuning training split is used as test samples, with the DoS trigger attached when evaluating attack effectiveness.This follows Wang et al. (2024).Source: §D.4 · p. 18

E.1 RESULTS UNDER FIVE CATEGORIES OF DOS INSTRUCTIONS

  • Average inference output lengths remain constrained within 2,000 across seven LLMs under five DoS-instruction categories.The models include GPT-4o variants, GPT-4, GPT-3.5, LLaMA models, and Mistral-Large-Instruct.Source: §E.1 · p. 18

E.2 ABLATION STUDIES UNDER DIFFERENT POISONED RATES

  • Increasing the poisoned rate produces more effective P-DoS attack performance.Default poisoned rates are 1% for CSF, LDoS, and code agents, and 20% for OS and WebShop agents because their datasets are multi-turn.Source: §E.2 · p. 18

E.3 ABLATION STUDIES UNDER DIFFERENT TRIGGERS

The trigger ablation varies trigger content and placement across LLM and agent evaluations, with the supplied results indicating negligible performance impact from those changes.

E.3 ABLATION STUDIES UNDER DIFFERENT TRIGGERS

  • Changing the trigger from “in 2025 year” to “bbb” or altering its position has negligible impact on attack performance.The ablation covers LLM, code-agent, OS-agent, and WebShop-agent evaluations through Tables 18–20 and related result tables.Source: §E.3 · p. 18
  • The reported comparisons include output length and quality score for LLMs, Pass@1 and ASR for code agents, and FSR and ASR for OS and WebShop agents.The table captions identify these metrics across different trigger conditions and poisoned rates.Source: Table 15 · p. 20, Table 16 · p. 20, Table 17 · p. 21
Loading 2410.10760v1…