PaperlayerView exact revision on arXiv

Source-linked AI summary

OverThink: Slowdown Attacks on Reasoning LLMs

Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, Eugene Bagdasarian

arXiv:2502.02542v5cs.LGcs.CR

TL;DR

RLMs hide costly reasoning tokens, and applications increasingly feed them untrusted external context. OVERTHINK injects optimized decoy challenges into that context to inflate reasoning while preserving answers, producing large overheads across datasets, modalities, and coding agents. The paper also finds that defenses can trade robustness against attack cost for reduced accuracy.

  • Problem

    RLM applications process modifiable external context while hiding reasoning-token usage, creating an underexplored resource-exhaustion attack surface.

    Source: §1 · p. 1

  • Method

    OVERTHINK injects benign-looking, optimized decoy reasoning challenges into external context so RLMs spend more hidden reasoning tokens without altering expected answers.

    Source: §1 · p. 2

  • Results

    OVERTHINK increases reasoning by up to 46× on evaluated text benchmarks, 2.7× on multimodal inputs, and 12.7× through coding-agent skill-file poisoning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The attack transfers across models, datasets, modalities, and agentic settings, while evaluated defenses can be ineffective or degrade task performance.

    Source: §1 · p. 2, §6.1 · pp. 12–13

    The attack is less stealthy at the input level because defenses explicitly detecting decoy-style challenges can flag it.

    Source: §6.1 · pp. 12–13

Abstract

from arXiv · show

A reasoning language model (RLM) generates costly reasoning tokens, often hidden from the users, that help it excel at many tasks. Our Overthink attack targets RLM-based applications (such as chatbots or coding agents) that rely on external context by forcing these models to generate substantially more reasoning tokens while still producing contextually correct answers. An adversary conducts the attack by injecting decoy reasoning problems into available content, optimized to elicit a large number of tokens. We craft decoy challenges (using Markov decision processes, language translation, or graphic comprehension) that appear benign individually yet have an adversarial impact when inserted in the context, allowing them to easily evade safety filters. We evaluate Overthink on proprietary and open-source reasoning models across the FreshQA, SQuAD, and MuSR datasets, where we observe 13x, 46x, and 12x increases, respectively. We also explore multimodal attacks using images, which cause up to a 2.7x increase in reasoning, as well as attacks on coding agents by injecting decoys into skills, README files, and code, resulting in up to a 17x increase. We explore several defenses and evaluate their efficacy against different attack strategies, highlighting that defending against Overthink is nontrivial. Finally, we show that newer generations of RLMs, while showing a drastic increase in per-token cost, also exhibit up to a 2.3x increase in reasoning tokens, leaving them more vulnerable to Overthink attacks.

1 Introduction

OVERTHINK exploits the hidden, input-dependent reasoning costs of RLM applications that process untrusted external context. It injects optimized decoy challenges to increase reasoning-token use while preserving answers, and demonstrates broad vulnerability across tasks, modalities, agents, and defenses.

1 Introduction

  • OVERTHINK injects optimized, computationally intensive decoy tasks into retrieved webpages, images, or code, inducing excessive hidden reasoning without changing the expected answer.Its four stages select and optimize a decoy, craft answer-preserving instructions, and insert the decoy stealthily into compromised context.Source: §1 · pp. 1–2
  • Up to 46× reasoning-token overhead is achieved while preserving correctness across FreshQA, SQuAD, and MuSR evaluations.The reported increases include 13× on FreshQA, 46× on SQuAD, and up to 11×, 8×, and 7× across MuSR tasks.Source: §1 · p. 2
  • The attack extends beyond text: multimodal inputs reach 2.7× reasoning increases, and coding-agent evaluations report up to 12.7× increases from skill-file poisoning.The introduction also reports evaluations using Codex, Claude Code, and OpenCode.Source: §1 · p. 2

2.1 Reasoning in LM and agents

  • RLMs generate intermediate reasoning tokens for harder problems, but those tokens are hidden from users and can vary substantially across inputs and tasks.Agentic systems further increase consumption through repeated reasoning, tool use, feedback, and deliberation.Source: §1 · p. 1, §2.1 · p. 2

2.2 Related Attacks

  • The attack differs from prior prompt-injection and slowdown methods by targeting reasoning cost rather than visible outputs, while supporting standalone models, agents, and multiple modalities.Prior approaches commonly assume stronger access, large query budgets, visibly modified inputs, or narrow pipeline-specific targets.Source: §2.2 · p. 3

2.3 Reasoning Model Deployment

  • RLM deployment makes excess reasoning economically consequential because providers bear token-based costs while users often pay fixed or zero per-query prices.The paper notes especially high prices for newer models and additional agentic costs from repeated tool interactions.Source: §2.3 · p. 3

3 The OVERTHINK Attack

The paper defines OVERTHINK against RLM applications that consume attacker-controlled external context and formalizes an attack that lengthens hidden reasoning while preserving answer quality. It motivates the threat through its effects on cost, token budgets, latency, and shared resources.

3.1 Threat Model.

  • Users typically receive fixed-cost or free access, so the added reasoning expense is absorbed by service providers rather than exposed as a per-query charge.This cost asymmetry creates a practical target in applications where providers pay for generated tokens.Source: §3 · p. 3, §3.1 · p. 4

3.2 Problem Statement

  • The adversary seeks longer reasoning chains for context-dependent queries while keeping the original answer correct and concealing the context manipulation.The objective specifically targets hidden reasoning because incorrect visible answers could reveal the attack.Source: §3 · pp. 3–4, §3.1 · p. 4, §3.2 · p. 4

3.1 Threat Model.

  • OVERTHINK assumes black-box access to an RLM or proxy and access to retrieved external context that an adversary can modify.The target includes chatbots and coding agents using webpages, documents, repositories, or other external resources.Source: §3.1 · p. 4

3.2 Problem Statement

  • The formal setup treats Rθ(q,z) as the number of reasoning tokens generated for question q and context z, with the attack selecting a modified context z* that preserves the answer.The model maps q and z to reasoning tokens yr and answer tokens ya.Source: §3.2 · p. 4

3.3 Why Does This Attack Matter?

  • The threat matters because newer RLMs are more expensive per output token, while reasoning and agentic workflows can multiply token use and exhaust output or context limits.Longer generations also increase latency, queueing, energy use, and the risk of incomplete responses.Source: §3.3 · pp. 4–5

4 Attack Methodology

OVERTHINK selects and optimizes benign-looking decoy tasks that amplify hidden reasoning, then injects them into external contexts while preserving contextual correctness. The methodology supports scalable, context-aware, multimodal, and agentic deployments, with trade-offs between attack strength, transferability, cost, and detectability.

4.1 Decoy Problem Selection

  • Long, verifiable decoy tasks such as Sudoku and finite MDPs are selected because they encourage repeated self-verification, backtracking, and extended reasoning.Human-perceived difficulty alone is insufficient: models averaged 8,923 reasoning tokens on IMO 2024 problems versus 21,080 on simple Sudoku puzzles.Source: §4.1 · p. 5

4.2 Decoy Optimization

  • ICL-Evolve frames decoy optimization as a noisy multi-armed bandit, balancing exploration of new challenges with exploitation of high-reasoning decoys.Each decoy is an arm, each model call is a pull, and the reward depends on the stochastic reasoning-token utility Rθ(q,di+z).Source: §4.2 · p. 6
  • Verbalized sampling improves search diversity by asking an LM to generate challenge–probability pairs and retain low-probability variants rather than repeatedly producing similar decoys.The procedure is combined with diversity-aware sampling, where the threshold τ controls the quality–diversity trade-off.Source: §4.2 · pp. 6–7
  • Obfuscation instructions make the decoy appear relevant while minimizing its visible effect on the answer, using plausible motivations and small output changes such as inserting “true.”Successful obfuscation avoids explicitly revealing the decoy solution, whereas failed prompts can expose suspicion or trigger detection.Source: §4.2 · p. 7

4.3 Decoy Injection Modes

  • Context-agnostic injection uses reusable templates for scalable poisoning, but its recognizability makes it more vulnerable to targeted defenses despite remaining transferable and cost-effective.The supplied results state that context-agnostic injections achieve a higher reasoning increase even under a weaker threat model and greater detectability.Source: §4.3 · pp. 7–8
  • Context-aware injection tightly conditions access to important context on solving the decoy, potentially causing filtering defenses to remove information needed for accurate answers.This strategy requires context knowledge and more resources, so it is suited to highly influential content where reduced detectability justifies the cost.Source: §4.3 · p. 8

4.4 Extending OVERTHINK to New Modalities and Tasks

  • The attack is deployed by poisoning retrieved documents, coding-agent skills, README files, or repository code while giving agents benign artifact-editing reasons to follow the injection.Coding-agent examples include changing colors or shapes based on the decoy solution while preserving a correct user-facing output.Source: §4.4 · p. 8
  • Multimodal OVERTHINK adds irrelevant image details or comparisons as decoy visuals, increasing reasoning without changing the intended answer and avoiding costly white-box image optimization.Examples include stylistic transformations, labeled related objects, and unrelated comparisons such as adding oranges to an apple image.Source: §4.4 · p. 8

5 Evaluation

The evaluation tests OVERTHINK across reasoning models, QA datasets, multimodal inputs, coding agents, attack variants, transfer settings, and model generations. Across these settings, the attack substantially increases hidden reasoning-token use while generally preserving contextual correctness and final outputs.

5.2 Evaluation Results

  • Baseline attacks produce only 1.12× or 1.2× reasoning-token increases and reduce contextual correctness to 50%, while TextGrad reduces reasoning tokens by 332 on average.All compared baselines reduce accuracy by 20% to 40%.Source: §5.2 · p. 9, Table 4 · p. 9
  • 27.7× reasoning-token usage is reached on GPT-5 with ICL-Evolve (Aware), while DeepSeek-R1 exceeds 23.3× under the same attack variant.The GPT-5 baseline is 560 tokens and the DeepSeek-R1 baseline is 550 tokens; ICL-Evolve is strongest in the aware setting.Source: §5.2 · pp. 9–10, Table 6 · p. 10
  • 2.7× higher reasoning-token use occurs in the image attack for Kimi-K2.6, compared with 2.2× for GPT-5 and 1.5× for Sonnet 5.The attack embeds decoy tasks in visually modified images while the user query remains unchanged.Source: §5.2 · p. 10, Table 7 · p. 10
  • 12.7× reasoning-token increases occur in skill-file attacks against OpenCode, compared with 4.0× for Codex and 2.8× for Claude Code.The final generated file remains similar, but agents produce more intermediate output while solving decoy challenges.Source: Table 8 · p. 10, §5.2 · p. 11
  • OVERTHINK amplifies reasoning-token usage while maintaining contextual consistency and accuracy across all evaluated models and datasets.Source: §5.2 · p. 11

5.3 Ablations

  • Transfer attacks increase reasoning tokens by 12× from o1-optimized contexts to DeepSeek-R1 and by 6.2× from o1 to o1-mini.DeepSeek-R1 also transfers to o1 with an 11.4× increase, showing cross-model transferability.Source: §5.3 · p. 12, Table 10 · p. 12
  • Newer model generations become more vulnerable: GPT-5 uses 3.6k attacked tokens versus 2.18k for o3-mini, while Fable 5 uses 3.9k versus 1.71k for Claude 3.7.These correspond to 1.3× and 2.3× increases across the OpenAI and Claude generations, respectively.Source: §5.3 · p. 12, Figure 4 · p. 12
  • 2–4 ICL samples with Top-p from 0.2–0.6 produce the most stable ICL-Evolve exploration, whereas one or five samples yield high variance and poor search quality.Beyond Top-p 0.6, ICL-Evolve generally fails to discover strong challenges except with one sample.Source: §5.3 · p. 12, Figure 5 · p. 12

6 Defenses

Existing prompt-injection detectors do not flag OVERTHINK, while filtering and paraphrasing reduce attack effectiveness with substantial accuracy or stealthiness trade-offs. Monitoring reasoning traces and adapting reasoning effort are proposed as additional defenses, but both face unresolved challenges.

6.1 Existing Defenses and Limitations

  • Existing open- and closed-source prompt-injection guardrails fail to flag or filter OVERTHINK.The attack avoids typical jailbreak signals such as arbitrary strings or policy-violating content.Source: §6.1 · pp. 12–13
  • Filtering reduces accuracy from 60% to 40% for context-aware attacks and to 0% for context-agnostic attacks.For context-aware attacks it largely removes the reasoning increase but reduces accuracy by roughly 50%; for context-agnostic attacks, about 6× reasoning growth remains.Source: §6.1 · p. 13
  • Paraphrasing degrades both attack variants, but context-aware attacks still produce a substantial reasoning-token increase after paraphrasing.The defense may disrupt decoy structure while largely preserving the original meaning of retrieved information.Source: §6.1 · p. 13

6.2 Potential Defenses

  • Reasoning-chain monitoring could detect solved decoys in the scratchpad, but more cryptic traces under prompting pressure leave its effectiveness unresolved.Adaptive effort selection is another option, though manipulated context can induce a more expensive effort level.Source: §6.2 · p. 13

7 Conclusion

OVERTHINK is a slowdown attack that injects optimized decoy challenges into untrusted context, increasing hidden reasoning costs while preserving expected answers. It affects text, images, and coding agents, while common defenses remain largely ineffective.

7 Conclusion

  • OVERTHINK increases reasoning tokens by up to 27× on FreshQA, 46× on SQuAD, and 12× on MuSR while preserving contextual correctness.The attack also reaches up to 2.7× on multimodal inputs and 12.7× on coding agents.Source: §7 · p. 13
  • Prompt-injection guardrails, filtering, and paraphrasing are largely ineffective against OVERTHINK.The conclusion identifies defense difficulty across the evaluated attack settings.Source: §7 · p. 13

Ethical Considerations

The study limits evaluation scale to reduce computational and infrastructure costs while aiming to preserve meaningful, reproducible insights and support defense adoption.

LLM Usage Disclosure

AI assistants were used only for writing polish and experimental code debugging, not for generating paper content or research ideas.

  • AI assistants polished writing and debugged experimental code but did not generate paper content or research ideation.Source: LLM Usage Disclosure · p. 14

8 Dataset Preparation

The evaluation uses factual, multi-hop, image, and coding-agent datasets to study how context length, task complexity, and injected decoys affect the attack.

Dataset Preparation

  • FreshQA contains 600 dynamic factual questions spanning stable, changing, and false-premise knowledge with single-hop and multi-hop reasoning.Source: §8 · p. 18
  • SQuAD contains more than 100k questions from over 500 Wikipedia articles, while MuSR contains 756 samples across murder-mystery, object-placement, and team-allocation tasks.Source: §8 · p. 18
  • The three text datasets enable analysis of attack efficacy as a function of context length and task complexity.Source: §8 · p. 18
  • The image dataset contains 30 mundane-object images, split evenly between benign images and images with inserted decoy elements.Source: §8 · pp. 18–19
  • The coding-agent dataset contains 90 benign and 90 adversarial samples across README files, skill files, and code.Source: §8 · p. 19
  • Coding tasks primarily involve Python applications, with inserted MDP decoys that agents may solve directly or through an additional script.Source: §8 · p. 19

9 Additional Results

Additional experiments show that Overthink remains effective across reasoning-effort settings, depends on obfuscation quality, and extends to multimodal and coding-agent contexts.

Reasoning Effort

  • Over 12× more reasoning tokens occur at high effort, while medium and low effort reach up to 14× under the context-agnostic attack.Source: §9 · p. 19, Table 12 · p. 19
  • The attack increases reasoning overhead across tasks of varying complexity, including tasks configured for low reasoning effort.Source: §9 · p. 19

Obfuscation Instruction Impact

  • Obfuscation Prompt 1 can trigger suspicion, whereas overly complex decoys in Prompt 2 can induce shortcuts and shorter reasoning.Source: §9 · p. 19, Table 13 · p. 20
  • Prompt 3 better balances completing the full decoy challenge with making the obfuscation instructions appear plausible.Source: §9 · p. 19

Multimodal Attacks

10 Chatbot Attack

This section illustrates chatbot-focused Overthink attacks by showing pre- and post-attack examples and prompts that insert decoy reasoning tasks into context. The examples include role-playing, output obfuscation, and a long MDP problem designed to precede the primary answer.

10 Chatbot Attack

11 Used Prompts and Examples

  • The attack uses role-playing instructions that require solving a decoy task before answering the contextual question.Source: §11 · p. 23, Sample Attack Prompt 1 · p. 24
  • Output obfuscation further instructs the model to compress the decoy solution into one word in the final response.Source: §11 · p. 23

Sample Attack Prompt 1

  • The sample MDP prompt supplies a discount factor, states, policy trajectories, and rewards before the model addresses the surrounding context.Source: Sample Attack Prompt 1 · p. 24

Sample Attack Prompt 2

This section presents additional decoy prompts and evaluation templates for measuring attack effects and contextual correctness. It covers MDP and Sudoku decoys, filtering and paraphrasing defenses, context-agnostic and context-aware templates, and scoring examples.

Sample Attack Prompt 2

Loading 2502.02542v5…