Source-linked AI summary
OverThink: Slowdown Attacks on Reasoning LLMs
Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, Eugene Bagdasarian
TL;DR
OverThink addresses the risk that reasoning models processing untrusted external context can be forced to spend costly, hidden reasoning effort without changing their answers. It injects benign, optimized decoy problems into retrieved content and evaluates the attack across models, datasets, and settings. The attack increases reasoning tokens substantially, while existing filtering and paraphrasing defenses have limited or costly effectiveness.
Problem
Reasoning tokens improve inference-time scaling but increase application costs, and external-context applications expose RLMs to indirect injections that can manipulate hidden reasoning effort.
Method
OverThink injects optimized benign decoy challenges into adversary-controlled external sources and crafts instructions that induce their solution while preserving the expected answer.
Results
Across multiple models, datasets, and attack strategies, OverThink increases reasoning tokens by up to 46× for context-agnostic attacks and up to 7.8× for context-aware attacks.
Takeaways & Limitations
OverThink can impose hidden inference overhead while remaining answer-stealthy, and its slowdown transfers across models and extends to multimodal settings.
Takeaways & Limitations
The attack is less stealthy at the input level because defenses that explicitly detect decoy-style challenges can flag it.
Abstract
from arXiv · showhide
Most flagship language models generate explicit reasoning chains, enabling inference-time scaling. However, producing these reasoning chains increases token usage (i.e., reasoning tokens), which in turn increases latency and costs. Our OverThink attack increases overhead for applications that rely on reasoning language models (RLMs) and external context by forcing them to spend substantially more reasoning tokens while still producing contextually correct answers. An adversary mounts an attack by injecting decoy reasoning problems into public content that is consumed by RLM at inference time. Because our decoys (e.g., Markov decision processes, Sudokus, etc.) are benign, they evade safety filters. We evaluate OverThink on both closed-source and open-source reasoning models across the FreshQA, SQuAD, and MuSR datasets. We also explore the attack in multi-modal settings by creating images that cause excessive reasoning. We show that the resulting slowdown transfers across models. Finally, we explore both LLM-based and systems-level defenses, and discuss the societal, financial, and energy implications of the OverThink attacks.
1 Introduction
OverThink exploits reasoning models’ reliance on untrusted external context by embedding benign, computationally intensive decoys that increase hidden reasoning effort without changing the final answer. The attack uses optimized decoys and tailored instructions, and substantially increases reasoning tokens across models, datasets, and strategies.
- Motivation: Reasoning tokens improve inference-time performance but increase output costs and create inefficient, largely invisible token usage.Providers may hide reasoning traces, while applications still incur their token costs.
- Evaluation scenarios: The attack targets both factual questions, where much retrieved context may be unnecessary, and logical deduction tasks, where external context supplies the relevant details.These scenarios test whether models attend to decoys under different levels of context utilization.
- Attack overview: OverThink injects benign computational decoys, such as Markov Decision Processes or Sudoku, into adversary-controlled public sources retrieved as model context.The model processes the poisoned context while preserving the user’s expected answer.
- Attack construction: The attack selects and optimizes decoy problems, crafts instructions to induce their solution, and integrates them into compromised sources using context-aware or general methods.ICL-Evolve is used to increase the reasoning tokens generated by decoys.
- Results: Up to 46× more reasoning tokens occur under large-scale context-agnostic attacks, while context-aware attacks produce increases up to 7.8×.The evaluation spans multiple state-of-the-art models, datasets, and attack strategies.
2 Background and Related Work
Reasoning models scale test-time computation through intermediate chains, but their deployment makes inference resources and token costs important attack targets. OverThink differs from prior prompt and slowdown attacks by increasing hidden reasoning effort while preserving answer correctness and stealthiness.
- Reasoning in LMs: Reasoning models generate intermediate reasoning tokens before answer tokens, enabling additional test-time computation and backtracking on harder problems.Longer reasoning traces can help models recognize errors and revert to a known-correct step.
- Related attacks: Prior prompt-injection and performance-degradation attacks commonly alter outputs, disrupt reasoning, or create visible malicious behavior.OverThink instead targets inference cost while maintaining answer stealthiness.
- Related attacks: OverThink introduces a slowdown attack that increases reasoning tokens without modifying the RLM’s answer, thereby inflating the financial cost of deployed reasoning models.Its novelty is the combination of indirect prompt injection, cost amplification, and answer preservation.
- Operational context: Inference-time scaling supports modern AI products, while cloud pricing and model size make reasoning-model inference computationally expensive.The paper gives GPT o1-pro pricing of $150 per million input tokens and $600 per million output tokens.
- Operational context: User-facing applications can shield users from per-query reasoning costs, leaving services responsible for expenses caused by extensive hidden reasoning.Free tiers and flat subscriptions decouple user payment from underlying token consumption.
3 The OverThink Attack
OverThink targets RLM applications that combine user questions with untrusted external context. An adversary modifies that context with optimized decoys so the model spends longer reasoning while preserving answer correctness and evading ordinary guardrails.
- Threat model: The attack increases inference cost by inducing unnecessary reasoning tokens while maintaining accuracy in generated answers.It relies on indirect prompt injection and input-dependent reasoning behavior.
- Threat model: The adversary targets applications that retrieve webpages, documents, emails, or other external resources and pays the operational cost of their RLM’s reasoning.The user may receive the service for free or at a fixed cost while the application bears model expenses.
- Attack objectives: OverThink applies to externally grounded queries but not context-independent instructions, because the attack requires manipulating context supplied to the RLM.The original query remains unchanged and the manipulated context is omitted from the answer.
- Adversary capabilities: The adversary assumes black-box access to the target RLM or a proxy RLM and access to the external context retrieved by the application.This enables optimization without requiring model parameters or gradients.
- Attack objectives: The formal objectives combine longer reasoning chains, answer stealthiness, and evasion of prompt-injection guardrails.The modified context should increase reasoning while keeping outputs similar to the benign-context result and avoiding detection.
- Why the attack is hard: Gradient-based and jailbreak-style black-box optimization are ineffective because slowdown attacks lack fixed target strings and may produce incoherent or budget-limited reasoning chains.This motivates a sample-efficient search over decoy tasks under noisy measurements.
4 Attack Methodology
OverThink selects and optimizes benign decoy challenges to amplify reasoning-token usage, then embeds them into external context through either scalable context-agnostic or stealthier context-aware injections.
- 4.1 Decoy Problem Selection: Reasoning LMs allocate different token budgets according to perceived task difficulty, so effective decoys must be difficult from the model’s perspective rather than merely for humans.IMO 2024 problems average 8,923 reasoning tokens, while simple Sudoku puzzles average 21,080 across three reasoning LMs.
- 4.1 Decoy Problem Selection: Sudoku puzzles and finite Markov decision processes are effective decoys because their many small, verifiable steps can trigger repeated backtracking.The design targets long tasks whose intermediate operations can be repeatedly checked against clear criteria.
- 4.2 Decoy Optimization: ICL-Evolve optimizes decoys by treating each candidate as a multi-armed-bandit arm whose noisy reward is the reasoning-token utility of a model call.The method balances exploitation of high-scoring decoys with exploration of new variants, using log-scaled utility to reduce sensitivity to noisy token counts.
- 4.3 Obfuscation: Effective attacks require obfuscation instructions that preserve contextual correctness and prevent outputs from visibly revealing that the decoy was solved.Without such instructions, the model may solve both tasks in its output, making the attack suspicious or ignore the decoy when instructed too directly.
- 4.4 Attack Scenarios: Context-agnostic injections use query-independent templates for scalable corpus poisoning, whereas context-aware injections adapt formatting and instructions to local content for greater stealth.The agnostic strategy is cheaper and more transferable but more recognizable; the context-aware strategy is harder to detect and filter.
5 Evaluation
OverThink substantially increases reasoning-token usage across models, attack variants, datasets, transfer settings, and multimodal inputs while preserving contextual correctness in reported evaluations. Results also show that stealthiness, transferability, and defenses vary substantially with attack design and model.
- Attack Variants Evaluation: 18× is the largest reasoning-token increase for o1 under ICL-Evolve (Agnostic), while context-agnostic and context-aware attacks achieve 9.7× and over 2×.For DeepSeek-R1, ICL-Evolve (Agnostic) increases reasoning tokens by more than 10× from a baseline of 711 ± 635 tokens.
- ICL-Evolve Hyperparameters: 2–4 ICL samples with Top-p 0.2–0.6 provide the most stable ICL-Evolve exploration, while one or five samples produce high variance and poor search quality.Except in the one-sample setting, ICL-Evolve struggles to discover strong challenges when Top-p exceeds 0.6.
- Attack Transferability: 12× transfer from o1 to DeepSeek-R1 exceeds the 10.5× increase from contexts optimized directly on DeepSeek-R1, while transfer gains vary across model pairs.Reported transfer increases include 6.2× from o1 to o1-mini, 11.4× from DeepSeek-R1 to o1, and 2.9× from o1-mini to o1.
- Reasoning Effort Tuning: Over 12× more reasoning tokens occur for o1 at high reasoning effort under the context-agnostic attack.The attack significantly increases reasoning tokens across all tested reasoning-effort levels.
- Multimodal Attacks: 2× and 2.9× reasoning-token increases occur for cat and apple images, respectively, while both outputs remain benign but differ in content selection.The cat decoy shifts content toward broader factual material, whereas benign and decoy apple outputs are highly similar.
6 Potential Defenses
The paper considers monitoring reasoning traces and adapting reasoning effort as defenses against OverThink. Trace monitoring can detect hidden objectives, while input-based effort control may be vulnerable to manipulated context.
- Reasoning Chain Monitoring: Reasoning-chain monitoring can detect OverThink because the decoy is solved in the model’s scratchpad despite benign external outputs.A smaller model or heuristics could score traces for manipulation signals.
- Adaptive Reasoning: Adaptive reasoning estimates how many reasoning tokens to spend from the model inputs instead of always using a fixed effort level.The trusted user question is proposed as a basis for estimating required effort.
- Adaptive Reasoning: Context-based effort selection can be manipulated because injected context may induce the application to choose a more expensive reasoning level.The passage contrasts untrusted context with trusted context such as the user’s question.
7 Conclusion
The conclusion presents OverThink as an indirect prompt-injection attack that increases reasoning effort while preserving expected answers. Across factual and logical benchmarks, it reports large token increases, transfer to multimodal settings, and limited effectiveness of common defenses.
- 7 Conclusion: OverThink injects computationally demanding decoy problems into untrusted data while preserving expected answers, making the manipulation difficult for users to detect.The attack targets applications in which reasoning language models process untrusted data.
- 7 Conclusion: Up to 18× more reasoning tokens were observed on FreshQA, 46× on SQuAD, and 12× on MuSR.These increases span factual question-answering and logical deduction evaluations.
- 7 Conclusion: Initial multimodal results indicate that OverThink can transfer beyond text-only inputs.The paper creates images that induce excessive reasoning in multimodal settings.
- 7 Conclusion: Prompt-injection guardrails, filtering, and paraphrasing show limited effectiveness against OverThink.The conclusion reports these defenses as common mitigations evaluated by the paper.
A Ethical Considerations
The authors frame the study around responsible research practices and limiting computational impact. They report restricting experiments, making materials public, and aiming to support sustainable, fair, and secure AI research.
- A Ethical Considerations: The study limited its use to a few million tokens for each model, a fraction of the services’ daily computational load.The restriction was motivated by computational and infrastructure costs associated with language models.
- A Ethical Considerations: The authors aimed to minimize infrastructure impact while still obtaining meaningful insights and enabling reproducible experiments.This describes the stated balance between research utility and resource use.
- A Ethical Considerations: The code and prompts were made public to facilitate adoption of defenses by applications relying on language-model reasoning.The authors connect disclosure to sustainable, fair, and secure advances in AI research.
A Dataset Preparation
The evaluation uses FreshQA, SQuAD, and MuSR to study OverThink across factual and logical tasks, varying context length and task complexity. The datasets differ substantially in scale, context size, and task structure.
- FreshQA: FreshQA contains 600 questions spanning stable, evolving, and false-premise knowledge, with single-hop and multi-hop reasoning linked to updated Wikipedia entries.The original queries average 11.6 tokens, while appended Wikipedia context averages 11278.2 input tokens.
- SQuAD: SQuAD contains more than 100k questions from more than 500 Wikipedia articles, with 11.5-token queries and 117.5-token average contexts.Its contexts are shorter and less variable than FreshQA’s.
- MuSR: MuSR contains 756 samples covering murder mysteries, object placement, and team allocation tasks that require deduction from contextual observations.The tasks involve detective reasoning, locating objects, and assigning people using social and preference information.
- Dataset Coverage: Using all three datasets enables analysis of how context length and task complexity affect OverThink’s efficacy.The datasets provide complementary factual and logical reasoning settings.
B Used Prompts and Examples
This section presents prompts and examples for attacking, filtering, paraphrasing, and evaluating context in the OverThink setup. It includes an example question, reasoning output, and contextual-correctness examples.
- The section uses the first-animal-on-the-Moon question as an example user query.
- The original chain-of-thought example reports a token count of 2228 for that question.
- Filtering Relevant Content: Filtering prompts instruct an assistant to retain only context sections highly relevant to the user’s query.
- Paraphrasing Context: Paraphrasing prompts instruct an assistant to preserve core meaning and retain important information while rephrasing context.
- Attacks and Evaluation: The materials include context-agnostic templates, a Mars-landing context-aware template, and prompts for accuracy and contextual-correctness evaluation.
- Contextual Correctness: Contextual correctness is scored as 1 when output uses the first context, 0.5 when it uses both contexts, and 0 otherwise.
- Contextual Correctness: The Zodiac Killer example illustrates an output assigned contextual-correctness score 1.
C Multi-Modal attacks
This section documents multimodal OverThink attack materials using images of apples and cats with different decoy visuals.
- Table 12 lists different apple images with different decoy visuals for multimodal OverThink attacks.
- Table 13 lists different cat images with different decoy visuals for multimodal OverThink attacks.
D Chatbot Attack
This section presents pre-attack and post-attack documents and examples.
- Pre-attack: The section includes a pre-attack document and a pre-attack example.
- Post-attack: The section includes a post-attack document and a post-attack example.