Source-linked AI summary
Silent Sabotage: Internal State Triggered Backdoor Attacks on LLM-Powered Robotic Systems
Doniyorkhon Obidov, Shivayogi Akki, Tan Chen, Kaichen Yang
TL;DR
Prior LLM-backdoor research in robotics has focused on external triggers, leaving history-based triggers in the robot’s own operational logic underexplored. This paper studies an instruction-embedded backdoor triggered by a sequence of past actions; two tested models achieved 10/10 attack success in every scenario. The findings point to internal state and memory as security concerns, although the attack was evaluated in simulation with a fixed trigger sequence.
Problem
Research on LLM security has focused mainly on external attack vectors, and instruction-based attacks had not yet been explored in physical robotics.
Source: §1 · p. 2Method
The paper embeds a conditional backdoor instruction in a robot controller, using a rare sequence of the robot’s past actions as the trigger.
Source: §1 · p. 3, Trigger Design · p. 8, Backdoor Instruction · p. 9Results
Llama-3.3-70B and Gemma-3-27B each achieved a 10/10 Attack Success Rate in every tested scenario.
Source: §5.2 · p. 10Takeaways & Limitations
The findings identify internal state and memory as security concerns beyond input sanitization for LLM-powered robots.
Source: §8 · p. 14The study is limited to simulation and uses a predefined, fixed action sequence as its trigger.
Source: §7.2 · p. 13
Abstract
from arXiv · showhide
The integration of Large Language Models (LLMs) into robotic control systems is enabling a new generation of autonomous agents capable of complex reasoning and planning. While this paradigm shift accelerates progress, it also introduces novel security risks that remain largely unexplored. Current research into LLM backdoors has focused on attacks triggered by external stimuli, such as specific words, visual objects, or environmental states. These attacks, while potent, overlook a more insidious class of vulnerability where the trigger is internal to the agent's own operational logic. This paper presents the first comprehensive study of history-based backdoor attacks on LLM-powered robotic systems. We demonstrate that an attacker can embed a stealthy backdoor into an LLM-based robot controller by manipulating its instructions. This backdoor is triggered not by an external cue, but by a specific, rare sequence of the robot's own past actions. It remains dormant during normal operation, preserving the robot's utility, but can be activated to induce a malicious behavior, such as a complete stop or a collision. Our experiments, conducted in a simulated environment with a variety of robots and LLMs, show that this history-based attack is highly effective, achieving a near-perfect attack success rate while remaining exceptionally difficult to detect. These findings reveal a critical and previously unaddressed vulnerability in autonomous systems and underscore the urgent need for security measures that account for an agent's internal state.
1 Introduction
Research on LLM backdoors in robotics has largely focused on external triggers, while instruction-based attacks had not been explored in physical robotics. The paper studies an instruction-embedded backdoor triggered by a robot’s own action history and targeting direct low-level commands.
1 Introduction
- Prior work focused on external triggers, and instruction-based attacks had not yet been explored in physical robotics.Existing embodied-agent triggers include prompt words, objects, or environmental scenarios; the paper identifies internal state and operational history as an overlooked vulnerability.Source: §1 · pp. 2–3
- The paper studies a history-based backdoor in a black-box robot controller, embedded through instructions without access to model weights, training data, or fine-tuning.Experiments use simulated quadruped robots and state-of-the-art LLMs for low-level command generation.Source: §1 · p. 3
- A rare sequence of the robot’s past actions triggers the backdoor, which targets direct low-level JSON commands rather than high-level plans.The trigger is internal to the robot’s operational history, not an external stimulus.Source: §1 · p. 3
- The trigger comparison contrasts external cues with the robot’s own action history, which the paper says is harder to detect through input-abnormality defenses.The attack targets behavior that cannot be identified by analyzing the LLM input for abnormalities.Source: Table 1 · p. 4
2 Related Work
Prior backdoor work spans NLP attacks and embodied systems, but robotic studies primarily rely on external triggers. This paper positions action-history triggers and direct reactive control as underexplored attack dimensions.
2.1 Backdoor Attacks on Language Models
- Earlier language-model backdoors used training-data poisoning and evolved toward less visible sentence, style, and character-level triggers.LLM-specific attack surfaces also include instruction tuning, RLHF, in-context demonstrations, and chain-of-thought reasoning.Source: §2.1 · pp. 4–5
- These language-model backdoor methods had largely been studied on traditional NLP tasks, while LLM robotics was emerging as a security research area.LLM robot control supports complex, long-horizon tasks.Source: §2.1 · p. 5, §2.2 · p. 5
2.2 Backdoor Attacks on LLM-Powered Robotics
- Prior robotic backdoors relied on fine-tuning or module poisoning and external textual, visual, or environmental triggers.Examples include a rare prompt phrase, a yellow CD, and a particular scenario.Source: §2.2 · p. 5
- The unexplored possibility was weaponizing an agent’s own internal state and action history, including its otherwise correct behavior in the physical domain.Existing embodied-agent triggers were external to the agent.Source: §2.2 · pp. 5–6
- The paper’s approach uses the robot’s past actions as an internal trigger and targets an immediate JSON-command control loop rather than high-level plans.It is training-free, directly modifies the system prompt, and differs from the few-shot-example manipulation described for ICL.Source: §2.2 · p. 6
3 Attack Scenario
The threat model considers a provider who can alter a black-box robot controller’s system instructions but not its model or training. The system uses recent action history, which allows a conditional instruction to leave ordinary navigation intact and redirect behavior when a trigger occurs.
3 Attack Scenario
- The attacker is the provider of a third-party controller offered as a black-box API, while the user can supply high-level instructions but cannot inspect the model or the provider’s full instructions.The scenario is likened to using a customized GPT from the GPT Store.Source: §3 · p. 6, §3.1 · p. 6
3.1 Threat Model
- The attacker can modify system instructions but not the model architecture, training data, or fine-tuning, and seeks a stealthy, hard-to-detect, effective backdoor.The backdoor should preserve normal task utility when its trigger is absent and cause a specific malicious behavior when activated, such as stopping or hitting a wall.Source: §3.1 · p. 6
- The proposed workflow embeds a malicious instruction in the black-box controller’s system prompt; a specific action-history trigger makes the LLM ignore its benign goal and execute the malicious command.The attacker’s control is over the prompt rather than the model’s parameters.Source: §3.1 · p. 6, Figure 2 · p. 7
3.2 System Architecture
- The simulated system navigates a quadruped toward the nearest room opening using LiDAR-derived opening summaries and direct JSON commands.It retains recent actions in memory to avoid loops and inform decisions, and the attack exploits that memory.Source: §3.2 · p. 7
4 Methodology
The methodology embeds a conditional backdoor instruction in the system prompt and supplies the controller with observations and recent actions. A rare, plausible ordered action sequence triggers the attacker’s command instead of the benign task behavior.
4 Methodology
- The attack modifies a black-box LLM controller’s system prompt by embedding a covert, history-based conditional instruction.The prompt combines the task instruction with the backdoor instruction.Source: §4 · p. 7, §4.1 · p. 8
4.1 Formalizing the Instruction-Based Attack
- At each time step, the controller receives environmental observations and its k most recent actions, then generates the next action from the resulting prompt.The prompt template formats the system prompt, observations, and action history; the implementation uses greedy decoding to select the highest-probability action.Source: §4.1 · p. 8
4.2 History-Based Backdoor Design
- The trigger is a specific ordered action sequence chosen to be rare enough to avoid accidental discovery but plausible in a complex real-world scenario.The backdoor instruction specifies the attacker’s desired action when the trigger appears in the history.Source: Trigger Design · p. 8, Backdoor Instruction · p. 9
- The conditional instruction overrides ordinary action selection when the trigger appears in the action history; otherwise, the controller follows its task prompt.The instruction is supplied directly as a “Special Instruction” rather than trained into the model; the example makes the robot stop after the specified sequence.Source: Backdoor Instruction · p. 9
5 Experiments
The experiments evaluate an instruction-based backdoor across simulated robots, LLMs, and room layouts, comparing triggered attacks with benign operation. Attack success was nearly perfect while benign task completion remained high.
5.1 Experimental Setup
- Trials used Go1, Go2, and A1 robots in Gazebo rooms with one, two, or four doors, testing three LLMs and a three-action trigger for emergency stop.Robot starting position and orientation were randomized; benign trials omitted the trigger, while attack trials injected it after the first 20 steps.Source: §5.1 · pp. 9–10, Figure 3 · p. 10
5.2 Results
- Benign TCR frequently reached 8/10, 9/10, or 10/10, indicating that the dormant backdoor preserved task performance in these trials.The authors note that this makes the vulnerability difficult to discover through standard performance or utility-based testing.Source: §5.2 · p. 11
- ASR was 10/10 for Llama-3.3-70B and Gemma-3-27B in every scenario, with only Qwen2-72B on Go1 in the 1-door map at 9/10.The lone deviation occurred because the robot escaped before the 20-step trigger-initialization threshold.Source: §5.2 · pp. 10–11, Table 2 · p. 11
6 Potential Countermeasures
The proposed instruction backdoor is difficult to defend against under a black-box threat model, where defenders lack access to model internals and prompts. The section outlines platform-level, runtime, and context-management strategies, alongside automated probing.
6 Potential Countermeasures
- The black-box setting rules out defenses that inspect or modify model weights, use gradients to invert triggers, or depend on manual system-prompt review.The authors describe proprietary models’ weights, architecture, and full system prompts as unavailable to end-users.Source: §6 · p. 12
- Platforms could scan system prompts for suspicious conditional logic, while runtime monitoring could flag low-probability action sequences and trigger safe mode or human review.The proposed runtime approach builds a probabilistic model of normal behavior.Source: §6 · p. 12
- Periodic context or action-history resets could disrupt long triggers but may reduce performance on legitimate long-horizon tasks; a separate red-team LLM could probe for hidden behaviors.The authors also propose automated searches over diverse and unexpected action sequences.Source: §6 · p. 12
7 Discussion
The discussion frames instruction-following and state maintenance as capabilities that also create an attack surface in LLM-powered robotics. It identifies implications for external defenses, then acknowledges simulation and fixed-trigger limits and outlines future work.
7 Discussion
- Complex instruction-following and state maintenance, capabilities useful in robotics, also create a new attack surface.The authors connect these capabilities to the security implications of LLM-powered robotic systems.Source: §7 · pp. 12–13
7.1 Implications of State-Based Attacks
- A history-based trigger can evade input sanitization and environmental scans because it arises from the robot’s own behavior and internal memory.The paper identifies possible consequences ranging from mission failure to physical harm.Source: §7.1 · p. 13
7.2 Limitations and Future Work
- The evaluation is limited to simulation, which does not capture all real-world sensor noise and physical-dynamics complexity, and uses a predefined fixed action sequence as its trigger.The authors identify these boundaries as limitations of the current study.Source: §7.2 · p. 13
- Future work includes physical-hardware validation, more dynamic or hybrid triggers, and implementation and testing of runtime anomaly detection and context-purging defenses.Planned validation includes a Unitree Go1 and G1; proposed triggers may depend on action order, environmental states, or sensor readings.Source: §7.2 · p. 13
8 Conclusion
The conclusion presents history-based backdoors as an internal-state vulnerability in LLM-powered robots. The demonstrated attack manipulates instructions and uses the robot’s own past actions as its trigger.
8 Conclusion
- The study demonstrates a stealthy, effective backdoor embedded through instruction manipulation and triggered by the robot’s own sequence of past actions.The authors call for security research to address vulnerabilities in agent internal state and memory, beyond input sanitization.Source: §8 · p. 14