PaperlayerView exact revision on arXiv

Source-linked AI summary

When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents

Jiawei Liu, Jiacheng Guo, Tian Zhang, Yiwei Xu, Juan Wang, Jinlin Fan, Bowen Xiao, Chi Guo, Keyan Guo, Hongxin Hu

arXiv:2608.16806v1cs.ROcs.AI

TL;DR

LLM-driven embodied agents expose a planner-visible state-semantic integrity gap because corrupted environmental evidence can influence physical actions without changing user instructions. ESTI evaluates this downstream threat and finds that representation-compatible state rewrites can alter plans and sometimes produce verifiable final-state consequences, while carrier compatibility and representation-level consistency strongly affect adoption.

  • Problem

    The paper studies whether polluted planner-visible environment-state semantics can propagate through embodied-agent planning to unintended physical consequences without modifying user instructions.

  • Method

    ESTI evaluates component-scoped, predicate-local, schema-preserving rewrites from one compromised state producer while keeping instructions, models, planners, and executors unchanged.

  • Results

    Across three embodied-agent settings, carrier compatibility and representation-level consistency strongly affect planning adoption, while planner adoption alone does not establish attack success.

  • Takeaways & Limitations

    Evaluating embodied-agent attacks requires tracing state-semantic manipulation through grounding, planning, execution, and final-state consequences rather than measuring planner adoption alone.

  • Takeaways & Limitations

    The study evaluates downstream adoption after delivery, not an end-to-end sensory or middleware compromise, which remains future work.

Abstract

from arXiv · show

Large Language Models (LLMs) have demonstrated capabilities in in-context learning, task decomposition, step-by-step reasoning, and code generation, driving their gradual evolution from text generation models into the core of agents capable of perceiving environments, invoking tools, and executing tasks. Traditional LLM Agents typically obtain information through webpages, documents, databases, or external tools and generate corresponding invocation sequences according to user goals; when this technology is further integrated with robotic systems, large language models begin to undertake functions such as task understanding, high-level planning, and behavioral decision-making. SayCan combines the task reasoning capability of language models with the affordances of robotic skills, while Code as Policies and ProgPrompt generate robot task plans through policy code and programmatic prompting, respectively, and VoxPoser uses language models and vision-language models to construct three-dimensional value maps to guide robotic manipulation \cite{6,7,8,9}. Vision-language-action models such as PaLM-E, RT-2, and GR00T N1 further strengthen the connection among language, visual perception, and robotic actions \cite{10,11,12}. In such LLM-driven embodied agents, the model not only needs to understand user instructions, but also needs to combine scene states, object attributes, spatial relations, and execution feedback to complete task grounding, and then hand the generated action plan to skill libraries, motion planners, or controllers for execution.

1 Introduction

LLM-driven embodied agents expand robotics capabilities while enlarging the security boundary because planner-visible state and execution feedback can influence physical actions. ESTI studies whether corrupted task-level state evidence is adopted as planning evidence and transferred through embodied execution under a constrained, representation-compatible threat model.

  • Motivation: LLM-generated embodied plans act on robots and their environments, expanding the security boundary beyond textual model outputs.Incorrect planning outputs can affect the robot itself and its surrounding environment.
  • Threat: Polluted state semantics in planner-visible information can alter judgments about target objects, locations, action sequences, and failure recovery, inducing unintended behavior.The affected information includes execution feedback treated as trusted environmental facts and planning evidence.
  • ESTI: ESTI isolates downstream integrity at the planner-visible semantic-state boundary rather than introducing a new corruption primitive or explaining how write access is obtained.It assumes one compromised state producer and studies whether encoded adversarial goals traverse grounding, preconditions, and execution constraints.
  • Threat model: ESTI constrains attacks to one producer’s authorized task-relevant records, with a preselected objective, predicate-local rewriting, schema preservation, and no adaptive access to planner outputs.The threat model excludes arbitrary high-privilege context writes and planner-output adaptation.
  • Evaluation: ESTI-Bench separates planning-level adversarial-objective adoption from execution-level realization of a verifiable final-state predicate, measuring the planning-to-execution transfer gap.The construction preserves the original user instruction, planner, and executor while encoding false evidence through state representations.
  • Findings: Carrier compatibility and representation-level consistency drive planning adoption under matched groundability, while a real-robot proof of concept tests downstream propagation.The proof of concept evaluates downstream propagation rather than an end-to-end sensory compromise.

2 Related Work

Prior work develops LLM-based embodied agents through high-level planning, tool use, unified vision-language-action models, and executable robotic programs. Related security research examines indirect prompt injection, perception and decision attacks, while state representations remain an underexamined security variable.

  • LLM capabilities support embodied-agent planning, while SayCan, Code as Policies, and ProgPrompt connect generated plans to robotic affordances, executable programs, and scene actions.These approaches use language-model reasoning and code generation to produce plans or programs aligned with available robotic actions.
  • PaLM-E, RT-2, and GR00T N1 increasingly unify visual observations, language instructions, and robotic action outputs.Such models transform perceptual observations or simulator states into representations consumed by an agent.
  • LLM-agent research broadens models from text generation to reasoning, tool invocation, web browsing, file processing, APIs, and interactive task execution.ReAct, Toolformer, ToolLLM, API-Bank, Gorilla, ToolBench, and later systems advance reasoning-action interleaving, tool selection, and multi-turn interaction.
  • Indirect prompt injection embeds executable commands in webpages, documents, or external content to exploit LLMs’ inability to distinguish data from instructions.Vanilla IPI provides a command-based baseline, whereas embodied deviations may fail to match state representations or satisfy task-resolution and execution constraints.
  • Robotic-security studies progress from perception and control attacks to adversarial task understanding, decision making, sensory-vector injection, and unsafe robot behavior, but state-level trust remains distinct.RIPA injects adversarial information through visual, audio, and LiDAR interfaces, while ESTI begins with a corrupted task-level record and evaluates its adoption by the planner and realization in actions.

3 ESTI: Environment State-Text Injection Attack

ESTI models an attack in which one compromised planner-facing state producer injects schema-compatible false semantics into an embodied agent’s unchanged planning–execution loop. The attack succeeds only when the false evidence induces a targeted, executable deviation rather than merely delivering text or causing arbitrary failure.

  • Overview: ESTI evaluates whether corrupted environment-state semantics propagate through an embodied planning–execution loop.The section defines the threat model, attack construction, and propagation mechanisms.
  • Agent pipeline: The planner combines user instructions with objects, relations, affordances, task-stage constraints, and execution feedback to generate plans that update the environment and next visible state.Planner-visible state is assembled from state-producing components and serialized for the high-level planner.
  • Threat model: A gray-box adversary compromises exactly one authorized state producer or its outgoing channel while leaving the user instruction, planner, executor, skills, controller, dynamics, and physical state unchanged.The attacker can alter only records emitted by the compromised component under the documented schema, without accessing hidden ground truth or intermediate planner outputs.
  • Success conditions: Targeted success requires both planner adoption of the adversarial goal and execution of the deviated plan through validity, reachability, affordance, precondition, and interface checks.Initial or stage-dependent evidence must propagate through the original grounding, planning, execution, feedback, and replanning loop; payload delivery or an unexecuted fragment does not count.
  • Attack construction: ESTI converts a fixed adversarial goal into schema-compatible false evidence by runtime re-grounding valid entities and matching payloads to native object, relation, affordance, stage, or feedback semantics.Unrelated records remain unchanged, and the rewrite preserves the native schema, fields, record count, serializer, and surrounding context while replacing existing semantic values.

4 Experiments and Evaluation · 4.1 Experimental Setup and Evaluation Protocol

The evaluation spans three embodied-agent environments and separates planning success from realized execution deviation. A matched clean/control/attack protocol and repeated comparisons are used to assess targeted state-semantic manipulation while reducing contextual and stochastic confounds.

  • 4.1 Experimental Setup and Evaluation Protocol: ESTI and ESTI-Bench are evaluated in ProgPrompt, VoxPoser, and AI2-THOR, covering programmatic planning, continuous-space manipulation, and interactive indoor execution.For AI2-THOR, qualifying iTHOR FloorPlans are selected, initial states are fixed, and outcomes are verified from simulator metadata.
  • 4.1 Experimental Setup and Evaluation Protocol: Each sample is tested under clean, control, and attack conditions to distinguish adversarial semantic manipulation from effects caused by additional context.Control adds benign state text matched to the attack payload in length and style, whereas attack introduces adversarial semantics into planner-visible state.
  • 4.1 Experimental Setup and Evaluation Protocol: P-ASR measures whether generated plans satisfy the adversarial objective, while E-ASR measures whether the corresponding observable deviation is realized after execution.Both metrics are computed over samples whose original task succeeds under the clean condition, enabling direct comparison between planning and execution.
  • 4.1 Experimental Setup and Evaluation Protocol: ESTI is compared with Vanilla IPI, EIRAD, and three BADROBOT variants under the same evaluation protocol.Vanilla IPI uses prevalidated adversarial goals as command-style text without runtime re-grounding, native-carrier construction, or cross-record consistency checks.
  • 4.1 Experimental Setup and Evaluation Protocol: EIRAD appends an adversarial suffix to the user task, while BADROBOT includes contextual jailbreak, safety misalignment, and conceptual deception.These methods provide the comparison categories used alongside ESTI and Vanilla IPI.
  • 4.1 Experimental Setup and Evaluation Protocol: Each experimental condition is independently repeated three times, and reported results are averaged over the three runs.RIPA is excluded as a numerically matched baseline because its primary experimental variable is the upstream channel through which adversarial input is delivered.

4.2 Overall Results

Conditional on state delivery, ESTI achieves the strongest planning- and execution-level attack success across ProgPrompt, VoxPoser, and AI2-THOR. Its state-semantic representation aligns with embodied-agent planning inputs, but successful planner adoption does not guarantee embodied attack success.

  • Overall attack performance: ESTI achieves the strongest P-ASR and E-ASR across ProgPrompt, VoxPoser, and AI2-THOR under conditional state delivery.Its P-ASR values are 100.00%, 97.37%, and 100.00%, while E-ASR values are 47.06%, 42.11%, and 48.08%, respectively.
  • Overall attack performance: Averaged across environments, ESTI reaches 99.12% P-ASR and 45.75% E-ASR, exceeding Vanilla IPI by 18.63 and 11.13 percentage points.Vanilla IPI averages 80.49% P-ASR and 34.62% E-ASR.
  • Baseline comparison: Baseline attacks are weaker or less stable because they are not specifically aligned with embodied agents’ state-dependent planning process.EIRAD perturbs user-side prompts, whereas BADROBOT variants rely on contextual jailbreak, safety misalignment, or conceptual deception without explicitly modeling object, relation, and action constraints.
  • Baseline comparison: Vanilla IPI produces relatively high planning-level success through direct competing instructions, but its lower E-ASR shows that planner changes do not necessarily yield valid embodied attacks.Its P-ASR reaches 66.18%, 86.84%, and 88.46%, while corresponding E-ASR values are 32.15%, 34.21%, and 37.50% across ProgPrompt, VoxPoser, and AI2-THOR.
  • ESTI mechanism: ESTI encodes objectives through native object attributes, spatial relations, task rules, and planner-facing feedback over prevalidated entities, matching the planner’s normal input format.A substantial P-ASR–E-ASR gap remains, including 100.00% versus 47.06% on ProgPrompt and 100.00% versus 48.08% on AI2-THOR.

4.3 Independent Graphical Analysis of ASD

ASD measures how strongly an attack changes an embodied agent’s action sequence, including deviations that do not satisfy the execution-level attack criterion. It complements P-ASR and E-ASR by separating behavioral perturbation from successful realization of the attack objective.

  • ASD as a complementary metric: ASD measures the overall difference between action sequences before and after an attack, capturing the extent of execution-behavior perturbation.Unlike ASR, ASD reflects whether the original behavior trajectory changed and how substantially.
  • ASD as a complementary metric: ASD captures behavioral deviations even when an attack changes the target object, action order, or movement path but later fails the E-ASR criterion.Thus, ASD reveals attack effects that execution-level success alone would not count.
  • Interpreting ASD: 4.59 ASD versus 3.01 ASD shows that Vanilla IPI can cause greater deviation than ESTI while achieving lower E-ASR: 32.15% versus 47.06%.On ProgPrompt with DeepSeek-V4-Pro, ASD magnitude does not directly indicate whether deviations align with the attacker’s objective.
  • Interpreting ASD: ASD is best interpreted as a complementary measure of behavioral perturbation, while P-ASR and E-ASR evaluate whether the attack objective is realized.Together, the metrics distinguish planning manipulation, behavioral deviation, and successful execution.

4.4 Planning-to-Execution Attack Transfer

ESTI effectively deceives LLM planners across most model–environment combinations, but planner deception does not consistently transfer to successful execution. The P-ASR–E-ASR gap captures transfer loss caused by plan quality and environment constraints.

  • Planner deception: ESTI achieves consistently high P-ASR across most model–environment combinations, reaching 99.12% for DeepSeek-V4-Pro and 97.04% for Qwen-3.6-Plus.P-ASR measures whether injected state semantics deceive the planner and induce an attacker-aligned plan.
  • Execution transfer: Successful planner deception does not necessarily produce successful execution because E-ASR also depends on generated-plan quality and simulation-environment constraints.Relevant constraints include infeasible actions, object availability, spatial reachability, action primitives, and interaction preconditions.
  • Execution transfer: DeepSeek-V4-Pro records 99.12% average P-ASR but 45.75% E-ASR, producing a 53.37-point gap, while GPT-5.6-luna has a higher transfer rate despite lower P-ASR.The P-ASR–E-ASR gap characterizes transfer loss from planner deception to execution realization rather than simply attack failure.

4.5 Representative Attack Processes in Simulation

ESTI induces target-object substitution in simulated embodied-agent tasks across AI2-THOR, VoxPoser, and ProgPrompt. The attack replaces each task’s instructed object with a different object, including tomato, block1, and cup substitutions.

  • Representative Attack Processes in Simulation: ESTI induces target-object substitution across AI2-THOR, VoxPoser, and ProgPrompt simulations.The visualization results cover Figures 4, 5, and 6.
  • Representative Attack Processes in Simulation: In AI2-THOR, the instructed bread target is replaced by a tomato.The clean task instructs the embodied agent to pick up bread.
  • Representative Attack Processes in Simulation: In VoxPoser, the instructed block0 target is replaced by block1.The clean task instructs the agent to move block0 to the center of the board.
  • Representative Attack Processes in Simulation: In ProgPrompt, the instructed book target is replaced by a cup.The clean task instructs the agent to place a book at a designated location.

4.6 Real-Robot Experiments · 4.7 Effect of the Planning Model

Real-robot experiments show that manipulated planner-visible state semantics can propagate into physical execution, though the setup manually supplies textual state rather than using closed-loop perception. Across planning models, ESTI remains highly effective at the planning level, while execution-level differences are smaller and do not scale proportionally with planning susceptibility.

  • 4.6 Real-Robot Experiments: The real-robot experiment tests whether state-semantic manipulation propagates into physical execution.It uses a humanoid robot following a predefined route once and returning to its starting position.
  • 4.6 Real-Robot Experiments: Under the clean condition, the robot completes the predefined route as intended.Representative execution stages are shown in Fig. 7, with red circles marking the robot’s position.
  • 4.6 Real-Robot Experiments: The experiment evaluates downstream state-to-execution propagation rather than an end-to-end perception attack.Environmental state is manually instantiated from the real scene and supplied to the LLM planner in textual form.
  • 4.6 Real-Robot Experiments: The clean and attack conditions differ in planner-visible state rather than visual observations.A fully closed-loop perception–planning–execution evaluation remains future work.
  • 4.7 Effect of the Planning Model: 99.12% average P-ASR on DeepSeek-V4-Pro and 97.04% on Qwen-3.6-Plus show high planning-level susceptibility to manipulated state semantics.GPT-5.6-luna has a lower average P-ASR of 79.06%, mainly because of 47.62% P-ASR on AI2-THOR.
  • 4.7 Effect of the Planning Model: GPT-5.6-luna nevertheless reaches 98.18% P-ASR on ProgPrompt and 91.38% on VoxPoser.These environment-specific results coexist with its lower overall average P-ASR.
  • 4.7 Effect of the Planning Model: 45.75%, 46.44%, and 47.77% are ESTI’s average E-ASR values for DeepSeek-V4-Pro, GPT-5.6-luna, and Qwen-3.6-Plus, respectively.GPT-5.6-luna reaches 63.64% E-ASR on ProgPrompt, while Qwen-3.6-Plus reaches 50.49% on AI2-THOR and 48.39% on VoxPoser.

4.8 Ablation Study

The ablation attributes attack propagation primarily to representation-level adoption: removing carrier compatibility or representation-level consistency sharply reduces P-ASR. Runtime re-grounding has only modest incremental effects because benchmark samples were already groundable.

  • Representation-level adoption: Removing carrier compatibility reduces P-ASR by 87.50 points, from 100.00% to 12.50%.This identifies carrier compatibility as a major representation-level factor in attack propagation.
  • Representation-level adoption: Removing representation-level consistency reduces P-ASR by 62.50 points, to 37.50%.E-ASR also falls because fewer samples first reach an attacker-aligned plan.
  • Runtime re-grounding: Removing runtime re-grounding changes P-ASR by only 1.92 points and E-ASR by 3.85 points.Object existence, candidate destinations, affordances, and key preconditions were already checked during benchmark construction.

5 Conclusion

ESTI shows that compromised planner-facing state producers can alter embodied-agent plans and sometimes cause verifiable final-state consequences, even when instructions, planners, and executors remain unchanged. The findings also narrow the role of grounding and motivate broader multimodal, provenance-aware, closed-loop evaluation.

  • Conclusion: ESTI studies how component-scoped, predicate-local, schema-preserving state rewrites create representation-compatible false records that alter high-level plans.The user instruction, planner, and executor remain unchanged.
  • Conclusion: Planner adoption alone is insufficient to characterize embodied attack success across ProgPrompt/VirtualHome, VoxPoser/RLBench, and AI2-THOR/iTHOR.The planning-to-execution gap can include verifiable final-state consequences.
  • Conclusion: With dataset-level groundability fixed, carrier compatibility and representation-level consistency strongly affect planning adoption, while runtime re-grounding provides only a small incremental benefit.This supports a narrower conclusion than a generic grounding claim.
  • Conclusion: The real-robot study is a downstream proof of concept rather than an end-to-end perception or middleware compromise.Its scope is limited to demonstrating downstream consequences.
  • Future Work: Future work will extend ESTI-Bench across platforms and modalities, add stronger defenses, and evaluate sim-to-real, provenance tracking, consistency checking, and execution-time verification.The agenda also includes adaptive corruption in long-horizon and dynamic tasks, where feedback may compound effects over time.
Loading 2608.16806v1…