PaperlayerView exact revision on arXiv

Source-linked AI summary

Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions

Lele Zheng, Tong Chen, Ke Cheng, Tao Zhang, Xingchi Liu, Ji He, Xutong Mu, Yulong Shen

arXiv:2609.27847v1cs.CR

TL;DR

Security and privacy research on large-model-driven embodied agents remains fragmented, while risks can extend through embodied closed loops beyond model outputs. This survey organizes the research by lifecycle stage and finds that attacks, defenses, and evaluation span multiple stages, but system-level solutions and unified evaluation remain incomplete.

  • Problem

    Research remains fragmented across lifecycle stages, while risks in embodied closed loops can extend beyond the model layer and affect the physical environment.

    Source: §1 · p. 2, §5 · p. 27

  • Method

    The survey defines the system boundary and organizes existing studies with a five-stage lifecycle framework covering attacks, defenses, and evaluation.

    Source: §1 · pp. 2–3

  • Results

    Attacks and consequences span training through deployment, defenses cover multiple layers but composable system-level solutions remain underdeveloped, and evaluation lacks a unified evidence standard.

    Source: §6 · p. 28

  • Takeaways & Limitations

    The lifecycle framework integrates scattered studies into a unified system process for analyzing attacks, defenses, evaluations, and research gaps throughout the lifecycle.

    Source: §6 · p. 28

    Existing studies have not yet formed a systematically validated end-to-end defense stack.

    Source: §4.6.2 · p. 27

Abstract

from arXiv · show

Large-model-driven embodied agents integrate foundation models with perception, reasoning, planning, and physical action, extending conventional model-level risks into embodied closed loops. Existing studies on their security and privacy remain fragmented across different system components and operational stages, making it difficult to understand how risks arise, propagate, and ultimately affect physical behavior or sensitive information. This survey presents a lifecycle-based analysis of security and privacy in large-model-driven embodied agents. We organize existing research into five stages: model construction and supply chain, multimodal input and interaction, semantic reasoning and task planning, action execution and physical feedback, and long-term deployment. Within this lifecycle, we systematically review representative attacks, defenses, and evaluation methods. Our analysis shows that attack entry, consequence realization, and defense intervention often occur at different stages of the embodied closed loop. It further reveals substantial gaps in end-to-end protection, real-world evaluation, and long-term privacy governance. This survey provides a unified perspective for understanding current progress and identifying critical directions for securing large-model-driven embodied agents.

1 Introduction

Large-model risks can propagate through embodied systems into physical behavior, while security and privacy research remains fragmented across stages. The survey defines a system boundary and uses a lifecycle perspective to organize this research and examine cross-stage risks and defenses.

1 Introduction

  • Risks can alter environmental perception, task planning, tool or API calls, trajectories, and control, extending from information space into the physical world.The survey argues that security must be understood along the full system chain rather than only through model inputs and outputs.Source: §1 · p. 2
  • Long-term embodied deployment raises privacy risks from persistent sensing and interaction, beyond training-data leakage.Collected information may support continual learning, decision optimization, and service provision.Source: §1 · p. 2
  • Research is distributed across lifecycle stages, and locally focused defense and evaluation studies offer limited visibility into the complete embodied closed loop.Different defense targets and evaluation goals make it difficult to compare defense coverage or relate results.Source: §1 · p. 2
  • The survey defines its scope as embodied agents substantially driven by large-model capabilities, distinguishing them from ordinary LLM security, software-agent security, and traditional robotics safety.Its analysis extends to the system boundary that supports agent operation.Source: §1 · p. 2
  • The survey organizes research into five lifecycle stages and analyzes system stages, attack and defense mechanisms, and evaluation methods from an operational perspective.The stages are model construction and supply chain; multimodal input and interaction; semantic reasoning and task planning; action execution and physical feedback; and deployment and human-centered impacts.Source: §1 · p. 3
  • The survey examines cross-stage risk propagation and identifies challenges in defense composition, system-level evaluation, real-world validation, and long-term deployment governance.It categorizes studies by the lifecycle stage where their main effect or impact occurs.Source: §1 · p. 3

2 Background and Scope

This section defines the survey’s analytical boundary around large-model-driven embodied agents and positions its lifecycle account against prior reviews.

2.1 Related Work and Survey Positioning

  • Table 1 compares prior surveys across scope, pipeline coverage, attack/defense taxonomy, evaluation, and security–privacy–deployment integration; this survey addresses the lack of a unified full-lifecycle account.Source: §2.1 · pp. 3–4, Table 1 · p. 4

2.2 Large-Model-Driven Embodied Agents

  • The survey focuses on agents whose core capabilities are substantially driven by large models, using the agent and its supporting software and hardware as the analysis boundary.For domains such as autonomous driving, UAVs, and multi-robot systems, it focuses on the large-model-driven agents and their operational-loop security and privacy problems.Source: §2.2 · pp. 4–5

2.3 Security, Privacy, Safety, Robustness, and Trustworthiness

  • Security concerns preserving intended functions, authority boundaries, and execution under malicious interference or unauthorized influence; embodied integration joins model and operational-control concerns.Source: §2.3 · p. 5
  • Privacy covers sensitive-information governance and improper access or leakage, extending in embodied agents to continuous sensing, memory storage, and deployed operation.Source: §2.3 · pp. 5–6
  • Safety concerns unacceptable real-world consequences, robustness concerns stable behavior under complex conditions, and trustworthiness concerns overall user and societal trust.The survey uses these concepts mainly to characterize consequences of security and privacy problems and requirements for deployment and governance.Source: §2.3 · p. 6

3 Five-Stage Lifecycle Taxonomy

The survey organizes security and privacy research around the embodied-agent lifecycle and the problem types studied within it. Its taxonomy shows that attack entry, downstream consequences, and defense intervention can span different lifecycle stages.

3.1 Lifecycle Stages

  • The five stages extend from model construction and supply chain through multimodal input, reasoning and planning, action execution, and long-term deployment.The framework extends the conventional perception–decision–execution loop with model construction and deployment ecosystems and impacts.Source: §3.1 · pp. 6, 8, Figure 2 · p. 7

3.2 Research Problem Types and Lifecycle-by-problem Matrix

  • The survey classifies studies as attacks, defenses, or evaluation, then combines those problem types with the five stages in a lifecycle-by-problem matrix.Source: §3.2 · p. 8, Table 3 · p. 9
  • Input attacks and runtime defenses form relatively clear research lines, while training and supply-chain security and long-term deployment governance remain less developed.Source: §3.2 · p. 8, Table 3 · p. 9

3.3 Cross-lifecycle Issues

  • Attack entry often differs from where consequences appear: training backdoors may affect later action generation, while physical prompts can progress from scene understanding to planning and execution.Some defenses intervene later, at planning or execution, to mitigate upstream risks.Source: §3.3 · p. 9
  • For problems with cross-stage propagation, single-stage analysis cannot fully capture the attack path, affected scope, or defense composition.The survey revisits these problems through a cross-lifecycle analysis of risk propagation and system governance.Source: §3.3 · p. 9

4.1 Model Construction, Training, and Supply Chain Security

The survey treats model construction, training, updates, and component integration as early lifecycle security boundaries. It reviews persistent behavior and privacy risks from attacks at this stage, alongside defenses that mitigate some risks but leave detection and evaluation gaps.

4.1 Model Construction, Training, and Supply Chain Security

  • Training data, model design, updates, action representations, infrastructure, and external interfaces can shape behavior before and after deployment.Risks introduced during construction may remain latent and later appear as action deviation, task failure, or privacy leakage under specific conditions.Source: §4.1 · pp. 9–10

4.1.1 Attacks.

  • Backdoors bind triggers to target behaviors, including long-horizon trajectories, low-level action primitives, state conditions, and visual scenes.Small trajectory deviations can accumulate into task failure, while manipulated action labels can induce open-gripper behavior, grasp failure, or object dropping.Source: §4.1.1 · pp. 10–11
  • Supply-chain compromise can alter robot behavior through external perception models, VLM/LVLM modules, or service dependencies, without access to policy training data or weights.TrojanRobot changes visual grounding, affecting target localization and action execution.Source: §4.1.1 · p. 11
  • Training-data inference can reveal whether samples or demonstrations were used, and action outputs may expose private schedules, object layouts, or user behavior patterns.Source: §4.1.1 · p. 11

4.1.2 Defenses.

  • Safety alignment and post-training behavior correction target unsafe behavior, while capability removal seeks to erase unwanted behaviors or sensitive knowledge.Current evidence for capability removal cannot strictly prove complete removal of demonstrations, object associations, or dangerous behavior.Source: §4.1.2 · pp. 11–12
  • Federated-training protections use encryption and proxy re-encryption for parameter transmission, but do not replace data provenance, backdoor auditing, or component certification.The cited methods also address robustness under node dropout.Source: §4.1.2 · p. 12
  • L1 evaluations use method-specific measures and varied benchmarks, leaving the stage without a shared evaluation protocol.Studies assess attack success, clean-task success, trigger stealthiness, privacy inference, safety violations, and recovery across benchmarks with differing tasks, action spaces, and evidence levels.Source: §4.1.3 · p. 12

4.2 Multimodal Input and Interaction Security

Multimodal inputs can shape scene interpretation and task representations, carrying risks beyond prompt-level misunderstanding into downstream planning and physical execution. The section surveys runtime attacks, defenses, and evaluations across this input-to-action path.

4.2.1 Attacks.

  • Input-stage attacks include sensor-level manipulation, adversarial perception, environmental prompt injection, and jailbreaks that elicit actions.Their physical or multimodal inputs can affect how agents interpret environments and generate actions.Source: §4.2.1 · p. 13, Table 6 · p. 13
  • Phantom Menace [52] transfers camera or microphone signal attacks from simulation to real robots, producing misgrasps, drops, collisions, and erratic execution.The attack assumes physical-signal access but not model or software access.Source: §4.2.1 · p. 13
  • Environmental text can act as an indirect command channel: CHAI [4] uses deceptive signs, while robotic prompt-injection attacks can induce incorrect navigation instructions [124].Jailbreaks also target executable behavior, including fixed action sequences [33] and physically malicious behavior [121].Source: §4.2.1 · p. 14

4.2.2 Defenses.

  • Input defenses range from moderation and jailbreak detection to latent-representation sanitization, perception-aware robustness, and interaction-level guardrails.Moderation cannot directly defend against physical sensor attacks; latent sanitization depends on reliably localizing dangerous concepts, while interaction guardrails are typically task-specific.Source: §4.2.2 · pp. 14–15
  • SAFE-Dict [96] weakens harmful concept coefficients in fused VLA representations without retraining the whole model, but cannot verify environmental-text sources or block physical sensor attacks.Other approaches improve perception with thermal sensing and control barrier functions [114], or assess certainty in speech-guided navigation [78].Source: §4.2.2 · p. 15

4.2.3 Evaluation.

  • L2 evaluations ask whether multimodal inputs change an agent’s perception, task understanding, or downstream behavior, spanning visual and instruction perturbations, natural-language red teaming, physical environmental changes, and hazardous-instruction diagnosis.Source: §4.2.3 · p. 15

4.3 Semantic Reasoning, Risk Awareness, and Task Planning Security

At the reasoning and planning stage, agents turn goals and environmental information into executable plans, so planning safety is distinct from merely satisfying a formal task objective. The section reviews attacks on plan generation, defenses before and during execution, and evaluations of safe planning and behavior.

4.3 Semantic Reasoning, Risk Awareness, and Task Planning Security

  • Planning feasibility and safety differ: a plan can meet its task objective yet violate safety constraints, physical conditions, or domain norms.Because plans feed downstream action generation and control, unsafe plans can become dangerous behavior.Source: §4.3 · p. 16

4.3.1 Attacks.

  • Decision-level adversarial prompting targets task decomposition and plan steps, potentially steering them away from user intent or toward an attacker’s goal.EIRAD [50] adapts GCG attacks to multi-step plan generation and tests whether steps match a target attack task.Source: §4.3.1 · pp. 16–17
  • Policy-executable jailbreaks seek outputs that are both harmful and acceptable to action interfaces; POEX [54] optimizes suffixes for executability, while ROBOPAIR [67] checks robotic API syntax.Attack success therefore depends on whether downstream modules can parse and execute the output.Source: §4.3.1 · pp. 16–17

4.3.2 Defenses.

  • Formal pre-execution verification checks plans against explicit constraints, but depends on complete constraints and reliable environment observation and symbolization.SafeGate [60] authorizes, defers, or rejects tasks using safety contracts and Z3 verification; Safety Chip [111] encodes requirements in LTL and supports replanning.Source: §4.3.2 · p. 17
  • Runtime safety logic checks plans and actions continuously, whereas risk reasoning and counterfactual correction offer flexible but non-guaranteed ways to identify and revise risks.Runtime logic depends on accurate predicates and contextual safety memory [52]; reasoning depends on knowledge and evaluator quality [54], and counterfactual correction on reasoning accuracy and scenario coverage [56].Source: §4.3.2 · p. 18

4.3.3 Evaluation.

  • At this stage, evaluation asks whether models transform task goals, environment states, safety constraints, and action requirements into safe plans, across planning safety, reasoning-chain localization, interactive execution, dynamic environments, and high-risk domains.Source: §4.3.3 · p. 18

4.4 Action Execution, Control Chain, and Physical Feedback Security

At execution, generated plans become control signals that interact with the physical environment, so safety depends on action constraints, monitoring, and feedback. The section covers action-sequence attacks, execution-layer defenses, and evaluations that look beyond task completion.

4.4.1 Attacks.

  • Action-level manipulation can hide risk in sequences of individually ordinary actions, whose combined physical consequences may cause harm.Blindfold’s Adversarial Proxy Planning [27] inserts cover actions into malicious sequences and checks symbolic constraints for executability.Source: §4.4.1 · p. 19

4.4.2 Defenses.

  • Runtime control shielding revises generated actions before execution, while language-conditioned MPC filtering and AEGIS [24] translate safety requirements or hazard grounding into control constraints.These methods depend on environment modeling and control accuracy [67]; multi-robot shielding also addresses inter-robot and obstacle distances [73].Source: §4.4.2 · p. 20
  • Constrained action decoding filters or reweights candidate actions before rollout; SafeDec [34] uses predicted states and STL specifications to enforce temporal safety.This addresses unsafe behavior that can accumulate during autoregressive action sampling.Source: §4.4.2 · pp. 20–21
  • Execution monitoring detects violations or predicts failure, but brief uncertainty peaks can be hidden by global average entropy, and unified recovery procedures remain underdeveloped.RoboCritics [39] checks motion traces and supplies feedback; SAFE [21] predicts failure scores, while uncertainty methods can support replanning or human takeover [80].Source: §4.4.2 · p. 21
  • Action-hallucination mitigation seeks to reduce actions inconsistent with visual observations while preserving multiple plausible execution paths; EOCD adjusts decoding using contrastive action distributions.Domain-specific compliance also matters: RoboChemist [127] evaluates both task success and adherence to experimental norms.Source: §4.4.2 · p. 21

4.4.3 Evaluation.

  • Execution evaluations assess temporal safety and fine-grained action quality, exposing unsafe behavior that binary task-success measures can miss.SAFEMANIP [25] monitors finite manipulation traces, mainly in simulated trajectories, while Eval-Actions and AutoEval [49] assess smoothness, safety, and execution-source authenticity.Source: §4.4.3 · pp. 21–22

4.5 Deployment, Multi-agent Ecosystems, and Human-centered Impacts

Long-term deployment extends security and privacy concerns to cloud-edge communication, multi-agent collaboration, permissions, and human interaction. The section reviews attacks on information flows and trust, system-level defenses, and evaluations of architecture and interaction trustworthiness.

4.5 Deployment, Multi-agent Ecosystems, and Human-centered Impacts

  • Deployment risks accumulate across communication links, collaborative messages, persistent sensing, and human-system trust, extending beyond the model itself.They can affect commands, spread malicious intent, build sensitive behavioral profiles, or contribute to over-authorization and mistaken reliance.Source: §4.5 · p. 22

4.5.1 Attacks.

  • Communication hijacking tampers with messages between agents, cloud models, or edge devices, potentially causing incorrect commands or ignored safety constraints.From Prompts to Motors shows how intercepted robot-API messages can alter obstacle information or override a user’s stop command.Source: §4.5.1 · pp. 22–23
  • Multi-agent propagation attacks exploit collaboration messages: InfectBot [30] uses one manipulated entry robot to forward unsafe intent through peer links.Defenses therefore need source verification, propagation limits, and controls on malicious forwarding.Source: §4.5.1 · p. 23
  • Trust-manipulation attacks perturb conditions that shape human trust, potentially changing collaboration rhythm and takeover behavior without directly changing control commands.Prior work models trust-aware collaboration and uses environmental perturbations to reduce collaboration efficiency and human trust.Source: §4.5.1 · p. 23

4.5.2 Defenses.

  • Deployment defenses emphasize provenance and orchestration, runtime access control and mission recovery, and optimization-mediated coordination that keeps LLMs at higher decision levels.These approaches address multi-party task accountability [96, 97], dynamic permissions and recovery [99, 100], and safety-constrained multi-robot action optimization with possible human feedback [101, 102].Source: §4.5.2 · pp. 23–24
  • Privacy and HRI governance address user data, trust calibration, and interaction risks; MALLM uses personalized agents and local processing in multi-user apartment environments to manage preference conflicts and protect privacy.Deployment-architecture evaluation examines communication latency, provenance integrity, and data sovereignty in an MQTT, Home Assistant, and Android edge-node chain [119]; an embodied-agent trustworthiness benchmark evaluates 150 interactive tasks.Source: §4.5.2 · p. 24, §4.5.3 · p. 25

4.6 Cross-lifecycle Risk Propagation, Defense Composition, and Future Directions

Risks can enter, remain latent, and produce consequences at different stages of the embodied closed loop. The section traces these propagation paths and proposes a layered defense composition, while noting that end-to-end validation remains incomplete.

4.6 Cross-lifecycle Risk Propagation, Defense Composition, and Future Directions

  • Attack entry, consequence realization, and defense intervention can occur at different lifecycle stages as dependencies carry effects from sensing through planning and action.Environmental feedback and deployment records can also influence later decisions.Source: §4.6.1 · p. 25

4.6.1 Cross-lifecycle Risk Propagation.

  • Training-time poisoning and backdoors can remain latent until later triggers alter action generation, including through visual, textual, proprioceptive, or physical-scene cues.Reported execution effects include object dropping, grasp failure, and trajectory drift.Source: §4.6.1 · p. 25
  • Sensor manipulation, environmental text, and rollout prompts can reach command or action interfaces and produce abnormal behavior or persistent action sequences.Phantom Menace reports abnormal real-robot behavior from poisoned camera and microphone inputs; CHAI describes scene text interpreted as control instructions.Source: §4.6.1 · p. 25
  • Planning attacks can produce harmful but executable policies or API-valid instructions, while deployment-message manipulation and peer communication can affect hardware outputs or spread unsafe intent.Source: §4.6.1 · pp. 25–26
  • Cross-boundary data flows connect system modules, allowing a local vulnerability to propagate downstream into planning decisions or physical execution outcomes.Source: §4.6.1 · p. 26
  • The proposed five-layer defense composition spans provenance and update control, input and message admission, semantic safety contracts, runtime constraint enforcement and recovery, and audit and governance.Source: §4.6.2 · p. 26

4.6.2 Cross-lifecycle Defense Composition.

  • Provenance records and update traceability support later localization and accountability, while input admission checks source, semantic role, and authority before information enters planning or control.Source: §4.6.2 · p. 26
  • Semantic contracts translate goals and constraints into checks; runtime defenses constrain actions and provide recovery, while audit and governance support incident reconstruction, but no end-to-end stack is systematically validated.The survey identifies constraint coverage, environmental grounding, recovery paths, and cross-layer coordination as challenges.Source: §4.6.2 · pp. 26–27

5 Gaps and Future Directions

The survey identifies lifecycle-wide gaps in evidence, defenses, and governance, with future work needed across each stage and in cross-stage evaluation.

5 Gaps and Future Directions

  • Research remains fragmented across lifecycle stages, uneven in evidentiary depth, and insufficient in system-level governance.Source: §5 · p. 27
  • Model construction lacks systematic toolchains for detecting latent backdoors and component contamination, while membership leakage and capability removal lack strong guarantees.Source: §5 · p. 27
  • Input-stage studies often isolate one channel, leaving compound multimodal effects and propagation into planning or execution underexplored.Source: §5 · p. 27
  • Planning research faces a disconnect between capability and safety awareness, and text-level rejection alone is insufficient for executable-plan and policy risks.Source: §5 · p. 27
  • Execution defenses often rely on control filtering and anomaly detection under assumptions such as reliable perception, approximate dynamics, or predefined constraints.The survey calls for closed-loop recovery frameworks that respond to and recover from anomalies.Source: §5 · p. 27
  • Deployment defenses remain platform- or task-specific, and long-term privacy evaluation lacks systematic quantification of risks emerging over prolonged use.Source: §5 · pp. 27–28
  • Cross-stage benchmarks lack sufficiently standardized evidence levels, making them difficult to compare or substitute directly.Source: §5 · p. 28

6 Conclusion

The survey defines a system-level scope and organizes security and privacy research into a five-stage lifecycle framework. Its synthesis finds broader attacks and defenses, alongside unresolved composability, evidence-standardization, and governance gaps.

6 Conclusion

  • The survey includes agents where large-model capabilities substantially participate in embodied closed-loop operation and organizes research across five lifecycle stages.The analysis extends from model outputs to the complete embodied-agent system.Source: §6 · p. 28
  • Attacks and consequences now span training, sensing, planning, execution, and deployment, while defenses cover alignment through governance; composable system-level solutions and unified evaluation standards remain underdeveloped.Source: §6 · p. 28
  • The survey concludes that research should move from pointwise protection toward systematic governance and support real-deployment risk assessment and accountability with clear evidence.Source: §6 · p. 28
Loading 2609.27847v1…