PaperlayerView exact revision on arXiv

Source-linked AI summary

PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents

Junchi Chen, Changtao Miao, Yuxiao Xiang, Zhenchao Jin, Haojie Yuan, Qi Chu, Tao Gong, He Liu, Bo Zhang, Jiansheng Cai, Zhe Li, Nenghai Yu

arXiv:2609.32801v1cs.AIcs.CRcs.CVcs.RO

TL;DR

Embodied multi-step plans can create physical risks that existing semantic guardrails and single-subtask detectors may miss. PlanGuard evaluates complete plans in their current environments before execution, using MSP-Safe and STAC-OPD to train compact detectors. PlanGuard-2B achieves 87.21% average F1 across the four test subsets.

  • Problem

    Physical risks from subtask dependencies and environment interactions can be missed by general-purpose semantic guardrails and detectors that assess subtasks in isolation.

    Source: §1 · p. 2, §2.2 · p. 4

  • Method

    PlanGuard assesses complete plans before execution, with MSP-Safe supervision and STAC-OPD transferring strong-teacher guidance along compact students’ on-policy trajectories.

    Source: §1 · p. 3, §3.3 · p. 8

  • Results

    87.21% average F1: PlanGuard-2B maintains strong performance across four test subsets and achieves 86.47% average F1 on unseen scene, hazard, and planner categories.

    Source: §4.2 · p. 11

  • Takeaways & Limitations

    PlanGuard-0.8B and PlanGuard-2B outperform evaluated general-purpose VLM and safety-guardrail baselines on average F1, while stepwise evaluation favors complete-plan assessment over isolated subtasks.

    Source: §4.2 · p. 11

Abstract

from arXiv · show

Embodied task planners may produce multi-step plans whose subtask dependencies and interactions with the environment create physical risks during execution. Yet existing safeguards overlook such compositional risks, as general-purpose guardrails focus on semantic harm and embodied safety detectors assess subtasks in isolation. To address this gap, we introduce PlanGuard, the first pre-execution detector that evaluates the physical safety of a complete multi-step plan in its current environment. For training and evaluation, we construct a Multi-Step Plan Safety (MSP-Safe) dataset through paired task construction, plan generation using diverse planners, and safety annotation by three judges. Task-oriented SFT on MSP-Safe establishes fundamental plan-safety assessment capabilities, yet a substantial gap remains between compact models suitable for real-time deployment and stronger but costlier large models. Accordingly, we propose Strong-Teacher Adaptive Compensation for On-Policy Distillation (STAC-OPD), which provides compact models with adaptive strong-teacher supervision along their on-policy trajectories. It combines token-level distribution transfer from a fine-tuned strong teacher with probability-routed sequence-level compensation, retaining student-generated targets when the student favors the reference safety decision and using teacher-reconstructed targets otherwise. Across all test subsets, PlanGuard-2B achieves average 87.15% ACC and 87.21% F1, demonstrating effective whole-plan physical-risk detection at compact model scale. Code and dataset will be publicly released.

1. Introduction

Multi-step plans can create physical risks through subtask dependencies and interactions with the environment, which existing safeguards may miss. PlanGuard assesses complete plans before execution, supported by the MSP-Safe dataset and STAC-OPD training for compact models.

  • Subtask dependencies and environment interactions can make an apparently safe plan physically hazardous, motivating pre-execution detection.Source: §1 · p. 2
  • PlanGuard detects physical risks in complete planner-generated multi-step plans before execution by assessing the instruction and current environment.Source: §1 · p. 3
  • MSP-Safe provides about 16K annotated plans, constructed through paired tasks, diverse planner generation, and three-judge safety annotation.Source: §1 · p. 3
  • STAC-OPD transfers a fine-tuned strong teacher’s token-level preferences along student trajectories and supplies routed sequence-level targets when the reference decision remains disfavored.Source: §1 · p. 3, Figure 2 · p. 4
  • The paper reports effective and generalizable complete-plan safety detection across the In-Domain and three category-held-out subsets.Source: §1 · p. 3

2. Related Work

Embodied planning methods generate executable subtask sequences, but existing benchmarks and guardrails do not provide a deployable way to detect physical risks across complete plans. PlanGuard addresses this gap with pre-execution whole-plan detection.

2.1. Task Planning for Embodied Agents

  • Embodied task planners include general VLMs, embodiment-pretrained VLMs, and VLAs that integrate planning with embodied perception and action.Source: §2.1 · p. 4

2.2. Safety of Embodied Task Planners

  • Existing benchmarks evaluate planner safety, while general-purpose guardrails focus on semantic content and single-subtask detectors miss cross-step dependencies and cumulative environmental effects.Source: §2.2 · p. 4
  • PlanGuard is proposed as a pre-execution detector for complete planner-generated multi-step plans in embodied agents.Source: §2.2 · p. 5

3. Methodology

The methodology has two components: constructing MSP-Safe and training PlanGuard with a two-stage optimization framework.

  • The methodology constructs the MSP-Safe dataset and trains the pre-execution detector using a two-stage optimization framework.Source: §3 · p. 5

3.1. MSP-Safe Dataset Construction

MSP-Safe pairs controlled task variants, generates complete plans from diverse planners, and annotates plan safety through a three-judge protocol. Expert checks and dataset statistics characterize the resulting supervision and evaluation splits.

3.1. MSP-Safe Dataset Construction

  • MSP-Safe contains environment image-instruction pairs, complete planner-generated plans, and structured binary safety labels with rationales.Source: §3.1 · p. 5

3.1.1. Paired Task Construction

  • Base/Risk tasks vary one controlled factor across instruction or environment, using 28 scenes and 12 hazard types to form 336 combinations.Source: §3.1.1 · p. 5

3.1.2. Diverse Plan Generation

  • Six planners from general VLM, embodiment-pretrained VLM, and VLA families generate complete plans with diverse structures, action granularities, and planning styles.Source: §3.1.2 · p. 5

3.1.3. Three-Judge Safety Annotation

  • Three heterogeneous judges assess complete plans for physical risk and provide binary decisions and explanations; majority voting determines the retained annotation.Source: §3.1.3 · p. 6

3.1.4. Expert Verification of Safety Annotations

  • On a random 1% training subset, three-judge majority vote achieved 96.0% agreement and 95.65% F1 against expert consensus; experts reviewed all test candidates.Source: §3.1.4 · p. 6

3.1.5. Dataset Statistics

  • MSP-Safe-Train contains 13,692 plans, while MSP-Safe-Test contains 1,992 expert-reviewed samples across in-domain and three category-held-out subsets.Source: §3.1.5 · p. 6

3.2. Task-Oriented Supervised Fine-Tuning

Task-Oriented SFT adapts pretrained models for complete-plan safety assessment using structured Result-Reason responses. Compact models are faster but converge below the 27B model, motivating knowledge transfer.

3.2. Task-Oriented Supervised Fine-Tuning

  • SFT trains models to generate structured Result-Reason responses for complete-plan safety assessment.Placing Result first allows a low-latency safety decision, with Reason generated only when an explanation is required.Source: §3.2 · pp. 6–7
  • Figure 4(a) shows compact-model performance saturating below the 27B model; compact models have substantially lower inference latency, further reduced by result-only generation, as Figure 4(b) shows.This performance-efficiency gap motivates STAC-OPD, which transfers the SFT-trained 27B teacher’s plan-safety knowledge to the SFT-trained compact students.Source: §3.2 · p. 7

3.3. Strong-Teacher Adaptive Compensation for OPD

STAC-OPD transfers a frozen strong teacher’s plan-safety knowledge to compact students through token-level supervision on student rollouts and adaptive complete-response targets. Its routing uses post-distillation decision probabilities to select student-generated or teacher-reconstructed targets.

3.3. Strong-Teacher Adaptive Compensation for OPD

  • STAC-OPD combines token-level distribution transfer on student-generated trajectories with probability-routed sequence-level compensation.The first stage supplies teacher preferences on student-induced states; the second gives complete-response supervision to samples that remain difficult.Source: §3.3 · p. 7

(a) Strong-Teacher On-Policy Distillation

  • Unlike fixed-target SFT, on-policy distillation supervises student-generated prefixes that may depart from annotated trajectories.Early generated errors can lead the student to states not directly supervised by fixed-target SFT.Source: §3.3 · pp. 7–8
  • On-policy distillation applies token-level reverse KL supervision from a frozen teacher to prefixes reached by the rollout student.The teacher’s distribution communicates relative preferences among alternative continuations on states the student is likely to encounter during inference.Source: §3.3 · p. 8

(b) Probability-Routed Hard Compensation

  • Post-distillation probabilities for the reference versus opposite safety decision determine which samples receive explicit sequence-level supervision.The probabilities are evaluated at the fixed output prefix immediately preceding the safety decision.Source: §3.3 · p. 9
  • When the student favors the reference decision, STAC-OPD retains a format-valid student response; otherwise, the frozen teacher reconstructs a complete target from the input.The teacher reconstructs without receiving the original annotation or reference decision, so its safety decision is not guaranteed to match the reference.Source: §3.3 · p. 9
  • The routed complete targets are optimized with a maximum-likelihood objective, combining student-target consolidation with teacher sequence-level distillation.The student-routed branch consolidates behavior already generated while favoring the reference decision; the teacher-routed branch supplies hard targets when that decision remains disfavored.Source: §3.3 · p. 9

4. Experiments

Experiments train PlanGuard models through SFT and STAC-OPD, evaluate on balanced in-domain and category-held-out data, compare against model and guardrail baselines, and report ACC and F1.

4.1.1. Training Details

  • Models are trained through SFT and STAC-OPD, with Qwen3.5-27B independently fine-tuned and frozen as the strong teacher.Experiments use eight NVIDIA A800 GPUs with 80 GB memory each.Source: §4.1.1 · pp. 9–10

4.1.2. Evaluation Data

  • MSP-Safe-Test has 1,992 expert-reviewed, Safe/Unsafe-balanced samples across In-Domain and scene-, hazard-, and planner-held-out subsets.Each sample includes a visual environment, task instruction, complete planner-generated plan, and structured safety annotation.Source: §4.1.2 · p. 10

4.1.3. Baseline Models

  • Baselines include general-purpose VLMs, safety guardrails, and EMBGuard-2B, which classifies a complete plan as Unsafe if any independently assessed subtask is predicted unsafe.All models use greedy decoding unless otherwise specified.Source: §4.1.3 · pp. 10–11

4.1.4. Metrics

  • Evaluation reports Accuracy and F1, mapping outputs to Safe or Unsafe and treating Unsafe as the positive class.Source: §4.1.4 · p. 11

4.2. Main Results

PlanGuard performs strongly across the four MSP-Safe-Test subsets, including held-out categories. Its compact variants also exceed the reported general-purpose VLM and safety-guardrail baselines on average F1.

4.2. Main Results

  • 87.21% average F1 for PlanGuard-2B, compared with 82.25% for PlanGuard-0.8B, across the four MSP-Safe-Test subsets.PlanGuard-2B also achieves 86.47% average F1 on unseen scene, hazard, and planner categories.Source: §4.2 · p. 11
  • 5.73 points is PlanGuard-2B’s average F1 improvement over Qwen3.5-Plus, and 31.27 points over GuardTrace-VL-3B.PlanGuard-0.8B improves average F1 by 0.77 and 26.31 points over those respective baselines.Source: §4.2 · p. 11
  • PlanGuard-0.8B and PlanGuard-2B exceed EMBGuard-2B by 13.29 and 18.25 average F1 points under the Stepwise protocol.The paper interprets the comparison as highlighting the importance of evaluating complete multi-step plans rather than isolated subtasks.Source: §4.2 · p. 11

4.3. Ablation Study

The ablations show gains from successive training stages and from evaluating complete plans rather than subtasks independently. The diagnostic set pairs plans whose individual subtasks appear safe despite risk emerging during ordered execution.

4.3.1. Training-Stage Ablation

  • 78.96% average F1 after Task-Oriented SFT, up from 64.38%; Strong-Teacher OPD raises it to 81.47%, and Hard Compensation adds 0.78 points to reach 82.25%.The additional Hard Compensation gains are 0.99 and 0.91 points on Scene-Heldout and Hazard-Heldout, respectively.Source: §4.3.1 · p. 11

4.3.2. Full-Plan versus Stepwise Safety Detection

  • The diagnostic set contains 50 neutral tasks paired with safe and unsafe plans whose subtasks appear individually safe, but whose ordered execution creates physical risk.The paired plans share an initial image and instruction; risks arise through intermediate state changes or cross-step dependencies.Source: §4.3.2 · pp. 11–12
  • 15.04 points is PlanGuard-2B’s F1 improvement for Full over Stepwise detection, versus 6.48 points for PlanGuard-0.8B; EMBGuard-2B shows no improvement.Full detection evaluates the complete plan in one pass, while Stepwise evaluates each subtask independently against the initial image and marks a plan unsafe if any subtask is predicted unsafe.Source: §4.3.2 · p. 12

5. Conclusion

PlanGuard detects physical risks in complete planner-generated plans before execution, and MSP-Safe provides structured supervision for training and evaluation. Experiments report strong performance across in-domain and category-held-out settings, while the full-plan analysis shows the value of capturing action order and cross-step dependencies.

5. Conclusion

  • PlanGuard is a pre-execution detector dedicated to complete multi-step plans from embodied task planners.It addresses physical risks arising from subtask dependencies, action order, and environment interactions.Source: §5 · p. 12
  • MSP-Safe supplies structured safety supervision over complete multi-step plans, while STAC-OPD enriches compact-model supervision at token and sequence levels.Task-oriented SFT first establishes basic plan-safety capabilities.Source: §5 · p. 12
  • Experiments report that PlanGuard outperforms evaluated general-purpose VLMs, safety guardrails, and prior subtask-level embodied safety detectors, while generalizing to held-out scene, hazard, and planner categories.The full-plan versus stepwise analysis reports that complete-plan supervision captures action order and cross-step dependencies that isolated subtask judgments cannot recover.Source: §5 · p. 12

6. Ethical Statement

MSP-Safe contains synthetic embodied tasks, images, and planner outputs, with no intentionally collected personal or sensitive data. Hazardous content is included solely for embodied-agent safety research and risk prevention.

6. Ethical Statement

  • MSP-Safe contains only synthetic embodied tasks, images, and planner outputs, with no intentionally collected personal or sensitive data.Source: §6 · p. 13
  • Hazardous content is included solely for embodied-agent safety research and risk prevention.Source: §6 · p. 13

Related papers

Loading 2609.32801v1…