PaperlayerView exact revision on arXiv

Source-linked AI summary

Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress

Christopher Agia, Rohan Sinha, Jingyun Yang, Zi-ang Cao, Rika Antonova, Marco Pavone, Jeannette Bohg

arXiv:2410.04640v2cs.ROcs.AIcs.LG

TL;DR

Sample-level OOD detection and uncertainty measures do not fully characterize closed-loop failures caused by time-correlated errors in multimodal generative policies. Sentinel combines statistical temporal action-consistency monitoring for erratic failures with VLM assessment of task progress for consistent failures. On Cover Object and Close Box shifts primarily causing task-progression failures, Sentinel achieved 93% TPR with a 7% increase in FPR; real-world Push Chair detection accuracy was 95%.

  • Problem

    Sample-level OOD detection and uncertainty measures do not fully characterize closed-loop failures caused by time-correlated errors in multimodal generative policies.

    Source: §1 · p. 2

  • Method

    Sentinel combines statistical temporal action-consistency monitoring for erratic failures with VLM assessment of task progress for consistent failures.

    Source: §1 · p. 2, §7 · p. 8

  • Results

    On Cover Object and Close Box shifts primarily causing task-progression failures, Sentinel achieved 93% TPR with a 7% increase in FPR; real-world Push Chair detection accuracy was 95%.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The results highlight the importance of assigning specialized detectors to complementary failure categories in the evaluated simulation and real-world domains.

    Source: §7 · p. 8

    The erratic and task-progression categories may not be exhaustive, and Sentinel does not provide formal guarantees on failure detection.

    Source: §7 · p. 8

Abstract

from arXiv · show

Robot behavior policies trained via imitation learning are prone to failure under conditions that deviate from their training data. Thus, algorithms that monitor learned policies at test time and provide early warnings of failure are necessary to facilitate scalable deployment. We propose Sentinel, a runtime monitoring framework that splits the detection of failures into two complementary categories: 1) Erratic failures, which we detect using statistical measures of temporal action consistency, and 2) task progression failures, where we use Vision Language Models (VLMs) to detect when the policy confidently and consistently takes actions that do not solve the task. Our approach has two key strengths. First, because learned policies exhibit diverse failure modes, combining complementary detectors leads to significantly higher accuracy at failure detection. Second, using a statistical temporal action consistency measure ensures that we quickly detect when multimodal, generative policies exhibit erratic behavior at negligible computational cost. In contrast, we only use VLMs to detect failure modes that are less time-sensitive. We demonstrate our approach in the context of diffusion policies trained on robotic mobile manipulation domains in both simulation and the real world. By unifying temporal consistency detection and VLM runtime monitoring, Sentinel detects 18% more failures than using either of the two detectors alone and significantly outperforms baselines, thus highlighting the importance of assigning specialized detectors to complementary categories of failure. Qualitative results are made available at https://sites.google.com/stanford.edu/sentinel.

1 Introduction

Generative robot policies can behave unpredictably on out-of-distribution test cases, while sample-level shift or uncertainty measures do not fully characterize closed-loop failures. Sentinel divides failure detection between erratic behavior and consistent behavior that fails to progress.

1 Introduction

  • Imitation-learned policies can encounter out-of-distribution test cases beyond controlled labs, motivating deployment-time monitoring for failures under distribution shift.The introduction notes generative policies’ early signs of generalization to novel environments and embodiments.Source: §1 · p. 2
  • OOD detection and uncertainty methods assess individual inputs and outputs but do not fully characterize closed-loop failures from time-correlated prediction errors.Multimodal actions can also vary greatly across timesteps, creating diverse runtime behaviors and failure modes.Source: §1 · p. 2
  • Sentinel detects erratic policy behavior with statistical temporal action consistency measures and task-progression failures with VLMs assessing progress from video.The VLM monitor targets failures that STAC cannot detect, such as consistent actions that do not solve the task.Source: §1 · p. 2
  • The framework combines complementary detectors, and the paper reports detection of over 97% of unknown diffusion-policy failures across simulated and real-world mobile-manipulation domains.The reported setup uses successful policy rollouts and a natural-language task description.Source: §1 · p. 2

2 Related Work

Prior work spans policy learning, OOD detection, and runtime monitoring, but the paper focuses on detecting unknown closed-loop failures in multimodal generative policies. Sentinel differs by combining direct monitoring of action-distribution consistency with zero-shot VLM assessment of task progress.

2 Related Work

  • Generative models learn multimodal action distributions, whose stochastic runtime behavior motivates characterizing generative robot policies for failure detection.The related-work discussion situates this focus amid advances in policy architectures, data collection, and high-capacity behavior policies.Source: §2 · p. 2
  • Some prior methods require known failure modes, while post hoc correction methods act after failures, leaving a role for runtime monitors.The paper frames runtime monitoring as complementary to methods that learn from corrective feedback.Source: §2 · p. 3
  • Prior monitoring approaches include OOD similarity, uncertainty, and symbolic-state methods, but face limits in failure relevance, computational expense, policy compatibility, or sensor assumptions.The paper notes that observation differences need not imply policy failure, while some symbolic methods assume multiple sensor modalities.Source: §2 · p. 3
  • Unlike prior VLM work cited here, Sentinel uses zero-shot assessment during task execution alongside policy-level detectors assigned to distinct failure categories.The cited comparison contrasts Sentinel with episode-level VLM success classification fine-tuned on a human-annotated dataset [63].Source: §2 · p. 3

3 Problem Setup

The paper frames failure detection as identifying, from observed states and sampled actions, whether a rollout will fail if execution continues. It considers stochastic policies that predict action chunks and diffusion policies, with calibration from successful validation trajectories.

3 Problem Setup

  • A failure detector classifies at each timestep whether a trajectory will fail if the policy continues through the remaining finite-horizon task.Its assessment uses only the history of observed states and sampled actions up to that timestep.Source: §3 · p. 3
  • A small dataset of successful validation trajectories can calibrate detector parameters and characterize nominal policy behavior near its training distribution.The policy is first trained, then validated on cases where it is expected to perform reliably.Source: §3 · p. 3
  • Performance is measured by true positive rate, true negative rate, and detection time, with detection time given by the earliest warning on a failed trajectory.A true negative is a successful trajectory on which the detector never warns.Source: §3 · p. 3
  • The policy predicts an h-step action sequence, executes its first k actions, then re-evaluates; successive predictions overlap for h−k timesteps.The paper visualizes the receding-horizon rollout for k = 1.Source: §3 · p. 4, Figure 2 · p. 4
  • The study specifies failure detection for diffusion policies, whose iterative denoising makes direct application of existing OOD methods challenging.The approach is described as generically applicable to several generative policy architectures.Source: §3 · p. 4

4 Proposed Approach: Sentinel

Sentinel assigns complementary failure categories to detectors with different timing needs: STAC flags erratic behavior, while VLMs assess whether consistent behavior is progressing toward the task. The method accumulates temporal action-distribution distances for STAC and uses video-based task reasoning for VLM monitoring.

4 Proposed Approach: Sentinel

  • The two categories are erratic failures, which call for rapid detection, and task-progression failures, which require observing behavior longer to judge progress.Erratic behavior may create costly resets, toppled objects, or safety hazards; consistent failure may include confidently placing an object incorrectly.Source: §4 · p. 4

4.1 STAC: Detecting Erratic Failures with Temporal Consistency

  • STAC compares statistical distances between overlapping action distributions at consecutive policy timesteps and accumulates the distances along the rollout.Cumulative scoring can detect both marginally elevated distances throughout an episode and more frequent inconsistency than in nominal scenarios.Source: §4.1 · p. 5
  • STAC raises a warning when its cumulative score exceeds a threshold calibrated from successful validation trajectories.The paper states that the false-positive rate on an i.i.d. successful test trajectory of length at most H is at most δ.Source: §4.1 · p. 5
  • Because diffusion policies make analytic distances challenging, the method approximates them by sampling batches of action sequences at each timestep.The samples are parallelized on a GPU.Source: §4.2 · p. 5

4.2 Detecting Task Progression Failures with VLMs

  • VLMs assess task progress from a video of image observations, the task description, elapsed time, and the task time limit, then classify the rollout as okay or failure.The prompt frames the assessment as chain-of-thought video question answering to reason over time and task context.Source: §4.2 · p. 6
  • Cloud VLM latency is less concerning for slower task-progression failures, while STAC handles urgent erratic failures; the detectors can run at different timescales.At the time of writing, GPT-4o’s mean cloud video-QA response time was 14.0 seconds.Source: §4.2 · p. 6

5 Experiments

Experiments evaluate failure detection in simulation and on a real robot, with test-time shifts in object dimensions or poses. In Close Box, Sentinel detects all erratic failures, while STAC is described as balancing failure detection with OOD generalization.

5 Experiments

  • Experiments span simulation and real-world mobile manipulation, including PushT, Close Box, Cover Object, and Push Chair.OOD scenarios randomize object scale and dimensions in PushT and Close Box, and object pose in Cover Object and Push Chair.Source: §5 · p. 6
  • Failure detectors share a calibration and evaluation protocol, with thresholds calibrated on in-distribution rollouts and standard detection metrics reported.Calibration uses 50 in-distribution rollouts per simulated task and 10 for the real-world task.Source: §5 · p. 6
  • Sentinel detects 100% of erratic policy failures in Close Box, with results averaged over 3 random seeds.The table caption says STAC accounts for both policy failures and OOD generalization, unlike embedding-based methods that associate state atypicality with failure.Source: Table 1 · p. 7

6 Results

Sentinel combines STAC and VLM monitoring to detect erratic and task-progression failures across simulation and real-world evaluations. Results show complementary strengths: STAC detects erratic failures and OOD successes, while VLMs identify temporally consistent task-progression failures.

STAC detects diffusion policy failures in multimodal domains

  • Only STAC exceeds 90% balanced accuracy in multimodal PushT, where output variance is high for both successful and failed policies.Embedding similarity also performs strongly in this domain, where state dissimilarity from the calibration dataset happens to correlate with failure.Source: §6 · p. 7, Figure 5 · p. 7

Statistical measures of action similarity enable temporal consistency detection

  • Statistical distance functions yield the strongest temporal-consistency detection; adding temporal consistency to baselines does not always improve performance.Non-statistical minimum distance performs worse than the baselines because it omits action multimodality.Source: §6 · p. 7, Figure 5 · p. 7

STAC accounts for OOD failures and generalization

  • STAC achieves 96% aggregate accuracy in Close Box and differentiates OOD successes from failures, unlike embedding similarity methods that yield false positives.The DDPM-loss and temporal-reconstruction baselines also perform well in this domain, possibly because action multimodality is lower than in PushT.Source: §6 · p. 7

Sentinel: Combining STAC and VLMs for system-level performance increases

  • 93% TPR with a 7% increase in FPR: Sentinel combines STAC and VLMs to detect task-progression failures in Cover Object and Close Box.STAC alone has 44% TPR for temporally consistent failures; the increase in detection time indicates that both the fast STAC and slower VLM contribute.Source: §6 · p. 8, Figure 6 · p. 8

Sentinel detects real-world, generative policy failures

  • 95% detection accuracy: Sentinel detects real-world failures in Push Chair across 10 successful and 10 failed rollouts.With calibration on 10 successful in-distribution rollouts, STAC detects 80% of failures at 90% TNR; the VLM reaches 90% TPR and 100% TNR.Source: §6 · p. 8

Discussion

  • Combining STAC and VLM monitoring provides a detector for both erratic and task-progression failures, while the baselines may have low accuracy or low TNR.The discussion notes that this makes it unclear how to combine the baselines with other detectors to outperform Sentinel.Source: §6 · p. 8

7 Conclusion

Sentinel reports 95% overall accuracy on the real-world Push Chair task and concludes that complementary detectors target distinct generative-policy failure categories. The authors note that the categories are not exhaustive and the approach does not provide formal guarantees or predict failures before they occur.

7 Conclusion

  • 95% overall accuracy on the real-world Push Chair task is reported for Sentinel.The result is described as strong real-world failure-detection performance.Source: Table 2 · p. 8
  • Sentinel combines statistical action-distribution change monitoring for erratic failures with VLM monitoring of task progression failures.The authors highlight targeting complementary failure categories with specialized detectors.Source: §7 · p. 8
  • The two failure categories may not be exhaustive; Sentinel offers no formal detection guarantees and focuses on detecting failures as they occur, not predicting them beforehand.Combining detector predictions may increase false alarms in the worst case.Source: §7 · p. 8

Appendix Overview: Unpacking Failure Modes of Generative Policies

  • The appendix provides implementation details, experiment descriptions, extended results and analysis, and supporting derivations.It also notes that qualitative results and a video abstract are available online.Source: Appendix Overview: Unpacking Failure… · p. 17

A Method Details: Sentinel

This section details Sentinel’s temporal action-consistency detector and VLM monitor, along with their implementations and evaluation settings in simulation and the real world. It also specifies the diffusion policies and failure-detection baselines, including their monitoring procedures and tradeoffs.

A Method Details: Sentinel

  • Sentinel runs STAC and a VLM monitor in parallel, assigning the former to erratic policy behavior and the latter to failures to progress on the task.STAC uses statistical temporal action-consistency measures; the VLM monitors task progress.Source: A Method Details: Sentinel · p. 18

A.1 Temporal Consistency Detection with STAC

  • STAC estimates changes between overlapping action-sequence distributions with empirical distances, including MMD and forward or reverse KL via KDE.Its batch size and kernel bandwidths are environment-specific; larger batches may be needed for highly multimodal domains, and filtering noisy or discrete actions or shortening horizons can improve scores.Source: §A.1 · pp. 18–19

A.2 Runtime Monitoring with Vision Language Models

  • The VLM receives a prompt and robot-camera image history, and can be queried less often than STAC because task-progression failures unfold over longer timescales and VLM calls may be costly.In experiments, the VLM was queried twice per episode; non-blocking calls also allow higher-frequency operation when needed.Source: §A.2 · pp. 19–20

Prompt Template (Video QA)

  • The Video QA prompt requests task-relevant questions, detailed answers about robot and object motion, progress analysis, and an ok-or-failure classification informed by elapsed time and the task deadline.The elapsed-time reference helps distinguish partial progress from stalling or progress too slow to finish within the time limit.Source: §A.2 · p. 20, Prompt Template (Video QA) · p. 20

A.2.1 Prompt Ensembling

  • Prompt ensembling uses majority voting across Video QA prompts to improve reliability when individual predictions are imperfect, and was used only in the Cover Object domain.That domain used three prompts: the base prompt plus variants with a successful rollout video or goal images; the variants use success data to help detect unknown failures.Source: §A.2.1 · pp. 21–22

Prompt Template (Video QA + Success Video; Video QA + Goal Images)

B.1.1 Simulation Domains

  • The simulation domains cover distinct failure settings: PushT tests multimodal actions and local-optimum failures, Close Box tests erratic failures in high-dimensional bimanual control, and Cover Object tests task-progression failures.Close Box uses a 14-DoF action space; Cover Object requires longer-timescale reasoning about premature release of the cover.Source: §B.1.1 · p. 23

B.1.2 Real-World Domains

  • In the real-world Push Chair domain, a mobile manipulator must tuck a chair under a table; randomized initial chair poses can lead to erratic rotation failures that are visually apparent.The task evaluates both STAC and the VLM in a dynamically complex real-world setting.Source: §B.1.2 · pp. 23–24

B.2 Diffusion Policies

  • Simulation policies use point-cloud inputs and a PointNet++ encoder, while real-world policies use EquiBot; policies are trained with 200 PushT, 50 Close Box and Cover Object, and 15 Push Chair demonstrations.Point clouds are used to make object position and scale shifts more salient; all policies use 100 denoising iterations.Source: §B.2 · p. 24

B.3 Baselines

  • Most evaluated detectors accumulate per-timestep scores and compare the running total with a calibrated threshold; the baselines vary in how they calculate each score.The baseline suite includes score functions introduced for generative diffusion policies and methods from OOD-detection literature.Source: §B.3 · pp. 24–25

B.3.1 STAC Baselines (Policy-Level Monitors)

  • Policy Encoder Embedding scores point-cloud dissimilarity with a Mahalanobis distance and leave-trajectory-out calibration; CLIP and ResNet baselines instead score image-embedding dissimilarity.The CLIP baseline uses an open-source, unfine-tuned CLIP model, and the ResNet baseline uses pretrained ResNet18 embeddings.Source: §B.3.1 · p. 25
  • Temporal Non-Distributional Minimum selects the closest sampled overlapping action sequence, while Diffusion Reconstruction scores re-noised sampled actions and requires repeated denoising.Diffusion Reconstruction evaluates reconstruction depths of 5, 10, 25, and 50 forward steps, with cost growing approximately with the number of reconstructions.Source: §B.3.1 · pp. 25–26
  • Temporal Diffusion Reconstruction and Temporal DDPM Loss condition re-noised action-sequence scores on the previous state, testing temporal consistency across policy-inference timesteps.Temporal Diffusion Reconstruction combines executed actions from timestep t with predicted actions from timestep t+k; Temporal DDPM Loss follows DDPM Loss hyperparameters.Source: §B.3.1 · p. 26
  • DDPM Loss estimates the diffusion policy’s training-task loss from 256 sampled action sequences and 10 sampled denoising iterations, without reconstructing actions over multiple denoising steps.Diffusion Output Variance is another alternative, thresholding the variance across sampled actions against calibration-set sample variances; it does not measure epistemic uncertainty.Source: §B.3.1 · pp. 26–27
  • Embedding baselines detect atypical states without accounting for policy behavior, reconstruction baselines are computationally expensive, and DDPM loss is cheap but diffusion-specific, unlike policy-formulation-agnostic STAC.The discussion characterizes DDPM loss as the next-best alternative to STAC among these baselines.Source: §B.3.1 · p. 27

B.3.2 VLM Baselines (Task-Level Monitors)

The VLM baselines compare video-based task-progress monitoring with image-only monitoring. Their prompt assesses progress against the task and time limit, while policy failure is defined by failure to meet the return threshold within the MDP horizon.

B.3.2 VLM Baselines (Task-Level Monitors)

  • Video QA analyzes the execution video up to the current timestep, while Image QA uses only the current image to assess task progress.The comparison evaluates video-based reasoning against single-image assessment.Source: §B.3.2 · p. 27

Prompt Template (Image QA)

  • The Image QA prompt asks for task-relevant questions, visual answers, elapsed-time assessment, and an overall ok-or-failure judgment.It discourages failure classification unless actions are unsafe or the task is unlikely to finish within the time limit.Source: Prompt Template (Image QA) · pp. 27–28

B.4.1 Definition: Policy Failure

  • Policy failure means the trajectory return does not exceed the success threshold by the MDP horizon.The online detector evaluates partial trajectories at each timestep to flag failure episodes as early as possible, without labeling the specific failure timestep.Source: §B.4.1 · p. 28

B.4.2 Constructing the Calibration Dataset

The calibration dataset grounds detectors in successful, nominal policy behavior. Its quality matters: unacceptable successes can normalize undesirable behavior, and calibrating directly on demonstrations increased false positive rates.

B.4.2 Constructing the Calibration Dataset

  • Calibration uses successful policy rollouts to ground test-time behavior in the policy’s nominal, in-distribution behavior.The dataset is intended to evaluate potentially failing test-time behavior against known nominal behavior.Source: §B.4.2 · p. 28
  • Calibration trajectories should exclude successful but unacceptable behavior, which could make detectors treat damaging behavior as nominal.For example, a rollout that closes the box while damaging its lids should not be included.Source: §B.4.2 · pp. 28–29
  • The authors collected 50 successful policy rollouts per simulation domain, manually filtering unacceptable behavior.They hypothesize that the needed dataset size varies by task; larger datasets may strengthen conformal guarantees on detector FPR.Source: §B.4.2 · p. 29
  • Calibrating STAC variants directly on demonstrations significantly increased detector FPR, which the authors attribute to imitation-learning covariate shift.They explain that prediction error increases quadratically on policy-induced states, causing successful test-time rollouts to be mislabeled as failures.Source: §B.4.2 · p. 29

B.4.3 Testing & Evaluation

The evaluation labels each rollout as success or failure using its final-state return, then applies detectors at every timestep to find the first warning. It reports classification rates, detection time, and accuracy measures, using balanced accuracy for imbalanced test sets.

B.4.3 Testing & Evaluation

  • Rollouts are labeled success or failure by thresholding the episode’s final-state return, and detectors are evaluated at each timestep to identify the first warning.This offline evaluation uses test datasets containing both successful and failed policy rollouts.Source: §B.4.3 · p. 29

B.4.4 Reported Metrics

  • A true positive is a warning anywhere in a failed trajectory, while a true negative is no warning anywhere in a successful trajectory.Detection time is the earliest warning timestep in a failed trajectory; false-positive and false-negative definitions follow accordingly.Source: §B.4.4 · p. 29
  • Reported metrics are TPR, TNR, FPR, detection time, accuracy, and balanced accuracy; balanced accuracy is used when positive and negative trajectories are notably imbalanced.TPR and TNR measure detected failures and successes, respectively, while FPR measures false alarms among successes.Source: §B.4.4 · p. 29

Does STAC’s performance depend on the policy’s prediction and execution horizon?

STAC’s performance depends more on the policy’s execution horizon than its prediction horizon, and statistical distance choices outperform a non-statistical alternative. Its standard horizon settings perform best in the reported comparisons.

Does STAC’s performance depend on the policy’s prediction and execution horizon?

  • STAC is relatively stable across prediction horizons in PushT, Close Box, and Push Chair, with the best result at the standard h = 16 setting.The authors suggest calibrating and deploying STAC with the same prediction horizon may normalize that parameter’s influence at test time.Source: Does STAC’s performance depend on the… · p. 30, Table 4 · p. 30
  • 61% TPR at k = 2, 78% at k = 4, and 95% at k = 8; STAC’s TNR stays above 95% across execution horizons.The standard setting k = 8 and h = 16 performs best; the authors attribute the k = 2 drop to too little environment change, making the compared action distributions similar and distances overly conservative.Source: Figure 8 · p. 30, Does STAC’s performance depend on the… · p. 30

How does STAC’s performance vary with the choice of statistical distance function?

  • STAC performs comparably with MMD using RBF kernels and KL-divergence via kernel density estimation, while all variants exceed 90% detection accuracy in Close Box.Using the non-statistical “Temporal Non-Distr. Min” instead causes a large performance drop, attributed to greater sensitivity to stochastic action-sampling variation and more false alarms.Source: How does STAC’s performance vary with… · p. 30

C.2 Extended Results: VLM Runtime Monitor

The extended results show that STAC and VLM monitoring cover different failure patterns, with performance varying by task and VLM. Sentinel combines their predictions by union, a choice motivated by detector specialization and differing runtime constraints.

C.2 Extended Results: VLM Runtime Monitor

  • In Close Box, STAC detects 99% of erratic failures and Sentinel detects 100%, while combining detectors raises false alarms to 13%.Video QA increases TNR across models; GPT-4o is more reliable than the other tested VLMs in this domain.Source: Table 5 · p. 31, §C.2 · p. 31
  • Image QA yields at most 7% TNR because a single image omits the initial state and robot actions needed to identify whether the box lids were closed.Video QA strictly increases TNR across tested models, though some VLMs confuse lid states or miss progress; the authors suggest recency bias and training-data representation as possible explanations.Source: §C.2 · pp. 31–32
  • 88% of Cover Object failures are detected with 11% false alarms by Sentinel using prompt-ensembled Claude 3.5 Sonnet, while STAC alone detects only 12% of task-progression failures.Claude 3.5 Sonnet Video QA achieves 79% TPR and 84% TNR before prompt ensembling raises TNR to 93%.Source: Table 6 · p. 32, §C.2 · p. 32

Discussion: Why combine failure detectors by taking the union of their predictions?

  • Sentinel uses the union of STAC and VLM warnings because specialized detectors address heterogeneous failures, while equal weighting avoids calibration data that may not represent deployment failures.The union can compound false-positive rates, but the authors argue low individual FPRs can keep the combined rate acceptably low and make contributions interpretable.Source: Discussion: Why combine failure… · pp. 32–33

C.2 Extended Results: VLM Runtime Monitor

  • 96% of Close Box task-progression failures are detected with 14% false alarms by Sentinel; STAC alone leaves 35% of failures undetected.Video-based reasoning is needed for high VLM TNR, and GPT-4o performs best in this domain.Source: §C.2 · p. 32, Table 7 · p. 33

Discussion: Why combine failure detectors by taking the union of their predictions?

  • Because STAC is fast and VLM monitoring is slow, warnings are combined only at overlapping timesteps, allowing detectors to run at different rates without synchronization.More sophisticated combination schemes may improve detection or scalability, but could be vulnerable to delays such as cloud-VLM network latency.Source: Discussion: Why combine failure… · p. 33

D Derivations

The derivation applies split conformal prediction to STAC to bound its false positive rate without distributional assumptions on trajectory rollouts. It calibrates on full successful trajectories to address sequentially correlated state-action pairs, while making no guarantee of failure detection or combined STAC–VLM performance.

D Derivations

  • Conformal prediction constructs a prediction set with user-defined coverage from i.i.d. calibration samples and a conformity score, without assumptions about the data-generating distribution.The validity claim holds regardless of the conformity score choice.Source: D Derivations · p. 34
  • Calibrating STAC on consistency scores from full trajectories permits an FPR bound despite sequentially correlated state-action pairs.The threshold γ is the empirical ⌈(M+1)(1−δ)⌉/M quantile of terminal STAC scores from successful validation trajectories.Source: D Derivations · p. 34
  • STAC’s false positive rate on a successful test trajectory of length H′ ≤ H is at most δ.The guarantee concerns false alarms on successful trajectories.Source: D Derivations · pp. 33–34
  • The proof uses the nonnegativity of the statistical distance to show STAC’s temporal score is increasing, so crossing γ implies the terminal score exceeds γ.The argument applies this monotonicity to establish the stated FPR bound.Source: D Derivations · p. 34
  • The analysis bounds only STAC’s FPR, not its ability to detect failures or the combined performance of STAC and the black-box VLM.The text says OOD-success false alarms cannot be ruled out without assumptions about OOD scenarios or failure data for calibration.Source: D Derivations · pp. 34–35
  • The conformal guarantee is marginal over calibration datasets, so it need not hold exactly for a particular dataset; limited random-seed repetitions leave empirical deviations from Eq. 5.Compute budgets restricted the experiments to a limited number of random seeds.Source: D Derivations · p. 35
Loading 2410.04640v2…