PaperlayerView exact revision on arXiv

Source-linked AI summary

Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies

Chen Xu, Tony Khuong Nguyen, Emma Dixon, Christopher Rodriguez, Patrick Miller, Robert Lee, Paarth Shah, Rares Ambrus, Haruki Nishimura, Masha Itkina

arXiv:2503.08558v3cs.ROcs.AIcs.LG

TL;DR

Generative imitation-learning policies can fail under stochastic sampling errors and OOD observations, while existing detectors often require failure data. FAIL-Detect learns scalar signals from successful demonstrations and applies time-varying conformal prediction thresholds; across simulation and hardware tasks, it detects failures accurately and quickly, with average best balanced accuracy of ∼78% in simulation and ∼72% on hardware. The authors identify limitations involving visual-feature use, false positives, temporal context, and additional sensing modalities.

  • Problem

    Generative imitation-learning policies may produce unreliable or dangerous actions under sampling errors or OOD observations, while many failure detectors require costly failure data.

  • Method

    FAIL-Detect learns scalar signals from successful trajectories and uses conformal prediction to construct time-varying thresholds for sequential OOD-based failure detection.

  • Results

    ∼78% average best balanced accuracy in simulation and ∼72% on robot hardware tasks, with accurate and fast detection across diverse scenarios.

  • Takeaways & Limitations

    Unseen robotic-policy failures can be detected without access to failure data, supporting the potential for safer and more reliable deployment.

  • Takeaways & Limitations

    Learned scores may favor simple robot-state cues over visual features, and false positives remain, especially in OOD settings; temporal history and multimodal sensing are not modeled.

Abstract

from arXiv · show

Recent years have witnessed impressive robotic manipulation systems driven by advances in imitation learning and generative modeling, such as diffusion- and flow-based approaches. As robot policy performance increases, so does the complexity and time horizon of achievable tasks, inducing unexpected and diverse failure modes that are difficult to predict a priori. To enable trustworthy policy deployment in safety-critical human environments, reliable runtime failure detection becomes important during policy inference. However, most existing failure detection approaches rely on prior knowledge of failure modes and require failure data during training, which imposes a significant challenge in practicality and scalability. In response to these limitations, we present FAIL-Detect, a modular two-stage approach for failure detection in imitation learning-based robotic manipulation. To accurately identify failures from successful training data alone, we frame the problem as sequential out-of-distribution (OOD) detection. We first distill policy inputs and outputs into scalar signals that correlate with policy failures and capture epistemic uncertainty. FAIL-Detect then employs conformal prediction (CP) as a versatile framework for uncertainty quantification with statistical guarantees. Empirically, we thoroughly investigate both learned and post-hoc scalar signal candidates on diverse robotic manipulation tasks. Our experiments show learned signals to be mostly consistently effective, particularly when using our novel flow-based density estimator. Furthermore, our method detects failures more accurately and faster than state-of-the-art (SOTA) failure detection baselines. These results highlight the potential of FAIL-Detect to enhance the safety and reliability of imitation learning-based robotic systems as they progress toward real-world deployment.

I. INTRODUCTION

FAIL-Detect addresses runtime failures in generative imitation-learning policies by detecting sequential OOD behavior using successful demonstrations alone. It combines scalar signals with conformal prediction to support faster, statistically controlled detection across diverse tasks.

  • Generative imitation-learning policies can fail because of poor stochastic action sampling or OOD observations, producing unreliable or dangerous actions.
  • Failure detection is difficult because observations are high-dimensional, reward functions are often absent, and manipulation tasks permit diverse failure modes.
  • Existing OOD-classification approaches often require costly, comprehensive failure data that may not generalize to unseen failures.
  • FAIL-Detect learns scalar signals from policy inputs or outputs using successful trajectories only, then applies conformal prediction to construct time-varying thresholds.
  • The framework evaluates diffusion and flow-matching imitation-learning architectures and investigates learned and post-hoc score candidates.
  • FAIL-Detect is designed to handle diverse test-time failures, including slipping or misplaced objects, while enabling faster inference than methods requiring multiple action samples.

III. PROBLEM FORMULATION

The problem formulation treats policy failure detection as a sequential decision over action-observation trajectories. FAIL-Detect constructs scalar scores from successful data and flags a rollout when its score exceeds a calibrated threshold.

  • III. PROBLEM FORMULATION: The generator produces H-step action sequences from observations, executes H′ actions, and replans repeatedly during rollout.
  • III. PROBLEM FORMULATION: Failure detection is a decision function over the current trajectory that flags a rollout when the policy fails to complete its task.
  • IV. FAILURE DETECTION FRAMEWORK: Stage 1 trains a scalar score model on action-observation pairs from successful trajectories only, while Stage 2 calibrates time-varying thresholds with conformal prediction.
  • IV. FAILURE DETECTION FRAMEWORK: The framework supports new score or threshold designs, and evaluates multiple scoring methods summarized in Table I.
  • A. Design of Scalar Scores: Scores are intended to be one-class, lightweight, and discriminative, using only the latest T_O=2 observation steps alongside future actions.
  • A. Design of Scalar Scores: Flow-based density scores assess whether observations are unlikely under successful data, with logpZO operating in latent noise space rather than the original observation space.
  • A. Design of Scalar Scores: Second-order candidates model uncertainty through probabilistic priors, including NatPN for discretized observations and DER for multivariate action regression.
  • A. Design of Scalar Scores: One-class discriminator candidates include RND’s prediction error and CFM’s trajectory-curvature consistency, while post-hoc candidates include SPARC and STAC.

B. Sequential Threshold Design with Conformal Prediction

FAIL-Detect uses functional conformal prediction to convert scalar score sequences into one-sided, time-varying thresholds. Scores above the calibrated upper band signal OOD behavior and potential failure.

  • Functional conformal prediction wraps scalar score time series and produces a distribution-free prediction band with significance level α.
  • Hardware scenarios test disturbance and OOD conditions, including crumpled towels, unseen distractors, and a novel towel color that typically produces low success rates.
  • The one-sided band focuses on high scores, which indicate OOD behavior, and is calibrated from scores collected on N successful rollouts.
  • The upper threshold is η_t=upper_t=μ_t+h_t, where the time-varying mean and band width define the conformal upper bound.
  • For a new successful rollout, the score remains within the band for the entire duration with probability at least 1−α under stated conditions.
  • The decision rule 1(D_M(A_t,O_t;θ)>η_t) controls the false-positive rate on successful trajectories at level α.

V. EXPERIMENTS

FAIL-Detect is evaluated across simulation and robot hardware tasks using diverse environments, baselines, scalar scores, and accuracy and speed metrics.

  • Tasks: Experiments span simulation benchmarks and two challenging bimanual robot tasks, including ID and OOD settings.Simulation uses Square, Transport, Can, and Toolhang; hardware uses FoldRedTowel and CleanUpSpill.
  • CP Calibration: Calibration uses successful rollouts, with simulation relying on ID-only bands and hardware comparing ID-only against setting-dependent bands.Hardware experiments use substantially fewer rollouts than simulation, creating a small-sample calibration regime.
  • Baselines: The evaluation compares learned and post-hoc scalar scores against STAC and PCA-kmeans, while omitting STAC from hardware because of real-time latency.PCA-kmeans uses K = 64 centroids; STAC generates batches of 256 action predictions per time step.
  • Evaluation Protocol: Failure detection is assessed with true-positive rate, true-negative rate, balanced accuracy, weighted accuracy, and detection time.Balanced accuracy equally represents classes, while weighted accuracy reflects the rollout distribution.

VI. RESULTS

The experiments test whether failure detection can work without enumerating failure scenarios and whether learned scores improve accuracy, speed, and qualitative separation. FAIL-Detect achieves strong accuracy across simulation and hardware while maintaining practical detection times.

  • Research Questions: The experiments ask whether failure detection is performant without failure data, how learned and post-hoc scores compare, and whether detections align with human intuition.Balanced accuracy, weighted accuracy, and detection time measure reliable and rapid identification.
  • A. How performant is failure detection without failure data?: FAIL-Detect’s average best balanced accuracy is approximately 78% in simulation and approximately 72% on robot hardware.These results are reported across the framework’s score candidates.
  • C. Do failure detections align with human intuition?: The learned metrics show tighter conformal prediction bands and greater failure/success separation than STAC and PCA-kmeans on Square ID rollouts.The qualitative comparison covers 150 test rollouts.
  • A. How performant is failure detection without failure data?: Average best detection time is faster than successful trajectory completion across the reported score designs.This supports practical early detection rather than detection only after a successful trajectory would have ended.

B. What is the impact of learned vs. post-hoc scores on failure detection?

Learned scalar scores generally outperform post-hoc scores in failure detection, with logpZO showing the most consistent performance across experiments. Learned methods also offer faster computation than STAC, while some post-hoc methods trade accuracy or reliability for speed.

  • Learned scores outperform post-hoc scores across simulation and hardware tasks, with logpZO achieving top-1 performance in 10/16 simulation cases.RND achieved top-1 performance in 5/16 simulation cases, while STAC led among post-hoc methods in 3/16 cases and PCA-kmeans was never best.
  • logpZO detected failures fastest in 3/8 cases, compared with 1/8 for PCA-kmeans, while STAC consistently exceeded practical detection-time limits.STAC's detection time surpassed the average success trajectory time.
  • logpZO remained the strongest hardware method, reaching top-1 highest balanced accuracy and weighted accuracy in 8/12 scenarios.PCA-kmeans ranked first in 4/12 scenarios, while RND never achieved top-1 performance but always ranked among the top three.
  • Across all experiments, logpZO was the most consistent score, whereas post-hoc methods varied between strong and poor performance depending on the setting.The reported qualitative trends also show better score separation between successful and failed trajectories for learned methods.
  • On an A6000 GPU with 50 rollouts, logpZO required 0.04 s for Square and 0.033 s for Transport per time step, versus 1.45 s for STAC.This corresponds to a 36-44 times slowdown for STAC; reducing its 256 action predictions could compromise statistical reliability.

C. Do failure detections align with human intuition?

FAIL-Detect’s score alerts generally align with observable physical failure events in simulation and on-robot tasks, while threshold calibration remains environment-sensitive. The framework also supports targeted post hoc analysis of failures, but false positives and representation limitations remain.

  • C. Do failure detections align with human intuition?: Score spikes often coincide with observable physical failures, including loss of the square, hammer slips, and unsuccessful towel folds.The correspondence appears in both simulation and real-world manipulation settings.
  • C. Do failure detections align with human intuition?: The method detects human-induced disruptions and OOD initial conditions that produce incomplete or improper towel folds on robot hardware.These include an incomplete second fold and an improper first fold.
  • C. Do failure detections align with human intuition?: Temporal windows around detected failures can support efficient identification of failure types for subsequent analysis and policy improvement.This reduces reliance on exhaustively enumerating failure modes in advance.
  • C. Do failure detections align with human intuition?: Environment-dependent threshold bands improve robot-hardware performance, whereas ID-only calibration can become overly conservative under OOD initial conditions.OOD successful trajectories may have slower, jitterier, or otherwise degraded behavior and higher scores than ID successes.
  • C. Do failure detections align with human intuition?: Learned scores may overuse simple robot-state information instead of higher-dimensional visual features, while false positives remain especially relevant in OOD settings.The authors suggest richer visual representations could improve detection performance.
  • C. Do failure detections align with human intuition?: The score candidates do not model long temporal sequences, and multimodal sensing could further improve timely failure detection.Suggested modalities include sound and tactile information.

APPENDIX

The logpZO score is constructed by fitting a flow-matching model between observations and Gaussian latent noise, then using predicted latent-noise density as an anomaly signal. In-distribution observations should map near standard-normal noise, whereas anomalous observations receive lower likelihood scores.

  • APPENDIX: Step 1 fits a flow-matching model between image-embedding/proprioception observations and latent noise sampled from N(0, I).The model defines a continuous interpolation parameterized by s ∈ [0, 1].
  • APPENDIX: Step 2 applies one-step prediction to a new observation Ot′ to estimate its corresponding latent noise.The estimated latent is denoted ZOt′ in the surrounding description.
  • APPENDIX: For in-distribution observations, the estimated latent noise should be close to samples from N(0, I).This provides the reference distribution for evaluating observation atypicality.
  • APPENDIX: Step 3 computes latent-noise density up to a constant using the squared norm of the estimated latent.The resulting quantity is used to form the logpZO score.

B. CP band construction

The CP band is calibrated from successful trajectories by modeling time-varying deviations from their mean and using quantiles to set a trajectory-wide upper threshold.

  • B. CP band construction: Calibration splits successful scores into DcalA and DcalB to estimate the mean trajectory, modulation function, and band width separately.The mean and modulation are computed on DcalA, while DcalB supplies maximum deviations for the final quantile.
  • B. CP band construction: The modulation function scales the band over time using either the average or maximum absolute deviation from the successful mean trajectory.It adapts band width to temporal variation in successful rollouts.
  • B. CP band construction: Maximum trajectory deviations form the calibration set S, whose (1 − α)-quantile defines the band width h.Taking the maximum ensures the CP band reflects the entire rollout trajectory.
  • B. CP band construction: The upper CP bound is upper_t = μ_t + h scalA(t), with α = 0.05 used throughout the experiments.This corresponds to a 95% confidence interval under the chosen significance level.

C. Experimental Details

Experiments evaluate failure detection across simulated Robomimic manipulation tasks and two long-horizon robot hardware tasks involving precision, coordination, and deformable objects.

  • C. Experimental Details: Simulation includes Square, Transport, Can, and Toolhang tasks with increasing demands for precision, coordination, dexterity, or assembly.Transport requires coordinated two-arm transfer, while Toolhang requires the greatest dexterity and precision among the four tasks.
  • C. Experimental Details: Test data are also used to evaluate failure detection methods, with selected hardware rollouts shuffled to include all failures when failures are scarce.The protocol marks cells with an asterisk when five or fewer failures occur among test rollouts.
  • C. Experimental Details: FoldRedTowel requires two arms to fold and push a towel, whereas CleanUpSpill combines cup lifting with towel-based spill removal.Both hardware tasks are long-horizon and require precise coordination with deformable objects.

2) Policy backbone and the calibration of CP bands:

The evaluation uses flow-matching and diffusion policy backbones, calibrates CP bands from successful rollouts, and compares learned and post-hoc scalar signals.

  • 2) Policy backbone and the calibration of CP bands:: Table II reports success rates across task-environment combinations, while Table III contains policy and CP-band hyperparameters.The evaluation protocol distinguishes policy-network settings from simulation and hardware CP calibration settings.
  • 2) Policy backbone and the calibration of CP bands:: Policy training uses flow matching and/or diffusion models, with a ResNet image encoder and task-specific training configurations.The policy backbone g is trained using the stated optimizer, learning-rate, scheduler, and epoch choices.
  • 2) Policy backbone and the calibration of CP bands:: Scalar signals are learned from policy inputs and outputs or computed post hoc, including density, random-network-distillation, curvature, smoothness, and PCA-kmeans scores.SPARC uses Fourier-spectrum arc length, while learned density scores use flow networks and instantaneous change of variables.
  • 2) Policy backbone and the calibration of CP bands:: On real FoldRedTowel trajectories, failed-rollout score spikes are more evident for learned logpZO and RND than for post-hoc PCA-kmeans and SPARC.The figure presents qualitative detection scores together with CP bands.
  • 2) Policy backbone and the calibration of CP bands:: The evaluation reports TPR and TNR against CP significance level in both simulation and hardware experiments.This comparison examines how detection and non-detection rates vary with the CP thresholding level.

D. Ablation

Ablations examine CP significance levels, calibration settings, and quantitative performance across simulation and hardware tasks, with logpZO repeatedly identified as a strong method.

  • D. Ablation: Higher α increases TPR and decreases TNR because more rollouts are flagged as failures; this trend is clearer in simulation than hardware.Hardware effects are muted by limited rollout counts and constant calibration quantiles for small α.
  • D. Ablation: The evaluation protocol specifies hardware and simulation training resources and hyperparameters in Table III.These include policy-network settings and CP-band calibration parameters.
  • D. Ablation: Simulation results identify logpZO and RND as the top two overall methods for FM policies.The figure reports best, second-best, and third-best quantitative results across simulation tasks.
  • D. Ablation: For DP policies, learned methods appear to have more capacity to detect failures than post-hoc methods, with logpZO and RND performing best.The comparison uses the same quantitative-results layout as the corresponding simulation analysis.
  • D. Ablation: logpZO remains the most robust method overall on both FoldRedTowel and CleanUpSpill hardware tasks.The hardware ablations compare two ways of computing the CP band.
Loading 2503.08558v3…