PaperlayerView exact revision on arXiv

Source-linked AI summary

Coverage Aware Active Evaluation for Failure Discovery with Paired Systems

Anjali Parashar, Rachel Luo, Apoorva Sharma, Sushant Veer, Edward Schmerling, Carson Sobolewski, Mingxin Yu, Chuchu Fan, Marco Pavone

arXiv:2608.13719v1cs.AIcs.RO

TL;DR

Discovering rare, diverse failures in autonomous systems is difficult when direct target-system testing is expensive and proxy failures may not transfer. This paper adaptively combines proxy guidance, limited target evaluations, local risk correction, and support-aware diversity selection, discovering up to twice as many target failures as the strongest baseline across multiple tasks.

  • Problem

    Direct testing is costly, while existing proxy-target methods typically estimate aggregate metrics rather than actively discovering diverse, scenario-specific target failures.

  • Method

    The method corrects biased proxy failure signals with a local control-variate target-risk predictor and combines it with support-aware mutual information to select likely, diverse scenarios.

  • Results

    Up to twice as many target-system failures are discovered as by the strongest baseline under the same evaluation budget across autonomous driving, manipulation, and quadruped tasks.

  • Takeaways & Limitations

    Cheap proxy evaluations improve failure discovery when used to guide adaptive target-system testing rather than replace target evaluation.

  • Takeaways & Limitations

    Performance depends on scenario representations capturing behaviorally relevant differences, and larger or more complex spaces may require larger target-evaluation budgets.

Abstract

from arXiv · show

Autonomous systems can fail in rare and heterogeneous ways, making real-world failure discovery difficult under limited testing budgets. Although cheaper proxies such as simulators, lower-fidelity systems, or related policies can be sampled extensively to find failures, proxy failures often do not transfer to the real world due to sim-to-real and system-to-system gaps. The key challenge is therefore to effectively leverage proxy system information for accurate prediction of severe target system failures. We propose an adaptive failure discovery method that combines proxy evaluations with limited target system results to guide scenario selection for target system testing. Our method learns a local predictor of target risk by correcting proxy failure signals using control-variate-inspired residual modeling. To find failures that are both likely and diverse, we combine this predictor with a support-aware mutual-information objective that favors realistic, well-supported regions while expanding coverage across failure modes. Across autonomous driving, manipulation, and quadruped velocity-tracking tasks, our method discovers up to 2$\times$ as many failures as random sampling and active-learning baselines, including severe and diverse failures missed by competing methods.

1 Introduction

Rare, severe, and heterogeneous failures are difficult to discover through direct target-system evaluation, while proxy failures may not transfer reliably. The proposed adaptive method uses proxy evaluations and limited target tests to discover diverse target failures under a fixed budget, achieving up to twice as many failures as the strongest baseline.

  • Motivation: Rare, severe, and heterogeneous failures arising in specific scenarios are important for evaluating autonomous systems, especially in safety-critical domains.A small number of unanticipated edge cases can cause unsafe behavior after deployment.
  • Limitations of Existing Approaches: Existing methods either search inexpensive simulations without reliable target transfer or directly test targets without exploiting abundant proxy evaluations.Proxy-target methods often estimate aggregate metrics from Monte Carlo data rather than scenario-specific failures needed for failure discovery.
  • Approach: The method formulates proxy-guided failure discovery as adaptive scenario selection using limited target evaluations to find many distinct target failure modes.Diversity covers distinct regions of scenario space, operating conditions, or failure modes.
  • Approach: A local, proxy-corrected failure predictor uses paired proxy-target evaluations to identify scenarios likely to fail on the target system rather than merely in the proxy evaluator.Proxy evaluations are treated as biased but informative signals.
  • Results: Up to twice as many target-system failures are discovered as by the strongest baseline under the same target-evaluation budget, with highest diversity across tasks and baselines.The method finds low- and high-severity failures, including high-severity failures completely missed by baselines, using a pairwise-distance-based coverage metric.

2 Related Work

Prior work spans scalable simulation-based testing, sample-efficient target-system testing, proxy-assisted estimation, and information-based acquisition. This work combines these directions to use proxy evaluations and limited target tests for diverse target-system failure discovery without assuming direct proxy-to-target transfer.

  • Simulation-based failure discovery: Simulation enables scalable, low-cost failure discovery, but sim-to-real gaps and model misspecification can make simulated safety unreliable on real systems.The method treats proxy evaluations as biased but informative signals and uses limited target evaluations to adaptively correct proxy-to-target differences.
  • Sample-efficient target-system testing: Target-system testing uses expert-designed scenarios, surrogate-based Bayesian optimization, or active learning under limited budgets, but surrogate methods can struggle with high-dimensional scenarios and complex failure landscapes.The proposed method does not depend on restrictive model choices while modeling target-system risk.
  • Combining proxy and target evaluations: Control variates and related variance-reduction methods use correlated proxy signals to improve aggregate real-world metric estimation, whereas this work targets rare-failure discovery through active scenario selection.Prior methods are typically designed for Monte Carlo estimation of global quantities rather than revealing many target-system failures.
  • Bayesian experimental design and information-based acquisition: Mutual-information-based experimental design selects informative evaluations, but uncertainty alone may target non-critical scenarios and exploitation alone may produce duplicate failures.This work combines support-aware mutual information for under-covered regions with a proxy-corrected risk estimate.
  • Positioning of this work: The work combines cheap proxy evaluations, limited target-system tests, and information-based acquisition to reveal diverse failure modes without assuming proxy failures transfer directly.It differs from target-only active testing by using proxies to guide search and from variance-reduced estimation by selecting evaluations instead of estimating an aggregate metric.

3 Problem Statement

The problem is to discover diverse, severe failures of an expensive black-box target system using cheaper paired proxy observations under a limited target evaluation budget. The adaptive acquisition balances failure likelihood against diversity through mutual information and Bayesian experimental design.

  • Problem formulation: The target is a black-box dynamical system mapping scenario parameters to trajectory rollouts, with failure measured by a safety metric where larger values indicate less safe behavior.A cheaper proxy system provides low-cost observations for the same scenarios, while stochasticity or environmental variation induces observation distributions.
  • Problem formulation: The objective is to identify a diverse set of scenarios whose expected target safety metric meets or exceeds the user-specified severity threshold γ under target evaluation budget B.Diversity spans scenarios or operating conditions, and failure is defined by µr(x) ≥γ.
  • Acquisition objectives: Mutual information promotes diversity by quantifying epistemic uncertainty from limited evaluations.The method uses MI to expand coverage across operating conditions while selecting failures.
  • Acquisition objectives: Failure discovery is formulated as adaptive scenario sampling with Bayesian Experimental Design under the target evaluation budget.At iteration k ≤B, the next scenario is selected using observed target evaluations Dk.
  • Acquisition objectives: The acquisition function selects scenarios that are likely to satisfy µr(x) ≥γ while covering diverse operating conditions through high mutual information.The next scenario is chosen as xk+1 = arg maxx∈X α(x).

4 Data acquisition strategy for meeting dual evaluation objectives

The strategy combines a local control-variate predictor using limited target evaluations and biased proxy signals with a support-aware mutual-information objective for diverse scenario selection. It then applies these objectives through severity-adjusted, batched sampling in the scenario space.

  • Control-variate prediction: The method combines limited target evaluations with biased, noisy proxy signals through a local control-variate predictor µCV for accurate target-failure prediction.The predictor corrects proxy-based estimates using local target information and is intended to estimate target failure statistics with minimum variance.
  • Support-aware diversity: A support-aware MI term promotes scenario diversity by identifying candidates outside the scenario space covered by observed target data.Observed scenarios are clustered, and candidates can be assigned to an existing cluster or a new cluster indicating uncovered support.
  • Support-aware diversity: The MI objective prioritizes candidates with ambiguous cluster assignments and candidates far from existing support.The support-discovery likelihood is low inside established clusters and high for new-cluster assignments.
  • Scalability: Unlike GP-based approaches, the method is agnostic to surrogate architecture and scales to high-dimensional scenarios beyond evaluation budgets of only tens of samples.Support directly encodes exploration, including unexplored regions under different supports induced by evaluated samples.
  • Batched acquisition: For discrete scenario spaces, the objectives become a severity-adjusted sampling rule that acquires b scenarios per iteration under budget B = Tb.Candidates are selected from scenarios with µCV ≥γ, with batch size b user-defined and scenarios chosen near cluster centers.

5 Experimental validation

The experiments validate the method across autonomous driving, manipulation, and quadruped velocity-tracking tasks using severity, coverage, and failure-count metrics. Results show improved discovery of diverse failures, while ablations clarify the roles of mutual-information exploration and proxy information.

  • Experimental setup: Experiments span nuPlan, SIMPLER, Quadruped, and KITTI across autonomous driving, manipulation, and quadruped velocity tracking.nuPlan uses open-loop proxy and closed-loop target evaluations in a 384-dimensional scenario space with TTC failures.
  • Evaluation protocol: The evaluation compares Random, BAMS, BNN-C/GP-C, and BNN-CV using cumulative severity, cumulative coverage, and positive-sample failure counts.BNN-CV removes MI-driven exploration, enabling direct assessment of exploration’s contribution.
  • nuPlan results: Our approach outperforms all baselines on nuPlan for cumulative coverage and discovered failures, despite a lower cumulative average than every baseline except BNN-CV.BNN-CV repeatedly samples the same failures, reducing diversity and failure count; nuPlan failures use closed-loop TTC < 0.3.
  • Quadruped results: Our method consistently discovers more diverse Quadruped failures than BAMS and Random and identifies more failures at the γ = 0.7 threshold.The surrogate captures two symmetric failure modes over (v_y, w_z), while BAMS concentrates near the threshold and misses high-severity failures.
  • Proxy ablation: Proxy benefits depend on proxy–surrogate correlation: weak correlation on nuPlan yields small gains, whereas ρ ≈0.5 on KITTI produces clearer gains in positive samples and utility.Proxy information can reduce target prediction risk by capturing target variation hidden by X, while additional qθ samples mainly reduce variance.

6 Discussion & Conclusion

The approach scales with evaluation budget and scenario complexity, while baseline comparisons reveal limitations in surrogate scalability, exploration, and proxy-failure transfer assumptions.

  • The approach scales gracefully with evaluation budget and scenario complexity, supporting its central hypothesis.
  • At 170 evaluations in nuPlan, BAMS degrades because its GP surrogates struggle to scale.The passage identifies this as a known limitation of GPs.
  • In SIMPLER, BNN-CV’s myopic acquisition collapses onto a narrow failure region despite a sparse failure metric requiring exploration.

7 Limitations

The approach depends on scenario representations that capture behaviorally relevant differences and requires target-system evaluations. Larger or more complex scenario spaces may therefore need larger budgets to learn accurate target-risk surrogates.

  • The method assumes scenario representations in which locality, support, and diversity are meaningful and behaviorally relevant.Performance may depend on how well these representations capture behaviorally relevant differences between scenarios.
  • Larger or more complex scenario spaces may require larger target-evaluation budgets to learn accurate target-risk surrogates.Like other black-box failure discovery methods, the approach requires target-system evaluations.

A Local minimum-variance property of the control-variate estimator

The control-variate estimator is minimum-variance within the class of local affine control-variate estimators, without reducing surrogate bias. Under the stated local sampling assumptions, its variance improves whenever proxy and target statistics are locally correlated, while the justification is limited by possible surrogate-induced bias.

  • Minimum-variance result: The estimator is the minimum-variance member of the local affine control-variate class.The result follows by optimizing the control-variate coefficient within the specified estimator class.
  • Estimator assumptions: Under independent paired and proxy-only samples from the same local neighborhood distribution, the estimator is unbiased for the local target statistic.The coefficient changes variance but not expectation under these assumptions.
  • Variance behavior: When local proxy and target statistics are correlated, the control-variate estimator strictly reduces variance; with zero correlation, it matches the target-only estimator.The variance reduction is established relative to the direct local Monte Carlo estimator using only surrogate-induced target-statistic samples.
  • Application to failure prediction: In the proposed setting, the optimized coefficient yields a minimum-variance local affine estimator of the surrogate-induced target failure statistic and supports proxy-corrected scenario selection.This interpretation is conditional on the learned surrogate qθ.
  • Practical limitation: The justification may fail in practice because active-learning data can make the surrogate-induced statistic biased rather than an unbiased representation of the target statistic.The method therefore focuses on selecting suitable training hyperparameters and surrogate architectures to bridge this gap.

B Motivation for MI-adjusted data acquisition

The method combines mutual information (MI) and failure discovery as distinct objectives for data acquisition, following an active-inference-based acquisition setup.

  • B Motivation for MI-adjusted data acquisition: MI and failure discovery serve as two distinct objectives in data acquisition.This setup arises naturally from active-inference-based data acquisition proposed by [27].

C Defining locality in the feature space · D Construction of terms for MI estimation · E nuPlan ablations

The method defines locality through an adaptive neighborhood radius and estimates mutual information with a support-aware model that emphasizes novel scenarios. Synthetic examples show that adaptive locality improves target-mean prediction, while the combined approach recovers diverse failure regions; the nuPlan ablation varies initialization size over 100 samples and four seeds.

  • C Defining locality in the feature space: Adaptive radius selection most accurately predicts the ground-truth target mean compared with fixed-region alternatives in the 1D synthetic task.Figure 7 compares fixed radii with adaptive Ra(x).
  • C Defining locality in the feature space: The neighborhood Bx controls the granularity of the local control-variate predictor µCV(x), ranging from a global estimator at Rmax to point-wise prediction as R approaches zero.Bx may be defined as a Euclidean ball with radius R.
  • C Defining locality in the feature space: The proposed sensitivity-aware heuristic selects smaller neighborhoods where prediction sensitivity is high and larger neighborhoods where it is low.The radius is adaptively adjusted within [Rmin, Rmax].
  • D Construction of terms for MI estimation: The MI estimator models the distribution over response variables Rx as a mixture over observed and new support clusters.Cluster-assignment priors may be observed from probabilistic clustering or manually designed for K-means.
  • D Construction of terms for MI estimation: Likelihoods assign higher novelty probability to scenarios far from existing clusters, while qk(x) is distance-weighted and capped by εexist.qk(x) is zero near an existing cluster and approaches εexist far away.
  • D Construction of terms for MI estimation: The support-discovery model concentrates p(Rx = 1) around novel scenarios and uses data-adaptive estimates of γ, ρ, and K to reduce hyperparameter dependence.K is selected from [Kmin, Kmax] using the elbow heuristic, while γ and ρ derive from current training-support geometry.
  • E nuPlan ablations: The nuPlan ablation compares different initialization sizes using 100 samples across four seeds, reporting average ± standard deviation.The supplied table caption specifies the evaluation design but no ablation values.
  • D Construction of terms for MI estimation: Both low-probability failure regions are recovered within B = 40 samples on the 2D synthetic task, indicating that MI promotes diverse failures while µCV bridges the sim-real gap.The experiment uses a GP surrogate and data acquisition with the proposed approach.

E.1 Ablation results for size of initial dataset … I Experiment details

The ablations show that performance is robust to small initial datasets but depends on proxy samples for coverage of severe failures. Across Quadruped, SIMPLER, and visualization analyses, the method identifies diverse failures, including failures unseen in simulation and obstacle-induced tracking failures.

  • E.1 Ablation results for size of initial dataset: N0 = 50 yields the lowest cumulative TTC, highest diversity, and highest failure count, while differences among N0 = 5, 10, and 20 are marginal.The method is therefore not substantially affected by insufficient initial data.
  • I Experiment details: Results are reported using B = 100 samples for initial-dataset ablations and four seeds for the β = 0 comparison.The initial-dataset study evaluates N0 = 5, 10, 20, and 50; the β = 0 study uses 20 initial samples and B = 150.
  • E.2 Ablation results for µCV: Removing sim samples with β = 0 marginally improves positive samples and cumulative TTC but compromises coverage, especially for TTC = 0.0 severe failures.The passage attributes this tradeoff to proxy scenarios being noisy estimates of the target system.
  • F Quadruped: Our method discovers more diverse Quadruped failures than BAMS, which tends to concentrate around failures found in early iterations.Scatterplots and failure-versus-success heatmaps provide qualitative evidence for broader failure discovery.
  • G SIMPLER: Our approach discovers SIMPLER failures unseen in simulation, including one scenario where both proxy and target fail and another where only the target fails.The comparison includes fail and non-fail video demonstrations alongside the baseline results.
  • H Visualization: Quadruped failures occur at high vx and vy when obstacle avoidance activates during tracking, causing velocity to become zero.The failures correspond to quadrupeds reaching near boxed obstacles and stopping.
  • H Visualization: Figure 12 provides additional examples of failure scenarios uncovered by the approach in the SIMPLER task.These examples supplement the broader SIMPLER failure demonstrations.

I.1 Model specifications · I.2 Data specifications · I.3 Scenario design and failure metrics for each task

The study uses task-specific surrogate models and offline initialization strategies, then defines scenario representations, proxy-target pairs, and failure metrics for driving, manipulation, and quadruped tracking. These specifications reflect differing stochasticity, proxy costs, scenario dimensions, and target-system conditions.

  • I.1 Model specifications: For Quadruped, the method uses an MLP surrogate because the velocity-tracking policy exhibits low observed stochasticity.
  • I.2 Data specifications: nuPlan uses a directly available proxy dataset, while SIMPLER and Quadruped use surrogate proxy models because querying proxy data can also be costly.SIMPLER and Quadruped proxy surrogates use BNN and MLP models with larger offline initial datasets of 50 and 2000 datapoints, respectively.
  • I.3 Scenario design and failure metrics for each task: For nuPlan, driving-log scenarios are encoded as 384-dimensional vectors, TTC is the failure metric, and open-loop versus closed-loop simulation defines proxy versus target.
  • I.3 Scenario design and failure metrics for each task: SIMPLER scenarios perturb visual specifications and object placements into 19-dimensional representations spanning tabletop positions, camera translations, distractor placements, yaw, task variants, and eight distractor objects.The task variants are pick_horizontal, pick_vertical, and pick_standing.
  • I.3 Scenario design and failure metrics for each task: SIMPLER failure means failing to grasp or lift the Coke can; proxy and target differ slightly in visual specifications, with extra target visual noise acting as occlusion.The passage states that this leads to worse target performance on several scenarios.
Loading 2608.13719v1…