Source-linked AI summary
Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving
Luís Marques, Rong Fang, Disha Kamale, Dmitry Berenson
TL;DR
Assessing planned-trajectory safety is difficult when other actors’ motion responds to ego motion, and raw VLM predictions are not reliable enough for safety-critical monitoring. The paper introduces SLLCP, a post-hoc calibration method for frozen VLMs; on unseen CARLA scenarios, it raises unsafe-label coverage over raw VLM alarms.
Problem
Estimating collision risk is difficult because actor motion can respond to the ego vehicle, while raw VLM safety predictions may be overconfident and lack assurances against missed collisions.
Source: §II · p. 2Method
SLLCP combines a learned nonconformity score with scene-localized and label-conditional conformal calibration over a frozen VLM.
Source: §I · p. 2, §III · p. 3Results
89.6% and 88.4% unsafe-label coverage for Qwen and Cosmos, respectively, versus raw VLM alarms on 4.6% and 39.1% of unsafe frames.
Source: §V · p. 4Takeaways & Limitations
SLLCP provides a planner-agnostic way to balance safety monitoring and false alarms without requiring VLM fine-tuning.
Source: §VI · p. 4
Abstract
from arXiv · showhide
Monitoring planned driving trajectories requires accurately estimating the collision likelihood with actors whose motion is itself impacted by the ego motion. Existing classical approaches are often limited by the quality of their forecasting model. Vision-language models (VLMs) have shown promise in reasoning about the consequences of high-level actions, yet their approximate predictions are unsuitable for safety-critical applications such as autonomous driving. Conformal prediction (CP) has emerged as a data-driven framework for quantifying the uncertainty of black-box model predictions. We propose Split Label-Localized Conformal Prediction (SLLCP), a post-hoc calibration layer over frozen VLMs that transforms their unreliable predictions into probabilistically calibrated safety prediction sets. We consider how the ability to estimate safety can depend on the observed driving scene and introduce a localized procedure that upweights relevant past experience when calculating uncertainty thresholds. We provide label-conditional finite-sample distribution-free coverage under exchangeability. Evaluated over 15k CARLA trajectories from unseen scenarios, SLLCP correctly flags 89.6% of collision-causing trajectories with a Qwen backbone and 88.4% with a Cosmos backbone, while the base VLMs only flagged 4.6% and 39.1% of the collision-causing trajectories, respectively. These results indicate that local, label-conditional calibration can reduce missed unsafe trajectories.
I. INTRODUCTION AND RELATED WORK
Assessing trajectory safety is difficult because other actors respond to ego motion, scene information can be incomplete, and VLM safety predictions can be unreliable. The paper proposes a post-hoc, scene-adaptive monitor that calibrates VLM predictions and evaluates its safety–false-alarm tradeoff on CARLA edge cases.
I. INTRODUCTION AND RELATED WORK
- Actor motion can depend on ego motion, while occlusions and unfamiliar scenarios complicate forecasting and safe trajectory assessment.A trajectory passing a pedestrian near the curb can be safe if they wait and unsafe if they step into the vehicle’s path.Source: §I · p. 1
- Raw VLM SAFE/UNSAFE predictions provide no assurance of a low rate of missed collisions because VLM spatial reasoning can be unreliable.The proposed calibration layer is intended to add probabilistic guarantees to the VLM’s common-sense reasoning.Source: §I · p. 1
- SLLCP calibrates a frozen VLM’s safety prediction post hoc using a planner-agnostic trajectory image, with the aim of probabilistically limiting missed unsafe trajectories.The method combines label-conditional calibration with scene-localized weighting of relevant calibration examples.Source: Figure 1 · p. 1, §I · p. 2
- The monitor is evaluated on edge-case CARLA scenarios for the tradeoff between missed unsafe trajectories and false alarms across two open-weight VLMs.The contribution reports using the same hyperparameter set for both models.Source: §I · p. 2
II. PROBLEM STATEMENT
The problem is to monitor each finite-horizon candidate trajectory using a set-valued safety prediction, despite uncertain outcomes and potentially unreliable forecasts. The monitor should bound missed unsafe trajectories while also limiting false alarms, under an iid held-out calibration-set assumption.
II. PROBLEM STATEMENT
- The monitor maps scene-and-trajectory input X to a set of SAFE and/or UNSAFE labels, with the goal of bounding the probability of missing an unsafe trajectory by user-specified α.UNSAFE means a collision occurs within H steps; SAFE means it does not.Source: §II · p. 2
- The monitor should also minimize false positives, because unnecessary alarms can hinder the agent’s task performance.Source: §II · p. 2
- The formulation assumes access to a held-out calibration set of labeled VLM input–outcome pairs that are iid with the test pair and uncorrupted.Source: §II · p. 2
III. BACKGROUND: CONFORMAL PREDICTION
Conformal prediction turns black-box scores into prediction sets with finite-sample guarantees under exchangeability. Label-conditional thresholds address undercoverage of difficult, underrepresented classes, while localized thresholds adapt to input difficulty and use a held-out correction.
Split conformal prediction
- SplitCP uses an exchangeable calibration set to set a nonconformity threshold for test-time prediction sets with probabilistic coverage guarantees.The threshold is the k-th smallest calibration score, where k = ⌈(n + 1)(1 − α)⌉.Source: §III · p. 2
Label-conditional prediction
- Label-conditional CP calibrates each class separately to address undercoverage when a difficult label is underrepresented in the calibration set.The per-label prediction set satisfies label-conditional coverage; including UNSAFE triggers a safety alarm.Source: §III · p. 2
Split localized conformal prediction
- Localized CP upweights calibration examples near the test input because nearby scenes may share prediction difficulty.A local score quantile approximates input-conditional coverage, which is impossible to guarantee with finite data without additional distributional assumptions.Source: §III · p. 3
- SLCP adds a held-out quantile of score residuals to the local threshold, but its guarantee is only marginal coverage.The calibration data are split between local fitting and held-out correction.Source: §III · p. 3
IV. METHOD: SLLCP
SLLCP combines label-conditional conformal prediction with localized calibration over a frozen VLM. It fits a probe and a low-dimensional representation for local score quantiles, then applies per-label held-out corrections to form the prediction set.
IV. METHOD: SLLCP
- SLLCP combines localized and label-conditional conformal prediction, using learned nonconformity scores instead of raw VLM scores for local quantile fitting.The method splits calibration data between learning the score and local quantiles, and per-label conformal corrections.Source: §IV · p. 3
Probe score and local representation
- A logistic probe estimates unsafe probability from frozen VLM hidden states, providing an alternative to Boolean-token probabilities without updating VLM weights.The authors report that Boolean-token probabilities may not expose all safety-relevant information in the hidden state.Source: §IV · p. 3
- SLLCP projects VLM embeddings into a two-dimensional UMAP space and uses projected distances to localize calibration examples.The projection and classifier are fixed independently of the correction set.Source: §IV · p. 3
Label-conditional local quantiles and correction
- For each label, SLLCP fits a weighted local score quantile and adds a held-out correction computed from examples of that label.The resulting prediction set includes labels whose nonconformity scores do not exceed their corrected local thresholds.Source: §IV · p. 3
- The monitor raises an alarm when UNSAFE is included in its prediction set, with label-conditional marginal coverage under within-label exchangeability.The paper notes that this assumption might not hold in its experiments, while reporting adequate empirical coverage in practice.Source: §IV · p. 3
V. EXPERIMENTAL RESULTS
SLLCP brought unsafe-label coverage on unseen CARLA scenarios near the 90% target for both VLM backbones, substantially above raw VLM binary alarms. Ablations expose a coverage–false-alarm tradeoff, while the evaluation also faces correlated frames and scenario shift.
V. EXPERIMENTAL RESULTS
- Evaluation reported unsafe-label coverage, false-positive rate on safe trajectories, and two-label ambiguity, using α = 0.1.Comparisons included raw VLM rules and ablations of locality, the linear probe, UMAP projection, and per-label correction.Source: §V · pp. 3–4
- The study collected 40,455 frames across 90 episodes and 30 scenario IDs, then evaluated on 15,321 frames from unseen scenario IDs.The two frozen open-weight backbones were Qwen3-VL-8B-Instruct and Cosmos-Reason2-8B; scenario seeds remained on one side of the calibration/evaluation split.Source: §V · p. 4
- SLLCP achieved 89.6% unsafe-label coverage for Qwen and 88.4% for Cosmos, versus raw VLM binary alarms at 4.6% and 39.1%, respectively.Both SLLCP results fell within the finite-sample Beta-binomial confidence interval of the 90% target.Source: §V · p. 4, Table I · p. 4
- Alarm FPRs rose to 51.8% for Qwen and 44.5% for Cosmos, with both-label ambiguity of 37.6% and 33.6%, respectively.The authors describe this as an efficiency–coverage tradeoff.Source: §V · p. 4
- Without UMAP, unsafe coverage fell to 84.3% and 82.4%; a label-independent threshold lowered it to 70.9% and 66.7%.Without the logistic probe, nearly all unsafe frames were included, but false alarms affected at least 95.6% of safe frames.Source: §V · p. 4
- Results are sensitive to UMAP projection and kernel hyperparameters; correlated frames break exchangeability, and scenario shift from calibration data affects empirical performance.The dataset’s frames were collected along full-episode SimLingo trajectories.Source: §V · p. 4
VI. CONCLUSION
On unseen Fail2Drive scenarios in CARLA, SLLCP achieved near-90% unsafe-label coverage for both evaluated VLMs, improving over raw VLM performance. The conclusion presents it as a planner-agnostic, no-fine-tuning approach and notes that extending it to semantic violations requires reliable ground-truth labels.
VI. CONCLUSION
- SLLCP achieved 89.6% and 88.4% unsafe-label coverage for Qwen and Cosmos, improving 85.0 and 49.3 percentage points, respectively, over raw VLM performance.The evaluation used unseen Fail2Drive scenarios in CARLA.Source: §VI · p. 4
- The approach combines a learned score from frozen VLM representations with local calibration in projected embedding space and label-conditional calibration.The authors describe it as planner-agnostic and requiring no VLM fine-tuning.Source: §VI · p. 4
- Future work could extend monitoring to semantic violations such as failing to yield or obey traffic signals, where reliable ground-truth labels are nontrivial to obtain.Source: §VI · p. 4