PaperlayerView exact revision on arXiv

Source-linked AI summary

RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving

Zhe Sun, Ziyi Luo, Yehao Lu, Lei Zhou, Xi Li

arXiv:2609.35078v1cs.CV

TL;DR

Driving VLAs rely heavily on successful demonstrations, leaving their own failures underused and difficult to learn from reliably. RefineDrive trains on simulator-grounded diagnoses and nearby safe corrections, then applies safety-layered GRPO. It achieves 91.7 PDMS on NAVSIM v1 and 89.4 EPDMS on NAVTEST scenes with the same checkpoint and no further training.

  • Problem

    VLA driving methods rely primarily on successful human behavior, while model-generated failures remain underused and are difficult to learn from because diagnoses, correction targets, and rewards can be poorly matched.

    Source: §1 · pp. 1–2

  • Method

    RefineDrive trains on simulator-verifiable diagnoses and nearby safe corrections retrieved for failed trajectories, then refines the policy with Safety-Layered GRPO.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    91.7 PDMS on NAVSIM v1 and 89.4 EPDMS on the original NAVTEST scenes under NAVSIM v2 extended metrics, using the same checkpoint without further training.

    Source: §5 · p. 11

  • Takeaways & Limitations

    Controlled ablations show diagnosis-and-correction supervision improves NC, DAC, and TTC after RL at some cost to progress, while Safety-Layered Reward improves all three at the same mean EP as direct PDMS optimization.

    Source: §5 · p. 11

    Samples are discarded when a collision object cannot be reliably localized in the current visual observation.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Vision-Language-Action (VLA) models for autonomous driving rely heavily on successful expert demonstrations, leaving model-specific failures underexploited. Learning from these failures is hindered by unreliable diagnoses, poorly matched correction targets, and coarse rewards. We propose RefineDrive, a failure-guided post-training framework that learns from self-generated failures through targeted supervision and safety-aware reinforcement learning. Reliable Diagnosis derives structured, verifiable feedback on collisions and drivable-area violations directly from simulator states. Minimum-Correction Target Retrieval searches a clustered human trajectory bank for nearby corrections that satisfy hard-safety constraints in the current scene, prioritizing preservation of the failed prediction's motion pattern. Conditioned on the driving context and failed trajectory, Correction SFT learns to generate the diagnosis followed by the retrieved correction as a training-only auxiliary task. We then apply GRPO with a Safety-Layered Reward that strictly prioritizes hard-safe trajectories, retains continuous safety feedback for both unsafe and hard-safe trajectories, and rewards driving progress only after hard safety is satisfied. At inference, the policy directly predicts trajectories from the driving context without an explicit diagnosis or repair stage. On NAVSIM v1, RefineDrive improves the 4B base SFT policy from 87.7 to 91.7 PDMS. Using the same checkpoint without additional training, RefineDrive achieves 89.4 EPDMS on the original NAVTEST scenes evaluated with NAVSIM v2 extended metrics. Controlled ablations support the benefits of structured diagnosis supervision, retrieved corrections, and safety-layered optimization for direct planning.

1 Introduction

Driving VLAs may improve by addressing residual weaknesses revealed by their own rollouts, rather than relying only on successful demonstrations. RefineDrive combines simulator-grounded diagnoses, minimally changed safe corrections, and safety-layered optimization.

1 Introduction

  • Failure-learning methods face unreliable diagnoses, over-corrective ground-truth targets, and coarse rewards that obscure safety violations and margins.These gaps matter because a failed trajectory may be only partially incorrect, while aggregated scores can miss the extent and timing of violations.Source: §1 · pp. 1–2
  • RefineDrive derives simulator-verifiable diagnoses and retrieves safe corrective trajectories close to failed predictions, reducing unnecessary behavioral changes.Correction SFT conditions on the driving context and failed trajectory; diagnosis and repair are training-only, with direct trajectory prediction at inference.Source: §1 · pp. 2–3
  • Safety-Layered GRPO prioritizes hard-safe trajectories, retains continuous safety feedback for unsafe and hard-safe trajectories, and rewards progress only after hard safety is satisfied.Source: §1 · pp. 2–3
  • RefineDrive uses failures from its own rollouts to target policy-specific residual weaknesses rather than repeatedly learning the full distribution of expert behavior.Source: §1 · pp. 2–3

2 Related Work

Related VLA work spans varied reasoning, representation, and planning approaches, while failure-learning methods explore teacher diagnoses, counterfactual supervision, and trajectory repair. RefineDrive distinguishes itself through simulator-grounded diagnoses and trajectory-conditioned corrective supervision used during training.

2.1 Vision-Language-Action Models for Autonomous Driving

  • VLA planners span language-space reasoning (DriveVLM [6]), aligned autoregressive actions (OpenDriveVLA [7]), semantic action tokens with reinforcement fine-tuning (AutoVLA), and VLM-plus-diffusion or unified representations (ReCogDrive; Qwen-Drive [10]).Source: §2.1 · p. 3
  • Unlike ReflectDrive’s inference-time waypoint editing and inpainting [11], RefineDrive retrieves complete scene-validated trajectories for offline Correction SFT.Source: §2.1 · p. 3

2.2 Learning from Failures

  • Failure-learning approaches include perturbed expert trajectories [12], teacher-generated diagnoses [13], counterfactual positives [14], and failure-triggered self-distillation [15]; RefineDrive instead grounds diagnoses in simulator evidence.Its diagnoses link violation times, involved objects, and boundary-exit directions to geometric and kinematic evidence.Source: §2.2 · p. 3
  • Where R2LPL [16] retrieves feasible anchors and trains an anchor-scoring planner, RefineDrive conditions Correction SFT on the failed prediction and a nearby scene-verified safe trajectory.The diagnosis and correction are training-only tasks, followed by Safety-Layered GRPO.Source: §2.2 · pp. 3–4, Figure 2 · p. 4

3 Method

RefineDrive improves an already capable driving policy by targeting safety-critical failures exposed by its own rollouts. It combines simulator-grounded diagnosis, nearby safe corrections, training-only Correction SFT, and Safety-Layered GRPO.

3.1 Overview

  • RefineDrive’s four stages are failure discovery, structured failure correction, Correction SFT, and Safety-Layered Reinforcement Learning.The framework focuses learning on weaknesses revealed by the policy’s own rollouts rather than repeatedly imitating behavior it has already mastered.Source: §3.1 · p. 4

3.2 Failure Discovery

  • The policy’s own simulator-reexecuted rollouts identify model-specific failures, focusing on at-fault collisions and departures from the drivable area.These failures expose modes produced by the current policy and serve as targeted cases for diagnosis and correction.Source: §3.2 · pp. 4–5

3.3 Structured Failure Correction

  • Structured failure correction pairs simulator-derived diagnoses with nearby feasible human trajectories, preserving the failed prediction’s motion pattern while avoiding unnecessary changes.Collision feedback records the first collision’s time, object category, relative position, and image bounding box; drivable-area feedback records the first exit time and boundary side. Samples are discarded when the collision object cannot be reliably localized.Source: §3.3 · p. 5
  • The correction search groups training-set human trajectories by motion pattern, searches the failed prediction’s cluster first, and re-evaluates every candidate for hard safety in the current scene.If needed, it searches other clusters by center proximity; no correction is assigned if none is feasible within the available candidates or search budget.Source: §3.3 · pp. 5–6

3.4 Correction SFT

  • Correction SFT trains on the driving context and failed trajectory, with a target consisting of the structured diagnosis followed by the retrieved correction.It is a training-only auxiliary task: inference directly generates a trajectory from driving context, without a failed trajectory or explicit diagnosis-and-correction pass.Source: §3.4 · p. 6

3.5 Safety-Layered Reward

  • Hard-safe trajectories receive strictly higher rewards, while continuous safety feedback is retained within both safety groups and progress is rewarded only after hard safety is met.Unsafe trajectories receive only the continuous safety reward; hard-safe trajectories also receive a feasibility bonus and progress reward.Source: §3.5 · pp. 6–7

4 Experiments

RefineDrive reaches 91.7 PDMS on NAVSIM v1 and 89.4 EPDMS on the original NAVTEST scenes under NAVSIM v2 extended metrics, using the same checkpoint without additional training. Controlled ablations report safety gains from diagnosis-and-correction supervision and Safety-Layered Reward, alongside progress trade-offs.

4.1 Experimental Setup

  • NAVTRAIN supplies training, failure mining, and reinforcement learning, while NAVTEST is reserved for evaluation; corrective trajectories come only from the training split.The study reports PDMS for overall comparison and analyzes NC, DAC, TTC, EP, and comfort as additional outcomes; all metrics are higher-is-better.Source: §4.1 · p. 7

4.2 Main Results

  • 91.7 PDMS: RefineDrive’s 4B model achieves this NAVSIM v1 score, with 98.7 NC, 98.2 DAC, and 95.9 TTC.Table 1 compares methods on NAVSIM v1; the reported result demonstrates overall and safety performance without relying on a substantially larger foundation model.Source: §4.2 · p. 8, Table 1 · p. 8
  • 89.4 EPDMS: RefineDrive achieves this score on the original NAVTEST scenes using NAVSIM v2 extended metrics and the same checkpoint without additional training.Table 2 covers the extended-metric evaluation on the original NAVTEST scenes.Source: §4.2 · p. 8, Table 2 · p. 9

4.3 Ablation Studies

  • With failure cases and diagnosis supervision fixed, retrieved minimal corrections raise PDMS from 88.2 with Human GT to 88.6, despite lower mean target PDMS (95.4 vs. 95.8).Removing diagnosis supervision while retaining the corrective targets lowers PDMS from 88.6 to 88.3.Source: §4.3 · p. 9, Table 3 · p. 9
  • Safety-Layered Reward reaches the same mean EP as direct PDMS optimization (87.2), while improving NC, DAC, TTC, and PDMS from 91.1 to 91.7.Compared with PDMS Reward, it improves NC, DAC, and TTC by 0.5, 0.4, and 1.0 percentage points, respectively.Source: §4.3 · p. 9, Table 4 · p. 10
  • After RL, full Correction SFT has the highest NC, DAC, and TTC among the compared variants, with lower EP; similar PDMS values do not imply similar safety performance.Against omitting Correction SFT, its safety gains are 0.9, 0.5, and 1.8 points, while EP decreases from 88.8 to 87.2.Source: §4.3 · p. 10, Table 5 · p. 10

4.4 Qualitative Analysis

  • On the same NAVTEST scenes, RefineDrive avoids the off-road and collision behaviors shown by the base policy in the qualitative examples.Figure 3 compares direct predictions from both policies against human trajectories; the policies predict independently from the same driving context.Source: Figure 3 · p. 10, §4.4 · p. 11

5 Conclusion

RefineDrive combines failure diagnosis, minimal-correction retrieval, and Safety-Layered GRPO without inference-time diagnosis or repair. It achieves 91.7 PDMS on NAVSIM v1 and 89.4 EPDMS on original NAVTEST scenes with the same checkpoint and no further training.

5 Conclusion

  • RefineDrive integrates Reliable Diagnosis, Minimum-Correction Target Retrieval, and Safety-Layered GRPO without inference-time diagnosis or repair.It is presented as a failure-guided post-training framework.Source: §5 · p. 11
  • After RL, diagnosis-and-correction supervision improves NC, DAC, and TTC at some cost to progress, while Safety-Layered Reward improves all three at the same mean EP as direct PDMS optimization.These are the conclusion’s reported controlled-ablation findings.Source: §5 · p. 11
  • 91.7 PDMS on NAVSIM v1 and 89.4 EPDMS on original NAVTEST scenes: RefineDrive reports both results with the same checkpoint and no further training.The NAVTEST evaluation uses NAVSIM v2 extended metrics.Source: §5 · p. 11

Related papers

Loading 2609.35078v1…