Source-linked AI summary
Unleashing VLA Potentials in Autonomous Driving via Explicit Learning from Failures
Yuechen Luo, Qimao Chen, Fang Li, Shaoqing Xu, Jaxin Liu, Ziying Song, Zhi-xin Yang, Fuxi Wen
TL;DR
VLA reinforcement learning can plateau when SFT-constrained exploration repeatedly fails in long-tail driving scenarios and scalar rewards cannot identify the failure source. ELF-VLA adds structured teacher diagnostics, uses them to generate corrected trajectories, and reinjects those samples into RL training. On Navsim, the method achieves state-of-the-art overall PDMS and high-level planning accuracy, while remaining dependent on teacher capability and evaluation in a non-reactive simulator.
Problem
SFT-constrained exploration and information-sparse scalar rewards leave VLA failures in long-tail scenarios undiagnosed across planning, reasoning, and trajectory execution.
Method
ELF-VLA uses teacher-generated structured failure reports to guide corrected high-reward trajectories that are re-injected into the GRPO training batch.
Results
ELF-VLA achieves state-of-the-art performance on Navsim for overall PDMS and high-level planning accuracy.
Takeaways & Limitations
Explicit failure diagnosis and feedback-guided refinement provide a targeted gradient for challenging scenarios that unguided exploration rarely resolves.
Takeaways & Limitations
The method depends on an external teacher model and has been evaluated only on the non-reactive Navsim benchmark.
Abstract
from arXiv · showhide
Vision-Language-Action (VLA) models for autonomous driving often hit a performance plateau during Reinforcement Learning (RL) optimization. This stagnation arises from exploration capabilities constrained by previous Supervised Fine-Tuning (SFT), leading to persistent failures in long-tail scenarios. In these critical situations, all explored actions yield a zero-value driving score. This information-sparse reward signals a failure, yet fails to identify its root cause -- whether it is due to incorrect planning, flawed reasoning, or poor trajectory execution. To address this limitation, we propose VLA with Explicit Learning from Failures (ELF-VLA), a framework that augments RL with structured diagnostic feedback. Instead of relying on a vague scalar reward, our method produces detailed, interpretable reports that identify the specific failure mode. The VLA policy then leverages this explicit feedback to generate a Feedback-Guided Refinement. By injecting these corrected, high-reward samples back into the RL training batch, our approach provides a targeted gradient, which enables the policy to solve critical scenarios that unguided exploration cannot. Extensive experiments demonstrate that our method unlocks the latent capabilities of VLA models, achieving state-of-the-art (SOTA) performance on the public NAVSIM benchmark for overall PDMS, EPDMS score and high-level planning accuracy.
1. Introduction
VLA autonomous-driving policies can plateau during RL because SFT limits exploration in safety-critical scenarios, while scalar rewards do not diagnose whether failures arise from planning, reasoning, or execution. ELF-VLA addresses this by using structured teacher feedback to guide corrected trajectories and achieve strong benchmark performance.
- VLA models map camera inputs to vehicle motion commands through SFT and RL, with intermediate reasoning intended to support explainable driving.
- After SFT, limited exposure to safety-critical scenarios constrains exploration, causing repeated zero-score rollouts in challenging situations.
- Scalar driving rewards reveal failure but do not identify whether its source is high-level planning, critical-target reasoning, or low-level trajectory execution.
- ELF-VLA uses a teacher to generate structured, ability-aligned diagnoses and has the policy produce a corrected trajectory from that feedback.
- Corrected high-reward samples are re-injected into GRPO batches, providing a goal-directed gradient for scenarios that unguided exploration rarely resolves.
- ELF-VLA achieves state-of-the-art performance on Navsim for overall PDMS and high-level planning accuracy.
2. Related Work
Related work places autonomous-driving VLA research across scene understanding, high-level reasoning, and end-to-end policy learning. Existing RL pipelines remain dependent on SFT quality and struggle with rare long-tail scenarios whose driving scores stay low during rollout.
- VLA research includes scene-understanding and high-level-reasoning systems, but improvements in actual driving performance have not been fully realized.
- Intermediate reasoning methods such as Chain-of-Thought improve trajectory-forecasting precision by exposing domain-specific cognitive processes.
- Current autonomous-driving VLA training typically uses SFT followed by GRPO-based RL, with rewards measured by driving scores such as PDMS.
- Because RL effectiveness depends heavily on preceding SFT, rare long-tail scenarios can retain extremely low driving scores across rollouts and cause learning stagnation.
3. Methods
ELF-VLA combines two-stage supervised fine-tuning with reinforcement learning enhanced by structured failure feedback. The framework trains trajectory prediction and refinement, then uses teacher-guided feedback to improve difficult rollouts and optimize them jointly.
- VLA Inputs Formulation: The model accepts base inputs containing camera imagery, navigation commands, ego state, and recent trajectory history, then produces trajectory responses with chain-of-thought reasoning.Responses are classified by a PDMS threshold into correct and wrong outputs for subsequent feedback construction.
- Failure Feedback: Correct responses receive rule-based positive feedback, whereas wrong responses are analyzed by a teacher model using the base inputs, erroneous trajectory, and ground-truth trajectory.Teacher feedback covers meta-action, thought-process, safety, and efficiency failures.
- Two-Stage SFT: ELF-VLA uses two-stage SFT: driving knowledge is learned first, followed by trajectory prediction and feedback-based refinement.The first stage uses driving-related Q&A data, while the second supervises both base and feedback inputs with ground-truth trajectories.
- Sample Curation: Efficient difficult-sample curation filters training data toward difficult and ambiguous scenarios before feedback-enhanced GRPO.The curation reduces the initial 85k training entries to 24k high-value scenarios.
- Reward Modeling: The reward combines PDMS trajectory quality, format compliance, and goal accuracy, with the overall reward defined as r = rtraj + rfmt + rgoal.The PDMS reward is continuous from 0 to 1, while the format reward is binary and the goal reward reflects endpoint distance.
- GRPO with Feedback: During GRPO rollouts, the policy samples responses, classifies them, generates feedback refinements, and adds higher-reward refinements to the optimization batch.Advantages are normalized over the combined original and feedback-generated rewards, while Policy Shaping addresses conditioning mismatch and low-probability refined outputs.
4. Experiment
ELF-VLA is evaluated on NAVSIM using trajectory prediction, high-level planning, baseline comparisons, failure-ratio analysis, qualitative refinement, and ablations. Across these evaluations, structured feedback and curated refinement improve driving performance and reduce persistent failures.
- Navsim Benchmark: ELF-VLA achieves 91.0 PDMS on NAVSIMv1 and 87.1 EPDMS on NAVSIMv2, establishing new state-of-the-art results.It improves over DriveVLA by 0.7 PDMS and DriveVLA-W0 by 1.0 EPDMS.
- Quantitative Evaluation: ELF-VLA outperforms conventional GRPO by 2.0 PDMS and surpasses GT-GRPO and Rule-GRPO by 1.8 and 1.4 PDMS, respectively.The method uses structured teacher feedback to generate refined, more easily optimizable trajectories.
- Total-Failure Ratio Analysis: ELF-VLA reduces the total-failure PDMS rate from 2.73% for GRPO to 1.08%, with similarly strong reductions for NC and DAC.The analysis measures samples for which all rolled-out trajectories fail simultaneously on each metric.
- High-Level Planning Evaluation: ELF-VLA reaches 80.3% overall high-level planning accuracy, 1.0% above conventional GRPO and 51.6% above Qwen2.5-VL-72B.It achieves the best results for both longitudinal Speed Accuracy and lateral Path Accuracy.
- On GRPO with Training Data: A curated 24k RL dataset reaches 91.0 PDMS, exceeding the full 85k dataset at 89.1 PDMS and a random 24k subset at 88.9 PDMS.The paper attributes this to concentrating training on complex scenarios with more valuable learning signals.
- On GRPO with Feedback: One refinement response achieves 91.0 PDMS, while four responses drop performance to 89.0; removing Policy Shaping lowers PDMS from 91.0 to 89.3.The results indicate that targeted refinement and Policy Shaping are important for effective optimization.
- Qualitative Analysis: In a complex left-turn example, teacher feedback corrects obstacle estimation and adjusts lateral position and longitudinal speed to produce a safer refined trajectory.The initial trajectory risked collision, whereas the refined trajectory avoided the obstacle.
5. Conclusion
ELF-VLA augments VLA reinforcement learning with teacher-generated diagnostic reports and corrected trajectories re-injected into training. Its experiments show strong NAVSIM performance, while the method remains dependent on the teacher model and has only been evaluated in the non-reactive NAVSIM environment.
- Conclusion: ELF-VLA diagnoses failure modes, synthesizes corrected high-reward trajectories, and re-injects them into RL batches to provide targeted gradients.This targets challenging scenarios that unguided exploration rarely overcomes.
- Conclusion: The method’s primary limitation is dependence on an external teacher model, which bounds student performance by the teacher’s analytical capabilities.The experiments are also limited to the non-reactive NAVSIM simulation environment.
6. Data Construction Details
The data construction combines scene-level driving semantics with trajectory-based feedback to train both reasoning and refinement. Dynamic interactions, static map constraints, and response evaluations support teacher diagnosis and corrective guidance.
- CoT Construction: The CoT data synthesizes future trajectories with scene-level semantics, including interacting agents, lane topology, road curvature, and lane centerlines.Dynamic entities are categorized by their interaction with the ego vehicle, while static elements define the drivable area.
- CoT Construction: CIPO-1, CIPO-2, and motion interaction represent leading vehicles, merging or cutting-in vehicles, and collision-risk entities, respectively.These categories encode distinct dynamic constraints on the ego vehicle.
- CoT Construction: The teacher model receives explicit environmental and task inputs to diagnose failure causes and generate detailed corrective guidance.The inputs integrate dynamic interactions, static constraints, and task requirements.
- SFT Data Construction: The feedback dataset samples 4k correct and 4k incorrect responses, pairing them with feedback according to a PDMS threshold s.Responses above the threshold are designated correct and receive rule-based feedback; the passage truncates the remainder of the procedure.
7. Method Details
The method uses multiple rewards to evaluate trajectory quality, enforce valid output structure, and align predicted endpoints with ground truth. Prompt designs distinguish base VLA inputs from teacher feedback inputs.
- Reward Design: PDMS provides a continuous trajectory reward from 0 to 1 based on collision, drivable-area, progress, time-to-collision, and comfort metrics.The simulator evaluates each predicted trajectory and returns PDMS as rtraj.
- Reward Design: The format reward assigns 0.5 points to the required <think> and <answer> sections and 0.5 points to machine-parsable trajectory syntax.The total format reward is 1.0 point.
- Reward Design: The goal reward promotes endpoint alignment with ground truth using a piecewise function based on L1 distance.Its formulation is specified in the paper following the stated design.
- Prompt Design: The VLA and teacher models use distinct prompt designs for base inputs and feedback inputs.The supplied figure captions identify the two prompt configurations but do not detail their contents.
8. Experiment Details
Experiments evaluate ELF-VLA with NAVSIM metrics, feedback comparisons, ablations, threshold sensitivity, and qualitative refinement examples. Results show gains from the training pipeline and structured feedback, while refinement addresses reasoning and trajectory failures.
- Evaluation Setup: PDMS and EPDMS serve as the primary closed-loop planning metrics for NAVSIMv1 and NAVSIMv2, respectively.PDMS combines collision, drivable-area, time-to-collision, comfort, and ego-progress components; EPDMS includes additional compliance and comfort components.
- Feedback Comparison: Rule-GRPO provides binary heuristic feedback, whereas ELF-VLA generates online, instance-specific diagnostics from fine-grained PDMS metrics.ELF-VLA uses these diagnostics to provide constructive guidance for trajectory correction.
- Training Pipeline: 91.0 PDMS is achieved after adding Feedback-GRPO, compared with 87.4 after pretraining and 85.3 with NAVSIM trajectory SFT alone.The ablation reports a 3.6-point gain from adding Feedback-GRPO and a 2.1-point gain from pretraining.
- Planning Labels: High-level action labels separate longitudinal and lateral planning, using sliding-window acceleration and trajectory–map-topology relationships for labeling.The figure describes both the discrete action categories and their labeling criteria.
- Pretraining Ablation: 90.0 PDMS without pretraining surpasses the 86.9-PDMS baseline by +3.1, indicating gains remain when pretraining data are removed.The passage attributes the remaining improvement primarily to the GRPO design.
- Threshold Sensitivity: Peak performance occurs at feedback threshold s = 0.8, while refining already high-quality responses in [0.8, 0.9) may introduce noise and destabilize optimization.The threshold controls when the teacher generates refinement guidance.
- Refinement Visualization: Qualitative examples show structured feedback validating high-level plans, correcting intermediate CoT reasoning, localizing obstacles, and diagnosing safety and efficiency failures.The resulting guidance produces concrete correction strategies for final trajectory refinement.