PaperlayerView exact revision on arXiv

Source-linked AI summary

RoG-DAgger: Rollout-Guided Post-Training for End-to-End Driving

Liangyu Zhong, Joachim Sicking, Fabian Hueger, Hanno Gottschalk

arXiv:2608.24525v1cs.RO

TL;DR

Fixed open-loop training leaves end-to-end driving policies vulnerable to policy-induced, safety-critical states, motivating closed-loop post-training with expert supervision. RoG-DAgger uses rollout-guided candidate planning, solvability-aware takeover, and field-of-view alignment, improving SimLingo across in-distribution, long-horizon, and out-of-distribution benchmarks. Its solvability estimates remain approximate because rollouts assume non-reactive surrounding actors and may omit feasible avoidance candidates.

  • Problem

    End-to-end driving policies trained mainly on fixed open-loop expert data remain vulnerable to distribution shift and safety-critical policy-induced states during closed-loop execution.

  • Method

    RoG-DAgger uses short-horizon kinematic rollouts to expand trajectory-and-speed supervision, time takeover near the estimated point of no return, and align expert and student fields of view.

  • Results

    RoG-DAgger improves SimLingo across complementary in-distribution, long-horizon, and out-of-distribution evaluations, including doubling Longest6 v2 driving score from 22 to 44 and raising Fail2Drive success rate from 55% to 66%.

  • Takeaways & Limitations

    Learning from policy-induced states with reliable expert supervision substantially improves closed-loop driving performance, particularly in challenging states underrepresented during pre-training.

  • Takeaways & Limitations

    Rollout-based solvability estimation is approximate because surrounding-actor behavior is assumed non-reactive and the candidate set may omit feasible collision-avoidance solutions.

Abstract

from arXiv · show

Recent end-to-end driving systems demonstrate strong performance on closed-loop benchmarks, yet are still predominantly trained on fixed expert-collected data using open-loop imitation learning. This training-inference mismatch leaves the policy vulnerable in policy-induced states, where accumulated errors can lead to safety-critical failures. A promising post-training approach to overcome this issue is Dataset Aggregation (DAgger), which gathers expert demonstrations in policy-induced states and subsequently fine-tunes the policy on the resulting aggregated dataset. Existing driving DAgger pipelines, however, face three challenges: i) the expert is restricted to a limited trajectory-and-speed solution space, ii) takeover may occur too early or too late relative to impending failures, and iii) privileged expert decisions may rely on information unavailable to the student. To address this, we introduce RoG-DAgger, a post-training framework that uses short-horizon kinematic rollouts to construct high-quality expert demonstrations in safety-critical states. Specifically, RoG-DAgger expands the expert's trajectory-and-speed solution space and evaluates candidate plans through rollout to construct preventive supervision. Moreover, it uses rollout solvability to time the takeover near the estimated point of no return. Lastly, it aligns the expert's field of view with that of the student to provide student-compatible supervision. Across in-distribution (including long-horizon) and out-of-distribution evaluations, RoG-DAgger improves the end-to-end model SimLingo by 5.3 driving-score points and 6.2 percentage points in success rate on Bench2Drive, doubles its driving score from 22 to 44 on Longest6 v2, and improves out-of-distribution success rate from 55\% to 66\% on Fail2Drive.

1 Introduction

SimLingo is trained mainly with fixed expert-collected data through open-loop imitation learning, but closed-loop execution exposes it to safety-critical states that are underrepresented during training. DAgger offers a closed-loop post-training approach by collecting expert demonstrations in policy-induced states, though existing driving pipelines face supervision and takeover limitations.

  • Closed-loop benchmarks execute predicted actions in the simulator, allowing them to influence subsequent observations.
  • SimLingo is predominantly trained on fixed expert-collected data using open-loop imitation learning.
  • SimLingo’s training data lacks safety-critical interactions, whereas such interactions occur frequently during closed-loop Bench2Drive inference.
  • This difference creates a substantial distribution shift between training and inference.
  • DAgger collects expert demonstrations in policy-induced states and fine-tunes the pretrained policy using the resulting aggregated data.

A. Kinematic Ego and Actor Rollout

RoG-DAgger uses kinematic rollouts to construct and assess trajectory-and-speed supervision, while addressing limitations in existing DAgger supervision and takeover timing. Its evaluation reports improvements across in-distribution, long-horizon, and out-of-distribution driving benchmarks.

  • RoG-DAgger uses kinematic rollouts to construct trajectory-and-speed supervision and evaluate candidate expert plans.
  • It expands the expert’s trajectory-and-speed solution space with multiple candidate geometric trajectories and target speeds, retaining rollout-solvable candidates satisfying driving constraints.
  • It triggers takeover near the estimated point of no return, where collision-free corrective candidates may no longer exist.
  • It aligns expert and student fields of view to reduce supervision that depends on information unavailable to the student.
  • On Longest6 v2, RoG-DAgger doubles SimLingo’s driving score from 22 to 44 and improves out-of-distribution success rate from 55% to 66% on Fail2Drive.

2 Related Work

Related work spans end-to-end driving systems and online post-training methods that adapt policies through closed-loop interaction. Driving DAgger methods differ in their expert supervision and takeover triggers, while RoG-DAgger uses rollout-based feasibility to provide state-specific preventive supervision.

  • End-to-End Driving Systems: End-to-end driving systems map sensory observations directly to planned trajectories within a differentiable pipeline.
  • End-to-End Driving Systems: Vision-language-action driving models improve alignment among visual observations, language instructions, and driving actions but remain primarily trained on fixed offline datasets.
  • Online Post-Training: Online post-training refines pretrained policies using data collected through closed-loop interaction, allowing policies to learn from their own closed-loop errors.
  • Driving DAgger: Existing driving DAgger methods differ in the supervision collected and the conditions used to trigger expert takeover.
  • Driving DAgger: RoG-DAgger constructs state-specific trajectory-and-speed supervision and triggers takeover when rollout evaluation indicates nominal emergency braking can no longer avoid an impending collision.

3 Rollout-Guided DAgger for E2E Driving

RoG-DAgger post-trains an end-to-end driving policy with rollout-guided expert supervision collected in policy-induced states. It combines kinematic rollouts, expanded trajectory-and-speed candidates, solvability-aware takeover, and student-aligned expert observations.

  • 3.1 Problem Formulation: DAgger queries expert supervision at states visited by the student policy, providing corrective training data for policy-induced states.
  • 3.2 Framework: RoG-DAgger performs separate infraction-based and solvability-aware closed-loop collection runs because solvability-aware takeover changes subsequent state distributions.
  • 3.3 Kinematic Ego and Actor Rollout: Kinematic ego and actor rollouts evaluate counterfactual future trajectories for candidate trajectory-and-speed plans, supporting preventive supervision and takeover decisions.
  • 3.4 Rollout-Guided Expert Supervision: 30% of pre-takeover samples remain collision-prone under rollout, motivating expanded solution spaces, better takeover timing, and student-compatible expert inputs.
  • Expanding the Trajectory-and-Speed Solution Space: The expert expands beyond a fixed route by evaluating laterally shifted trajectories and target speeds through rollouts, retaining candidates that are collision-free and satisfy driving constraints.
  • Solvability-Aware Takeover Trigger: Solvability-aware takeover targets the latest state with at least one collision-free candidate, approximating the estimated point of no return.
  • Field-of-View-Aligned Planning: RoG-Expert discards pre-takeover samples whose responsible actor is outside the student’s field of view while retaining post-takeover recovery samples.

4 Experiments

RoG-DAgger is evaluated through closed-loop in-distribution, long-horizon, and out-of-distribution benchmarks, alongside qualitative and ablation studies. Across these evaluations, post-training improves SimLingo and each major design component contributes to performance.

  • Evaluation Setup: RoG-DAgger is evaluated on Bench2Drive, Longest6 v2, and Fail2Drive, covering in-distribution, long-horizon, and out-of-distribution driving.Bench2Drive measures diverse short routes, Longest6 v2 stresses routes approximately 10× longer, and Fail2Drive tests unseen assets and novel scenario configurations.
  • In-Distribution and Long-Horizon Results: 5.3 p.p. DS and 6.2 p.p. SR improvements over SimLingo on Bench2Drive bring RoG-DAgger to 90.34 DS and 73.51 SR.RoG-DAgger also outperforms TakeAD and MindDrive and performs comparably to TakeVLA.
  • In-Distribution and Long-Horizon Results: RoG-DAgger doubles SimLingo’s Longest6 v2 DS from 22 to 44 and improves RC from 70 to 88.It substantially outperforms other non-privileged methods, despite using the weaker RoG-Expert rather than PDM-Lite.
  • Out-of-Distribution Results: RoG-DAgger improves Fail2Drive generalization over SimLingo by about 3 p.p. in DS and 11 p.p. in SR while remaining comparable on base scenarios.In FullyBlocked scenarios, it slows down and stops when the road is obstructed, whereas SimLingo often maintains speed and collides.
  • Ablation Studies: Removing trajectory-speed expansion, solvability-aware takeover, or FoV alignment degrades both Bench2Drive DS and SR relative to the full method’s 90.34 DS / 73.51 SR.Trajectory-speed expansion causes the largest SR drop, to 68.94; disabling solvability-aware takeover or FoV alignment reduces SR to 69.86.
  • Ablation Studies: Removing pre-takeover samples reduces Bench2Drive SR from 73.51 to 68.04, indicating that preventive supervision contributes to failure prevention.A conventional PDM-Lite DAgger configuration also lowers DS to 87.85 and SR to 66.66, despite PDM-Lite’s stronger standalone performance.

5 Discussion

RoG-DAgger improves pretrained driving policies through closed-loop expert supervision and performs especially strongly in long-horizon and out-of-distribution settings. Its rollout-based solvability estimation and CARLA dependence define important scope boundaries.

  • RoG-DAgger combines expanded trajectory-and-speed candidates, solvability-aware triggering, field-of-view alignment, and kinematic rollout validation.These components target distribution shift while providing reliable supervision in policy-induced states.
  • RoG-DAgger consistently improves closed-loop performance and is substantially stronger under long-horizon and out-of-distribution evaluation.
  • Rollout-based solvability estimation remains approximate because non-reactive actor forecasts and a finite candidate set can misrepresent feasible collision avoidance.The estimated point of no return is therefore not a physical feasibility boundary.
  • CARLA closed-loop simulation limits direct application of RoG-DAgger to real-world data.The paper identifies high-fidelity data-driven simulators as a future direction.

A Model Architecture and Pre-Training

The model uses SimLingo’s front-facing camera setup and jointly predicts trajectory and speed waypoints. Fine-tuning follows the SimLingo recipe with specified optimization settings and a Qwen3-VL backbone replacement.

  • Model Architecture: SimLingo uses a front-facing camera with a 110° horizontal field of view and outputs 10 trajectory waypoints plus 10 corresponding speed waypoints.Together, these outputs represent the planned spatial path and desired speed profile.
  • Pre-Training and Fine-Tuning: Fine-tuning runs for 5 epochs at a 1×10^-4 learning rate with effective batch size 48 across 4 GPUs.LoRA uses rank r = 32 and scaling factor α = 64.
  • Pre-Training and Fine-Tuning: The experiments disable chain-of-thought output to reduce inference latency and replace InternVL with Qwen3-VL-2B-Instruct.The reported comparison finds no significant B2D performance difference between the two backbones.
  • Implementation Details: The appendix describes trigger conditions, solution-space expansion, qualitative examples, data filtering, and other implementation settings.

B.1 Preliminary DAgger Experiment with PDM-Lite

The preliminary DAgger experiment evaluates PDM-Lite’s counterfactual plans with controller-in-the-loop kinematic rollouts and expands them when braking alone is insufficient. It also motivates solvability-aware takeover and filtered, observable supervision.

  • Preliminary Diagnostic: The diagnostic run queries PDM-Lite for counterfactual trajectory-and-speed plans and tests them with a controller-in-the-loop kinematic rollout model.The rollout determines whether the plan overlaps predicted surrounding-actor trajectories.
  • Data Collection: RoG-DAgger uses three trigger types in two independent data-collection runs, including rollout-based solvability criteria.Table 7 identifies R-TTC as rollout-model time-to-collision.
  • Solution-Space Expansion: When full braking is insufficient, the method generates geometric candidates and searches target speeds and lateral offsets for collision-free, lane-compliant plans.Offsets are tested in 0.2 m increments up to dmax = 3 m, prioritizing higher target speeds and smaller deviations.
  • Solution-Space Expansion: Local trajectory shifts begin 5 m plus vehicle half-extents before predicted overlap and extend 2 m plus half-extents beyond it.A cosine ease-in/ease-out transition over 2 m avoids abrupt geometric changes.
  • Solvability-Aware Takeover: Solvability-aware takeover activates near the estimated point of no return while the state remains recoverable, allowing RoG-Expert to correct the trajectory before collision.
  • Data Filtering: Collected samples are filtered for meaningful policy-expert disagreement, and pre-takeover collision frames are retained only when the relevant actor is camera-visible.The thresholds include speed disagreement above 0.5 m/s, a current-target speed gap above 10%, or steering deviation of at least 0.2 rad.

C Distribution of Post-Training Data

The RoG-DAgger dataset is characterized by takeover phase and high-level trigger, with most samples collected after takeover. Trigger groups include infractions and solvability-aware outcomes.

  • Most RoG-DAgger post-training samples originate from the post-takeover phase rather than before takeover.
  • The dataset is also grouped by high-level collection trigger and corresponding event types, including collisions, red-light and out-of-lane infractions, and solvability-aware outcomes.

D Additional RoG-DAgger Results

Additional results illustrate how RoG-DAgger improves expert intervention and reports broader evaluation coverage across closed-loop driving settings.

  • Expert supervision: Without an expanded trajectory-and-speed solution space, an expert relying primarily on full braking can fail to prevent collisions.The failure occurs when braking along the nominal route is insufficient.
  • Expert supervision: RoG-Expert searches alternative geometric trajectories when braking along the nominal route cannot prevent an impending collision.This expands the available trajectory-and-speed solution space in safety-critical states.
  • Takeover timing: Solvability-aware takeover activates when rollout predicts that an impending collision is near the boundary of preventability.In the example, a predicted TTC of 0.05 s triggers a 0.5 m lateral shift and a target speed of 50% of current speed.
  • Dataset composition: The post-training dataset is characterized from both takeover phase and collection-trigger or event-type perspectives.Percentages are normalized separately within the two table panels.
  • Evaluation coverage: Additional evaluations cover efficiency, comfort, multi-capability performance, and out-of-distribution results at scenario-class and individual-scenario levels.These results are reported for Bench2Drive and Fail2Drive, respectively.

D.1 Details of Evaluation Metrics

The paper evaluates route progress, driving compliance, and route-level success, then reports complementary performance and implementation details.

  • Evaluation metrics: Route Completion measures the percentage of the prescribed route traversed by the autonomous driving agent.It quantifies progress toward the destination.
  • Evaluation metrics: Driving Score combines route progress with a multiplicative penalty for unsafe or invalid driving behavior.Penalized behaviors include collisions, red-light violations, lane departures, and scenario timeouts.
  • Bench2Drive results: RoG-DAgger generally outperforms SimLingo on additional Bench2Drive efficiency, comfort, and multi-ability metrics.Its performance is reported as comparable to TakeVLA across these metrics.
  • Fail2Drive results: RoG-DAgger improves harmonic mean by 30 p.p. on Fail2Drive Behavior scenarios but performs 6 p.p. worse on Robustness scenarios.The paper associates the robustness reduction with conservative behavior near construction sites, which can impede progress and cause timeout penalties.
  • Implementation: Experiments use CARLA 0.9.15 for closed-loop simulation on Azure Standard_NC96ads_A100_v4 instances with four NVIDIA A100 GPUs.The environment uses Python 3.10, PyTorch 2.8, CUDA 12.6, and cuDNN 9.10.

E.2 Compute Time

The supplementary material combines compute-time reporting with qualitative examples of RoG-DAgger’s behavior across in-distribution and out-of-distribution scenarios.

  • Compute time: SimLingo-Qwen3VL-2B pre-training takes approximately 35 hours using four NVIDIA A100 GPUs.The reported duration uses the paper’s specified training schedule.
  • Compute time: Data collection requires approximately 15 hours with six parallel CARLA instances per GPU, while post-training takes an additional 3–4 hours.These durations are reported alongside the pre-training time.
  • Fail2Drive results: Per-scenario Fail2Drive results report Driving Score and Success Rate on Base and Generalization splits.RoG-DAgger performs particularly well on novel Construction Pedestrian and Fully Blocked configurations but still fails on unseen-asset scenarios such as Wall.
  • Fail2Drive examples: In Construction Pedestrian, SimLingo steers toward oncoming traffic despite a free right side and may collide after a late evasive maneuver.The behavior is described as consistent with a training bias toward construction zones on the right side.
  • Fail2Drive examples: In Fully Blocked, SimLingo accelerates toward a fire truck and collides, whereas RoG-DAgger is post-trained on policy-induced safety-critical scenarios.The comparison spans the time sequence from t1 through t4.
  • Bench2Drive examples: On Bench2Drive, RoG-DAgger merges into traffic during a left turn and uses emergency braking to avoid a lead-vehicle collision.Its predicted trajectory also adapts laterally as a pedestrian crosses the route.
  • Fail2Drive examples: On Fail2Drive, RoG-DAgger brakes before an unseen animal and navigates around an unseen children’s slide by changing lanes.These examples involve scenes absent from both pre-training and post-training.
Loading 2608.24525v1…