PaperlayerView exact revision on arXiv

Source-linked AI summary

READ: Learning Risk-Informed Fields for End-to-End Autonomous Driving

Zhiyuan Liu, Yuanxin Tian, Zehong Ke, Jinhao Li, Hao Cheng, Zhenhua Xu, Wenhao Yu, Jianqiang Wang

arXiv:2609.12371v1cs.RO

TL;DR

Learned planners often leave implicit how road structure and agent motion influence candidate actions, while classical safety fields prescribe scene-independent risk shapes and weights. READ learns a continuous, planning-aligned risk field from geometric and behavioral constraints. It reports consistent gains with matched end-to-end backbones and strong performance in a VLA setting on NAVSIM.

  • Problem

    Learned planners often leave environmental influences on candidate actions implicit, while classical safety fields prescribe risk shapes and weights in advance rather than adapting them to each scene.

    Source: §I · p. 1

  • Method

    READ learns a continuous, differentiable spatiotemporal risk field from complementary geometric and behavioral constraints and integrates it with end-to-end and VLA planners.

    Source: §I · pp. 1–2

  • Results

    Matched NAVSIM comparisons improve PDMS from 84.0 to 84.6 with TransFuser and from 89.1 to 90.0 with Drive-JEPA; READ reaches 90.7 PDMS in the reported VLA setting.

    Source: B. Main Results · p. 5

  • Takeaways & Limitations

    The learned field provides an explicit, adaptable spatial-risk representation that can be queried along trajectories and optionally used to refine them.

    Source: §I · p. 2, §V · p. 7

    The current dense spatiotemporal field may introduce unnecessary computation in large-scale settings.

    Source: §V · pp. 7–8

Abstract

from arXiv · show

Autonomous driving requires more than recognizing what is present in a scene: a planner must determine how road structure, surrounding agents, and their motion states should influence a future maneuver. Existing learning-based planners can capture these influences through latent scene features and trajectory decoders, but the relationship between environmental factors and candidate actions often remains implicit. This limits the ability to inspect, diagnose, or refine how scene context affects the safety of a predicted trajectory. Classical safety fields provide an explicit spatial representation of this relationship, but their risk shapes and relative weights are prescribed in advance and do not adapt to each scene. We introduce READ, a framework that learns an explicit, planning-aligned risk representation from complementary geometric and behavioral constraints. READ instantiates this representation as a continuous spatiotemporal field, enabling differentiable queries along candidate trajectories. The learned field connects scene understanding with action selection by encouraging predicted trajectories to align with low-risk regions, while retaining a differentiable interface for trajectory evaluation and refinement. READ integrates with both end-to-end planners and Vision-Language-Action models. Experiments on NAVSIM show consistent gains across matched end-to-end backbones and strong performance in a VLA setting; READ also achieves competitive results on NAVSIM v2. These results establish learned spatial risk as an explicit, adaptable representation for safe planning.

I. INTRODUCTION

Planners need to relate road structure and agent motion to the safety of future maneuvers, but learned planners often leave that relationship implicit. READ learns an explicit, scene-adaptive risk field from geometric and behavioral constraints and integrates it with end-to-end and VLA planning.

I. INTRODUCTION

  • Classical safety fields make spatial and temporal risk explicit, but require risk terms and parameters that may not suit each scene’s geometry and interactions.Their amplitudes, spatial decay, anisotropy, and relative source strengths are specified in advance.Source: §I · p. 1
  • Learned scene features and trajectory decoders capture geometric, temporal, and semantic priors, yet often leave environmental influences on candidate actions implicit.The same semantic region can have different motion implications, such as free space beside a fast-moving vehicle versus an empty lane.Source: §I · p. 1
  • READ learns an explicit, planning-aligned continuous spatiotemporal risk field from geometric and behavioral constraints rather than prescribing complete field geometry.Non-drivable regions and future agent occupancy provide geometric anchors; expert trajectories and deviations from inertial motion provide behavior-dependent evidence.Source: §I · p. 1
  • The shared field adapts its spatial support to each scene while encouraging predicted trajectories to align with low-risk regions.It retains continuity and locality, including different levels of influence in physically unoccupied regions.Source: §I · p. 2, Figure 1 · p. 2
  • READ augments end-to-end planners and VLA models, and its differentiable field can assess trajectories or optionally guide selective refinement.The paper reports consistent gains with matched end-to-end backbones and strong results in a VLA setting on NAVSIM.Source: §I · p. 2

II. RELATED WORK

Prior driving planners use learned scene representations, occupancy, or structured fields to connect perception with action. READ builds on this work by learning scene-conditioned field geometry while retaining a continuous cost that can be queried along trajectories.

End-to-end autonomous driving

  • End-to-end, predictive, and multimodal driving systems learn powerful planning features, but spatial risk is often implicit or represented only through geometric occupancy.READ adds a continuous cost representation that can regularize shared features and be queried along arbitrary trajectories.Source: §II · p. 2

Vision-Language-Action models for driving

  • Driving VLA approaches connect scene understanding and planning through language, visual reasoning, latent world-action models, or scene-to-goal hierarchies.READ complements these mechanisms with risk-structured latent tokens that retain continuous spatial meaning.Source: §II · p. 2

Safety fields and learned planning costs

  • Classical and learned safety fields make spatial influence useful for planning, but analytic kernels and coefficients remain prescribed rather than scene-conditioned.One-4-All learns a geodesic potential for image-goal navigation, while FlowDrive uses risk-potential and lane-attraction fields for driving [14].Source: §II · p. 2

III. METHOD

READ jointly learns a continuous spatiotemporal risk field and a planning policy from partial geometric and behavioral evidence. Its smooth, differentiable representation is designed to remain useful across planning architectures and for trajectory queries or refinement.

III. METHOD

  • READ learns a continuous spatiotemporal risk field jointly with the planning policy, integrating it with two representative architectures and optionally refining trajectories.The same field supplies structured gradients to end-to-end and VLA planners.Source: §III · p. 2, Figure 2 · p. 3

A. Preliminaries

  • Unlike a complete hand-designed analytic field, READ uses selected map, occupancy, demonstrated-path, and kinematic-residual constraints while learning the remaining geometry with planning.This retains continuity, differentiability, and spatial locality while placing fewer assumptions on how risk decays around each source.Source: A. Preliminaries · p. 3

B. Adaptive Spatiotemporal Risk Field

1) Gaussian Parameterization:

  • Anisotropic Gaussian primitives on a grid produce smooth, differentiable field queries, with learned scales controlling the spatial extent of risk.Tighter influence can represent static structure and broader influence uncertain moving agents; time-indexed weights and scales let the landscape evolve with predicted interactions.Source: 1) Gaussian Parameterization · p. 3

2) Partial Safety Constraints:

READ combines geometric anchors and behavior-derived samples to shape risk without assigning a target value to every location. The constraints jointly guide the field while sparsity discourages unsupported high risk.

2) Partial Safety Constraints:

  • Map annotations impose high risk on non-drivable cells, while projected future agent boxes impose high risk at their annotated time-specific locations.These provide objective geometric anchors without prescribing how far risk extends into nearby free space.Source: 2) Partial Safety Constraints · pp. 3–4
  • A global sparsity prior discourages the degenerate high-risk-everywhere solution but does not label unannotated regions as safe.Expert waypoints are sampled low-risk behavioral constraints, not evidence that every unvisited location is unsafe.Source: 2) Partial Safety Constraints · pp. 3–4
  • A constant-velocity counterfactual path supplies a kinematic-residual constraint whose distance from the demonstration is normalized using a training-set percentile scale.The residual target is matched at the queried point and down-weighted for heading changes, since intentional turns can create large residuals without indicating hazards.Source: 2) Partial Safety Constraints · p. 4
  • The supervision losses balance evidence sources, while predicted Gaussian weights and scales—not the loss coefficients—determine spatial influence.Joint prediction reconciles constraints in a common surface whose support can change with object arrangement, time horizon, and planning context.Source: 2) Partial Safety Constraints · pp. 3–4

C. Integration with Driving Models

READ uses the same field definition in two architectures to assess whether it is useful beyond a single planner design.

C. Integration with Driving Models

1) Risk-Field Auxiliary Learning for E2E Planning:

For BEV-based planners, READ adds a structured field objective to a shared representation while keeping trajectory prediction as a direct inference head.

1) Risk-Field Auxiliary Learning for E2E Planning:

2) Risk-Structured Latent Supervision for VLA:

READ supervises risk latents and trajectory queries in a VLA model, and its continuous field can also guide post-hoc trajectory refinement.

2) Risk-Structured Latent Supervision for VLA:

D. Differentiable Trajectory Refinement

  • READ’s continuous field also permits post-hoc guidance of trajectories from READ or an external planner.Source: D. Differentiable Trajectory Refinement · p. 4
  • Refinement is limited to trajectories whose initial average risk exceeds ρ and optimizes waypoint positions and headings without retraining or re-querying the policy.The trajectory loss preserves the initial prediction while enforcing heading consistency, smoothness, acceleration and jerk regularity, and forward progress.Source: D. Differentiable Trajectory Refinement · p. 4

IV. EXPERIMENTS

The experiments evaluate READ on NAVSIM and NAVSIM v2, using aggregate planning metrics and controlled representation and constraint ablations on the original NAVSIM protocol.

A. Experimental Setup

  • NAVSIM’s PDMS aggregates no at-fault collision, drivable-area compliance, ego progress, time-to-collision, and comfort.Source: A. Experimental Setup · p. 4
  • Representation and constraint ablations use the original NAVSIM protocol so supervision changes share the same evaluator.Source: A. Experimental Setup · p. 4
  • PDMS combines collision and drivable-area penalties with weighted driving-quality terms for progress, time-to-collision, and comfort.Source: A. Experimental Setup · p. 4
  • Table I identifies PDMS as the primary aggregate metric and marks rows that add READ to a corresponding architecture.READ’s representation is risk fields rather than dense semantic BEV supervision.Source: Table I · p. 5
  • The study trains BEV-based E2E planners and a Qwen3-VL-2B VLA model, with shared field and trajectory representations optimized jointly.E2E models train for 40 epochs and the VLA model for 20 epochs.Source: A. Experimental Setup · p. 5

B. Main Results

READ improves matched end-to-end planning results and performs strongly in VLA and NAVSIM v2 settings. The ablations support scene-adaptive field learning and complementary risk constraints, while trajectory evaluation and refinement provide additional uses.

B. Main Results

  • Matched E2E gains raise PDMS from 84.0 to 84.6 and 89.1 to 90.0; READ reaches 90.7 PDMS in VLA and 90.1 EPDMS on NAVSIM v2, where matched field supervision raises EPDMS by 0.7.On NAVSIM v2, the matched comparison also improves TTC, NC, DAC, DDC, and LK; EP, TLC, and EC remain close.Source: B. Main Results · p. 5, Table II · p. 6

C. More Analysis and Ablations

  • Adding a READ decoder changes PDMS from 89.06 to 89.23, while eight-step BEV prediction reaches 89.13 over the same four-second planning window.These controls test whether the gain follows merely from an additional decoder or a longer prediction horizon.Source: C. More Analysis and Ablations · pp. 5–6, Table III · p. 6
  • Under the same ViT planner, the rule-based field reaches 89.49 PDMS, while READ reaches 90.02 and adds 0.53 over that fixed-field result.Both field approaches improve on the BEV baseline of 89.06; READ learns scene-dependent geometry from partial constraints.Source: C. More Analysis and Ablations · p. 6, Table III · p. 6
  • Map and future-agent terms anchor hazards, expert paths anchor low risk, kinematic residuals query deviations, and sparsity discourages unsupported activation without dense free-space labels.Unqueried free space is not forced into a binary safe-or-unsafe class.Source: C. More Analysis and Ablations · p. 6
  • Removing any risk-field constraint lowers aggregate score; TTC falls from 96.36, with the largest drop to 95.76 when the future-agent term is removed.The authors identify TTC as the strongest and most consistent constraint-ablation effect.Source: C. More Analysis and Ablations · p. 6, Table IV · p. 6
  • Figure 4 contrasts semantic reconstruction with continuous risk and shows field gradients moving a selected high-risk prediction while trajectory regularization limits deviation.READ represents graded, scene-conditioned influence around static structure and dynamic agents.Source: Figure 4 · p. 7
  • A sparsity weight of 0.5 performs best; most predicted trajectories end in near-zero-risk regions, but this distribution is a consistency diagnostic, not calibrated collision probability or a safety guarantee.The remaining high-risk cases are identified as candidates for selective refinement.Source: Figure 3 · p. 6, C. More Analysis and Ablations · p. 7, Table V · p. 7
  • Post-hoc field refinement improves TransFuser PDMS from 84.14 to 84.75 and DiffusionDrive from 88.08 to 88.43, with gains in every displayed component.The external planner is frozen and only its output waypoints are optimized.Source: C. More Analysis and Ablations · p. 7

D. Qualitative Analysis

READ represents planning-relevant influence as a continuous field rather than a reconstruction of semantic categories, and the qualitative analysis illustrates how that field can guide trajectory refinement.

D. Qualitative Analysis

  • READ represents continuous planning-relevant influence around static structure and dynamic agents, unlike BEV supervision, which reconstructs semantic categories.The field extends into nearby free space and can express graded margins that vary with the surrounding configuration.Source: D. Qualitative Analysis · p. 7
  • The field’s scene-conditioned margins can be queried at arbitrary trajectory points, unlike a fixed analytic profile applied to every object.The comparison with semantic occupancy highlights that READ adapts its field geometry to local driving context rather than merely producing a denser reconstruction.Source: D. Qualitative Analysis · p. 7
  • In the refinement visualization, successive local-gradient updates reduce exposure to a high-value region while regularization preserves smoothness and forward progress.The procedure is optional, operates on trajectories from frozen planners, and illustrates optimization without changing the planner; it does not establish field calibration or closed-loop safety.Source: D. Qualitative Analysis · p. 7, Table VI · p. 7

V. CONCLUSION

READ learns a differentiable spatiotemporal risk field from geometric and behavior-derived constraints for integration into end-to-end and VLA planners. Matched NAVSIM experiments show consistent improvements, while the current dense field may add unnecessary computation at large scale.

V. CONCLUSION

  • READ learns a differentiable spatiotemporal field from geometric and behavior-derived constraints and integrates with end-to-end and VLA planners.The field is presented as a compact planning-aligned representation beyond dense BEV reconstruction and can optionally refine trajectories from external planners.Source: §V · pp. 7–8
  • Matched NAVSIM experiments show consistent improvements, and rule-based control confirms the value of explicit field supervision and adaptive field geometry.The conclusion also identifies optional trajectory refinement for external planners as a use of the learned field.Source: §V · p. 7
  • The dense spatiotemporal field may introduce unnecessary computation in large-scale settings; richer supervision such as intervention or takeover data is proposed for future work.The proposed signals are intended for modeling safety-critical driving behaviors.Source: §V · pp. 7–8
Loading 2609.12371v1…