PaperlayerView exact revision on arXiv

Source-linked AI summary

Predictive Control for Driving under Uncertain Road Geometry Estimated from Onboard Vision

Jelena Trisovic, Andrea Carron, Melanie N. Zeilinger

arXiv:2508.02494v2eess.SYcs.ROcs.SY

TL;DR

When road-geometry perception error is large relative to lateral margins, treating the estimated reference as certain can undermine safety constraints. The paper combines curvature-based road estimation with sampled geometric uncertainty in constrained control. Its estimation results show improved curvature accuracy and 100% success on both tracks for the initialized proposed method.

  • Problem

    When geometric estimation errors approach admissible lateral margins, treating perception-derived references as certain may leave safety constraints unsatisfied.

    Source: §I · pp. 1–2

  • Method

    The framework estimates road curvature from centerline points and represents perception uncertainty through plausible geometry samples used in shared, constrained control.

    Source: §I · pp. 1–2

  • Results

    The initialized proposed curvature estimator matches the naive baseline in Cartesian reconstruction accuracy, improves curvature MAE, and is the only configuration with a 100% success rate on both tracks.

    Source: 1) Simulation Results · p. 11

  • Takeaways & Limitations

    Simulation and real-world validation support curvature-based centerline estimation compatible with downstream control and the use of multiple centerline realizations to account for perception errors.

    Source: §IX · p. 14

    Real-time problem sizing is supported for moderate Ns; curvature estimation runs slower than the controller and is triggered only when sufficient new information is available.

    Source: §IX · p. 14

Abstract

from arXiv · show

Autonomous vehicles driving on unknown roads must estimate the road geometry from onboard sensors and follow the resulting reference while respecting road boundaries. When the perception-induced estimation error is non-negligible relative to the lateral constraint margins, treating the estimated reference as ground-truth can lead to safety constraint violations. This paper proposes a perception-based control framework that integrates road geometry estimation and the resulting geometric uncertainty with constrained control. A parametric model of the road curvature is identified from RGB-D measurements via constrained nonlinear optimization, enforcing geometric consistency. The residual perception uncertainty is then captured by constructing a set of curvature profiles consistent with measurements by perturbing the estimated parameters. Using the Frenet-frame vehicle model, the curvature uncertainty is propagated through the model via a scenario model predictive control scheme as parametric model uncertainty, thus enforcing constraints across all sampled curvature realizations. The curvature estimation module is evaluated both in high-fidelity simulation and on real-world image data, confirming reliable operation under realistic visual and depth noise. The full perception-to-control pipeline is validated in simulation, demonstrating that the uncertainty-aware controller maintains tighter adherence to road boundaries compared to its nominal counterpart.

I. INTRODUCTION

When perception error approaches the available lateral margin, treating an estimated road reference as certain can compromise constraint satisfaction. The paper integrates curvature estimation, geometric uncertainty representation, and constrained control, addressing gaps in prior curvature-estimation and uncertainty-aware control work.

I. INTRODUCTION

  • When geometric estimation error is comparable to admissible lateral deviation, certainty-equivalence control may fail to satisfy safety constraints.The separation may be adequate when errors are small relative to road width, as on well-marked highways.Source: §I · p. 1
  • Frenet-frame dynamics treat reference curvature as a parameter, allowing perception error to enter control as parametric model uncertainty.Raw images are not inputs to the controller; it uses the perception-derived curvature model as an uncertain parameter.Source: §I · p. 1
  • The framework fits a geometrically consistent, bounded, twice-differentiable curvature model to centerline points and samples plausible profiles by perturbing its parameters.The model attenuates perception noise and is designed for compatibility with optimization-based control.Source: §I · p. 2
  • A scenario formulation optimizes shared control while imposing state constraints across sampled road geometries,.This integrates perception-induced geometric uncertainty into a shared control optimization.Source: §I · p. 2

II. RELATED WORK

  • Prior curvature methods can lack differentiable, bounded estimates needed by optimization-based control, and the cited literature does not jointly address estimation and downstream control,.The paper also notes that some methods assume one curvature value per image, limiting their fit to rapidly varying roads.Source: §II · p. 2
  • Compared with related robust curvilinear MPC approaches, the method adapts uncertainty to locally estimated geometry but does not provide formal guarantees.The cited alternatives require worst-case assumptions over full road geometry or explicit vehicle-model uncertainty sets [31],.Source: §II · p. 3

III. PROBLEM STATEMENT AND PRELIMINARIES

The problem is safe motion along an unknown planar road using locally perceived geometry despite uncertainty in that estimate. Curvature provides both a road representation and a reference that can be reconstructed in Cartesian coordinates.

III. PROBLEM STATEMENT AND PRELIMINARIES

  • The setup assumes known vehicle pose, a static environment without dynamic obstacles or occlusions, constant known road width, and bounded road curvature.Safety means remaining within road boundaries despite uncertainty in sensor-derived road geometry.Source: §III · pp. 3–4
  • The Frenet vehicle model uses road curvature as a parameter, linking estimated geometry to dynamics; the controller constrains road boundaries while maximizing progress.The uncertainty-aware controller extension is presented in Section VII.Source: §III · p. 4

A. Curvature-Based Road Representation

  • A curve is represented by curvature κ(s), the derivative of tangent angle with respect to arc length, and reconstructed in Cartesian coordinates by integration.The reconstruction uses initial coordinates and tangent angle; its discretization is written as c_k+1 = f_Δ(c_k, κ(s_k)).Source: A. Curvature-Based Road Representation · p. 4

B. Bicycle Model of a Car in the Frenet Frame

The bicycle model expresses car motion relative to a reference path in Frenet coordinates. Reference curvature enters the dynamics, while steering and drivetrain rates are the modeled control inputs.

B. Bicycle Model of a Car in the Frenet Frame

  • The Frenet state contains progress, lateral distance, heading, longitudinal and lateral velocities, yaw rate, steering angle, and drivetrain input.Progress is arc length along the reference; lateral distance and heading are defined relative to the path.Source: B. Bicycle Model of a Car in the… · p. 4
  • The vehicle dynamics include reference curvature κ(s), which enters the model alongside the vehicle’s physical dynamics.The tire forces use a simplified Pacejka model.Source: B. Bicycle Model of a Car in the… · p. 4
  • The modeled input is the rate of change of steering angle and drivetrain, enabling input-rate constraints that prevent abrupt control behavior.The physical inputs are steering angle and drivetrain input.Source: B. Bicycle Model of a Car in the… · p. 4

C. Model Predictive Contouring Control in Frenet Frame

The Frenet-frame controller treats uncertain road geometry as model uncertainty and uses scenario MPC to enforce constraints across plausible centerlines. The framework derives those centerlines from RGB-D measurements and represents perception uncertainty at the reconstructed-centerline level.

C. Model Predictive Contouring Control in Frenet Frame

  • The controller maximizes progress along the reference path while respecting road boundaries and vehicle dynamics.Its stage cost encourages progress while keeping lateral and orientation errors low; constraints also account for physical state and input limits.Source: C. Model Predictive Contouring Control… · p. 5
  • Scenario MPC enforces constraint satisfaction across all plausible road geometries consistent with perception accuracy.A shared control input is optimized across sampled curvature scenarios.Source: C. Model Predictive Contouring Control… · p. 5, §IV · p. 5

IV. METHOD OVERVIEW

  • RGB-D processing extracts centerline pixels, unprojects them to 3D, and transforms them into ordered 2D world-frame points under a flat-road assumption.The resulting point cloud is noisy, and analytical propagation of pixel-level noise through the perception pipeline is intractable.Source: §IV · p. 5
  • Perception uncertainty is represented as bounded discrepancy in reconstructed centerlines because curvature error maps nonlinearly to Cartesian error.The centerline is the quantity directly enforcing lateral constraint satisfaction.Source: §IV · p. 6
  • A nominal curvature model is fitted, a bounded neighborhood captures residual geometric uncertainty, and the resulting centerlines enter Frenet dynamics and scenario MPC.The bounded neighborhood accounts for imperfect model fitting or insufficient modeling expressiveness.Source: §IV · p. 6

V. CURVATURE ESTIMATION

The curvature-estimation pipeline fits a finite-horizon parametric road model to centerline measurements and updates it as new road segments are observed. Its sigmoid representation encodes curvature transitions while providing a differentiable, bounded profile for uncertainty sampling and control.

V. CURVATURE ESTIMATION

  • The estimation pipeline fits a nominal centerline from measurements and updates it with newly observed points while preserving temporal consistency.It uses constrained Cartesian-coordinate fitting at initialization and combines existing geometry with new observations at later steps.Source: §V · p. 6

A. Road Curvature Model

  • Curvature is modeled as a sum of sigmoid functions, each approximating one differentiable curvature transition.The sigmoid parameters have direct geometric interpretations, and the profile includes an additional curvature offset.Source: A. Road Curvature Model · p. 6
  • The sigmoid profile is twice differentiable and bounded, supporting well-posed Frenet coordinates and compatibility with contouring MPC.Its low-dimensional parameters have admissible ranges based on road-design specifications, enabling explicit geometric constraints and direct parameter-space sampling.Source: A. Road Curvature Model · p. 6
  • The representation reconstructs Cartesian centerline points from curvature and associates each point with its tangential angle.Its finite spatial horizon and maximum number of sigmoids are selected based on road scale, sensor field of view, and expected vehicle speed.Source: A. Road Curvature Model · pp. 6–7

B. Initial Curvature Estimation

Initial curvature estimation fits a constrained parametric road model to noisy centerline observations. It jointly estimates curvature parameters, initial tangent angle, and map points so reconstructed points approximate measurements while satisfying geometric consistency.

B. Initial Curvature Estimation

  • At initialization, the optimization estimates sigmoid curvature parameters, initial tangential angle, and map points constrained to the parametric road model while approximating noisy observations.The objective combines measurement fit with optional curvature-parameter regularization.Source: B. Initial Curvature Estimation · p. 7
  • Geometric constraints order curvature transitions, bound their arc-length locations, and keep curvature changes and absolute curvature within valid ranges.The number of sigmoids is inferred from detected curvature transitions or fixed as a hyperparameter.Source: B. Initial Curvature Estimation · p. 7
  • Constraining reference curvature to at most 1/ηmax satisfies the condition for a unique Frenet-frame representation.ηmax is the vehicle's maximal allowed lateral distance.Source: B. Initial Curvature Estimation · p. 7

C. Receding-Horizon Map Update

The map update uses a finite receding spatial horizon rather than a global map. It conditionally combines retained map points with new measurements, keeping the active representation bounded and avoiding accumulation of long-term estimation errors.

C. Receding-Horizon Map Update

  • A receding spatial horizon prevents unbounded growth in model parameters and avoids long-term estimation-error accumulation associated with global mapping.Continuous updating with current perception data avoids drift effects associated with global map building.Source: C. Receding-Horizon Map Update · p. 7
  • An update proceeds only when measurements contain enough overlapping points for consistency and enough novel points to add information.Otherwise, the existing curvature estimate is retained; successful updates optimize over retained map points and new observations.Source: C. Receding-Horizon Map Update · pp. 7–8
  • Successful updates retain only the road-ahead map segment and discard curvature transitions behind the vehicle, keeping active basis functions bounded.If updates do not succeed for a prolonged period, the map can be discarded and estimation reinitialized from the current observation.Source: C. Receding-Horizon Map Update · p. 7

D. Curvature Estimation Summary

The curvature estimator updates a locally retained map from new centerline measurements, then represents geometric uncertainty with bounded centerline samples generated by parameter perturbations. The sampling uses Hausdorff distance to enforce the geometric bound and provides finite scenarios for control.

D. Curvature Estimation Summary

  • At each time step, the estimator updates the curvature model from sufficiently novel centerline points while retaining relevant map segments and skipping uninformative measurements.It outputs the curvature model and reconstructed centerline for uncertainty sampling.Source: D. Curvature Estimation Summary · p. 8

VI. UNCERTAINTY SAMPLING BASED ON HAUSDORFF DISTANCE

  • Candidate centerlines are generated by perturbing curvature parameters and retained only when their Hausdorff distance from the nominal centerline satisfies the prescribed geometric bound.Representative samples may be selected by favoring candidates farthest from the nominal centerline.Source: §VI · p. 8
  • Perturbing all curvature parameters within the spatial horizon avoids assuming where uncertainty arises, while the receding-horizon map bounds sampling dimensionality.Nrep governs candidate generation and Ns the scenarios passed to the controller, exposing a trade-off between coverage and computational complexity.Source: §VI · pp. 8–9
  • The uncertainty model is set-based rather than probabilistic: the geometric bound Dk controls practical robustness, while covariance ΣΘk structures local parameter-space exploration.Dk and ΣΘk are fixed offline and should be scaled together to cover the uncertainty tube efficiently.Source: §VI · pp. 8–9
  • The finite, geometrically consistent centerlines provide a tractable way to propagate perception uncertainty, but formal theoretical connections to the scenario framework remain future work.Adapting Dk online from perception confidence and co-adapting ΣΘk is identified as a natural extension.Source: §VI · p. 9

VII. UNCERTAINTY-AWARE CONTROL

The controller applies one control sequence to parallel vehicle-model realizations associated with sampled centerlines, imposing constraints across their predicted trajectories. The formulation uses Frenet-frame structure to limit the growth in optimization size and tunes conservatism through the geometric uncertainty bound.

VII. UNCERTAINTY-AWARE CONTROL

  • Vehicle states are projected onto each sampled centerline once per time step, and the resulting Frenet states are reused throughout the optimization.The control formulation accepts any finite set of bounded, differentiable curvature profiles, regardless of how they were generated.Source: §VII · p. 9
  • A single control sequence is optimized across parallel vehicle realizations, each driven by a different sampled curvature profile, so constraints apply across plausible centerlines.The cost averages across the trajectories, and the formulation adds bounds that keep predicted progress within the currently perceived road segment.Source: §VII · p. 9
  • The geometric bound Dk governs conservatism: larger values expand admissible centerlines and produce more conservative actions, while smaller values reduce conservatism.The paper presents this as a trade-off based on estimated perception accuracy.Source: §VII · pp. 9–10

VIII. NUMERICAL RESULTS

The numerical evaluation uses synthetic tracks and a simplified vehicle-perception setting to isolate perception-induced geometric uncertainty. The study also reports that its full simulation pipeline reduces lateral deviation from the true centerline relative to the nominal controller.

VIII. NUMERICAL RESULTS

  • The full simulation pipeline reports reduced lateral deviation from the true road centerline when the controller incorporates multiple centerline realizations rather than using the nominal controller.The section overview identifies this as the uncertainty-aware controller's result.Source: §VIII · p. 10

A. Simulation Setup

  • The evaluation uses two Blender-rendered tracks with piecewise-constant curvature, known constant width, visible centerlines, and no clutter or dynamic agents.The tracks include sharp turns and frequent curvature changes; only partial track information is available ahead, often requiring reduced speed.Source: A. Simulation Setup · p. 10, Figure 5 · p. 10
  • Vehicle pose is assumed known, and the tracks use simplified visual conditions to isolate perception-induced uncertainty in closed-loop control.The estimation errors in simulation arise from imperfect centerline pixel extraction and coordinate quantization; realistic image noise and depth inaccuracies are assessed separately on real-world data.Source: A. Simulation Setup · p. 10

B. Centerline Estimation

The initialized parametric curvature estimator performs best overall in the reported ablation, matching the naive baseline in Cartesian reconstruction accuracy while improving curvature error and completing estimation on both tracks. The evaluation also describes its point-cloud inputs and contrasts the continuous model with discrete finite-difference baselines.

B. Centerline Estimation

  • Centerline point clouds extracted from RGB-D measurements using edge drawing are processed by the curvature estimation procedure in simulation and on real-world images.The section describes edge drawing as demonstrated to generalize to realistic road imagery.Source: B. Centerline Estimation · pp. 10–11

1) Simulation Results:

  • The ablation evaluates ten perturbed trajectories per track and reports mean Hausdorff distance, mean absolute curvature error, and the fraction of trajectories achieving complete track estimation.The proposed method is compared with naive and smooth-naive baselines, with and without initialization.Source: 1) Simulation Results · p. 11
  • The naive baselines compute discrete pointwise curvature by finite differences, with the smooth variant first applying a Savitzky-Golay filter; neither directly supplies a bounded continuous curvature profile for MPC.Smoothing curvature itself is not pursued because it yields poor and unpredictable Cartesian reconstruction error.Source: 1) Simulation Results · p. 11
  • The initialized proposed method matches the naive baseline in Cartesian reconstruction accuracy, improves curvature MAE, and is the only configuration with 100% success on both tracks.The results also identify initialization as critical to the nonlinear optimization; the fitted curvature remains differentiable and bounded for Frenet-frame compatibility.Source: 1) Simulation Results · p. 11

2) Real-World Validation:

Real-world RGB-D tests show that curvature-based centerline estimation reduces large raw measurement errors, while scenario MPC accounts for residual uncertainty. Across tested configurations, the uncertainty-aware controller reduces lateral deviations and the calibrated Hausdorff bound yields the lowest worst-case deviation.

2) Real-World Validation:

  • The real-world test used offline RGB-D recordings from an F1TENTH vehicle, with only overlap thresholds and maximum map length adjusted from simulation.The laboratory track is 1m wide, approximately 20m long, and has piecewise-constant curvature; the adjustments accommodate its larger scale and noisier measurements.Source: 2) Real-World Validation · p. 12
  • The estimated centerline has 14cm mean Hausdorff distance, 0.4 m−1 mean absolute curvature error, and 29cm worst-case Hausdorff distance from ground truth.The worst-case distance over the first 2m ahead is 22cm; raw measurements have 15cm mean and 1.5m worst-case Hausdorff distance.Source: 2) Real-World Validation · p. 12

C. Sampling of Centerline Realizations

  • Sampling perturbed curvature parameters produces admissible centerline realizations; Dk = 10cm bounds accepted candidates, and control evaluation uses Nrep = 100.Smaller Dk tightens the admissible set, while larger values allow less reliable perception; Nrep = 500 is used for visualization.Source: C. Sampling of Centerline Realizations · p. 12, Figure 10 · p. 13

D. Uncertainty-Aware Control

  • As the number of centerline scenarios increases, both average and maximum lateral deviations from the ground-truth centerline decrease.Compared trajectories show the uncertainty-aware controller staying closer to the estimated centerline and away from regions where perception errors could cause constraint violations.Source: D. Uncertainty-Aware Control · p. 13, Table II · p. 14, Figure 11 · p. 14
  • Dk = 10cm yields the lowest worst-case lateral deviation; maximum lateral deviation η is 22.56cm at 5cm, 21.63cm at 15cm, and 22.36cm at 20cm.At Dk = 10cm, average velocity is 1.47m/s versus 1.49m/s for the nominal controller on Track A.Source: D. Uncertainty-Aware Control · pp. 13–14, Table II · p. 14

IX. DISCUSSION AND LIMITATIONS

The framework incorporates geometric uncertainty into control and demonstrates empirical robustness, but it does not provide formal safety guarantees. The discussion also identifies real-time feasibility results and future extensions for localization and broader road geometry.

IX. DISCUSSION AND LIMITATIONS

  • The framework combines curvature-based centerline estimation with multiple centerline realizations in control to account for perception errors.The paper reports simulation and real-world validation of the estimation approach and simulation validation of the full perception-to-control pipeline.Source: §IX · p. 14
  • The controller offers empirical robustness to bounded road-model deviations, not formal worst-case or probabilistic safety guarantees.Flat-road and known-width assumptions are not inherent to the control formulation; estimated width or 3D geometry could be incorporated, while banking requires a structurally different model.Source: §IX · p. 14
  • For Ns = 1, 5, and 10, average solve times are 4.6ms, 8.1ms, and 15.8ms, respectively, all below the 33ms sampling period.The simulation experiments use a Python/IPOPT prototype that is not optimized for real-time execution.Source: §IX · p. 14
  • Future work could relax the known-pose assumption using RGB-D geometric cues, point-cloud registration, or joint arc-length estimation.The discussion also identifies formal robustness or probabilistic safety guarantees under perception-induced uncertainty as an open direction.Source: §IX · pp. 14–15

Related papers

Loading 2508.02494v2…