PaperlayerView exact revision on arXiv

Source-linked AI summary

Silence Is Not Assurance: When Production Lane Keeping Assist Does Not Warn, and What It Costs

Tianjia Yang, Yuxin Ding, Ke Li, Kaidi Liang, Ruwen Qin, Xianbiao Hu

arXiv:2610.31899v1cs.ROcs.HCcs.SYeess.SY

TL;DR

Whether production LKA’s displayed lane status reflects its perception has not been examined in real-world driving, where proprietary perception and the absence of a per-second ground-truth label complicate evaluation. CANDID-LKA compares the display with independent perception confidence and driver response. It finds that the display misses many degraded-perception seconds, and silent seconds with the driver not steering are associated with greater lateral-position variability.

  • Problem

    Whether production LKA’s driver-facing lane-detection status reflects the system’s perception has not been evaluated in real-world driving, and lane-marking perceivability has no per-second ground-truth label.

    Source: INTRODUCTION · pp. 1–2

  • Method

    CANDID-LKA compares the displayed status with independent lane-perception confidence and CAN-recorded driver response, estimating status completeness with a latent-class model.

    Source: INTRODUCTION · p. 2, External Test · p. 7

  • Results

    The status reports only 47% of degraded seconds as not detected, versus 80% for third-party low perception confidence, and misses not-detected reporting in half of low-confidence episodes.

    Source: CONCLUSIONS · p. 17

  • Takeaways & Limitations

    A detected status does not guarantee reliable lane perception, and the paper identifies silence while the driver is not in control as the case needing most attention.

    Source: INTRODUCTION · pp. 1–2

    The latent-class model assumes independent errors across three signals, although the production and third-party systems observe the same road and may have correlated errors.

    Source: CONCLUSIONS · p. 17

Abstract

from arXiv · show

Production lane keeping assist (LKA) systems tell the driver whether they detect the lane markings, but whether this displayed status reflects what the system perceives, and what happens when it does not, has not been examined in real-world driving. This study introduces CANDID-LKA, a CAN-based Diagnosis of the Indicated Detection status in LKA. It evaluates a proprietary driver-facing status using three signals: the displayed status and the steering torque of the driver, both recorded from the controller area network (CAN) bus of the vehicle, and the lane perception confidence of a third-party vision system observing the same road. The framework examines how the status responds to driving conditions and estimates how completely it reports degraded lane perception by comparing it with the other two signals in a latent-class model. We apply it to 39,800 seconds of real-world driving from 13 vehicle models of three manufacturers. The results show that the status does not respond consistently to the conditions that make markings harder to see, and that the three makes differ widely in how often they report not detected. The latent-class model shows that the displayed status is incomplete: it reports only 47% of degraded seconds as not detected, compared with 80% for the third-party low perception confidence, and in half of the low-confidence episodes it never reports not detected. In the silent seconds in which the status reads detected while the perception confidence is low and the driver is not in control, the standard deviation of lateral position over the next 3 s is about 60% larger than when the status and the perception confidence agree. A detected status thus does not guarantee reliable lane perception, and silence with the driver not in control is the case that most needs attention.

INTRODUCTION

The paper asks whether production LKA’s driver-facing lane-detection status reflects the system’s perception, despite proprietary perception systems and no per-second ground truth for lane-marking perceivability. CANDID-LKA compares the status with independent perception confidence and driver response, finding incomplete reporting and greater lateral-position variability during some silent seconds.

INTRODUCTION

  • A status that stays detected after perception degrades leaves the driver without a signal when lane departure is most likely; false not-detected reports instead risk nuisance alerts and eroded credibility.The paper describes the silent, degraded-perception case as more damaging than reporting not detected when perception is adequate.Source: INTRODUCTION · p. 2
  • The evaluation addresses a gap: prior work examined LKA perception and driver trust, but not whether the production status shown to drivers accurately reflects perception.The proprietary internal perception is unavailable, and lane-marking perceivability varies continuously, leaving no per-second ground-truth label.Source: INTRODUCTION · p. 2
  • CANDID-LKA evaluates the displayed status against independent lane-perception confidence and the driver’s CAN-recorded control response, using a latent-class model to estimate how completely it reports degraded perception.The framework also examines status response to difficult marking conditions and the outcomes of silent seconds.Source: INTRODUCTION · pp. 2–3
  • Half of sustained low-confidence episodes receive no not-detected report, and 3 s SDLP is about 60% larger in silent seconds when the driver is not steering than after agreement seconds.The study applies CANDID-LKA to 39,800 seconds of naturalistic driving from 13 production vehicle models.Source: INTRODUCTION · p. 3
  • CANDID-LKA separates how often the status switches to not detected from how long it remains there, because the share of not-detected time can conceal long outages or frequent short ones.The authors suggest differences in reporting across manufacturers partly reflect status-reporting logic, and that more complete reporting is achievable.Source: INTRODUCTION · p. 3

LITERATURE REVIEW

Prior research has assessed when lane perception degrades and how drivers respond to automation feedback, but has not established whether production LKA’s displayed detection status reflects that perception.

Lane Keeping Performance

  • Road tests, simulations, and naturalistic datasets examine lane-perception performance under marking, weather, and other conditions, but do not evaluate what the vehicle reports to the driver.Studies span controlled and instrumented-road testing, rare or extreme simulated conditions, and large-scale production-vehicle datasets.Source: Lane Keeping Performance · pp. 3–4

Perception Monitoring and Driver Trust

  • Perception-monitoring research flags unreliable perception within software or against curated benchmarks, while driver studies document reliance, miscalibrated trust, and mode confusion; the accuracy of the displayed status remains unestablished.The two lines address complementary requirements: signaling when lane perception fails and driver action on that signal.Source: Perception Monitoring and Driver Trust · p. 4

Multi-Source Reliability Estimation

CANDID-LKA evaluates the driver-facing lane status using vehicle, driver, and independent-vision signals. Its internal test examines whether not-detected reports track driving conditions, including how often they begin and how long they last.

Multi-Source Reliability Estimation

ANALYTICAL FRAMEWORK

  • For each second, the framework uses the vehicle’s displayed status and steering torque alongside third-party vision-system signals observing the same road.Those vision signals include lane perception confidence, lane position, and forward video.Source: ANALYTICAL FRAMEWORK · p. 5, Variables · p. 5

Internal Test

  • The internal test estimates not-detected reporting under driving conditions with logistic regression, fit across all systems and separately by make.Model fit is reported using AUC and McFadden’s pseudo-R2.Source: Internal Test · p. 5
  • The internal test also models transitions into and out of not-detected status, distinguishing how often reports begin from how long they persist.A reported outage is a maximal consecutive run of not-detected seconds, with mean duration 1/λrec(x).Source: Internal Test · pp. 5–6

External Test

The external test estimates status trustworthiness by comparing it with independent perception confidence and driver response. It also characterizes missed low-confidence episodes and examines lane-position outcomes after silent seconds.

External Test

  • Three model-free measures supplement the latent-class estimate: status-confidence agreement, episode miss rate, and driver-response risk ratio.Confidence intervals use a route-level bootstrap with 2,000 resamples.Source: External Test · pp. 7–8

Trustworthiness

  • A latent-class model treats displayed status, perception confidence, and driver response as imperfect indicators of degraded lane-marking perceivability.No signal is treated as a reference; the model assumes their errors are independent given the latent state.Source: External Test · p. 7
  • The model estimates displayed-status sensitivity and specificity from the observed agreement pattern among the three signals.The model is fit by maximum likelihood with an expectation-maximization algorithm from 10 random starts.Source: External Test · p. 7

External Test

  • A low-confidence episode is a consecutive stretch with confidence below 0.5 lasting at least two seconds; its miss rate is the share never reported not detected.Cohen’s kappa summarizes status-confidence agreement, with 0 indicating chance-level agreement and 1 perfect agreement.Source: External Test · p. 7

Silence outcome analysis

  • Silence-outcome comparisons measure 3 s SDLP and lateral excursion after silent versus agreement seconds, split by whether the driver is in control.Models adjust for speed, planned lateral acceleration, and vehicle model; not-in-control seconds receive further breakdowns by road conditions.Source: External Test · p. 8

DATA DESCRIPTION

The analysis uses synchronized naturalistic driving data from OpenLKA-Failure and retains 13 vehicle models from three manufacturers after screening. The dataset combines vehicle-bus signals, openpilot vision outputs, and scene labels.

DATA DESCRIPTION

  • OpenLKA-Failure contains 4,947 one-minute clips, about 82 hours, recorded with a windshield-mounted comma 3X device.Recordings combine a Tampa rental-vehicle campaign with community-contributed drives across the United States.Source: DATA DESCRIPTION · p. 8
  • Each clip synchronizes vehicle CAN data, openpilot lane-perception outputs, and scene labels extracted from forward video.Vision outputs include per-lane-line confidence, lane geometry, and lane-change state.Source: DATA DESCRIPTION · p. 8
  • Of 41 dataset models, 16 from three manufacturers broadcast a decodable driver-facing status; screening leaves 13 models for analysis.Toyota and Lexus provide discrete states, Hyundai an ordinal confidence level, and Honda a continuous probability.Source: DATA DESCRIPTION · pp. 8–9
  • After screening, the sample comprises 39,800 seconds across 853 clips and 89 routes.The retained data include 8 Toyota and Lexus models, 4 Hyundai models, and 1 Honda model.Source: DATA DESCRIPTION · p. 9

Response of the Displayed Status to Driving Conditions

Across systems, not-detected reports do not consistently rise under conditions that make markings harder to see, and vehicle model explains more status variation than the measured driving conditions. Responses also differ across makes.

Response of the Displayed Status to Driving Conditions

  • Vehicle-model-only prediction has AUC 0.73, compared with 0.80 when all driving conditions are included, indicating most status variation comes from vehicle model.Across all systems, neither road-curvature term is significant.Source: Response of the Displayed Status to… · pp. 9–10
  • On faded markings, Hyundai reports not detected in about 23% of seconds versus 14% on clear markings, while Honda reports 9% versus 5% and Toyota 21% versus 31%.Honda also reports more often on low-contrast markings (odds ratio 1.92, 95% CI 1.16 to 3.16).Source: Response of the Displayed Status to… · p. 9, Table 2 · p. 10
  • Occlusion has the largest all-systems odds ratio at 1.94, but its confidence interval includes 1; low-contrast and faded markings show little association with not-detected reports.Reports are significantly less frequent with a lead vehicle and at higher speed; the night-driving decrease is not statistically significant.Source: Response of the Displayed Status to… · p. 9, Table 2 · p. 10
  • On occluded markings, Toyota reports not detected in 55% of seconds versus 31%, while Honda reports it less often; perception confidence is low more often on every degraded marking type.Low confidence occurs in 18% of low-contrast, 13% of faded, and 24% of occluded seconds, versus 8% on clear markings.Source: Response of the Displayed Status to… · p. 10, Table 2 · p. 10

Switching Dynamics of the Displayed Status

The displayed status switches into and out of not-detected at different rates across marking conditions and makes, so time spent not detected does not directly measure marking difficulty.

Switching Dynamics of the Displayed Status

  • Occlusion is the only condition with a clear all-system effect: loss rate ratio 4.10 (95% CI 1.95 to 8.60), while recovery rate ratio 1.84 (0.78 to 4.33) is not statistically significant.Outages on occluded markings are frequent but comparatively short.Source: Switching Dynamics of the Displayed… · p. 10, Table 3 · p. 11
  • Toyota’s occlusion loss rate ratio is 8.10 (95% CI 3.73 to 17.60), and its mean observed outage shortens by half to about 12 s.Hyundai’s faded-marking outages have a median duration of 14 s versus 5 s on clear markings; 63% remain ongoing at clip end versus 49%.Source: Switching Dynamics of the Displayed… · pp. 10–11, Table 3 · p. 11
  • Honda outages last a median 3 to 4 s across conditions, while loss is more frequent on low-contrast markings (rate ratio 2.34, 95% CI 1.24 to 4.43).Its faded-marking loss rate ratio is 2.33 (0.73 to 7.44), not statistically significant.Source: Switching Dynamics of the Displayed… · p. 11, Table 3 · p. 11
  • Toyota’s faded-marking not-detected share falls from 31% to 21% despite a similar loss rate (0.95, 95% CI 0.43 to 2.06), as recovery is faster (1.57, 0.89 to 2.78).The recovery difference is not statistically significant; low-contrast and faded markings show little all-system association with switching rates.Source: Switching Dynamics of the Displayed… · pp. 10–11, Table 3 · p. 11

Trustworthiness of the Displayed Status

The status is an incomplete indicator of degraded lane-marking perceivability, and its agreement and missed-episode rates vary by make. Driver steering response also reflects road demand, not just what the status reports.

Trustworthiness of the Displayed Status

  • The displayed status detects only 47% of degraded seconds (95% CI 0.34 to 0.62), versus 80% for perception confidence; status sensitivity is 0.46 for Toyota, 0.44 for Hyundai, and 0.79 for Honda.The status specificity is 0.84; only 30% of not-detected reports fall in degraded seconds, while 92% of detected reports fall in non-degraded seconds.Source: Trustworthiness of the Displayed Status · pp. 11–12, Table 4 · p. 12
  • Status–perception agreement is low overall (Cohen’s kappa 0.21, 95% CI 0.13 to 0.30) and differs by make: 0.12 for Toyota, 0.26 for Hyundai, and 0.57 for Honda.The passage describes the overall agreement as only slightly above chance.Source: Trustworthiness of the Displayed Status · p. 12, Figure 1 · p. 13
  • In half of low-confidence episodes the status never reports not detected: this occurs in 50% of Toyota and 54% of Hyundai episodes, versus 10% for Honda.When a report occurs within an episode, it is generally immediate; the median delay is 0 s overall and 1 s for Honda.Source: Trustworthiness of the Displayed Status · pp. 12–13
  • A high-torque input follows silent seconds in 27% of cases versus 6% of agreement seconds, but after accounting for required lateral acceleration the ratio is 0.89 (95% CI 0.62 to 1.27), not statistically significant.The adjusted ratio after a not-detected report is 0.83 (0.67 to 1.02), also not statistically significant; the driver’s steering follows road requirements rather than the status.Source: Trustworthiness of the Displayed Status · p. 13

Outcomes of the Silent Seconds

Silent seconds are divided according to whether the driver is in control over the next three seconds. Lateral position varies substantially more after silence when the driver is not in control, while the in-control comparison is largely shaped by turns versus straight roads.

Outcomes of the Silent Seconds

  • Silent seconds are grouped as driver-controlled when mean torque over the following 3 s exceeds the vehicle’s median; otherwise the driver is classified as not in control.The threshold separates low torque while lane keeping steers from higher torque when the driver steers.Source: Outcomes of the Silent Seconds · p. 14
  • The driver is in control in 78% of silent seconds versus 48% of agreement seconds; these in-control silent seconds are mostly sharp turns (65%), unlike agreement seconds, which are mostly straight roads (56%).Adjusted 3 s SDLP is 5.2 cm higher after in-control silent seconds, and excursions exceed 50 cm in 75% of windows versus 22%.Source: Outcomes of the Silent Seconds · p. 14, Table 5 · p. 14
  • When the driver is not in control, median 3 s SDLP is 12.2 cm after a silent second versus 4.2 cm after agreement, an adjusted difference of +5.4 cm (95% CI 3.8 to 7.0), about 60% more.Lateral excursion exceeds 50 cm in 46% of silent windows versus 6% of agreement windows; on straight roads, SDLP is 6.4 cm higher and excursions exceed 50 cm in 36% versus 3%.Source: Outcomes of the Silent Seconds · p. 15, Table 6 · p. 15
  • No-control silent seconds occur mostly on urban roads (59%), and low-contrast markings account for 31% of them versus 18% of all driving time; added SDLP is 4 to 8 cm across condition classes.The extra variability is similar across marking, lighting, and road-type classes, with overlapping confidence intervals.Source: Outcomes of the Silent Seconds · p. 15

Sensitivity Analysis

Across four sensitivity analyses, the main conclusions largely persist under changes to the confidence threshold, sampling interval, response window, and assumed dependence between signals. Correlated errors could make the displayed status appear more sensitive than it is.

Sensitivity Analysis

  • Changing the low-confidence threshold alters estimated sensitivity by at most 0.05 for all systems, Toyota, and Hyundai, while Honda remains between 0.71 and 0.86.A stricter 0.3 threshold raises sensitivity slightly; a looser 0.7 threshold lowers it.Source: Sensitivity Analysis · pp. 15–16
  • Every-fifth-second sampling yields 0.49 sensitivity across all systems, versus 0.47 in the full sample, with Toyota and Hyundai changing by no more than 0.02.Honda’s reduced-sample estimate of 0.95 is less stable because few low-confidence seconds remain.Source: Sensitivity Analysis · p. 16
  • Across 2-, 3-, and 5-second response windows, all-system status sensitivity stays between 0.46 and 0.47, while driver-response risk ratios are 1.72, 1.65, and 1.56.The silent-to-agreement ratios are 4.8, 4.3, and 3.7 across those same windows; window length changes measure sizes but not conclusions.Source: Sensitivity Analysis · p. 16
  • Allowing correlated errors lowers estimated status sensitivity by at most 0.12 for all systems, Toyota, and Hyundai, and 0.21 for Honda; Honda remains most sensitive.If the signals tend to be wrong together, the status misses at least about half of degraded seconds.Source: Table 7 · p. 16, Sensitivity Analysis · p. 16

Implications

The displayed status misses many degraded-perception seconds, and silent seconds when the driver is not in control coincide with greater lateral variation. The paper outlines how CANDID-LKA can assess these gaps and notes boundaries on the findings and directions for further validation.

Implications

  • CANDID-LKA can assess proprietary driver-facing status using displayed status, windshield-camera confidence, and steering, including outcomes after silent seconds.The paper suggests consumer ratings, regulatory testing, post-deployment audits, and monitoring as possible uses.Source: Implications · p. 17

CONCLUSIONS

  • The evaluation combines condition-sensitive status checks with a latent-class comparison against separate camera confidence and driver steering signals in 39,800 seconds of naturalistic driving.Silent-second outcomes are compared separately when the driver is and is not in control.Source: CONCLUSIONS · p. 17
  • The status reports only 47% of degraded seconds as not detected, versus 80% for third-party low perception confidence, and in half of low-confidence episodes never reports not detected.Sensitivity ranges from 0.44 to 0.46 for Hyundai and Toyota and is 0.79 for Honda; 22% of silent seconds occur when the driver is not in control, with 3 s SDLP about 60% larger than in agreement seconds.Source: CONCLUSIONS · p. 17
  • The latent-class model assumes independent errors, although shared road conditions may correlate the production and third-party systems’ errors.Lateral-movement results are interpreted as associations because lane-line readings are noisier at low confidence.Source: CONCLUSIONS · pp. 17–18
  • Future work proposes human-labeled scene validation, independent vehicle-path measurement, larger fleets and longer recordings, and applying the framework to other driver-facing reports.Examples include adaptive cruise control and automatic emergency braking availability indicators.Source: CONCLUSIONS · p. 18
Loading 2610.31899v1…