Source-linked AI summary
Using Automated Vehicles Operational Data to Confirm Safety and Anticipate Threats
Riccardo Donà, Espedito Rusciano, Germana Trentadue, Anastasios Tsakalidis, Maria Cristina Galassi
TL;DR
ADS safety assessment faces residual unknowns that traditional type approval and limited public-road testing cannot fully address. The paper examines ISMR, drawing on regulatory practices in nuclear energy and transportation, to use operational data for confirmation, scenario discovery, and shared safety learning. The framework is intended to close the safety-assessment loop and add proactivity, while its data specifications, event taxonomy, and scalability remain under development.
Problem
Traditional type approval and public-road testing provide limited coverage for the unknowns and confirmatory evidence required in ADS safety assessment.
Method
The paper examines an ISMR framework combining monitoring, reporting, and investigation of ADS operational data after type approval.
Results
ISMR is presented as a way to close the safety-assessment loop with real-world data, identify critical occurrences, and issue safety recommendations.
Takeaways & Limitations
Operational feedback can support proactive ADS safety assessment and collaboration between manufacturers and authorities.
Takeaways & Limitations
Detailed data elements, reporting frequency, event taxonomy, and use-case-specific ISMR approaches remain challenging to define.
Abstract
from arXiv · showhide
European Union (EU) policymakers adopted revolutionary data collection provisions for Automated Driving Systems (ADS) in the recently approved regulation that allows driverless vehicles to be operated on public roads. The framework is inspired by best practices developed at the United Nations Economic Commission for Europe(UNECE) level: the In-Service Monitoring and Reporting (ISMR); and by similar operational data collection regulatory approaches in nuclear energy production and transportation fields. The collection of real-world data will enable the competent safety authorities to gather the information needed to confirm the homologation safety target. Safety-relevant driving scenarios discovered during the real-world operation of a given ADS can also be stored in a scenario catalogue to investigate how other ADS types might have addressed such a traffic conflict. Moreover, lessons learnt deriving from the data collected can be shared among original equipment manufacturers (OEMs) and safety authorities. Ultimately, the ISMR is recognised as a necessary tool to properly tackle the challenges associated with ADS safety assessment given the number of unknowns that might remain undisclosed by leveraging the traditional homologation validation scheme only.
1 Introduction
ADS regulation is moving beyond traditional type approval by incorporating operational data collection through the ISMR framework. Real-world experience is intended to confirm safety, identify relevant scenarios, and support shared safety learning.
- Open-road testing alone would require billions of kilometres to gather confirmatory safety evidence, making the approach poorly scalable.
- Operational data collection enables a learning phase after certification, which is especially important because ADS are difficult to validate through traditional type approval alone.
- ADS experience can be shared across manufacturers to anticipate possible failures, adding a proactive component to safety assessment.
- The NATM begins with analysis of the ADS Operational Design Domain, including roadway type, weather, geography, and time of day.
- After requirements are demonstrated through audits and simulation, track, and real-world tests, ISMR collects operational data to confirm safety and identify new validation scenarios.
- The paper emphasizes data collection procedures as a source of confirmatory safety evidence unavailable through traditional testing methods alone.
2 Literature review
The literature review surveys monitoring and reporting practices in nuclear, aviation, railway, maritime, and road transportation. These precedents motivate ADS operational data collection while revealing limits of existing road-vehicle event-recording tools.
- Nuclear monitoring and reporting practices inspired ADS ISMR principles, including continuous monitoring, relevant-occurrence reporting, and distribution of lessons learned.
- ECCAIRS supports transportation-sector reporting through predefined taxonomies and pre-compiled templates, while road reporting remains more limited and historically affected by harmonisation issues.
- Aviation combines Flight Data Monitoring with reporting regulations, databases, taxonomies, and statistical safety reviews that can support safety recommendations.
- EDR/DSSAD can provide objective evidence for safety-critical events but remain limited by triggering conditions and their liability-assignment purpose.
- EDR/DSSAD cannot analyze near misses or proactively anticipate critical events, although ADS-collected data may support anticipative corrective actions.
- California and NHTSA ADS reporting requirements have begun providing authorities with information about ADS interaction with the transportation network.
3 Methodology
The proposed ISMR framework collects operational ADS data through monitoring, reporting, and investigation to address residual safety unknowns while remaining scalable and protecting confidential information.
- 3 Methodology: Operational data collection is needed because public-road testing has limited coverage and ADS operation involves unknown conditions outside controlled environments.The framework targets unknown unknowns that emerge after an ADS demonstrates compliance with NATM requirements.
- 3 Methodology: The ISMR combines continuous monitoring, event-based reporting, and independent investigation to collect operational evidence after ADS approval.These methods support the NATM framework and assign investigation of safety-critical events to an impartial competent body.
- 3.1 Monitoring: Monitoring continuously collects relevant data during normal operation to identify emerging trends, unsafe ODD conditions, and events in which the ADS prevents incidents.These uses support proactive safety evidence and positive lesson sharing.
- 3.1 Monitoring: Because storing all raw camera frames is not viable, monitoring uses processed sensor information while research continues on sufficient data elements and recording specifications.The design must provide evidence without overloading authorities’ data-processing capacity.
- 3.2 Reporting: A centralized database screens submissions for duplicates and relevant patterns, shares information across contracting parties, and preserves industry-sensitive confidentiality.The reporting scheme is intended to support a just culture rather than assign liability or blame.
- 3.2 Reporting: Reporting records specified occurrences through short-term and periodic approaches, with short-term reports due within one month and intended to support prompt remedial action.Short-term submissions combine a textual event summary with recorded ADS data, while structured templates aim to improve consistency.
4 Conclusions and Future Perspective
The paper presents ISMR as a way to address ADS safety-assessment challenges through operational data, while identifying unresolved implementation and scalability issues.
- ISMR addresses the need for statistical evidence to confirm achieved ADS safety levels and the scalability challenges of operational monitoring.
- The proposed AV scheme uses monitoring, reporting, and investigation while defining responsibilities for authorities and contracting parties.
- Real-world operational data closes the safety-assessment loop and supports identifying critical occurrences, issuing safety recommendations, and collaboration between manufacturers and authorities.
- Guideline development remains ongoing, with data-element specifications, collection frequency, event taxonomy, and differing AV use cases still challenging implementation.
List of abbreviations and definitions
The abbreviation list defines the principal systems, organizations, schemes, and data concepts used in the paper's discussion of automated-vehicle safety monitoring.
- ADS means Automated Driving System, while ALKS means Automated Lane Keeping System and AVP means Automated Valet Parking.
- DSSAD, FDM, ADREP, CADaS, and ECCAIRS denote data-storage, monitoring, reporting, or accident-information systems and datasets.
- ISMR means In-Service Monitoring and Reporting, NATM means New Assessment/Test Method, and ODD means Operational Design Domain.
- SMS denotes Safety Management System, OEM denotes Original Equipment Manufacturers, and AI denotes Artificial Intelligence.
- UNECE, IAEA, EASA, JRC, and ERCS denote organizations or schemes supporting the paper's regulatory and safety context.