This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Evaluating Automated Vehicle Scenario Navigation Using the Operational Safety Assessment (OSA) Methodology
Technical Paper
2023-01-0797
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
The operational safety of Automated Driving System-equipped vehicles (AVs) is a critical issue with AVs being deployed on public roads. Methodologies for evaluating the operational safety are therefore necessary to maintain public safety. One possible approach is a safety case established by the AV developer that uses evidence to support a structured argument that the AV exhibits a given level of operational safety. One of the key components of a safety case for AVs is a set of testing results showing behavioral competency in a variety of scenarios within the AV’s operational design domain (ODD). The Institute of Automated Mobility (IAM) has previously published operational safety assessment (OSA) metrics along with a means to evaluate the severity of violations of the safety envelope-type OSA metrics for navigation of individual scenarios in the proposed OSA Methodology. The objective of the OSA Methodology is to objectively quantify the safety performance of each scenario navigated by the AV, with the aggregate quantifications to be included in the AV’s safety case. This work expands on the introduction of the OSA Methodology to (1) include more OSA metrics (beyond the safety envelope type) with clearly defined formulations for the severity of metric violation formulations, and (2) propose a methodology for interpreting the various OSA metric violation measurements to result in the desired quantification of the AV’s safety performance of a scenario navigation that is composed of evaluations of collision events, near-miss events, and non-collision events. The OSA Methodology is implemented in various simulation software and in real-world testing results at an IAM testbed to validate the concept. The results show a promising test result interpretation methodology that can be incorporated into a safety case framework to help ensure AV operational safety.
Citation
Como, S. and Wishart, J., "Evaluating Automated Vehicle Scenario Navigation Using the Operational Safety Assessment (OSA) Methodology," SAE Technical Paper 2023-01-0797, 2023, https://doi.org/10.4271/2023-01-0797.Also In
References
- Automated Vehicle Safety Consortium AVSC Information Report for Adapting a Safety Management System (SMS) for Automated Driving System (ADS) SAE Level 4 and Level 5 Testing and Evaluation SAE Industry Technologies Consortia 2021
- SAE International 2022
- International Organization for Standardization 2018
- International Organization for Standardization
- Shalev-Shwartz , S. , Shammah , S. and Shashua , A. 2017
- Weng , B. , Rao , S. , Deosthale , E. , Schnelle , S. and Barickman , F. 2020
- https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/disengagement-reports/
- Wishart , J. , Como , S. , Elli , M. , Russo , B. et al. Driving Safety Performance Assessment Metrics for ADS-Equipped Vehicles SAE Int. J. Adv. & Curr. Prac. Mobility 2 5 2020 2881 2899 https://doi.org/10.4271/2020-01-1206
- Automated Vehicle Safety Consortium AVSC Best Practice for Metrics and Methods for Assessing Safety Performance of Automated Driving Systems (ADS) SAE Industry Technologies Consortium 2021
- SAE International
- Como , S. , Wishart , J. , Elli , M. , and Kidambi , N. Evaluating the Severity of Safety Envelope Violations in the Proposed Operational Safety Assessment (OSA) Methodology for Automated Vehicles SAE Technical Paper 2022-01-0819 2022 https://doi.org/10.4271/2022-01-0819
- Viano , D.C. and Parenteau , C.S. Severe Injury to Near- and Far-Seated Occupants in Side Impacts by Crash Severity and Belt Use Traffic Injury Prevention 11 2010 69 78
- https://community.geotab.com/s/product-guide?language=en_US
- University of California PATH Program 2016 https://www.nspe.org/sites/default/files/resources/pdfs/Peer-Review-Report-IntgratedV2.pdf
- Elli , M. , Wishart , J. , Como , S. , Dhakshinamoorthy , S. et al. Evaluation of Operational Safety Assessment (OSA) Metrics for Automated Vehicles in Simulation SAE Technical Paper 2021-01-0868 2021 https://doi.org/10.4271/2021-01-0868
- Lu , D. , Jammula , V.C. , Como , S. , Wishart , J. , et al. CAROM - Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures International Conference on Robotics and Automation (ICRA) 2021
- Como , S. Operational Safety Assessment Methodology Framework: An Approach to Quantifying Automated Vehicle Safety Arizona State University 2022