Evaluating Automated Vehicle Scenario Navigation Using the Operational Safety Assessment (OSA) Methodology

The operational safety of Automated Driving System-equipped vehicles (AVs) is a critical issue with AVs being deployed on public roads. Methodologies for evaluating the operational safety are therefore necessary to maintain public safety. One possible approach is a safety case established by the AV developer that uses evidence to support a structured argument that the AV exhibits a given level of operational safety. One of the key components of a safety case for AVs is a set of testing results showing behavioral competency in a variety of scenarios within the AV’s operational design domain (ODD). The Institute of Automated Mobility (IAM) has previously published operational safety assessment (OSA) metrics along with a means to evaluate the severity of violations of the safety envelope-type OSA metrics for navigation of individual scenarios in the proposed OSA Methodology. The objective of the OSA Methodology is to objectively quantify the safety performance of each scenario navigated by the AV, with the aggregate quantifications to be included in the AV’s safety case. This work expands on the introduction of the OSA Methodology to (1) include more OSA metrics (beyond the safety envelope type) with clearly defined formulations for the severity of metric violation formulations, and (2) propose a methodology for interpreting the various OSA metric violation measurements to result in the desired quantification of the AV’s safety performance of a scenario navigation that is composed of evaluations of collision events, near-miss events, and non-collision events. The OSA Methodology is implemented in various simulation software and in real-world testing results at an IAM testbed to validate the concept. The results show a promising test result interpretation methodology that can be incorporated into a safety case framework to help ensure AV operational safety.