This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Evaluation of Operational Safety Assessment (OSA) Metrics for Automated Vehicles Using Real-World Data
ISSN: 0148-7191, e-ISSN: 2688-3627
Published March 29, 2022 by SAE International in United States
Annotation ability available
Assurance of the operational safety of automated vehicles (AVs) is crucial to enable commercialization and deployment on public roads. The operational safety must be quantified without ambiguity using well-defined metrics. Several efforts are in place to establish an appropriate set of metrics that can quantify the operational safety of AVs in a technology-neutral way, including the Operational Safety Assessment (OSA) metrics proposed by the Institute of Automated Mobility (IAM). The focus of this work is to compute real-world measurements of the relevant safety envelope OSA metrics in car-following scenarios. This allows for an analysis of the impact of different parameters and thresholds and for an evaluation of the individual usefulness of the safety envelope OSA metrics. The current work complements prior IAM work involving evaluating the safety envelope OSA metrics in car-following scenarios in simulation. Video data were collected from infrastructure-based cameras at a traffic intersection in Anthem, AZ. Pairs of vehicles that either interact with each other or influence each other’s decision-making were identified. A methodology was developed using computer vision to localize the vehicles using the video data and fusing them with a map representation to obtain vehicle-vehicle relations and the maneuvers in which they are involved. Longitudinal conflicts in car-following scenarios were filtered to compute the safety envelope OSA metrics. Analysis of the safety envelope OSA metrics results were conducted to identify the usefulness of the various metrics in the car-following scenarios and to make a comparison to the observations from simulation.
CitationJammula, V., Wishart, J., and Yang, Y., "Evaluation of Operational Safety Assessment (OSA) Metrics for Automated Vehicles Using Real-World Data," SAE Technical Paper 2022-01-0062, 2022, https://doi.org/10.4271/2022-01-0062.
- Wishart , J. , Como , S. , Elli , M. , Russo , B. et al. Driving Safety Performance Assessment Metrics for ADS-Equipped Vehicles SAE Technical Paper 2020-01-1206 2020 10.4271/2020-01-1206
- Automated Vehicle Safety Consortium(AVSC) AVSC Best Practice for Metrics and Methods for Assessing Safety Performance of Automated Driving Systems (ADS) SAE Industry Technologies Consortia 2021
- Como , S. , Wishart , J. , Elli , M. , and Kidambi , N. Operational Safety Assessment (OSA) Methodology: A Proposed Approach to Quantifying the Safety Envelope for Automated Vehicle Safety 2022 SAE WCX
- Underwriters Laboratories (UL) ANSI/UL 4600 - Standard for Safety For the Evaluation of Autonomous Products, 2020
- Automated Vehicle Safety Consortium (AVSC) 2021
- Elli , M. , Wishart , J. , Como , S. , Dhakshinamoorthy , S. et al. Evaluation of Operational Safety Assessment (OSA) Metrics for Automated Vehicles in Simulation SAE Technical Paper 2021-01-0868 2021 10.4271/2021-01-0868
- Wassim , N. , Ranganathan , R. , Srinivasan , G. , Smith , J. et al. 2013
- Shalev-Shwatz , S. , Shammah , S. , and Shashua , A. 2017
- Dosovitskiy , A. , Ros , G. , Codevilla , F. , Lopez , A. et al. 2017
- Github.com github.com/carla-simulator/scenario_runner
- Altekar , N. , Como , S. , Lu , D. , Wishart , J. et al. Infrastructure-based Sensor Data Capture Systems for Measurement of Operational Safety Assessment (OSA) Metrics SAE Technical Paper 2021-01-0175 2021 10.4271/2021-01-0175
- Lu , D. , Jammula , V. , Como , S. , Wishart , J. et al. CAROM - Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on the Road Infrastructures IEEE International Conference on Robotics and Automation 2020
- Ding , W. , Li , S. , Zhang , G. , Lei , X. et al. Vehicle Pose and Shape Estimation Through Multiple Monocular Vision IEEE International Conference on Robotics and Biomimetics (ROBIO) 2018
- https://webapp.connectedvcs.com/isd/ https://webapp.connectedvcs.com/isd/