This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Contributing Factors to Consider While Defining Acceptance Criteria and Validation Targets for Assuring SOTIF in Autonomous Vehicles
Technical Paper
2022-01-0065
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
Two major steps involved in SOTIF analysis are defining acceptance criteria and estimating the validation target. While acceptance criteria aids in determining if we have an acceptable residual risk corresponding to a hazardous scenario, the validation target specifies the amount of testing effort (in hours or representative miles) that is needed to ensure that the acceptance criteria are met. The current approaches for defining acceptance criteria heavily rely on existing fatality databases or naturalistic driving study data sets. The criterion is selected based on average number of fatalities or crashes per mile or per one hour of operation. The validation target is then calculated based on acceptance criteria. However respective validation targets., are these values really reflecting the acceptable risk criteria and targets? According to statistics, for a given data set and a random sample derived from the dataset, only the mean of population of the data set and the sample can be considered equal. In the case of autonomous vehicles, this implies we can only generalize the mean of crash statistics of all vehicles in a country to a mean of crash statistics of a sample fleet of vehicles. However, the current acceptance criteria consider arithmetic mean across states but not vehicles, and often do not take into account the operational design domain (ODD) factors, the driver characteristics, and justifications behind their selection. Similarly, while the validation target is defined from the acceptance criteria there are no minimal requirements with respect to scenarios that are stated which needs be met within the target. To overcome these limitations, in this paper, we discuss the various factors that need to be weighed in for defining acceptance criteria and validation target. We illustrate with an example how the results can vary drastically when the factors we defined are considered and compare our results with the currently adopted methods.
Authors
Topic
Citation
Madala, K., Krishnamoorthy, J., Avalos Gonzalez, C., Shivkumar, A. et al., "Contributing Factors to Consider While Defining Acceptance Criteria and Validation Targets for Assuring SOTIF in Autonomous Vehicles," SAE Technical Paper 2022-01-0065, 2022, https://doi.org/10.4271/2022-01-0065.Also In
References
- Kalra , N. and Paddock , S.M. Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? Santa Monica, CA RAND Corporation 2016 https://www.rand.org/pubs/research_reports/RR1478.html
- Böddeker , B. , Wendorff , W. , Nguyen , N. , Diehl et al. Automated Driving Safety-The Art of Conscious Risk Taking-Minimum Lateral Distances to Pedestrians 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE) 2021 1466 1471
- Kläs , M. , Adler , R. , Jöckel , L. , Groß , J. et al. 2021
- Ringdorfer , M. and Griessnig , G. Speed-Up Testing by Application of Semiformal Notations and Automation European Conference on Software Process Improvement 457 472 2021
- Zaeske , W. and Durak , Y. Leveraging Semi-formal Approaches for DepDevOps International Conference on Computer Safety, Reliability, and Security 2020 217 222
- Gauerhof , L. , Munk , P. , and Burton , S. Structuring Validation Targets of a Machine Learning Function Applied to Automated Driving International Conference on Computer Safety, Reliability, and Security 2018 45 58
- Xu , Z. , Zhang , K. , Min , H. , Wang , Z. et al. What Drives People to Accept Automated Vehicles? Findings From a Field Experiment Transportation Research Part C: Emerging Technologies 95 2018 320 334
- Zhang , T. , Tao , D. , Qu , X. , Zhang , X. et al. Automated Vehicle Acceptance in China: Social Influence and Initial Trust Are Key Determinants Transportation Research Part C: Emerging Technologies 112 2020 220 233
- Nastjuk , I. , Herrenkind , B. , Marrone , M. , Brendel , A.B. et al. What Drives the Acceptance of Autonomous Driving? An Investigation of Acceptance Factors from an End-User's Perspective Technological Forecasting and Social Change 161 2020 120319
- Rezaei , A. and Caulfield , B. Examining Public Acceptance of Autonomous Mobility Travel Behaviour and Society 21 2020 235 246
- Berrada , J. , Mouhoubi , I. , and Christoforou , Z. Factors of Successful Implementation and Diffusion of Services Based on Autonomous Vehicles: Users’ Acceptance and Operators’ Profitability Research in Transportation Economics 83 2020 100902
- NHTSA https://www.nhtsa.gov/research-data/fatality-analysis-reporting-system-fars
- SHRP2 https://insight.shrp2nds.us/login/auth
- Schäbe , H. Different Principles Used for Determination of Tolerable Hazard Rates Conference Proceeding 1 2001 435 442
- Kron , H. On the Evaluation of Risk Acceptance Principles 19th Dresden Conference on Traffic and Transportation Science 2003
- https://www.iihs.org/topics/fatality-statistics
- https://www.bts.gov/content/number-us-aircraft-vehicles-vessels-and-other-conveyances
- Milne , A. The Centric Systematic Area-Sample Treated as a Random Sample Biometrics 15 2 1959 270 297
- WSDOT https://remoteapps.wsdot.wa.gov/highwaysafety/collision/data/portal/public
- US DOT https://www.volpe.dot.gov/news/how-much-time-do-americans-spend-behind-wheel
- Wallace , D.L. Conditional Confidence Level Properties The Annals of Mathematical Statistics 30 4 1959 864 876
- NHTSA https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/813141
- FHA https://www.fhwa.dot.gov/policyinformation/statistics/2019/mv1.cfm
- Census , U.S. https://data.census.gov/cedsci/table?tid=ACSDP5Y2019.DP04&g=0400000US53_1600000US5363000
- Balk , G. Seattle Times https://www.seattletimes.com/seattle-news/data/seattles-car-population-has-finally-peaked./