This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Bayesian Test Design for Reliability Assessments of Safety-Relevant Environment Sensors Considering Dependent Failures
Technical Paper
2017-01-0050
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
With increasing levels of driving automation, the perception provided by automotive environment sensors becomes highly safety relevant. A correct assessment of the sensors’ perception reliability is therefore crucial for ensuring the safety of the automated driving functionalities. There are currently no standardized procedures or guidelines for demonstrating the perception reliability of the sensors. Engineers therefore face the challenge of setting up test procedures and plan test drive efforts. Null Hypothesis Significance Testing has been employed previously to answer this question. In this contribution, we present an alternative method based on Bayesian parameter inference, which is easy to implement and whose interpretation is more intuitive for engineers without a profound statistical education. We show how to account for different environmental conditions with an influence on sensor performance and for statistical dependence among perception errors. Additionally, we study the impact of error dependence among several sensors on the perception reliability of a redundant multi-sensor system. To this end, we simplify the sensor data fusion with a majority voting scheme, which implies that the multi-sensor system’s perception fails whenever more than half of the individual sensors commit unacceptable errors. For a redundant multi-sensor system, in which error occurrence is weakly dependent, it can be shown that empirical reliability assessments are feasible. While the presented method does not encompass entirely the full complexity of the problem, it provides an initial systematic estimate of the necessary test drive effort and facilitates the use of sound statistical methods for test effort estimation.
Authors
Citation
Berk, M., Kroll, H., Schubert, O., Buschardt, B. et al., "Bayesian Test Design for Reliability Assessments of Safety-Relevant Environment Sensors Considering Dependent Failures," SAE Technical Paper 2017-01-0050, 2017, https://doi.org/10.4271/2017-01-0050.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 |
Also In
References
- Beiker , S. Deployment Scenarios for Vehicles with Higher-Order Automation Maurer , M. , Gerdes , J.C. , Lenz , B. , and Winner , H. Autonomous Driving Springer Berlin Heidelberg, Berlin, Heidelberg 978-3-662-48845-4 193 211 2016
- Google Google Self-Driving Car Project http://static.googleusercontent.com/media/www.google.com/en/us/selfdrivingcar/ February 25 2016
- Audi Mission accomplished: Audi A7 piloted driving car completes 550-mile automated test drive https://www.audiusa.com/newsroom/news/press-releases/2015/01/550-mile-piloted-drive-from-silicon-valley-to-las-vegas February 25 2016
- Pink , O. , Becker , J. , and Kammel , S. Automated driving on public roads: Experiences in real traffic it - Information Technology 57 4 2015 10.1515/itit-2015-0010
- Broggi , A. , Debattisti , S. , Grisleri , P. , and Panciroli , M. The deeva autonomous vehicle platform 2015 IEEE Intelligent Vehicles Symposium (IV) Seoul, South Korea 692 699
- Aeberhard , M. , Rauch , S. , Bahram , M. , Tanzmeister , G. et al. Experience, Results and Lessons Learned from Automated Driving on Germany's Highways IEEE Intell. Transport. Syst. Mag . 7 1 42 57 2015 10.1109/MITS.2014.2360306
- Franke , U. , Pfeiffer , D. , Rabe , C. , Knoeppel , C. et al. Making Bertha See 2013 IEEE International Conference on Computer Vision Workshops (ICCVW) Sydney, Australia 214 221 2013
- Kammel , S. , Ziegler , J. , Pitzer , B. , Werling , M. et al. Team AnnieWAY's autonomous system for the 2007 DARPA Urban Challenge J. Field Robotics 25 9 615 639 2008 10.1002/rob.20252
- Blevis , B. Losses due to rain on radomes and antenna reflecting surfaces IEEE Trans. Antennas Propagat . 13 1 175 176 1965 10.1109/TAP.1965.1138384
- Ishimaru , A. Wave propagation and scattering in random media and rough surfaces Proc. IEEE 79 10 1359 1366 1991 10.1109/5.104210
- Rasshofer , R.H. , Spies , M. , and Spies , H. Influences of weather phenomena on automotive laser radar systems Adv. Radio Sci . 9 49 60 2011 10.5194/ars-9-49-2011
- Weitzel , A. , Winner , H. , Peng , C. , Geyer , S. et al. Absicherungsstrategien für Fahrerassistenzsysteme mit Umfeldwahrnehmung: [Bericht zum Forschungsprojekt FE 82.0546/2012] Berichte der Bundesanstalt für Straßenwesen Fahrzeugtechnik 98 Fachverl. NW, Bremen 978-3-95606-118-9 2014
- Wachenfeld , W. and Winner , H. The Release of Autonomous Vehicles Maurer , M. , Gerdes , J.C. , Lenz , B. , and Winner , H. Autonomous Driving Springer Berlin Heidelberg, Berlin, Heidelberg 978-3-662-48845-4 425 449 2016
- Winner , H. ADAS, Quo Vadis? Winner , H. , Hakuli , S. , Lotz , F. , and Singer , C. Handbook of Driver Assistance Systems Springer International Publishing Cham 978-3-319-12351-6 1557 1584 2016
- Bock , F. , Siegl , S. , and German , R. Mathematical Test Effort Estimation for Dependability Assessment of Sensor-based Driver Assistance Systems 2016 42st Euromicro Conference on Software Engineering and Advanced Applications (SEAA) Limassol, Cyprus
- Greenland , S. , Senn , S.J. , Rothman , K.J. , Carlin , J.B. et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations European journal of epidemiology 31 4 337 350 2016 10.1007/s10654-016-0149-3
- Cohen , J. The earth is round (p < .05) American Psychologist 49 12 997 1003 1994 10.1037/0003-066X.49.12.997
- Nuzzo , R. Scientific method: statistical errors Nature 506 150 152 2014 10.1038/506150a
- Baker , M. Statisticians issue warning over misuse of P values Nature 531 151 2016 10.1038/nature.2016.19503
- Wasserstein , R.L. and Lazar , N.A. The ASA's Statement on p -Values: Context, Process, and Purpose The American Statistician 70 2 129 133 2016 10.1080/00031305.2016.1154108
- Hamada , M.S. , Wilson , A.G. , Reese , C.S. , and Martz , H.F. Bayesian Reliability Springer Series in Statistics 1st Springer-Verlag 978-0-387-77950-8 2008
- Rausand , M. and Høyland , A. System reliability theory: Models, statistical methods, and applications Wiley series in probability and statistics Applied probability and statistics 2nd Wiley-Interscience Hoboken, NJ 978-0-471-47133-2 2004
- Fitzgerald , M. , Martz , H.F. , and Parker , R.L. Bayesian Single-Level Binomial And Exponential Reliability Demonstration Test Plans Int. J. Rel. Qual. Saf. Eng . 06 02 123 137 1999 10.1142/S0218539399000139
- Singh , H. , Cortellessa , V. , Cukic , B. , Gunel , E. et al. A Bayesian approach to reliability prediction and assessment of component based systems 12th International Symposium on Software Reliability Engineering. ISSRE 2001 Hong Kong, China 27-30 Nov. 2001 12 21
- Brender , D.M. The Bayesian Assessment of System Availability: Advanced Applications and Techniques IEEE Trans. Rel . R-17 3 138 147 1968 10.1109/TR.1968.5216927
- Martz , H.F. and Wailer , R.A. Bayesian reliability analysis of complex series/parallel systems of binomial subsystems and components Technometrics 32 4 407 416 1990
- Guida , M. and Pulcini , G. Automotive reliability inference based on past data and technical knowledge Reliability Engineering & System Safety 76 2 129 137 2002 10.1016/S0951-8320(01)00132-6
- Gotzig , H. and Geduld , G. Automotive LIDAR Winner , H. , Hakuli , S. , Lotz , F. , and Singer , C. Handbook of Driver Assistance Systems Springer International Publishing Cham 978-3-319-12351-6 405 430 2016
- Winner , H. Automotive RADAR Winner , H. , Hakuli , S. , Lotz , F. , and Singer , C. Handbook of Driver Assistance Systems Springer International Publishing Cham 978-3-319-12351-6 325 403 2016
- Dietmayer , K. Predicting of Machine Perception for Automated Driving Maurer , M. , Gerdes , J.C. , Lenz , B. , and Winner , H. Autonomous Driving Springer Berlin Heidelberg, Berlin, Heidelberg 978-3-662-48845-4 407 424 2016
- Kottegoda , N.T. and Rosso , R. Applied statistics for civil and environmental engineers 2nd Blackwell Pub Oxford, UK, [Malden, MA] 978-1-4051-7917-1 2008
- Vidgen , B. and Yasseri , T. P-Values: Misunderstood and Misused Front. Phys . 4 e124 2016 10.3389/fphy.2016.00006
- Morey , R.D. , Hoekstra , R. , Rouder , J.N. , Lee , M.D. et al. The fallacy of placing confidence in confidence intervals Psychonomic bulletin & review 23 1 103 123 2016 10.3758/s13423-015-0947-8
- VanderPlas , J. Frequentism and bayesianism: a python-driven primer arXiv preprint arXiv:1411.5018 2014
- Jaynes , E.T. and Kempthorne , O. Confidence Intervals vs Bayesian Intervals Harper , W.L. and Hooker , C.A. Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science Springer Netherlands, Dordrecht 978-90-277-0619-5 175 257 1976
- Gelman , A. Induction and deduction in Bayesian data analysis Rationality, Markets and Morals 2 67 78 2011
- Darms , M. Data Fusion of Environment-Perception Sensors for ADAS Winner , H. , Hakuli , S. , Lotz , F. , and Singer , C. Handbook of Driver Assistance Systems Springer International Publishing Cham 978-3-319-12351-6:549-566 2016
- Winner , H. Fundamentals of Collision Protection Systems Winner , H. , Hakuli , S. , Lotz , F. , and Singer , C. Handbook of Driver Assistance Systems Springer International Publishing Cham 978-3-319-12351-6 1149 1176 2016
- Straub , D. Lecture Notes in Engineering Risk Analysis 2013
- Gelman , A. , Carlin , J.B. , Stern , H.S. , and Rubin , D.B. Bayesian data analysis Texts in statistical science 2nd Chapman & Hall Boca Raton, Fla. 9781584883883 2004
- Jeffreys , H. Theory of Probability 3rd Oxford University press London 1961
- Kass , R.E. and Wasserman , L. The selection of prior distributions by formal rules Journal of the American Statistical Association 91 435 1343 1370 1996
- Durrant-Whyte , H. and Henderson , T.C. Multisensor Data Fusion Siciliano , B. and Khatib , O. Springer Handbook of Robotics Springer Berlin Heidelberg, Berlin, Heidelberg 978-3-540-23957-4:585-610 2008
- Hisakado , M. , Kitsukawa , K. , and Mori , S. Correlated binomial models and correlation structures Journal of Physics A: Mathematical and General 39 50 15365 2006
- Rosner , B. Beta-Binomial Distribution Armitage , P. and Colton , T. Encyclopedia of Biostatistics John Wiley & Sons, Ltd Chichester, UK 047084907X 2005
- Maharry , T.J. PROPORTION DIFFERENCES USINGTHE BETA-BINOMIAL DISTRIBUTION Dissertation Oklahoma State University Stillwater, Oklahoma 2006
- Paul , S.R. Applications of the beta distribution Gupta , A.K. and Nadarajah , S. Handbook of beta distribution and its applications CRC press 423 436 2004
- Gupta , R.C. and Tao , H. A generalized correlated binomial distribution with application in multiple testing problems Metrika 71 1 59 77 2010 10.1007/s00184-008-0202-7
- Bahadur , R.R. A representation of the joint distribution of responses to n dichotomous items Studies in item analysis and prediction 6 158 168 1961
- Kupper , L.L. and Haseman , J.K. The use of a correlated binomial model for the analysis of certain toxicological experiments Biometrics 69 76 1978