An Analysis of ISO 26262: Machine Learning and Safety in Automotive Software

Authors

• Rick Salay - University of Waterloo
• Rodrigo Queiroz - University of Waterloo
• Krzysztof Czarnecki - University of Waterloo

Topic

• Machine learning
• Computer software and hardware
• Vehicle drivers
• Certification

Citation

Also In

References

1. Spanfelner, B.,Richter, D.,Ebel, S.,Wilhelm, U.,Branz, W., andPatz, C., “Challenges in applying the ISO 26262 for driver assistance systems,” Tagung Fahrerassistenz, München, 15(16), 2012.
2. Koopman, P. andWagner, M., “Challenges in Autonomous Vehicle Testing and Validation,” SAE International Journal of Transportation Safety 4(1):15-24, 2016, doi:10.4271/2016-01-0128.
3. International Organization for Standardization, “ISO 26262: Road Vehicles - Functional Safety”, 2011.
4. Varshney, K. R., “Engineering safety in machine learning,” arXiv preprint arXiv:1601.04126, 2016.
5. Amodei, D.,Olah, C.,Steinhardt, J.,Christiano, P.,Schulman, J., andMané, D., “Concrete problems in AI safety,” arXiv preprint arXiv:1606.06565, 2016.
6. Tickle, A.B.,Andrews, R.,Golea, M., andDiederich, J., “The Truth will Come to Light: Directions and Challenges in Extracting the Knowledge Embedded within Trained Artificial Neural Networks,” IEEE Transactions on Neural Networks 9(6):1057-1068, 1998, doi:10.1109/72.728352.
7. Peterson, G. E., “Foundation for neural network verification and validation,” in Optical Engineering and Photonics in Aerospace Sensing, 196-207, International Society for Optics and Photonics, 1993.
8. Rodvold, D. M., “A software development process model for artificial neural networks in critical applications,” International Joint Conference on Neural Networks, 5:3317-3322, 1999, doi:10.1109/IJCNN.1999.836192.
9. Hendricks, L. A.,Akata, Z.,Rohrbach, M.,Donahue, J.,Schiele, B., andDarrell, T., “Generating visual explanations,” in European Conference on Computer Vision, 3-19, Spring, 2016, doi:10.1007/978-3-319-46493-0 1.
10. Huang, X.,Kwiatkowska, M.,Wang, S., andWu, M., “Safety verification of deep neural networks,” arXiv preprint arXiv:1610.06940, 2016.
11. Katz, G.,Barrett, C.,Dill, D.,Julian, K., andKochenderfer, M., “Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks,” arXiv preprint arXiv:1702.01135, 2017.
12. Schumann, J.,Gupta, P., andLiu, Y., “Application of neural networks in high assurance systems: A survey,” in Applications of Neural Networks in High Assurance Systems, 1-19, Spring, 2010, doi:10.1007/978-3-642-10690-3.
13. Pullum, L. L.,Taylor, B. J., andDarrah, M. A., Guidance for the Verification and Validation of Neural Networks, vol. 11. John Wiley & Sons, 2007, doi:10.1002/9781119134671.
14. Bedford, D.,Morgan, G., andAustin, J., “Requirements for a standard certifying the use of artificial neural networks in safety critical applications,” . In: Proceedings of the International Conference on Artificial Neural Networks. (1996).
15. Kurd, Z.,Kelly, T., andAustin, J., “Developing Artificial Neural Networks for Safety Critical Systems,” Neural Computing and Applications 16(1):11-19, 2007, doi:10.1007/s00521-006-0039-9.
16. Martin, H.,Tschabuschnig, K.,Bridal, O., andWatzenig, D., “Functional Safety of Automated Driving Systems: Does ISO 26262 Meet the Challenges?,” in Automated Driving, 387-416, Spring, 2017, doi:10.1007/978-3-319-31895-0_16.
17. Burton, S.,Gauerhof, L., andHeinzemann, C., “Making the Case for Safety of Machine Learning in Highly Automated Driving,” in International Conference on Computer Safety, Reliability, and Security, 5-16, Spring, 2017, doi:10.1007/978-3-319-66284-8_1.
18. Henzel, M.,Winner, H., andLattke, B., “Herausforderungen in der Absicherung von Fahrerassistenzsystemen bei der Benutzung maschinell gelernter und lernenden Algorithmen,” . In: Proceedings of 11th Workshop Fahrerassistenzsysteme Und Automatisiertes Fahren (FAS). (2017), 136-148.
19. SAE International, “SAE J3016: Taxonomy and Definitions for Terms Related to on-Road Motor Vehicle Automated Driving Systems”, 2017.
20. Parasuraman, R. andRiley, V., “Humans and automation: Use, misuse, disuse, abuse,” Human Factors: The Journal of the Human Factors and Ergonomics Society 39(2):230-253, 1997, doi:10.1518/001872097778543886.
21. Brookhuis, K. A.,De Waard, D., andJanssen, W. H., “Behavioural Impacts of Advanced Driver Assistance Systems-An Overview,” EJTIR, 1(3):245-253, 2001.
22. Sullivan, J.M., “Flannagan,” (Pradhan, A. K., and Bao, S., Literature Review of Behavioral Adaptations to Advanced Driver Assistance Systems. AAA Foundation for Traffic Safety, M. J, 2016).
23. Goodrich, M.A. andSchultz, A.C., “Human-Robot Interaction: A Survey,” Foundations and Trends in Human-Computer Interaction 1(3):203-275, 2007, doi:10.1561/1100000005.
24. van denBrule, R.,Dotsch, R.,Bijlstra, G.,Wigboldus, D. H., andHaselager, P., “Do Robot Performance and Behavioral Style Affect Human Trust?,” International Journal of Social Robotics, 6(4):519-531, 2014, doi:10.1007/s12369-014-0231-5.
25. Chakarov, A.,Nori, A., andRajamani, S., “Sen, S., and Vijaykeerthy,” D., “Debugging machine learning tasks,” arXiv preprint arXiv 1603:07292, 2016.
26. Nushi, B. andKamar, E., “Horvitz, E., and Kossmann,” D., “On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems,” arXiv preprint arXiv 1611:08309, 2016.
27. Ponti Jr, M.P., “Combining Classifiers: From the Creation of Ensembles to the Decision Fusion,” in 24th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials (SIBGRAPI-T), 1-10,”IEEE, 2011, doi:10.1109/SIBGRAPIT.2011.9.
28. Breiman, L., “Bagging Predictors,” Machine Learning 24(2):123-140, 1996, doi:10.1007/BF00058655.
29. Freund, Y.,Schapire, R.E. et al., “Experiments with a new boosting algorithm,” . In: ICML, 96. (1996), 148-156.
30. Sha, L., “Using Simplicity to Control Complexity,” IEEE Software 18(4):20-28, 2001.
31. Phan, D.,Yang, J.,Clark, M.,Grosu, R. et al., D., “A component-based simplex architecture for high-assurance cyber-physical systems,” arXiv preprint arXiv 1704:04759, 2017.
32. Perkins, T.J. andBarto, A.G., “Lyapunov Design for Safe Reinforcement Learning,” Journal of Machine Learning Research 3:803-832, 2002.
33. Fisac, J.F.,Akametalu, A.K.,Zeilinger, M.N.,Kaynama, S. et al., J., “A general safety framework for learning-based control in uncertain robotic systems,” arXiv preprint arXiv 1705:01292, 2017.
34. Bhattacharyya, S.,Cofer, D.,Musliner, D.,Mueller, J., andEngstrom, E., “Certification Considerations for Adaptive Systems,” in Unmanned Aircraft Systems (ICUAS), 2015 International Conference on, 270-279,”IEEE, 2015, doi:10.1109/ICUAS.2015.7152300.
35. Rouder, J.N. andRatcliff, R., “Comparing Exemplar and Rule-Based Theories of Categorization,” Current Directions in Psychological Science 15(1):9-13, 2006, doi:10.1111/j.0963-7214.2006.00397.x.
36. Ammann, P. andOffutt, J., “Introduction to Software Testing,” Cambridge University Press, 2016, doi:10.1017/CBO9780511809163.
37. International Organization for Standardization, “ISO/AWI PAS 21448: Road Vehicles - Safety of the Intended Functionality”. (under development).
38. Bojarski, M.,Del Testa, D.,Dworakowski, D.,Firner, B. et al., Zhang, J., et al., “End to end learning for self-driving cars,” arXiv preprint arXiv 1604:07316, 2016.
39. Shalev-Shwartz, S. and Shashua, A., “On the sample complexity of end-to-end training vs. semantic abstraction training,” arXiv preprint arXiv 1604:06915, 2016.
40. Takanami, I.,Sato, M., andYang, Y.P., “A Fault-Value Injection Approach for Multiple-Weight-Fault Tolerance of MNNs,” Proceedings of the IEEE-INNS-ENNS,” International Joint Conference on Neural Networks 3:515-520, 2000, doi:10.1109/IJCNN.2000.861360.

Cited By