Multilevel Concept for Verification of Automated Driving Systems by Using Model in the Loop Simulations in Early Development Phases

Authors

• Wolfgang Sinz - Magna Steyr Fahrzeugtechnik AG & Co KG
• Bernhard Angrosch - Magna Steyr Fahrzeugtechnik AG & Co KG
• Jens Gächter - Magna Steyr Fahrzeugtechnik AG & Co KG
• Bernhard Putsche - Magna Steyr Fahrzeugtechnik AG & Co KG
• Branko Rogic - Magna Steyr Fahrzeugtechnik AG & Co KG
• Stefan Bernsteiner - Magna Steyr Fahrzeugtechnik AG & Co KG

Topic

• Automated driving systems
• Automated vehicles
• Simulation and modeling
• Road tests

Citation

Also In

References

1. World Health Organization 2018
2. NHTSA 2017
3. Sander , U. , Lubbe , N. , and Pietzsch , S. Intersection AEB Implementation Strategies for Left Turn Across Path Crashes Traffic Injury Prevention 20 sup1 2019 119 125 10.1080/15389588.2019.1602728
4. Zavodjančík , T. , Kasanický , G. , and Demčáková , L. Reduction of Pedestrian Accidents - Automated Road Vehicles Transportation Research Procedia 55 2021 1605 1612 10.1016/j.trpro.2021.07.150
5. Rajabli , N. , Flammini , F. , Nardone , R. , and Vittorini , V. Software Verification and Validation of Safe Autonomous Cars: A Systematic Literature Review IEEE Access 9 2021 4797 4819 10.1109/ACCESS.2020.3048047
6. Albrecht , H. , Barickman , F. , and Schnelle , S. Advanced Test Tools for ADAS and ADS Washington, D.C. 20590, 1200 New Jersey Avenue, US 2021
7. SAE International/ISO SAE J3016: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles Geneva, Switzerland 2021
8. Koopman , P. and Wagner , M. Challenges in Autonomous Vehicle Testing and Validation SAE International Journal of Transportation Safety 4 1 2016 15 24 10.4271/2016-01-0128
9. Bargende , M. , Reuss , H.-C. , Wagner , A. , and Wiedemann , J. 20. Internationales Stuttgarter Symposium: Automobil- und Motorentechnik - Band 1 Michael Bargende, Hans-Christian Reuss, Andreas Wagner ISBN 978-3-658-29942-2 2020 10.1007/978-3-658-29943-9
10. Schlager , B. , Muckenhuber , S. , Schmidt , S. , Holzer , H. et al. State-of-the-Art Sensor Models for Virtual Testing of Advanced Driver Assistance Systems/Autonomous Driving Functions SAE International Journal of Connected and Automated Vehicles 3 2020 233 261 10.4271/12-03-03-0018
11. Batsch , F. , Kanarachos , S. , Cheah , M. , Ponticelli , R. et al. A Taxonomy of Validation Strategies to Ensure the Safe Operation of Highly Automated Vehicles Journal of Intelligent Transportation Systems 0 0 2020 1 20 10.1080/15472450.2020.1738231
12. Hermans , T. , Ramaekers , P. , Denil , J. , de Meulenaere , P. , and Anthonis , J. Incorporation of AUTOSAR in an Embedded Systems Development Process: A Case Study 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications IEEE ISBN 978-1-4577-1027-8 2011 10.1109/SEAA.2011.45
13. Thorn , E. , Kimmel , S. , and Chaka , M. DOT HS 812 623 - A Framework for Automated Driving System Testable Cases and Scenarios Washington, DC, US 2018
14. Sotelo , M.A. Lateral Control Strategy for Autonomous Steering of Ackerman-Like Vehicles Robotics and Autonomous Systems 45 3-4 2003 223 233 10.1016/j.robot.2003.09.002
15. Kong , J. , Pfeiffer , M. , Schildbach , G. , and Borrelli , F. Kinematic and Dynamic Vehicle Models for Autonomous Driving Control Design 2015 IEEE Intelligent Vehicles Symposium (IV) Institute of Electrical and Electronics Engineers Seoul, Korea 1094 1099 2015 10.1109/IVS.2015.7225830
16. Viswanath , P. , Mody , M. , Nagori , S. , Jones , J. et al. Virtual Simulation Platforms for Automated Driving: Key Care-About and Usage Model Electronic Imaging 2018 17 2018 1 6 10.2352/ISSN.2470-1173.2018.17.AVM-164
17. Meng , X. , Gan , H. , and Yan , Y. Simulation Test of Driving Assistance System Based on Virtual Scene Journal of Physics: Conference Series 12047 2019 10.1088/1742-6596/1419/1/012047
18. Aeberhard , M. Object-Level Fusion for Surround Environment Perception in Automated Driving Applications Dortmund 2017
19. Hartstern , M. , Rack , V. , Kaboli , M. , and Stork , W. Simulation-based Evaluation of Automotive Sensor Setups for Environmental Perception in Early Development Stages 2020 IEEE Intelligent Vehicles Symposium (IV) IEEE Las Vegas, USA ISBN 978-1-7281-6673-5 858 864 2020 10.1109/IV47402.2020.9304771
20. Berk , M.J. Safety Assessment of Environment Perception in Automated Driving Vehicles Munich, Germany 2019
21. Rosique , F. , Navarro , P.J. , Fernández , C. , and Padilla , A. A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research Sensors 19 3 2019 648 10.3390/s19030648
22. Yeong , D.J. , Velasco-Hernandez , G. , Barry , J. , and Walsh , J. Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review Sensors 21 6 2021 10.3390/s21062140
23. Hasirlioglu , S. 2020
24. Holder , M.F. , Linnhoff , C. , Rosenberger , P. , Popp , C. , and Winner , H. Modeling and Simulation of Radar Sensor Artifacts for Virtual Testing of Autonomous Driving 9. Tagung Automatisiertes Fahren Lehrstuhl für Fahrzeugtechnik mit TÜV SÜD Akademie Munich, Germany 2019
25. Schnelle , S. , Salaani , K. , Rao , S. , Barickman , F. , and Elsasser , D. 2019
26. Son , T.D. , Hubrechts , J. , Awatsu , L. , Bhave , A. , and van der Auweraer , H. A Simulation-Based Testing and Validation Framework for ADAS Development 7th Transport Research Arena - TRA Transport Research Arena Vienna, Austria 2018
27. Maurer , M. , Gerdes , J.C. , Lenz , B. , and Winner , H. Autonomous Driving: Technical, Legal and Social Aspects Springer Nature 2016 10.1007/978-3-662-48847-8 ISBN 978-3-662-48845-4
28. Kalra , N. and Paddock , S.M. Driving to Safety: How Many Miles of Driving Would it Take to Demonstrate Autonomous Vehicle Reliability? Transportation Research Part A: Policy and Practice 94 2016 1 15
29. Saigol , Z. and Peters , A. Verifying Automated Driving Systems in Simulation: Framework and Challenges 25th ITS World Congress 1 10 2018
30. Riedmaier , S. , Ponn , T. , Ludwig , D. , Schick , B. et al. Survey on Scenario-Based Safety Assessment of Automated Vehicles IEEE Access 8 2020 87456 87477 10.1109/ACCESS.2020.2993730
31. Khastgir , S. , Brewerton , S. , Thomas , J. , and Jennings , P. Systems Approach to Creating Test Scenarios for Automated Driving Systems Reliability Engineering & System Safety 215 2021 1 14 10.1016/j.ress.2021.107610
32. Koopman , P. , Kane , A. , and Black , J. Credible Autonomy Safety Argumentation 27th Safety-Critical Systems Club Symposium Safety-Critical System Club Bristol, UK 2019
33. Bolte , J.-A. , Bar , A. , Lipinski , D. , and Fingscheidt , T. Towards Corner Case Detection for Autonomous Driving 2019 IEEE Intelligent Vehicles Symposium (IV) Institute of Electrical and Electronics Engineers Paris, France 438 445 2019
34. Fahrenkrog , F. , Schneider , M. , Naujoks , F. , Tango , F. , et al. L3Pilot - Draft Code-of-Practice for Automated Driving and Results from Pilot Application of Draft CoP Brussels, Belgium 2020 10.13140/RG.2.2.28149.40161
35. Submitted by the experts of OICA 2019
36. Kalisvaart , S. , Svanberg , E. , Barnard , Y. , Innamaa , S. , et al. 2021
37. Riedmaier , S. , Schneider , D. , Watzenig , D. , Diermeyer , F. et al. Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles Applied Sciences 11 1 2021 1 24 10.3390/app11010035
38. Ma , X. , Hu , X. , Weber , T. , and Schramm , D. Traffic Simulation of Future Intelligent Vehicles in Duisburg City Inner Ring Applied Sciences 11 1 2021 1 12 10.3390/app11010029
39. Nalic , D. , Li , H. , Eichberger , A. , Wellershaus , C. et al. Stress Testing Method for Scenario-Based Testing of Automated Driving Systems IEEE Access 8 2020 1 12 10.1109/ACCESS.2020.3044024
40. Hoque , M.A. , Hong , X. , and Ahmed , M.S. Parallel Closed-Loop Connected Vehicle Simulator for Large-Scale Transportation Network Management: Challenges, Issues, and Solution Approaches IEEE Intelligent Transportation Systems Magazine 11 4 2018 62 77 10.1109/MITS.2018.2879163
41. Hallerbach , S. , Xia , Y. , Eberle , U. , and Koester , F. Simulation-Based Identification of Critical Scenarios for Cooperative and Automated Vehicles SAE International Journal of Connected and Automated Vehicles 1 2018-01-1066 2018 93 106 10.4271/2018-01-1066
42. Hallerbach , S. Simulation-Based Testing of Cooperative and Automated Vehicles Germany Oldenburg 2020
43. Smirnov , N. , Liu , Y. , Validi , A. , Morales-Alvarez , W. et al. A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios Sensors1 21 4 2021 1 19 10.3390/s21041523
44. Tian , R. , Li , N. , Kolmanovsky , I. , Yildiz , Y. et al. Game-Theoretic Modeling of Traffic in Unsignalized Intersection Network for Autonomous Vehicle Control Verification and Validation IEEE Transactions on Intelligent Transportation Systems 2020 1 16
45. Sippl , C. , Bock , F. , Wittmann , D. , Altinger , H. , and German , R. From Simulation Data to Test Cases for Fully Automated Driving and ADAS IFIP International Conference on Testing Software and Systems 2016 10.1007/978-3-319-47443-4_12
46. Stellet , J.E. , Woehrle , M. , Brade , T. , Poddey , A. , and Branz , W. Validation of Automated Driving - A Structured Analysis and Survey of Approaches 13. Workshop Fahrerassistenz und automatisiertes Fahren RWTH Aachen University 2020
47. de Gelder , E. , Saberi , A.K. , and Elrofai , H. A Method for Scenario Risk Quantification for Automated Driving Systems - Paper Number: 19-0129 26th International Technical Conference on the Enhanced Safety of Vehicles (ESV): Technology: Enabling a Safer Tomorrow National Highway Traffic Safety Administration 2019
48. Raffaelli , L. , Vallée , F. , Fayolle , G. , de Souza , P. , et al. 2016
49. Weber , H. , Bock , J. , Klimke , J. , Roesener , C. et al. A Framework for Definition of Logical Scenarios for Safety Assurance of Automated Driving Traffic Injury Prevention 20 1 2019 65 70 10.1080/15389588.2019.1630827
50. Pütz , A. , Zlocki , A. , Bock , J. , and Eckstein , L. System Validation of Highly Automated Vehicles with a Database of Relevant Traffic Scenarios 12th ITS European Congress Strasbourg, France E5 2017
51. Nalic , D. , Mihalj , T. , Baeumler , M. , Lehmann , M. , et al. Scenario Based Testing of Automated Driving Systems: A Literature Survey - F2020-ACM-096 FISITA - Web Congress 2020 2020
52. Roesener , C. , Fahrenkrog , F. , Uhlig , A. , and Eckstein , L. A Scenario-Based Assessment Approach for Automated Driving by Using Time Series Classification of Human-Driving Behaviour 19th International Conference on Intelligent Transportation Systems Institute of Electrical and Electronics Engineers Rio de Janeiro, Brazil 1360 1365 2016 10.1109/ITSC.2016.7795734
53. Nalic , D. , Pandurevic , A. , Eichberger , A. , and Rogic , B. Design and Implementation of a Co-Simulation Framework for Testing of Automated Driving Systems Sustainability 12 24 2020 10.3390/su122410476
54. Lattarulo , R. , Pérez , J. , and Dendaluce , M. A Complete Framework for Developing and Testing Automated Driving Controllers IFAC Papers On Line 50 1 2017 258 263 10.1016/j.ifacol.2017.08.043
55. Rudigier , M. , Nestlinger , G. , Tong , K. , and Solmaz , S. Development and Verification of Infrastructure-Assisted Automated Driving Functions Electronics 10 17 2021 1 13 10.3390/electronics10172161
56. Hussein , A. , Diaz-Álvarez , A. , Armingol , J.M. , and Olaverri-Monreal , C. 3CCoAutoSim: Simulator for Cooperative ADAS and Automated Vehicles 21st International Conference on Intelligent Transportation Systems (ITSC) Institute of Electrical and Electronics Engineers Maui, HI, US 3014 3019 2018 10.1109/ITSC.2018.8569512
57. Wood , M. , Knobel , C. , Garbacik , N. , Wittmann , D. , et al. 2019
58. 2019
59. Berger , C. and Rumpe , B. 2014 10.1007/978-0-85729-772-3_10
60. Singh , V. , Hari , S.K.S. , Tsai , T. , and Pitale , M. Simulation Driven Design and Test for Safety of AI Based Autonomous Vehicles Conference on Computer Vision and Pattern Recognition Workshops Institute of Electrical and Electronics Engineers Nashville, TN, USA 122 128 2021 10.1109/CVPRW53098.2021.00022
61. Bernhard , J. , Esterle , K. , Hart , P. , and Kessler , T. BARK: Open Behavior Benchmarking in Multi-Agent Environments International Conference on Intelligent Robots and Systems Institute of Electrical and Electronics Engineers Las Vegas, NV, USA 6201 6208 2020 10.1109/IROS45743.2020.9341222
62. Zhang , L. , Ding , W. , Chen , J. , and Shen , S. Efficient Uncertainty-Aware Decision-Making for Automated Driving Using Guided Branching International Conference on Robotics and Automation (ICRA) 2020 3291 3297 10.1109/ICRA40945.2020.9197302
63. IPG Automotive GmbH https://ipg-automotive.com/en/ 2021
64. The MathWorks, Inc. https://de.mathworks.com/ 2021
65. PTV Planung Transport Verkehr AG https://company.ptvgroup.com/en/ 2021
66. Stern , P. 2019
67. International Standard - ISO 2018
68. Bargende , M. , Reuss , H.-C. , Andreas , W. , and Jochen , W. 19. Internationales Stuttgarter Symposium - Automobil- und Motorentechnik Wiesbaden Springer Fachmedien Wiesbaden 2019 10.1007/978-3-658-25939-6 ISBN 978-3-658-25938-9
69. Heidecker , F. , Breitenstein , J. , Rösch , K. , Löhdefink , J. , et al. 2021
70. Wang , J. , Zhang , C. , Liu , Y. , and Zhang , Q. Traffic Sensory Data Classification by Quantifying Scenario Complexity IEEE Intelligent Vehicles Symposium (IV) Institute of Electrical and Electronics Engineers Changshu, China 1543 1548 2018 10.1109/IVS.2018.8500669
71. Huang , L. , Xia , Q. , Xie , F. , Xiu , H.-L. , and Shu , H. Study on the Test Scenarios of Level 2 Automated Vehicles IEEE Intelligent Vehicles Symposium (IV) Institute of Electrical and Electronics Engineers Changshu, China 49 54 2018 10.1109/IVS.2018.8500600
72. Federal Minister of Transport, I. and T 2016
73. Riedmaier , S. , Nesensohn , J. , Gutenkunst , C. , Düser , T. , et al. Validation of X-in-the-Loop Approaches for Virtual Homologation of Automated Driving Functions 11th Graz Symposium Virtual Vehicle Virtual Vehicle Research GmbH Graz, Austria 2018
74. Ponn , T. , Fratzke , D. , Gnandt , C. , and Lienkamp , M. Towards Certification of Autonomous Driving: Systematic Test Case Generation for a Comprehensive but Economically-Feasible Assessment of Lane Keeping Assist Algorithms 5th International Conference on Vehicle Technology and Intelligent Transport Systems SCITEPRESS - Science and Technology Publications Heraklion, Greece ISBN 978-989-758-374-2 2019 10.5220/0007678603330342
75. Page , Y. , Fahrenkrog , F. , Fiorentino , A. , Fränzle , M. , et al. A Comprehensive and Harmonized Method for Assessing the Effectiveness of Advanced Driver Assistance Systems by Virtual Simulation: The P.E.A.R.S. Initiative - Paper Number 15-0370 24th International Technical Conference on the Enhanced Safety of Vehicles (ESV): Traffic Safety Through Integrated Technologies National Highway Traffic Safety Administration Gothenburg , Sweden 12 2015

Cited By