This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Test Scenario Generation for Driving Simulators Using Constrained Randomization Technique
ISSN: 0148-7191, e-ISSN: 2688-3627
Published March 28, 2017 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
The advent of Advanced Driver Assistance Systems (ADAS) and automated driving has offered a new challenge for functional verification and validation. The explosion of the test sample space for possible combinations of inputs needs to be handled in an intelligent manner to meet cost and time targets for the development of such systems.
This paper addresses this research gap by using constrained randomization techniques for the creation of the required test scenarios and test cases. Furthermore, this paper proposes an automated constrained randomized test scenario generation framework for testing of ADAS and automated systems in a driving simulator setup. The constrained randomization approach is deployed at two levels: 1) test scenario randomization 2) test case randomization. The novelty of the proposed approach is in applying the constrained randomization method to generate test scenarios and test cases for automotive system and system of systems in a driving simulator environment. Test scenario randomization is used to automatically create the base scenario w.r.t., possible path trajectories of vehicles, environment variables and traffic variables. Test case randomization is achieved by real-time communication between the driving simulator and the constrained randomization tool via a TCP/IP HiL interface (client-server interface). Constrained randomization is then used to intelligently explore the possible sample space to find the corner cases for which an ADAS or an automated system may fail. For the use case discussed in this paper, 150 test scenarios were created, with 20 test cases for each test scenario, i.e., 3000 test runs were executed which were generated using a constrained randomization method.
The test setup comprises of the 3xD Simulator for Intelligent Vehicles at WMG, University of Warwick, UK, which has been integrated with the Vitaq® tool, which is a constrained randomization test automation tool developed by Vertizan Limited.
CitationKhastgir, S., Dhadyalla, G., Birrell, S., Redmond, S. et al., "Test Scenario Generation for Driving Simulators Using Constrained Randomization Technique," SAE Technical Paper 2017-01-1672, 2017, https://doi.org/10.4271/2017-01-1672.
Data Sets - Support Documents
|Unnamed Dataset 1|
- Charette RN 2009 This car runs on code IEEE Spectr 46 3
- Khastgir S , Birrell S , Dhadyalla G , Jennings P 2015 Identifying a Gap in Existing Validation Methodologies for Intelligent Automotive Systems : Introducing the 3xD Simulator Proc. of the 2015 IEEE Intelligent Vehicles Symposium (IV) IEEE Seoul, South Korea 648 653
- Klockwork W.P. 2012 Software on Wheels October http://automotive-apps2013.we-conect.com/cms/media/uploads/events/501/dokumente/klocwork-paper-software-on-wheels.pdf October 14 2016
- Malek S 2017 What Software Developers Can Learn From the Latest Car Recalls CISQ http://it-cisq.org/what-software-developers-can-learn-from-the-latest-car-recalls/ January 3 2017
- Altinger H , Wotawa F , Schurius M 2014 Testing methods used in the automotive industry: results from a survey Proc. of the 2014 Workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-Based Testing - JAMAICA 2014 10.1145/2631890.2631891
- Khastgir , S. , Dhadyalla , G. , and Jennings , P. Incorporating ISO 26262 Concepts in an Automated Testing Toolchain Using Simulink Design Verifier™ SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 9 1 59 65 2016 10.4271/2016-01-0032
- Zorian Y 1999 Testing the monster chip IEEE Spectr 54 60
- Wagner I , Austin T 2005 StressTest : An Automatic Approach to Test Generation Via Activity Monitors Proc. of the 42nd Annual Design Automation Conference Anaheim, CA, USA 783 788
- Miranda B , Masini H , Reis R 2015 Using Simulink Design Verifier for Automatic Generation of Requirements-Based Tests FM 2015: Formal Methods Bjørner N , Boer F de Springer International Publishing 601 604
- Rajeev , A. , Mohalik , S. , and Ramesh , S. Design Verification of Automotive Controller Models SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 6 2 419 424 2013 10.4271/2013-01-0428
- Etienne JF , Fechter S , Juppeaux E 2010 Using Simulink Design Verifier for Proving Behavioral Properties on a Complex Safety Critical System in the Ground Transportation Domain Complex Systems Design & Management Aiguier M , Bretaudeau F , Krob D Springer Berlin Heidelberg 61 72
- Nörenberg R , Reissing R , Weber J 2010 ISO 26262 conformant Verification Plan INFORMATIK 2010 Fähnrich PDK-P , Franczyk PDB 515 520
- Conrad M 2012 Verification and Validation According to ISO 26262 : A Workflow to Facilitate the Development of High-Integrity Software Proc ERTS 2012 Embed Real Time Softw ans Syst.
- Carlsson D 2013 Development of an ISO 26262 ASIL D compliant verification system Dissertation Linköping University
- ISO 2011 Road vehicles — Functional safety (ISO 26262)
- ISO 2011 Road vehicles — Functional safety (ISO 26262): Part 6
- Vertizan Vitaq software test methodology Bletchley, UK http://www.vertizan.com/ October 13 2016
- Schmidt , T. , Jin , S. , Rogalli , J. , Rogier , T. Efficient Testing Framework for Simulink Models with MTCD and Automated Test Assessments in the Context of ISO 26262 SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 7 1 166 177 2014 10.4271/2014-01-0306
- Schloßer A , Khastgir S , Jentges M , Jakoby B , Richenhagen J 2014 Test Driven Model Based Series Software Deveopment for Automotive Systems Proc. of the FISITA World Automotive Congress 2014 Maastricht, Netherlands
- Medikeri , M. , Tasky , T. , and Richenhagen , J. Automated Verification and Validation Methods for Transmission Control Software SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 8 1 99 105 2015 10.4271/2015-01-0163
- Cohda Wireless: White Papers 2016 http://cohdawireless.com/WhyCohda/Whitepapers.aspx October 13 2016
- Kalra N , Paddock SM 2016 Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transp Res Part A Policy Pract 94 December 182 193