This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Test Scenario Generation for Driving Simulators Using Constrained Randomization Technique
ISSN: 0148-7191, e-ISSN: 2688-3627
Published March 28, 2017 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
The advent of Advanced Driver Assistance Systems (ADAS) and automated driving has offered a new challenge for functional verification and validation. The explosion of the test sample space for possible combinations of inputs needs to be handled in an intelligent manner to meet cost and time targets for the development of such systems.
This paper addresses this research gap by using constrained randomization techniques for the creation of the required test scenarios and test cases. Furthermore, this paper proposes an automated constrained randomized test scenario generation framework for testing of ADAS and automated systems in a driving simulator setup. The constrained randomization approach is deployed at two levels: 1) test scenario randomization 2) test case randomization. The novelty of the proposed approach is in applying the constrained randomization method to generate test scenarios and test cases for automotive system and system of systems in a driving simulator environment. Test scenario randomization is used to automatically create the base scenario w.r.t., possible path trajectories of vehicles, environment variables and traffic variables. Test case randomization is achieved by real-time communication between the driving simulator and the constrained randomization tool via a TCP/IP HiL interface (client-server interface). Constrained randomization is then used to intelligently explore the possible sample space to find the corner cases for which an ADAS or an automated system may fail. For the use case discussed in this paper, 150 test scenarios were created, with 20 test cases for each test scenario, i.e., 3000 test runs were executed which were generated using a constrained randomization method.
The test setup comprises of the 3xD Simulator for Intelligent Vehicles at WMG, University of Warwick, UK, which has been integrated with the Vitaq® tool, which is a constrained randomization test automation tool developed by Vertizan Limited.
CitationKhastgir, S., Dhadyalla, G., Birrell, S., Redmond, S. et al., "Test Scenario Generation for Driving Simulators Using Constrained Randomization Technique," SAE Technical Paper 2017-01-1672, 2017, https://doi.org/10.4271/2017-01-1672.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
- Charette RN (2009) This car runs on code. IEEE Spectr 46(3).
- Khastgir S, Birrell S, Dhadyalla G, Jennings P (2015) Identifying a Gap in Existing Validation Methodologies for Intelligent Automotive Systems : Introducing the 3xD Simulator. Proc. of the 2015 IEEE Intelligent Vehicles Symposium (IV) (IEEE, Seoul, South Korea), pp 648–653.
- Klockwork W.P. (2012) Software on Wheels. (October). Available at: http://automotive-apps2013.we-conect.com/cms/media/uploads/events/501/dokumente/klocwork-paper-software-on-wheels.pdf [Accessed October 14, 2016].
- Malek S (2017) What Software Developers Can Learn From the Latest Car Recalls. CISQ. Available at: http://it-cisq.org/what-software-developers-can-learn-from-the-latest-car-recalls/ [Accessed January 3, 2017].
- Altinger H, Wotawa F, Schurius M (2014) Testing methods used in the automotive industry: results from a survey. Proc. of the 2014 Workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-Based Testing - JAMAICA 2014 doi:10.1145/2631890.2631891.
- Khastgir, S., Dhadyalla, G., and Jennings, P., "Incorporating ISO 26262 Concepts in an Automated Testing Toolchain Using Simulink Design Verifier™," SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 9(1):59–65, 2016, doi:10.4271/2016-01-0032.
- Zorian Y (1999) Testing the monster chip. IEEE Spectr: 54–60.
- Wagner I, Austin T (2005) StressTest : An Automatic Approach to Test Generation Via Activity Monitors. Proc. of the 42nd Annual Design Automation Conference (Anaheim, CA, USA), pp 783–788.
- Miranda B, Masini H, Reis R (2015) Using Simulink Design Verifier for Automatic Generation of Requirements-Based Tests. FM 2015: Formal Methods, eds Bjørner N, Boer F de (Springer International Publishing), pp 601–604.
- Rajeev, A., Mohalik, S., and Ramesh, S., "Design Verification of Automotive Controller Models," SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 6(2):419–424, 2013, doi:10.4271/2013-01-0428.
- Etienne JF, Fechter S, Juppeaux E (2010) Using Simulink Design Verifier for Proving Behavioral Properties on a Complex Safety Critical System in the Ground Transportation Domain. Complex Systems Design & Management, eds Aiguier M, Bretaudeau F, Krob D (Springer Berlin Heidelberg), pp 61–72.
- Nörenberg R, Reissing R, Weber J (2010) ISO 26262 conformant Verification Plan. INFORMATIK 2010, eds Fähnrich PDK-P, Franczyk PDB, pp 515–520.
- Conrad M (2012) Verification and Validation According to ISO 26262 : A Workflow to Facilitate the Development of High-Integrity Software. Proc ERTS 2012 Embed Real Time Softw ans Syst.
- Carlsson D (2013) Development of an ISO 26262 ASIL D compliant verification system. Dissertation (Linköping University).
- ISO (2011) Road vehicles — Functional safety (ISO 26262).
- ISO (2011) Road vehicles — Functional safety (ISO 26262): Part 6.
- Vertizan Vitaq software test methodology (Bletchley, UK) Available at: http://www.vertizan.com/ [Accessed October 13, 2016].
- Schmidt, T., Jin, S., Rogalli, J., Rogier, T. ., "Efficient Testing Framework for Simulink Models with MTCD and Automated Test Assessments in the Context of ISO 26262," SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 7(1):166–177, 2014, doi:10.4271/2014-01-0306.
- Schloßer A, Khastgir S, Jentges M, Jakoby B, Richenhagen J (2014) Test Driven Model Based Series Software Deveopment for Automotive Systems. Proc. of the FISITA World Automotive Congress 2014 (Maastricht, Netherlands).
- Medikeri, M., Tasky, T., and Richenhagen, J., "Automated Verification and Validation Methods for Transmission Control Software," SAE Int. J. Passeng. Cars – Electron. Electr. Syst. 8(1):99–105, 2015, doi:10.4271/2015-01-0163.
- Cohda Wireless: White Papers (2016) Available at: http://cohdawireless.com/WhyCohda/Whitepapers.aspx [Accessed October 13, 2016].
- Kalra N, Paddock SM (2016) Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transp Res Part A Policy Pract 94(December):182–193.