This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Evolutionary Safety Testing of Embedded Control Software by Automatically Generating Compact Test Data Sequences
Technical Paper
2005-01-0750
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
Whereas the verification of non-safety-related, embedded software typically focuses on demonstrating that the implementation fulfills its functional requirements, this is not sufficient for safety-relevant systems. In this case, the control software must also meet application-specific safety requirements.
Safety requirements typically arise from the application of hazard and/or safety analysis techniques, e.g. FMEA, FTA or SHARD. During the downstream development process it must be shown that these requirements cannot be violated. This can be achieved utilizing different techniques. One way of providing evidence that violations of the safety properties identified cannot occur is to thoroughly test each of the safety requirements.
This paper introduces Evolutionary Safety Testing (EST), a fully automated procedure for the safety testing of embedded control software. EST employs extended evolutionary algorithms in an optimization process which aggressively tries to find test data sequences that cause the test object to violate a given safety requirement.
A compact description formalism for input sequences for safety testing is presented, which is compatible with description techniques used during other test process stages. This compact description allows 1) an efficient application of evolutionary algorithms (and other optimization techniques) and 2) the description of long test sequences necessary for the adequate stimulation of real-world systems. The objective function is designed in such a way that optimal values represent test data sequences which violate a given safety requirement. By means of repeated input sequence generation, software execution and the subsequent evaluation of the objective function each safety requirement is extensively tested.
The use of EST for the safety testing of automotive control software is demonstrated using safety requirements of an adaptive cruise control (ACC) system.
The EST approach can easily be integrated into an overall software test strategy which combines different test design techniques with specific test objectives.
Recommended Content
Authors
Citation
Pohlheim, H., Conrad, M., and Griep, A., "Evolutionary Safety Testing of Embedded Control Software by Automatically Generating Compact Test Data Sequences," SAE Technical Paper 2005-01-0750, 2005, https://doi.org/10.4271/2005-01-0750.Also In
SAE 2005 Transactions Journal of Passenger Cars: Mechanical Systems
Number: V114-6; Published: 2006-02-01
Number: V114-6; Published: 2006-02-01
References
- Baresel, A. Conrad, M. Sadeghipour, S. The Interplay between Model Coverage and Code Coverage Proc. 11. Europ. Int. Conf. on Software Testing, Analysis and Review (EuroSTAR 03) 2003
- Baresel, A. Pohlheim, H. Sadeghipour, S. Structural and Functional Sequence Testing of Dynamic and State-Based Software with Evolutionary Algorithms Proc. Genetic and Evolutionary Computation Conf. (GECCO 03) 2428 2441 2003
- Baresel, A. Sthamer, H. Schmidt, M. Fitness Function Design to improve Evolutionary Structural Testing Proc. Genetic and Evolutionary Computation Conf. (GECCO 02) 1329 1336 2002
- Beizer, B. Software Testing Techniques New York Van Nostrand Reinhold 1983
- Conrad, M. Fey, I. Sadeghipour, S. Systematic Model-Based Testing of Embedded Control Software: The MB3T Approach Proc. ICSE 2004 Workshop W14S on Software Engineering for Automotive Systems (SEAS 04) 17 25 2004
- Conrad, M. Hötzer, D. Selective Integration of Formal Methods in the Development of Electronic Control Units Proc. 2. IEEE Int. Conf. on Formal Engineering Methods (ICFEM 98) IEEE Computer Society 144 155 1998
- Conrad, M. Beschreibung von Testszenarien für Steuergerätesoftware - Vergleichskriterien und deren Anwendung 1646 VDI Verlag 381 398 2001
- Conrad, M. A Systematic Approach to Testing Automotive Control Software Proc. Convergence 2004, SAE paper 2004-21-0039 2004
- Conrad M. Modell-basierter Test eingebetteter Software im Automobil - Auswahl und Beschreibung von Testszenarien Deutscher Universitäts-Verlag 2004
- Conrad, M. Sadeghipour, S. Wiesbrock, H.-W. Automatic Evaluation of ECU Software Tests Proc. SAE World Congress 2005, SAE paper 2005-01-1659 2005
- OSC - Embedded Systems AG
- Fenelon, P. McDermid, J.A. Nicholson, M. Pumfrey, D.J. Towards Integrated Safety Analysis and Design ACM Computing Reviews Aug. 1994 21 32 1994
- Grieskamp, W. Heisel, W. Doerr, H. Specifying embedded systems with statecharts and Z - An agenda for cyclic software components Proc. Formal Aspects of Software Engineering (FASE 98) Springer-Verlag 1998
- Grönberg, R. Conrad. M. Werkzeugunterstützung für Sicherheitsanalysen von Hardware- und Software-Systemen DaimlerChrysler AG, Forschung und Technologie Berlin, Germany 2000
- Harman, M. Hu, L. Munro, M. Zhang, X. Side-Effect Removal Transformation Proc. IEEE Int. Workshop on Program Comprehension (IWPC) Toronto, Canada 2001
- Hahn, G. Philipps, J. Pretschner, A. Stauner, T. Prototype-Based Tests for Hybrid Reactive Systems Proc. 14. IEEE Int. Workshop on Rapid System Prototyping San Diego, US 2003
- Horstmann, M. Schnieder, E. Mäder, P. Nienaber, S. Schulz, H.-M. A framework for interlacing Test and/with Design Proc. ICSE 2004 Workshop W14S on Software Engineering for Automotive Systems (SEAS 04) 2004
- Hermes, T. Schultze, A. Predelli, O. Software Quality is not a Coincidence - A Model-Based Test Case Generator Proc. SAE World Congress 2005, SAE paper 2005-01-1664 2005
- Hohler, B. Villinger, U. Normen und Richtlinien zur Qualitätssicherung von Steuerungssoftware Informatik-Spektrum 21 63 72 1998
- Analysis techniques for system reliability - Procedure for failure mode and effects analysis (FMEA)
- Fault tree analysis (FTA) 2004
- Jones B.-F. Sthamer, H. Eyres, D. Automatic structural testing using genetic algorithms Software Engineering Journal 11 5 299 306 1996
- Klein, T. Conrad, M. Fey, I. Grochtmann, M. Modellbasierte Entwicklung eingebetteter Fahrzeugsoftware bei DaimlerChrysler Lecture Notes in Informatics (LNI) P-45 Köllen Verlag 31 41 2004
- Korel, B. Automated Test Data Generation IEEE Transactions on Software Engineering 16 8 870 879 1990
- Lamberg, K. Beine, M. Eschmann, M. Otterbach, R. Conrad, M. Fey, I. Model-based Testing of Embedded Automotive Software using MTest Proc. SAE World Congress 2004, SAE paper 2004-01-1593 2004
- Linder, P. Modellbasiertes Testen von eingebetteter Software - Ein Ansatz auf der Grundlage von Signalflussplänen Proc. Automotive Safety & Security 2004 Stuttgart, Germany 2004
- Mackenthun, R. Kelling, C. An Agenda for the Safety Analysis of Cyclic Software Components ESPRESS: Final Reports Berlin, Germany June 1999
- dSPACE GmbH http://www.dspaceinc.com
- Papadopoulos, Y. McDermid J. Mavrides, A. Scheidler S. Maruhn, M. Model-Based Semiautomatic Safety Analysis Of Programmable Systems In Automotive Applications Proc. Int. Conf. on Advanced Driver Assistance Systems (ADAS 01) 2001
- Pohlheim, H. Evolutionäre Algorithmen - Ver-fahren, Operatoren, Hinweise aus der Praxis Berlin, Heidelberg Springer-Verlag 1999 http://www.pohlheim.com/eavoh/index.html
- Pohlheim, H. GEATbx - Genetic and Evolutionary Algorithm Toolbox for Matlab http://www.geatbx.com/ 1994-2005
- Ranville, S. MCDC Unit Test Vectors From Matlab Models - Automatically Proc. Embedded Systems Conference 2003
- Rau, A. Verwendung von Zusicherungen in einem modellbasierten Entwicklungsprozess 137 144 2002
- Rau, A. Model-based Development of Embedded Automotive Control Systems http://dissertation.de 2003
- Reactis Tester / Validator Reactive Systems Inc.
- Stürmer, I. Conrad, M. Test Suite Design for Code Generation Tools 18. IEEE Int. Conf. on Automated Software Engineering (ASE 03) 2003
- Simulink/Stateflow The MathWorks Inc. http://www.mathworks.com/products
- The MathWorks Inc. www.mathworks.com/products/simverification
- Sthamer, H. The Automatic Generation of Software Test Data Using Genetic Algorithms University of Glamorgan Pontyprid, UK 1996
- Tracey, N. Clark, J. Mander, K. McDermid, J. An Automated Framework for Structural Test Data Generation Proc. 13. IEEE Conf. on Automated Software Engineering 1998
- Computational Intelligence, Evolutionary algorithms - Terms and definitions VDI/VDE guideline VDI/VDE 3550, Part 3 Verein Deutscher Ingenieure 2003
- Wegener, J. Evolutionärer Test des Zeitverhaltens von Realzeit-Systemen Shaker-Verlag 2001
- Wegener, J. Sthamer, H. Baresel, A. Evolutionary Test Environment for Automatic Structural Testing Special Issue of Information and Software Technology 43 851 854 2001
- Wegener, J. Sthamer, H. Jones, B. Eyres, D. Testing Real-time Systems using Genetic Algorithms Software Quality Journal 6 2 127 135 1997