This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Real-Time Sequence Testing of an Automotive Electric Machine Control Systems
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 03, 2018 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
Automotive electronic control systems are expected to respond to input demands in real-time (circa: milliseconds) to ensure occupant and road user safety and comfort. System complexity and real-time computing requirements create significant challenges in proving the robustness of control systems; here robustness is the degree to which a system can function correctly in the presence of unexpected inputs. Evidence shows that faults still escape to customers incurring large warranty costs. Existing test methods can be ineffective in testing robustness with the primary focus being on requirements validation. Evidence from other industries such as IT and medical suggests faults that are difficult to find, manifest due to complex interactions and sequences of events. Research in model based software design, test optimization and formal methods - mathematical based approaches to prove robustness, is abundant in literature. However, modelling and simulation has scalability challenges being computationally intensive, requiring prohibitively excessive time to implement. Model based approaches also need to abstract the concept of “time” making them unsuitable for real-time testing. Yet there is little evidence within the literature pointing to effective sequence interaction testing within real-time test environments. A novel methodology for creating and running t-way input sequence interaction test suites in real-time is presented. For our real-world case study, a safety monitoring system deployed in a prototype embedded electric machine control system is tested with 2688 3-way event sequence interaction tests that trigger a fault in 210 sequences.
CitationDhadyalla, G., Yang, C., Marco, J., and Jennings, P., "Real-Time Sequence Testing of an Automotive Electric Machine Control Systems," SAE Technical Paper 2018-01-0004, 2018, https://doi.org/10.4271/2018-01-0004.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
|[Unnamed Dataset 2]|
|[Unnamed Dataset 3]|
|[Unnamed Dataset 4]|
|[Unnamed Dataset 5]|
- Mössinger, J. , “Software in Automotive Systems,” IEEE Software (2):92-94, 2010.
- Charette, R.N. , “This Car Runs on Code,” IEEE Spectrum 3, 2009.
- V. S. B. VOSA , “Vehicle Recalls Data,” 2013, http://www.dft.gov.uk/vosa/apps/recalls/default.asp, accessed Dec. 08, 2013.
- WarrantyDirect , “Manufacturer Ratings Electrical,” 2014, http://www.reliabilityindex.com/manufacturer/Electrical, accessed Nov. 13, 2014.
- Power, J.D. , “2015 U.S. Vehicle Dependability Study Results,” 2015, http://autos.jdpower.com/content/study-auto/I0X6eMP/2015-u-s-vehicle-dependability-study-results.htm, accessed Apr. 22, 2015.
- Smith, D.C. , “Overview of NHTSA and Its Activities Related to Unintended Acceleration and Vehicle Electronics,” NHTSA Associate Administrator of Enforcement, presentation to the National Academy of Sciences, http://onlinepubs.trb.org/onlinepubs/UA/100630DOTSlidesSmith.pdf2010, http://onlinepubs.trb.org/onlinepubs/UA/100630DOTSlidesSmith.pdf, accessed Mar. 12, 2015.
- NHTSA , “Technical Assessment of Toyota Electronic Throttle Control (ETC) Systems,” US Department of Transport - National Highway Traffic Safety Administration, http://www.nhtsa.gov/staticfiles/nvs/pdf/NHTSA_report_execsum.pdf2011, http://www.nhtsa.gov/staticfiles/nvs/pdf/NHTSA_report_execsum.pdf, accessed Mar. 12, 2015.
- Grottke, M. and Trivedi, K.S. , “A Classification of Software Faults,” Journal of Reliability Engineering Association of Japan 27(7):425-438, 2005.
- C. C. IEEE Standards, IEEE Computer Society., E. Institute of Electrical and Electronics, I. S. Board, and I. American National Standards , “IEEE Standard Glossary of Software Engineering Terminology,” (in English), IEEE Std 610.12-1990, p. 83, 1990.
- Kuhn, R., Kacker, R., Lei, Y., and Hunter, J. , “Combinatorial Software Testing,” IEEE Computer 42(8):94-96, 2009.
- Wallace, D.R. and Kuhn, D.R. , “Failure Modes in Medical Device Software: An Analysis of 15 Years of Recall Data,” International Journal of Reliability, Quality and Safety Engineering 8(04):351-371, 2001.
- Kuhn, D.R. and Okum, V. , “Pseudo-Exhaustive Testing for Software,” Software Engineering Workshop, 2006. SEW'06. 30th Annual IEEE/NASA, 2006, 153-158, IEEE.
- Oshana, R. and Kraeling, M. , “Software Engineering for Embedded Systems: Methods, Practical Techniques, and Applications,” (United States of America, Newnes, 2013), 1150.
- Kopetz, H. , “Real-Time Systems: Design Principles for Distributed Embedded Applications,” (Springer, 2011).
- Rao, C.R. , “Factorial Experiments Derivable from Combinatorial Arrangements of Arrays,” Supplement to the Journal of the Royal Statistical Society 9(1):128-139, 1947.
- Cohen, D.M., Dalal, S.R., Fredman, M.L., and Patton, G.C. , “The AETG System: An Approach to Testing Based on Combinatorial Design,” IEEE Transactions on Software Engineering 23(7):437-444, 1997.
- Cohen, D.M., Dalal, S.R., Parelius, J., and Patton, G.C. , “The Combinatorial Design Approach to Automatic Test Generation,” Software, IEEE 13(5):83-88, 1996.
- Chee, Y.M., Colbourn, C.J., Horsley, D., and Zhou, J. , “Sequence Covering Arrays,” SIAM Journal on Discrete Mathematics 27(4):1844-1861, 2013.
- Kuhn, D.R., Higdon, J.M., Lawrence, J.F., Kacker, R.N. et al. , “Combinatorial Methods for Event Sequence Testing,” 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (ICST), 2012, 601-609, IEEE.
- Colbourn, C.J. , “Combinatorial Aspects of Covering Arrays,” Le Matematiche (Catania) 58:121-167, 2004.
- Bryce, R.C., Colbourn, C.J., and Cohen, M.B. , “A Framework of Greedy Methods for Constructing Interaction Test Suites,” Proceedings of the 27th International Conference on Software Engineering, 2005, 146-155, ACM.
- Colbourn, C.J. and Dinitz, J.H. , “Handbook of Combinatorial Designs,” (London, Chapman & Hall/CRC, 2007).
- Sarkar, K. and Colbourn, C.J. , “Two-Stage Algorithms for Covering Array Construction,” arXiv preprint arXiv:1606.06730, 2016.
- Sarkar, K. and Colbourn, C.J. , “Upper Bounds on the Size of Covering Arrays,” arXiv preprint arXiv:1603.07809, 2016.
- Farchi, E., Segall, I., Tzoref-Brill, R., and Zlotnick, A. , “Combinatorial Testing with Order Requirements,” Software Testing, Verification and Validation Workshops (ICSTW), 2014 IEEE Seventh International Conference on, 2014, 118-127, IEEE.
- Erdem, E., Inoue, K., Oetsch, J., Pührer, J. et al. , “Answer-Set Programming as a New Approach to Event-Sequence Testing,” 2011.
- Brain, M. et al. , “Event-Sequence Testing Using Answer-Set Programming,” International Journal on Advances in Software 5(3-4):237-251, 2012.
- Margalit, O. , “Better Bounds for Event Sequencing Testing,” Software Testing, Verification and Validation Workshops (ICSTW), 2013 IEEE Sixth International Conference on, 2013, 281-284, IEEE.
- Rahman, M., Othman, R.R., Ahmad, R.B., and Rahman, M.M. , “Event Driven Input Sequence T-way Test Strategy Using Simulated Annealing,” Intelligent Systems, Modelling and Simulation (ISMS), 2014 5th International Conference on, 2014, 663-667, IEEE.
- Petke, J., Cohen, M.B., Harman, M., and Yoo, S. , “Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection,” IEEE Transactions on Software Engineering 41(9):901-924, 2015.
- Becci, G., Dhadyalla, G., Mouzakitis, A., and Marco, J. , “Robustness Testing of Real-Time Automotive Systems Using Sequence Covering Arrays,” SAE Int. J. Passeng. Cars - Electron. Electr. Syst. 6(1):287-293, 2013, doi:10.4271/2013-01-1228.
- dSPACE , “Real-Time Testing,” 2017,https://www.dspace.com/en/inc/home/products/sw/test_automation_software/automated_test_execution_in_re.cfm, accessed Oct. 29, 2017.