Precisely defined terms and metrics as well as a reliable base of information are the foundation for integrated maturity assessment of complex electronic systems. In this paper, a method is proposed for defining and quantifying multiple relations between systems, functions, requirements, and test cases, allowing to improve traceability in the specification documents and to define terms like test progress, test coverage and system maturity formally.
The method is based on an extended generic data model replacing commonly used hierarchical data models for storing function specifications and test artifacts. The key feature are allocation matrices for realizing weighted n:m-relations between systems, requirements, and test cases. Theses matrices represent the distributed systems according to the vehicle network architecture, as well as the requirement coverage of the specified test cases. The method includes classifications to simplify data collection for the allocation matrices and default initializations for accelerating the practical use. Upon test execution, relevant metrics like test progress, test coverage, level of confidence, and system maturity can be generated by simple matrix operations. The final maturity assessment can be broken down to single test cases and relations, while the 100% maturity level is clearly defined by the available specifications and allocations. Additional benefits include ranking of test cases according to system coverage and priority based test planning.
The method has been developed based on practical experience in dozens of E/E integration test campaigns for premium passenger cars and commercial vehicles. It supports the implementation of recognized software quality standards like ISO/IEC 14598 [9], ISO/IEC 25000 [10], and IEEE 1061 [8]. Results and examples from recent vehicle projects illustrate the capabilities of this approach.