Model Release Process using Standardized Error Metrics for Validation of X-in-the-Loop Simulation Models

The current automotive market is dynamic, leading to complex functionalities being incorporated into the control software of various components like engine, gearbox, battery, E-motor etc. This results in utilization of virtual environments for software testing to reduce the development time. The virtual platforms under the category X-in-the-Loop (XiL) e.g. Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL) use simulated models to achieve a desired test goal. These component models must be rigorously validated to ensure the quality of XiL-Testing. Thus, it is essential to define a model release process that maintains model quality irrespective of the modeling approach used and the user. One of the challenges is to choose an appropriate Error Metric (EM) that sets criteria for model release. This paper proposes a combination of Theil’s Inequality Coefficient (TIC) and Unscaled Mean Bounded Relative Absolute Error (UMBRAE) as the EM. TIC is used to characterize the steady and non-dynamic variables. In contrast, UMBRAE is used to characterize transient dynamic variables. The entire operating range of the component model is validated using this combination of EM. This paper presents the process to validate an engine plant model against test-bench measurements. The quality level of each output signal is categorized as “excellent” if EM is less than 1, “good” if EM is less than 5, or “acceptable” if the EM is less than 10. The mean of EM of all signals represents the overall model quality and maximum of EM represents the worst-case scenario. The inaccurate signals and corresponding functionalities can be promptly identified through the process. These models can be iteratively improved to ensure that the quality demands are met for all the signals.