This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Model Release Process using Standardized Error Metrics for Validation of X-in-the-Loop Simulation Models
Technical Paper
2021-01-1148
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
The current automotive market is dynamic, leading to complex functionalities being incorporated into the control software of various components like engine, gearbox, battery, E-motor etc. This results in utilization of virtual environments for software testing to reduce the development time. The virtual platforms under the category X-in-the-Loop (XiL) e.g. Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL) use simulated models to achieve a desired test goal. These component models must be rigorously validated to ensure the quality of XiL-Testing. Thus, it is essential to define a model release process that maintains model quality irrespective of the modeling approach used and the user. One of the challenges is to choose an appropriate Error Metric (EM) that sets criteria for model release. This paper proposes a combination of Theil’s Inequality Coefficient (TIC) and Unscaled Mean Bounded Relative Absolute Error (UMBRAE) as the EM. TIC is used to characterize the steady and non-dynamic variables. In contrast, UMBRAE is used to characterize transient dynamic variables. The entire operating range of the component model is validated using this combination of EM. This paper presents the process to validate an engine plant model against test-bench measurements. The quality level of each output signal is categorized as “excellent” if EM is less than 1, “good” if EM is less than 5, or “acceptable” if the EM is less than 10. The mean of EM of all signals represents the overall model quality and maximum of EM represents the worst-case scenario. The inaccurate signals and corresponding functionalities can be promptly identified through the process. These models can be iteratively improved to ensure that the quality demands are met for all the signals.
Recommended Content
Journal Article | Prediction of Friction Drive Limit of Metal V-Belt |
Ground Vehicle Standard | Model Architecture and Interfaces Recommended Practice for Ground Vehicle System and Subsystem Dynamical Simulation |
Technical Paper | A Study on the Torque Capacity of a Metal Pushing V-Belt for CVTs |
Authors
Citation
Narasimha, V., Nayak, M., and Bick, A., "Model Release Process using Standardized Error Metrics for Validation of X-in-the-Loop Simulation Models," SAE Technical Paper 2021-01-1148, 2021, https://doi.org/10.4271/2021-01-1148.Also In
References
- Caraceni , A. , De Cristofaro , F. , Ferrara , F. , Scala , S. et al. Benefits of Using a Real-Time Engine Model During Engine ECU Development SAE Technical Paper 2003-01-1049 2003 https://doi.org/10.4271/2003-01-1049
- Köhl , S. and Jegminat , D. How to Do Hardware-in-the-Loop Simulation Right SAE Technical Paper 2005-01-1657 2005 https://doi.org/10.4271/2005-01-1657
- Niggemann , O. , Otterbach , R. , Fleischer , D. , and Jogi , S. Using Software Architecture Models in Automotive Development Processes SAE Technical Paper 2008-01-2664 2008 https://doi.org/10.4271/2008-01-2664
- Allen , J. Simulation and Test Systems for Validation of Electric Drive and Battery Management Systems SAE Technical Paper 2012-01-2144 2012 https://doi.org/10.4271/2012-01-2144
- Himmler , A. , Lamberg , K. , and Beine , M. Hardware-in-the-Loop Testing in the Context of ISO 26262 SAE Technical Paper 2012-01-0035 2012 https://doi.org/10.4271/2012-01-0035
- Lee , S. , Andert , J. , Neumann , D. , Querel , C. et al. Hardware-in-the-Loop-Based Virtual Calibration Approach to Meet Real Driving Emissions Requirements SAE Int. J. Engines 11 6 2018 1479 1504 https://doi.org/10.4271/2018-01-0869
- Xia , F. , Lee , S.-Y. , Andert , J. , Kampmeier , A. et al. Crank-Angle Resolved Real-Time Engine Modelling SAE International Journal of Engines 11 6 2018 1385 1398
- Santos , J. , Vasconcelos , F. , Silva , E. , Guimarães , A. et al. A Method to Calculate Hardware in the Loop Applications Representativeness SAE Technical Paper 2018-36-0171 2018 https://doi.org/10.4271/2018-36-0171
- Dong , Y. , Li , M. , and Josey , R. Model Based Software Development for Automotive Electronic Control Units SAE Technical Paper 2004-21-0038 2004
- Venkitachalam , H. , Granrath , C. , Gopalakrishnan , B. , and Richenhagen , J. Metric-based Evaluation of Powertrain Software Architecture SAE Int. J. Passeng. Cars - Electron. Electr. Syst. 10 1 2017 194 208 https://doi.org/10.4271/2017-01-1615
- Romero , V. Comparison of Several Model Validation Conceptions Against a Real Space End-to-End Approach SAE International Journal of Materials and Manufacturing 4 1 2011 396 420
- De Gooijer , J.G. and Hyndman , R.J. 25 Years of Time Series Forecasting International journal of forecasting 22 3 2006 443 473
- Hyndman , R.J. and Koehler , A.B. Another Look at Measures of Forecast Accuracy International Journal of Forecasting 22 4 2006 679 688
- Armstrong , J.S. Principles of Forecasting: a Handbook for Researchers and Practitioners 30 Springer Science & Business Media 2001
- Schnaars , S.P. Long-Range Forecasting: From Crystal Ball to Computer: J. scott armstrong, 2nd ed. (wiley, new york, 1985) [uk pound]22.95 (paper), pp. 689 International Journal of Forecasting 2 3 1986 387 390
- Goodwin , P. and Lawton , R. On the Asymmetry of the Symmetric Mape International Journal of Forecasting 15 4 1999 405 408
- Davydenko , A. and Fildes , R. Measuring Forecasting Accuracy: The Case of Judgmental Adjustments to Sku-Level Demand Forecasts International Journal of Forecasting 29 3 2013 510 522
- Murray-Smith , D. Testing and Validation of Computer Simulation Models: Principles, Methods and Applications. Simulation Foundations, Methods and Applications Springer International Publishing 2015
- Chen , C. , Twycross , J. , and Garibaldi , J.M. A New Accuracy Measure Based on Bounded Relative Error for Time Series Forecasting PloS one 12 3 2017 e0174202
- Botchkarev , A. 2018