This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Assessment of Driving Simulators for Use in Longitudinal Vehicle Dynamics Evaluation
Technical Paper
2022-01-0533
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
In the last decade, the use of Driver-in-the-Loop (DiL) simulators has significantly increased in research, product development, and motorsports. To be used as a verification tool in research, simulators must show a level of correlation with real-world driving for the chosen use case. This study aims to assess the validity of a low-cost, limited travel Vehicle Dynamics Driver-in-Loop (VDDiL) simulator by comparing on-road and simulated driving data using a statistical evaluation of longitudinal and lateral metrics. The process determines if the simulator is appropriate for verifying control strategies and optimization algorithms for longitudinal vehicle dynamics and evaluates consistency in the chosen metrics. A validation process explaining the experiments, choice of metrics, and analysis tools used to perform a validation study from the perspective of the longitudinal vehicle model is shown in this study.
Validity is measured statistically, where a simulator is said to have absolute validity if the chosen metrics are not statistically different between on-road and simulator driving. Metrics showing statistically significant differences but with a consistent trend imply relative validity. In this study, an instrumented vehicle is used to collect on-road driving data in a defined urban route with 14 drivers with no professional driving experience. The vehicle model used in the simulator was matched to the actual vehicle in the longitudinal dynamics using CarSim. The urban route used for on-road driving was replicated in the virtual world with geometric accuracy.
VDDiL was rated to have moderate consistency with the real-world driving experience implying medium physical fidelity with a combination of absolute and relative validity for longitudinal metrics. On average, the speed-based metrics in the simulator were approximately 5 mph higher, with acceleration and jerk-based metrics about 1.5 - 3 times of corresponding on-road metrics.
Authors
Citation
Sekar, R., Jacome, O., Chrstos, J., and Stockar, S., "Assessment of Driving Simulators for Use in Longitudinal Vehicle Dynamics Evaluation," SAE Technical Paper 2022-01-0533, 2022, https://doi.org/10.4271/2022-01-0533.Also In
References
- Aykent , B. , Chardonnet , J.-R. , Merienne , F. , and Kemeny , A. Angular Velocity Perception Threshold and Sense of Presence for a Three Degrees of Freedom (DOF) Driving Simulator in Virtual Environment SAE Technical Paper 2018-01-5043 2018 https://doi.org/10.4271/2018-01-5043
- Bassani , M. . Catani , L. , Ignazzi , A. , and Piras , M. Validation of a Fixed-Base Driving Simulator to Assess Behavioural Effects of Road Geometrics Proceedings of the Driving Simulation Conference 2018 Europe VR Driving Simulation Association Antibes, France 2018 101 108
- Bella , F. Validation of a Driving Simulator for Work Zone Design Transportation Research Record: Journal of the Transportation Research Board 1937 1 2005 136 144 https://doi.org/10.1177/0361198105193700119
- Bella , F. Driving Simulator for Speed Research on Two-Lane Rural Roads Accident Analysis & Prevention 40 3 2008 1078 1087 https://doi.org/10.1016/j.aap.2007.10.015
- Boer , E.R. , Girshick , A.R. , Kuge , N. , and Yamamura , T. Experiencing the Same Road Twice: A Driver Centered Comparison between Simulation and Reality DSC 2000: Driving Simulation Conference. Conference Proceedings Paris, France 2000
- Boonsiripant , S. Speed Profile Variation as a Surrogate Measure of Road Safety Based on GPS-Equipped Vehicle Data Georgia Institute of Technology 2009
- Deshmukh , A. , Mulani , B. , Jadhav , N. , and Parihar , A. Study of Frequency Characteristics of Vehicle Motions for the Derivation of Inherent Jerk SAE Int. J. Passeng. Cars - Mech. Syst. 9 1 2016 419 423 https://doi.org/10.4271/2016-01-1681
- de Groot , S. , de Winter , J.C. , Mulder , M. , and Wieringa , P.A. Nonvestibular Motion Cueing in a Fixed-Base Driving Simulator: Effects on Driver Braking and Cornering Performance Presence 20 2 2011 117 142 https://doi.org/10.1162/pres_a_00039
- Huang , Q. and Wang , H. Fundamental Study of Jerk: Evaluation of Shift Quality and Ride Comfort SAE Technical Paper 2004-01-2065 2004 https://doi.org/10.4271/2004-01-2065
- Hussain , Q. , Alhajyaseen , W.K.M. , Pirdavani , A. , Reinolsmann , N. et al. Speed Perception and Actual Speed in a Driving Simulator and Real-World: A Validation Study Transportation Research Part F: Traffic Psychology and Behaviour 62 2019 637 650 https://doi.org/10.1016/j.trf.2019.02.019
- Knapper , A. , Christoph , M. , Hagenzieker , M. , and Brookhuis , K. Comparing a Driving Simulator to the Real Road Regarding Distracted Driving Speed European Journal of Transport and Infrastructure Research 15 2 2015 https://doi.org/10.18757/EJTIR.2015.15.2.3069
- Panerai , F. , Droulez , J. , Kemeny , A. , Balligand , E. et al. Speed and Safety Distance Control in Truck Driving: Comparison of Simulation and Real-World Environment 2001
- Parduzi , A. , Venrooij , J. , Peller , M. , Forster , Y. et al. Evaluation of a 3-DOF Mid-Size Simulator Concept: A Behavioural Validation Study Proceedings of the Driving Simulation Conference 2019 Europe VR Driving Simulation Association Strasbourg, France 2019 57 64
- Plouzeau , J. , Paillot , D. , Aykent , B. , and Merienne , F. Vibrations in Dynamic Driving Simulator: Study and Implementation CONFERE 2013 France 2013 1 8
- Reed , M.P. and Green , P.A. Comparison of Driving Performance On-Road and in a Low-Cost Simulator Using a Concurrent Telephone Dialing Task Ergonomics 42 8 1999 1015 1037 https://doi.org/10.1080/001401399185117
- McGehee , D.V. , Mazzae , E.N. , and Baldwin , G.H.S. Driver Reaction Time in Crash Avoidance Research: Validation of a Driving Simulator Study on a Test Track Proceedings of the Human Factors and Ergonomics Society Annual Meeting 44 2000 3-320 3-323 https://doi.org/10.1177/154193120004402026
- Romano , R. , Markkula , G. , Boer , E. , Jamson , H. et al. An Objective Assessment of the Utility of a Driving Simulator for Low Mu Testing Transportation Research Part F Traffic Psychology and Behaviour 65 2019 https://doi.org/10.1016/j.trf.2019.07.001
- Reymond , G. , Kemeny , A. , Droulez , J. , and Berthoz , A. Role of Lateral Acceleration in Curve Driving: Driver Model and Experiments on a Real Vehicle and a Driving Simulator Human Factors 43 3 2001 483 495 https://doi.org/10.1518/001872001775898188
- Rubanraj , S. 2020 http://rave.ohiolink.edu/etdc/view?acc_num=osu1576801483307636
- Rubanraj , S. , Chrstos , J.P. , Dupaix , R. , and Romano , R. Assessing the Impact of 3DOF Motion on Driver Performance in a Small DiL DSC 2020 Europe VR
- Santos , J. , Merat , N. , Mouta , S. , Brookhuis , K. et al. The Interaction between Driving and In-vehicle Information Systems: Comparison of Results from Laboratory, Simulator and Real-World Studies Transportation Research Part F: Traffic Psychology and Behaviour 8 2 2005 135 146 https://doi.org/10.1016/j.trf.2005.04.001
- Shechtman , O. , Classen , S. , Awadzi , K. , and Mann , W. Comparison of Driving Errors between On-the-Road and Simulated Driving Assessment: A Validation Study Traffic Injury Prevention 10 4 2009 379 385 https://doi.org/10.1080/15389580902894989
- Siegler , I. , Reymond , G. , Kemeny , A. , and Berthoz , A. Sensorimotor Integration in a Driving Simulator: Contributions of Motion Cueing in Elementary Driving Tasks Proceedings of Driving Simulation Conference 2001
- Mullen , N. , Charlton , J. , Devlin , A. , and Bédard , M. Simulator Validity: Behaviors Observed on the Simulator and on the Road Handbook of Driving Simulation for Engineering, Medicine, and Psychology 2011
- Godley , S.T. , Triggs , T.J. , and Fildes , B.N. Driving Simulator Validation for Speed Research Accident Analysis & Prevention 34 5 2002 589 600 https://doi.org/10.1016/S0001-4575(01)000562
- Wilkerson , J. 2020 http://rave.ohiolink.edu/etdc/view?acc_num=osu1595262540712316
- Williams , D. , Nhila , A. , and Sherwin , K. Lane Keeping Evaluation of Various Steering Systems SAE Technical Paper 2016-01-8012 2016 https://doi.org/10.4271/2016-01-8012
- Wynne , R.A. , Beanland , V. , and Salmon , P.M. Systematic Review of Driving Simulator Validation Studies Safety Science 117 2019 138 151 https://doi.org/10.1016/j.ssci.2019.04.004
- Zeng , X. , Cui , H. , Song , D. , Yang , N. et al. Jerk Analysis of a Power-Split Hybrid Electric Vehicle Based on a Data-Driven Vehicle Dynamics Model Energies. 11 2018 1537 https://doi.org/10.3390/en11061537
- Grant , P.R. and Haycock , B. Effect of Jerk and Acceleration on the Perception of Motion Strength Journal of Aircraft 45 4 2008 1190 1197 https://doi.org/10.2514/1.33757
- Miller , S. 2001
- Auckland , R.A. , Manning , W.J. , Carsten , O.M.J. , and Jamson , A.H. Advanced Driver Assistance Systems: Objective and Subjective Performance Evaluation Vehicle System Dynamics 46 S1 2008 883 897 https://doi.org/10.1080/00423110802037131
- Auckland , R. , Carsten , O. , Levesley , M. , Crolla , D. et al. Dynamic Performance of Adaptive Cruise Control Vehicles SAE Technical Paper 2005-01-0403 2005 https://doi.org/10.4271/2005-01-0403
- Bham , G.H. , Leu , M.C. , Vallati , M. , and Mathur , D.R. Driving Simulator Validation of Driver Behavior with Limited Safe Vantage Points for Data Collection in Work Zones Journal of Safety Research 49 53 2014 e1 e60 https://doi.org/10.1016/j.jsr.2014.02.012
- Klee , H. , Bauer , C. , Radwan , E. , and Al-Deek , H. Preliminary Validation of Driving Simulator Based on Forward Speed Transportation Research Record 1689 1 1999 33 39 https://doi.org/10.3141/1689-05
- Alm , H. Driving Simulators as Research Tools: A Validation Study Based on the VTI Driving Simulator Swedish National Road and Transport Research Institute 1996
- Harms , L. Driving Performance on a Real Road and in Driving Simulator: Results of a Validation Study Gale , A.G. , Brown , I.D. , Haslegrave , C.M. , Taylor , S.P. Vision in Vehicles Amsterdam Elsevier 1996 19 26
- MATLAB 2020
- CarSim Mechanical Simulation Ann Arbor, MI CarSim 2019
- https://www.sensodrive.de
- D-BOX Technologies Inc. https://www.d-box.com
- Vehicle Dynamics International https://www.vehicledynamicsinternational.com
- Tire Technology International https://www.tiretechnologyinternational.com
- © OpenStreetMap Contributors https://www.openstreetmap.org/copyright
- IEEE Standard for System, Software, and Hardware Verification and Validation IEEE Std. 1012-2016 (Revision of IEEE Std. 1012-2012/Incorporates IEEE Std. 1012-2016/Cor1-2017) 1 260 2017 https://doi.org/10.1109/IEEESTD.2017.8055462
- Kennedy , R.S. , Lane , N.E. , Berbaum , K.S. , and Lilienthal , M.G. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness The International Journal of Aviation Psychology 3 3 1993 203 220 https://doi.org/10.1207/s15327108ijap0303_3