Virtual EET for Autonomous Ground Vehicles: Integrating Qualitative and Quantitative Analyses in Autonomous Vehicle Simulation

Spurred by the constraints of the COVID-19 pandemic, virtual testing is becoming an increasingly essential method for verification and validation of autonomous ground vehicle simulation tools. The Mobility Systems Branch (MSB) of the US Army Corps of Engineers Engineering Research and Development Center (ERDC) Geotechnical and Structures Laboratory (GSL) has developed a new approach in physics-based virtual testing of autonomous ground vehicle systems through the incorporation of both qualitative and quantitative data in congruency with ERDC’s Software-in-the-Loop laboratory. Virtual testing of autonomous vehicles combines simulation tools consisting of vehicle and sensor models represented in a virtual scene with both performer observations and modeling and simulation observations. The first iteration of a Virtual Engineering Evaluation Test (V-EET) for robotic and autonomous ground vehicle systems took place in 2021 at the ERDC in Vicksburg, Mississippi. Virtual testing took place over the course of several months with remote researchers participating from across the country. Researchers used a combination of quantitative and qualitative methods to identify discrepancies between traditional field Engineering Evaluation Test data collection and V-EET data collection. Also identified were issues within testing protocols and difficulties associated with overwhelming and complex data sets. Building on findings, researchers developed a new virtual testing framework that addressed these issues and included more versatility. This new framework included streamlined and efficient data collection and analysis, standardization of observation collection techniques, and objectification of qualitative data to be used across relevant products with visual or human components. This will then provide the most efficient and robust products possible and improve situational awareness for autonomous vehicle assessment in complex on- and off-road environments.