Selection of Surrogate Models with Metafeatures

Modeling and simulation of ground vehicles can be a computationally expensive problem due to the complexity of high-fidelity vehicle models. Often to determine mobility metrics, multiple stochastic simulations need to be evaluated. Surrogate models, or models of models, offer a means to reduce the computational cost of these simulation efforts. Since various types of surrogate models are available to the user, choosing the best surrogate model for a simulation is mostly the challenging process. In this paper, the process of selecting surrogate models and its uses based on model metafeatures is presented. The approach formulates this decision as a trade-off among three main drivers, required dataset size (how much information is necessary to compute the surrogate model), surrogate model accuracy (how accurate the surrogate model must be) and total computational time (how much time is required for the surrogate modeling process). Using an automated surrogate model selection, different model metafeatures were employed to train a Classification and Regression Tree (CART). This is performed by exhaustively evaluating several surrogate model types versus a set of training examples. The framework then uses the resulting CART classifier to select future surrogate models based on the classification learned from the training data set - hence forming a predictive estimation framework. The optimization process involves the use of codes to allow surrogates with varying numbers of candidate and hyperparameter kernels to be considered. The effectiveness of the CART classifier is then evaluated by contrasting its performance with a different set of surrogate models that were not a part of the training process.