Nested vs. Non-Nested Sampling: Definition of an Infilling Strategy for Multi-Fidelity Bayesian Optimization Based on Data Correlation

The multi-fidelity Bayesian optimization of large models with multiple parameters can be computationally very expensive and thereby the proper choice of the infilling strategy is crucial to minimize the required convergence time. If, on one hand, nested infilling guarantees a better performance of the multi-fidelity Gaussian algorithm, on the other hand the new samples added at every iteration and at all levels represent a non-negligible cost factor. In this paper the two alternative infilling approaches, nested and non-nested sampling, are analyzed for a family of data sets, each one characterized by a different correlation factor. The aim is to establish for each case the best option that minimizes the computational time and thus a connection between correlation level of the data and infilling strategy. Three data sets are investigated: low, medium and highly correlated data in accordance with their correlation matrix calculated ex ante knowing the exact solutions in the entire domain. The total process time is given by the summation for each iteration of the time required for the single acquisition and this variable depends not only on the infilling strategy, single or multilevel, but also on the single level acquisition itself assuming that adding points on the high-fidelity level is more expensive than on the low-fidelity level. For this reason a weighted selection of the acquisition function (AQF) is defined. The exact data correlation factor (DCF) is in real cases unknown but the partial data correlation factor based on the available data can already provide useful information. The present study has general validity but it has been especially designed as preliminary investigation for the optimization of a laser welding process.