Multi-Objective Bayesian Optimization Supported by Deep Gaussian Processes

A common scenario in engineering design is the evaluation of expensive black-box functions: simulation codes or physical experiments that require long evaluation times and/or significant resources, which results in lengthy and costly design cycles. In the last years, Bayesian optimization has emerged as an efficient alternative to solve expensive black-box function design problems. Bayesian optimization has two main components: a probabilistic surrogate model of the black-box function and an acquisition functions that drives the design process. Successful Bayesian optimization strategies are characterized by accurate surrogate models and well-balanced acquisition functions. The Gaussian process (GP) regression model is arguably the most popular surrogate model in Bayesian optimization due to its flexibility and mathematical tractability. GP regression models are defined by two elements: the mean and covariance functions. In some modeling scenarios, the prescription of proper mean and covariance functions can be a difficult task, e.g., when modeling non-stationary functions and heteroscedastic noise. Motivated by recent advancements in the deep learning community, this study explores the implementation of deep Gaussian processes (DGPs) as surrogate models for Bayesian optimization in order to build flexible predictive models from simple mean and covariance functions. The proposed methodology employs DGPs as the surrogate models and the Euclidean-based expected improvement as the acquisition function. This approach is compared with a strategy that employs GP regression models. These methodologies solve two analytical problems and one engineering problem: the design of sandwich composite armors for blast mitigation. The analytical problems involve non-convex and segmented Pareto fronts. The engineering problem involves expensive finite element simulations, three design variables, and two expensive black-box function objectives. The results show that the architecture of the DGP model plays an important role in the performance of the optimization approach. If the DGP architecture is adequate, the implementation of DGPs produces satisfactory results; otherwise, the use of GP regression models is preferable.