The Effect of Multi-Additional Sampling for Multi-Fidelity Efficient Global Optimization

Abstract

Powerful computer-aided design tools are presently vital for engineering product development. Efficient global optimization (EGO) is one of the most popular methods for design of a high computational cost problem. The original EGO is proposed for only one additional sample point. In this work, parallel computing is applied to the original EGO process via a multi-additional sampling technique. The weak point of the multi-additional sampling is it has slower convergence rate when compared with the original EGO. This paper applies the multi-fidelity technique to the multi-additional EGO process to see the effect of the number of multi-additional sampling points and the converge rate. A co-kriging method and a hybrid RBF/Kriging surrogate model are selected for the surrogate model in the EGO process to show the advantage of the multi-additional EGO process compared with the single-fidelity Kriging surrogate model. In the experiment, single-additional sampling points and two to four number of multi-additional sampling per iteration are tested with symmetry and asymmetry mathematical test functions. The results show the hybrid RBF/Kriging surrogate model can obtain the similar optimal points when using the multi-additional sampling EGO.

1. Introduction

Presently, computer-aided optimization is one of the most important computer-aided design tools for product development and improvement. In recent years, the optimization design for expensive function was carried out using a combination of global optimization techniques. Due to high-cost computation function, a surrogate model and an optimization technique are combined to increase the efficiency of the optimization algorithm. The optimization with a surrogate model is popular and used in the various fields such as, mechanical engineering [1,2], chemical engineering [3,4], aerospace engineering [5,6], etc. One of the popular methods that combined the surrogate model and the optimization technique to solved an expensive function is an efficient global optimization (EGO) [7] because it can reduce the computational cost for an optimization process. The EGO process starts with generating an initial set of sampling points. The objective of this process is to increase the diversity of the sampling points. The most popular method for generating the initial sampling is the Latin Hypercube Sampling (LHS) [8]. Most of researchers try to increase the efficiency of this process [9,10,11]. Then, the surrogate model is used to estimate the actual function value. The original EGO uses the Kriging method, which the accuracy of this function directly affects the efficiency of the optimization method. Finally, the optimization method is used to find the optimal point of the surrogate model and that point is used to evaluate the next additional sampling point to increase the accuracy of the surrogate model and find the better optimal solution. Therefore, the important process of the EGO is to find additional points. The original EGO is designed for finding single additional sampling points per one iteration. However, the time required for completing a single simulation is often one day or more, thus, finding numerous sampling points for the original EGO will take a longer time. Much research work has solved this problem by using the parallel computing technique [12]. For example, Ginsbourger et al. [13,14] extended the original EGO to parallel computing by applying the multivariate expected improvement (q-EI) and implementing it via Monte Carlo simulation. Then, Frazier [15] proposed a q-EI stochastic gradient approach for calculation the q-EI. Masahiro M. [16] proposed a method for adding various points in combination with maximization of EI by using a multi-additional sampling (MAs) method.

The weak point of the multiple additional sampling EI is it requires more experiments or the expensive data than the single additional sampling [17] because it can use less data to predict a function with the surrogate model. The target to increase the efficiency of the multiple additional sampling points by EGO is to increase the accuracy of the surrogate model. Fortunately, in real-world design problems, it is possible to select a variety of physical computation models to tackle a design problem. For example, an efficiency of a helicopter rotor blade [18] could be calculated by using the blade element momentum method (BEMT) or using Navier–Stokes equations. However, the advantage of the BEMT is that it can calculate the efficiency within a few minutes but it has lower efficiency than calculation via the Navier–Stokes equation which takes significantly longer computing time. The efficiency of an airfoil could be calculated using a potential flow equation [19] and the Navier–Stokes equation [20]. For the structural analysis, the Finite Element Method (FEM) and a novel numerical implementation of the asymptotic homogenization method (NIAH) [21] could be used to predict the buckling analysis. With this advantage, many researchers try to combine the data of several fidelity levels to increase the accuracy of the surrogate model for the EGO process. Forrester, Sóbester and Keane [22] proposed the co-Kriging method for the multi-fidelity optimization. Huang [23] used the co-Kriging method as the multi-fidelity surrogate model to design the aircraft. Brooks [24] also used the co-Kriging method to design the transonic compressor rotor. Sun [25] used the radial basis function (RBF) and the response surface hybrid surrogate model to find the optimal shape of the honeycomb structure. Liu [26] proposed the Gaussian/RBF surrogate model for multi-fidelity optimization. In addition, Ariyarit [18,27] proposed the RBF/Kriging surrogate model applied with the EGO to find the optimal shape of the helicopter rotor blade and the airfoil. However, most of the proposed multi-fidelity surrogate models are designed for single-additional sampling points.

In this study, a multi-additional multi-fidelity EGO, which could improve the efficiency in searching for an optimum point, is proposed and investigated by implementing it to symmetry and asymmetry mathematical test problems. The co-Kriging and RBF/Kriging hybrid surrogate model are selected to show which surrogate model is most suitable for the proposed method. The results of the proposed method are compared with the original multi-additional sampling EGO.

2. Efficient Global Optimization

The EGO [7] procedure, shown in Figure 1, starts with the generated samples for low fidelity and high-fidelity using the design of experiment technique (DoE) [28]. In this research, the LHS [8] is selected for this process, because it could maintain the diversity of the data and can fix the number of experiments by a user. The next process is to construct the surrogate model, in which the Kriging method is selected for a single-fidelity optimization process and co-Kriging and RBF/Kriging hybrid surrogate model are selected for a multi-fidelity optimization process. The schematic diagram of the single-fidelity and multi-fidelity surrogate models is shown in Figure 2. After the surrogate model is constructed, an additional sample point for an optimization method can be found by maximizing the EI [7] via genetic algorithms [29].

The EI($E\left[I\right(\mathbf{x}\left)\right]$) at point $\mathbf{x}$ can be expressed as

$$I\left(\mathbf{x}\right)=\max [f_{\mathrm{ref}}-\widehat{y}\left(\mathbf{x}\right),0],$$

(1)

$$E\left[I\left(\mathbf{x}\right)\right]={\int }_{\infty }^{f_{\mathrm{ref}}}(f_{\mathrm{ref}}-\widehat{y}\left(\mathbf{x}\right))\varphi \left(\widehat{y}\left(\mathbf{x}\right)\right)dy,$$

(2)

where $\varphi$ is the probability density function representing uncertainty about $\widehat{y}\left(\mathbf{x}\right)$, which predicts a function value from a surrogate model. Additional sampling points based on using the EI is repeated until an objective function converges.

2.1. Kriging Surrogate Model

An ordinary Kriging model [30] can predict the unknown function $\widehat{y}\left(\mathbf{x}\right)$ as

$$\widehat{y}\left(\mathbf{x}\right)=\mu \left(\mathbf{x}\right)-\epsilon \left(\mathbf{x}\right)$$

(3)

where $\mu \left(\mathbf{x}\right)$ and $\epsilon \left(\mathbf{x}\right)$ denote a global model and a local model, respectively. The global model ($\mu \left(\mathbf{x}\right)$) is expressed as

$$\mu =\frac{{\mathbf{1}}^T{\mathbf{R}}^{-1}\mathbf{F}}{{\mathbf{1}}^T{\mathbf{R}}^{-1}\mathbf{1}}$$

(4)

where $\mathbf{R}$ is a matrix denoting the correlation between the sample points, and $\mathbf{F}$ is a vector containing the evaluation value of each sampling point. For the Kriging surrogate model, $\mu$ denotes the constant global model. The local model ($\epsilon \left(\mathbf{x}\right)$) is expressed as

$$\epsilon \left(\mathbf{x}\right)=\mathbf{r}{\left(\mathbf{x}\right)}^{-1}{\mathbf{R}}^{-1}(\mathbf{F}-\mathbf{1}\mu )$$

(5)

where $\mathbf{r}\left(x\right)$ is a vector written in terms of $\mathbf{x}.\mathbf{r}\left(x\right)$ is a vector of sampling points. The correlation between $\epsilon \left(\mathbf{x}\right)$ and $\epsilon \left({\mathbf{x}}^i\right)$ is a function of the distance between $\mathbf{x}$ and ${\mathbf{x}}^i$. In the Kriging surrogate model, the local derivation at an unknown point $\mathbf{x}$ is expressed using stochastic processes. Several design points are generated as sampling points and then a surrogate model is constricted using a Gaussian random function as the correlation function for the estimation of the trend using the stochastic process.

2.2. Co-Kriging Surrogate Model

A Co-Kriging surrogate model [22] is the extension of a Kriging surrogate model by combining the low-fidelity data and high fidelity data. A co-Kriging method can be predicted an unknown function $\widehat{y}\left(\mathbf{x}\right)$ by Equation (3). However, the different point of these method is the co-Kriging model estimates the global model $\mu \left(\mathbf{x}\right)$ and the local model $\epsilon \left(\mathbf{x}\right)$ by combining data of the low-fidelity and the high-fidelity data by using the scaling parameter ($\rho$) of the different model ($\mathbf{d}$), which is defined as:

$$\mathbf{d}={\mathbf{F}}_h-\rho {\widehat{\mathbf{y}}}_l\left({\mathbf{x}}_h\right)$$

(6)

where ${\mathbf{F}}_h$ is a vector that contains the high-fidelity evaluation value of each high-fidelity sample points, and ${\widehat{\mathbf{y}}}_l\left({\mathbf{x}}_h\right)$ is the estimated value via the Kriging surrogate model of each high-fidelity sample points.

2.3. Rbf/Kriging Multi-Fidelity Surrogate Model

A RBF/Kriging multi-fidelity surrogate model [18] is the combination surrogate model between a Radial Basis Function (RBF) [31] and the Kriging surrogate model. In this surrogate model, the RBF is used to predicted the local model ($\mu \left(x\right)$) of the Kriging surrogate model in Equation (3). The different point of the RBF/Kriging hybrid surrogate model and co-Kriging surrogate model is the hybrid surrogate model uses the low-fidelity data to predict the global model ($\mu \left(x\right)$), but the co-Kriging method uses the low-fidelity data to predict the local model ($\epsilon \left(\mathbf{x}\right)$).The hybrid surrogate model can predict the unknown function $\widehat{y}\left(\mathbf{x}\right)$ as

$$\widehat{y}\left(\mathbf{x}\right)=\mu +f_{RBF}\left(\mathbf{x}\right)+\mathbf{r}{\left(\mathbf{x}\right)}^T{\mathbf{R}}^{-1}\left({\mathbf{F}}_h-\mathbf{1}\mu -\mathbf{1}{f}_{RBF}\left(\mathbf{x}\right)\right)$$

(7)

In this study, $f_{RBF}\left(\mathbf{x}\right)$ is calculated using the data of low-fidelity data and the $f_{RBF}\left(\mathbf{x}\right)$ at an unknown design point ($\mathbf{x}$) can be expressed as

$$f_{RBF}\left(\mathbf{x}\right)=\sum _{i=1}^N{w}_i\exp (-\beta |\mathbf{x}-{\mathbf{x}}_i{|}^2)$$

(8)

where N is the number of sampling design solutions, $w_i$ denotes the weight coefficient of the ith design sampling point, and $\beta$ represents a positive coefficient.

3. Multiple Additional Sampling for Multi-Fidelity Optimization

The original multi-fidelity EGO can be obtained one additional sample in each iteration where the process of the original multi-fidelity EGO is illustrated in Figure 1b. A designer can only employ computational resources for one evaluation in this case, whereas they can use a parallel environment in the initial sampling. Here, available computational resources are not employed during the additional sampling stage, as illustrated in Figure 3a.

Here, multi-additional sampling (MAs) for multi-fidelity optimization is proposed. In EGO with MAs, a sub-iteration is additionally included, as illustrated in Figure 3b. In this sub-iteration, an additional sampling solution (obtained by an iterative approach) is included to construct an updated multi-fidelity surrogate model by using the predicted point $(\mathbf{x},\widehat{y}\left(\mathbf{x}\right))$ as a temporal function value. Another additional sampling solution is obtained through EI maximization.

A schematic illustration of additional sampling in the proposed MAs method is illustrated in Figure 4. This process begins by having ${\mathbf{x}}_{EI\max 1}$ from using EI maximization on the initial surrogate model. The predicted value ${\widehat{y}}_{a1}$ is also computed, and then the model is then updated temporally using (${\mathbf{x}}_{EI\max 1},{\widehat{y}}_{a1}$). As the EI value around ${\mathbf{x}}_{EI\max 1}$ should not be greater, the next additional point (${\mathbf{x}}_{EI\max 2}$) can be obtained. This process is repeated until an arbitrary number of additional sampling solutions are obtained. Thereafter, the exact values $y_{a1},y_{a2},\dots$ are calculated using a parallel evaluation environment for ${\mathbf{x}}_{EI\max 1},{\mathbf{x}}_{EI\max 2},\dots$ by using the expensive function. The set of these additional sampling points is finally added to the data set for the improvement of the model.

In this sub-iteration, only the values $\widehat{y}\left(\mathbf{x}\right)$ predicted by the surrogate model are required. Therefore, multiple additional samples can be rapidly obtained. In the main iteration, the exact value for these additional sampling solutions can be computed using a parallel evaluation. As a result, a design using this EGO method can be completed more quickly compared to the original EGO. The number of sub-iterations can be found using the number of parallel evaluations, where the value of EI or the number of experiments could be the termination condition of the EGO process.

4. Investigation Problems

Here, the proposed method is investigated by solving test functions. The effect of multi-fidelity surrogate model, which are co-Kriging, RBF/Kriging surrogate model, for the multi-sampling EGO process results are compared with results given by the single-fidelity multi-sampling EGO to demonstrate the efficiency of these methods.

In this study, four test functions [18,32,33] are used. $y_h$ denotes a high-fidelity function and $y_l$ denoted a low-fidelity function.

• The six-hump camel-back function (SC) is defined as follows:

  $$\mathrm{Minimize}:y_h(x_1,x_2)=sc(x_1,x_2)$$

  (9a)

  $$y_l(x_1,x_2)=sc(0.7x_1,0.7x_2)+x_1{x}_2-65$$

  (9b)

  $$sc(x_1,x_2)=4x_1^2-2.1x_1^4+\frac{x_1^6}{3}+x_1{x}_2-4x_2^2+4_2^4$$

  (9c)

  $$x_1\in [-2,2],x_2\in [-2,2]$$
• The Himmelblau function (HIM) is defined as follows:

  $$\mathrm{Minimize}:y_h(x_1,x_2)=him(x_1,x_2)$$

  (10a)

  $$y_l(x_1,x_2)=him(0.5x_1,0.8x_2)+x_2^3-{(x_1+1)}^2$$

  (10b)

  $$him(x_1,x_2)={(x_1^2+x_2-11)}^2+{(x_2^2+x_1-7)}^2$$

  (10c)

  $$x_1\in [-5,5],x_2\in [-5,5]$$
• The Rosenbrock function (ROS) is defined as follows:

  $$\mathrm{Minimize}:y_h(x_1,x_2,x_3,x_4)=\sum _{i=1}^3{\left[100{(x_{i+1}-x_i^2)}^2-{(x_i-1)}^2\right]}^2$$

  (11a)

  $$y_l(x_1,x_2,x_3,x_4)=\sum _{i=1}^3\left[100{(0.5x_{i+1}-0.6x_i^2)}^2-{(0.5x_i-0.5)}^2\right]$$

  (11b)

  $$x_1\in [0,2],x_2\in [0,2],x_3\in [0,2],x_4\in [0,2]$$
• The Colville function (COL) is defined as follows:

  $$\mathrm{Minimize}:y_h(x_1,x_2,x_3,x_4)=Col(x_1,x_2,x_3,x_4)$$

  (12a)

  $$f_l(x_1,x_2,x_3,x_4)=Col(0.8x_1,0.8x_2,0.5x_3,0.5x_4)$$

  (12b)

  $$\begin{array}{cc}\hfill Col(x_1,x_2,x_3,x_4)& =100{(x_1^2-x_2)}^2+{(x_1-1)}^2+{(x_3-1)}^2+90{(x_3^2-x_4)}^2\hfill \\ \hfill & +10.1\left({(x_2-1)}^2+{(x_4-1)}^2\right)+19.8(x_2-1)(x_4-1)\hfill \end{array}$$

  (12c)

  $$x_1\in [0,1],x_2\in [0,1],x_3\in [0,1],x_4\in [0,1]$$

In each investigation, 10 initial high-fidelity points $y_h$ and 200 low-fidelity point $y_l$ were acquired using LHS. The number of additional sampling points was set as 12 for all test functions and 2–4 multi-additional sampling were selected to test the effect of multi-additional sampling compared with the results of single-additional sampling for the single-fidelity and multi-fidelity EGO. The function of the low-fidelity data is slightly different from the high-fidelity data because in the real world computation there are some errors between these functions. However, the characteristic of the low-fidelity and high-fidelity function is they, to some extent, have similar function landscapes. To simulate the situation, the low fidelity function is always set as its high fidelity counterpart multiplied by a factor. This implies that the high fidelity function is always more accurate.

5. Results

5.1. Minimization Problem of SC Function

Figure 5 shows the comparison of additional sampling of the SC problem across the RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO. The results in each graph show the single additional sampling (SAS), 2 multi-additional sampling (2-MAs), 3 multi-additional sampling (3-MAs) and 4 multi-additional sampling (4-MAs). The optimum value obtained by RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO with SAS, 2-MAs, 3-MAs and 4-MAs are shown in

Table 1. Optimum value of the SC problem.

	SAS	2-MAs	3-MAs	4-MAs
RBF/Kriging	−1.03	−1.03	−0.93	−0.91
Co-Kriging	−1.02	−0.92	−0.9	−0.86
Kriging	−0.93	−0.84	−0.70	−0.89

. As shown in Table 1, the optimum value of RBF/Kriging hybrid multi-fidelity EGO is the best optimum solution solved by SAS, and the optimum value by 2-MAs, 3-MAs and 4-MAs were obtained the similar optimum solution. The results from the Co-Kriging method was a good optimum value when solved with the SAS, 2-MAs, and 3-MAs and 4-MAs. However, the results from the RBF/Kriging hybrid multi-fidelity EGO obtained better solution than the Co-Kriging method because the results from the the Co-Kriging method obtained the similar solution as shown in Figure 6 when solving with the MAs shown in Figure 5. As shown in Figure 5b, the results of additional sampling for the same iteration mostly get similar values as that shown in Figure 6. The results of the Co-Kriging method shows that the Co-Kriging multi-fidelity EGO cannot increase the diversity of the additional sampling when solving with the MAs. The results from the ordinary Kriging method obtained the worst solution when solving the optimum problem with the MAs method. The cross-validation results [34] are shown in Figure 7 and

Table 2. Cross-validation results for the SC problem.

	Slope	Offset	${\mathit{R}}^2$
RBF/Kriging
- SAS	0.9539	0.0204	0.9796
- 2-MAs	0.9311	0.0655	0.9886
- 3-MAs	0.876	0.0755	0.9755
- 4-MAs	0.839	0.0659	0.9857
Co-Kriging
- SAS	0.9315	0.0872	0.9604
- 2-MAs	0.7843	0.2377	0.9625
- 3-MAs	0.9021	0.2132	0.9676
- 4-MAs	0.5473	0.3995	0.815
Kriging
- SAS	0.5568	1.2856	0.8051
- 2-MAs	0.5688	1.128	0.7588
- 3-MAs	0.6417	1.0784	0.8261
- 4-MAs	0.6199	0.9272	0.8900

. The cross-validation results show the RBF/Kriging hybrid multi-fidelity surrogate model obtains the high accuracy of the surrogate model for all cases, where the slope of the regression line can maintain a slope close to 1.0 and $R^2$ greater than 0.95 for all cases. However, for the Co-Kriging multi-fidelity surrogate model results, the surrogate model of 4-MAs cannot maintain the accuracy. The reason for these results is that the additional sampling of this case has low diversity. The Kriging surrogate model results show that the accuracy of this surrogate model is slightly increased when using higher multi-additional sampling, but the accuracy of the Kriging model is still less than the accuracy of the Co-Kriging and RBF/Kriging multi-fidelity surrogate models.

5.2. Minimization Problem of Him Function

Figure 8 shows the comparison of additional sampling of the HIM problem across the RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO. The optimum value obtained by RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO with SAS, 2-MAs, 3-MAs and 4-MAs are shown in

Table 3. Optimum value of the HIM problem.

	SAS	2-MAs	3-MAs	4-MAs
RBF/Kriging	0.82	3.31	3.24	1.41
Co-Kriging	0.01	0.31	0.53	5.14
Kriging	1.02	2.39	4.27	4.20

. As shown in Table 3, the optimum value of Co-Kriging multi-fidelity EGO obtained a good optimum solution with solving by SAS, 2-MAs and 3-MAs. However, the optimum solution of the 4-MAs obtained slightly worse results. For this case, the 4-MAs was not effective because the Co-Kriging method obtained the similar solution when solving with the MAs shown in Figure 8b. This effect was similar when solving the SC problem. The results from the RBF/Kriging multi-fidelity EGO shows the additional sampling has a higher diversity than the results from the Co-Kriging multi-fidelity EGO, but it still obtained good solution for all cases. The results from the ordinary Kriging method show the Kriging method was not good to solve this problem with high MAs. These results could be concluded the multi-fidelity optimization could be helpful to solve the EGO with the MAs because the original EGO will have lower accuracy of the surrogate model. The cross-validation results of the HIM are shown in

Table 4. Cross-validation results for the HIM problem.

	Slope	Offset	${\mathit{R}}^2$
RBF/Kriging
- SAS	0.8912	24.721	0.9896
- 2-MAs	0.8671	25.157	0.9086
- 3-MAs	0.8504	25.519	0.9055
- 4-MAs	0.8633	23.476	0.9557
Co-Kriging
- SAS	0.9081	22.909	0.9801
- 2-MAs	0.8954	23.303	0.9545
- 3-MAs	0.8925	20.245	0.9024
- 4-MAs	0.7514	35.575	0.3132
Kriging
- SAS	0.9124	24.626	0.8512
- 2-MAs	0.8521	30.2157	0.7502
- 3-MAs	0.7215	35.243	0.6554
- 4-MAs	0.7034	60.338	0.3323

. It is seen that the Co-Kriging multi-fidelity EGO method obtained the best solution because the Co-kriging multi-fidelity surrogate model obtained the highest accuracy for SAS, 2-MAs and 3-MAs. However, the Co-Kriging multi-fidelity surrogate model could not maintain the accuracy of the surrogate model for the case 4-MAs. On the other hand, the RBF/Kriging hybird multi-fidelity surrogate model could not obtain the highest accuracy for this test problem, but it can maintain the accuracy of the surrogate model for all cases, while the slope of the regression line can maintain a slope close to 1.0 and $R^2$ greater than 0.90 for all cases. The results of the Kriging surrogate model reveal that the accuracy of the surrogate model is decreased when using the higher number of MAs. This test problem shows that the multi-fidelity technique could be helpful when using the MAs technique because it could be maintain the accuracy of the surrogate model when using the higher number of MAs.

5.3. Minimization Problem of Ros Function

Figure 9 shows the comparison of additional sampling of the ROS problem across the RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO. The optimum value that obtained by RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO with SAS, 2-MAs, 3-MAs and 4-MAs are shown in

Table 5. Optimum value of the ROS problem.

	SAS	2-MAs	3-MAs	4-MAs
RBF/Kriging	1.11	1.90	1.12	2.46
Co-Kriging	3.11	3.33	3.31	5.18
Kriging	1.58	1.89	5.48	20.38

. As shown in Table 5, the optimum value of the RBF/Kriging obtained the best solution for SAS, 3-MAs and 4-MAs. The results from the Co-Kriging obtained the worst optimal solution when solving by SAS, but the solution of higher number of multiple-additional sampling could be accepted. On the other hand, the results from the original EGO shows the original EGO could be solved this problem well when using the SAS, but it cannot get a good solution when using higher number of multiple-additional sampling points. The cross-validation results of the ROS are shown in

Table 6. Cross-validation results for the ROS problem.

	Slope	Offset	${\mathit{R}}^2$
RBF/Kriging
- SAS	0.9245	14.561	0.9796
- 2-MAs	0.9285	10.73	0.9886
- 3-MAs	0.9185	16.773	0.9755
- 4-MAs	0.9197	20.486	0.9857
Co-Kriging
- SAS	0.7121	30.575	0.7873
- 2-MAs	0.7051	39.681	0.7354
- 3-MAs	0.6954	55.248	0.7535
- 4-MAs	0.5232	100.157	0.5348
Kriging
- SAS	0.5533	270.86	0.5895
- 2-MAs	0.797	35.396	0.8971
- 3-MAs	0.5334	156.77	0.6457
- 4-MAs	0.4923	201.51	0.5895

. The RBF/Kriging multi-fidelity EGO gives the best solution because the RBF/Kriging multi-fidelity surrogate model obtained the highest accuracy for all cases, where the slope of the regression line can maintain a slope close to 1.0 and $R^2$ greater than 0.95 for all cases. The Co-Kriging multi-fidelity surrogate model could not obtained the high accuracy for this test problem, but it still have higher accuracy than the Kriging surrogate model. In addition, the accuracy of the Co-Kriging multi-fidelity surrogate model depends on the number of MAs. For this case, the accuracy the Co-Kriging multi-fidelity surrogate model is decreased when using the higher number of MAs. In the same way, The results of the Kriging surrogate model show that the accuracy of the surrogate model is decreased when using the higher number of MAs. This test problem still shows the multi-fidelity technique could be helpful with the use of the MAs technique because it could maintain the accuracy of the surrogate model when using the higher number of MAs.

5.4. Minimization Problem of Col Function

Figure 10 shows the comparison of additional sampling of the ROS problem across the RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO. The optimum value obtained by RBF/Kriging hybrid multi-fidelity EGO, Co-Kriging multi-fidelity EGO and original EGO with SAS, 2-MAs, 3-MAs and 4-MAs are shown in

Table 7. Optimum value of the COL problem.

	SAS	2-MAs	3-MAs	4-MAs
RBF/Kriging	4.35	4.22	3.85	4.29
Co-Kriging	1.54	2.83	8.69	11.57
Kriging	5.54	7.48	10.44	6.67

. The optimum value obtained by the Co-Kriging multi-fidelity EGO could obtain the best solution when solving this problem with SAS, but it cannot obtain a good solution when solving with 3-MAs and 4-MAs. These results show the Co-Kriging has high impact for multi-additional sampling. The RBF/Kriging hybrid multi-fidelity cannot obtain the best solution when solving with the SAS. However, this method could obtain a similar optimum solution when solving with all methods. The optimal solutions by the original EGO received results lower than the RBF/Kriging hybrid multi-fidelity EGO and the Co-Kriging method except when solving with 4-MAs. The cross-validation results of the COL are shown in

Table 8. Cross-validation results for the COL problem.

	Slope	Offset	${\mathit{R}}^2$
RBF/Kriging
- SAS	0.9512	10.245	0.9654
- 2-MAs	0.9214	12.2484	0.9532
- 3-MAs	0.9324	11.245	0.9512
- 4-MAs	0.9134	13.575	0.9212
Co-Kriging
- SAS	0.9823	10.5751	0.9742
- 2-MAs	0.9624	15.215	0.9124
- 3-MAs	0.8124	20.2484	0.8512
- 4-MAs	0.7143	24.557	0.7512
Kriging
- SAS	0.5568	12.251	0.8721
- 2-MAs	0.5688	18.425	0.8212
- 3-MAs	0.6417	20.746	0.7521
- 4-MAs	0.6199	15.358	0.8145

. The results show that the Co-Kriging multi-fidelity EGO method gives the best solution because the Co-kriging multi-fidelity surrogate model obtained the highest accuracy for SAS and 2-MAs. However, the Co-Kriging multi-fidelity surrogate model could not maintain the accuracy of the surrogate model for the case 3-MAs and 4-MAs. On the other hand, the RBF/Kriging hybird multi-fidelity surrogate model could not obtain the highest accuracy for this test problem, but it can maintain the accuracy of the surrogate model for all cases, where the slope of the regression line can maintain a slope close to 1.0 and $R^2$ greater than 0.90 for all cases. The results of the Kriging surrogate model were not good for this test problem. This test problem can show the multi-fidelity technique could be helpful for using the MAs technique because it could maintain the accuracy of the surrogate model when using the higher number of MAs.

5.5. Discussion

According to the results from four optimization test functions, the multi-fidelity optimization could reduce the effect of the multi-additional sampling process because the multi-fidelity optimization could obtain higher accuracy of the surrogate model. From the cross-validation results in Table 2, Table 4, Table 6, and Table 8, it is shown that the accuracy of the multi-fidelity surrogate model could obtain higher accuracy when solving the problem with the higher number of MAs. From some test functions, the Co-Kriging multi-fidelity surrogate model gives the highest accuracy for the SAS case, but it cannot maintain the accuracy of the surrogate model when solving the solution with the higher number MAs. The RBF/Kriging hybrid multi-fidelity surrogate model shows that it is able to maintain the good accuracy when using the higher number of MAs. It can be concluded that the multi-fidelity technique could increase the efficiency of the MAs EGO process because it can maintain the accuracy of the surrogate model when using the higher number of MAs.

6. Conclusions

This study investigated the effect of multi-fidelity technique to the multi-additional EGO process. The RBF/Kriging hybrid multi-fidelity EGO and CO-Kriging multi-fidelity EGO are selected to test the effect of this framework. The MAs technique could help the EGO process to run the parallel computing that can reduce the computation time. However, the weak point of the MAs technique combined with the EGO results in the accuracy of the surrogate model being reduced, which leads to the use of resources to find the optimum solution. Consequently, the multi-fidelity technique is proposed to combineEGO with MAs to increase the efficiency of the EGO process by increasing the accuracy of the surrogate model.

To examine the multi-fidelity EGO framework, four test functions are solved, and the results are compared with those obtained by the ordinary Kriging EGO. The optimum results for the test functions show that the RBF/Kriging hybrid multi-fidelity EGO has the lowest effect when using the RBF/Kriging hybrid multi-additional sampling framework because the RBF/Kriging hybrid multi-fidelity surrogate model can maintain the accuracy of the surrogate model when it runs with higher number of MAs. For some of the number of MAs test functions the Co-Kriging could receive the best optimal solution when solving with the SAS but it cannot obtain a good solution when solving via multi-additional sampling framework. The ordinary EGO has a high-effect when using the multi-additional sampling framework. The optimal solution of the ordinary EGO gets worse results when solving the optimal solution via higher number of multi-additional sampling points. This results suggest that the hybrid RBF/Kriging multi-fidelity EGO is the best choice for the MAs framework because it can maintain the accuracy of the surrogate model that affects the efficiency of the optimization process.