Skip to main content
SpringerLink
Log in

Managing computational complexity using surrogate models: a critical review

  • Original Paper
  • Published:
Research in Engineering Design Aims and scope Submit manuscript

Abstract

In simulation-based realization of complex systems, we are forced to address the issue of computational complexity. One critical issue that must be addressed is the approximation of reality using surrogate models to replace expensive simulation models of engineering problems. In this paper, we critically review over 200 papers. We find that a framework for selecting appropriate surrogate modeling methods for a given function with specific requirements has been lacking. Having such a framework for surrogate model users, specifically practitioners in industry, is very important because there is very limited information about the performance of different models before applying them on the problem. Our contribution in this paper is to address this gap by creating practical guidance based on a trade-off among three main drivers, namely, size (how much information is necessary to compute the surrogate model), accuracy (how accurate the surrogate model must be) and computational time (how much time is required for the surrogate modeling process). Using the proposed guidance a huge amount of time is saved by avoiding time-consuming comparisons before selecting the appropriate surrogate model. To make this contribution, we review the state-of-the-art surrogate modeling literature to answer the following three questions: (1) What are the main classes of the design of experiment (DOE) methods, surrogate modeling methods and model-fitting methods based on the requirements of size, computational time, and accuracy? (2) Which surrogate modeling method is suitable based on the critical characteristics of the requirements of size, computational time and accuracy? (3) Which DOE is suitable based on the critical characteristics of the requirements of size, computational time and accuracy? Based on these three characteristics, we find six different qualitative categories for the surrogate models through a critical evaluation of the literature. These categories provide a framework for selecting an efficient surrogate modeling process to assist those who wish to select more appropriate surrogate modeling techniques for a given function. It is also summarized in Table 4 and Figs. 2, 3. MARS, response surface models, and kriging are more appropriate for large problems, acquiring less computation time and high accuracy, respectively. Also, Latin Hypercube, fractional factorial designs and D-Optimal designs are appropriate experimental designs. Our contribution is to propose a qualitative evaluation and a mental model which is based on quantitative results and findings of authors in the published literature. The value of such a framework is in providing practical guide for researchers and practitioners in industry to choose the most appropriate surrogate model based on incomplete information about an engineering design problem. Another contribution is to use three drivers, namely, computational time, accuracy, and problem size instead of using a single measure that authors generally use in the published literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Mean arctangent absolute percentage error.

Abbreviations

ANN:

Artificial Neural Network

CCD:

Central composite design

CFD:

Computational fluid dynamics

CPU:

Central processing unit

DOE:

Design of experiments

DST:

Dempster–Shafer theory

EA:

Evolutionary algorithm

FD:

Factorial design

FEA:

Finite element analysis

FFD:

Fractional factorial design

GSME:

Generalized mean square error

KRG:

Kriging

LS:

Least squares

MAE:

Mean absolute error

MAPE:

Mean absolute percentage error

MARS:

Multivariate Adaptive Regression Splines

MAXE-CV:

Maximum absolute cross-validation error

MSEGO:

Multiple surrogate efficient global optimization

MEMO:

Multimodal-based evolutionary multiple-objective

NSGA-II:

Non-dominated sorting genetic algorithm II

OA:

Orthogonal array

PNN:

Polynomial Neural Network

POF:

Pareto optimal front

PRESS:

Predicted residual error sum of squares

PRS:

Polynomial response surface

PSO:

Particle swarm optimization

RBDO:

Reliability-based design optimization

RBF:

Radial basis function

RMSE:

Root Mean Square Error

RSM:

Response surface models

SE:

Square error

SVM:

Support vector machines

WAS:

Weighted average surrogate

WLSR:

Weighted least square regression

References

  • Acar E (2015) Effect of error metrics on optimum weight factor selection for ensemble of metamodels. Expert Syst Appl 42(5):2703–2709

    Article  Google Scholar 

  • Adams, MJ, Silverman AK, Turner CJ (2015) Pseudo elimination of geometry dependence in surrogate models of distributed knee loads from an explicit dynamic finite element analysis. International design engineering technical conferences. DETC2015-47850: V01AT02A055

  • Adhav R, Samad A, Kenyery F (2015). Design optimization of electric centrifugal pump by multiple surrogate models. SPE middle east oil and gas show and conference. Manama, Bahrain, Society of Petroleum Engineers

  • Ajetunmobi AM, Turner CJ, Crawford RH (2008). Robust optimization With NURBS HyPerModels. International design enginggering technical conferences. DETC2008-49961: 317–327

  • Al-Juboori M, Datta B (2019) Reliability-based optimum design of hydraulic water retaining structure constructed on heterogeneous porous media: utilizing stochastic ensemble surrogate model-based linked simulation optimization model. Life Cycl Reliab Saf Eng 8(1):65–84

    Article  Google Scholar 

  • Alizadeh R, Lund PD, Beynaghi A, Abolghasemi M, Maknoon R (2016a) An integrated scenario-based robust planning approach for foresight and strategic management with application to energy industry. Technol Forecast Soc Chang 104:162–171

    Article  Google Scholar 

  • Alizadeh R, Khodaei R, Mknoon R (2016b) A combined model of scenario planning and assumption-based planning for futurology, and robust decision making in the energy sector. J Energy Plan Policy Res 2(2):7–32

    Google Scholar 

  • Alizadeh R, Jia L, Nellippallil AB, Wang G, Hao J, Allen JK, Mistree F (2019) Ensemble of surrogates and cross validation for rapid and accurate predictions using small data sets. Artif Intell Eng Des Anal Manuf 33(4):484–501

    Article  Google Scholar 

  • Amit Y, Geman D (1997) Shape quantization and recognition with randomized trees. Neural Comput 9(7):1545–1588

    Article  Google Scholar 

  • Anderson D, Turner CJ (2015). Applying NURBs-based surrogate models for performance forecasting in manufacturing systems. International mechanical engnieering congress and expo. IMECE2015-51862: V011T014A042

  • Arias-Montano A, Coello CAC, Mezura-Montes E (2012) Multi-objective airfoil shape optimization using a multiple-surrogate approach. Evolutionary computation (CEC), 2012 IEEE congress on, IEEE

  • Audoux Y, Montemurro M, Pailhes J (2018) A surrogate model based on non-uniform rational B-splines hypersurfaces. Procedia CIRP 70:463–468

    Article  Google Scholar 

  • Babaei M, Pan I (2016) Performance comparison of several response surface surrogate models and ensemble methods for water injection optimization under uncertainty. Comput Geosci 91:19–32

    Article  Google Scholar 

  • Badhurshah R, Samad A (2015) Multiple surrogate based optimization of a bidirectional impulse turbine for wave energy conversion. Renew Energy 74:749–760

    Article  Google Scholar 

  • Balling R, Wilkinson C (1997) Execution of multidisciplinary design optimization approaches on common test problems. AIAA J 35(1):178–186

    Article  MATH  Google Scholar 

  • Bandler JW, Madsen K (2001) Surrogate modelling and space mapping for engineering optimization. Optim Eng 2(4):367–368

    Article  Google Scholar 

  • Bandler WJ, Koziel S, Madsen K (2008) Editorial—Surrogate modeling and space mapping for engineering optimization. Optim Eng 9:307–310

    Article  MATH  Google Scholar 

  • Barthelemy J-FM, Haftka RT (1993) Approximation concepts for optimum structural design—a review. Struct Optim 5(3):129–144

    Article  Google Scholar 

  • Barton RR (1994). Metamodeling: a state of the art review. Proceedings of winter simulation conference, IEEE

  • Basudhar A (2012) Selection of anisotropic kernel parameters using multiple surrogate information. 12th AIAA aviation technology, integration, and operations (ATIO) conference and 14th AIAA/ISSMO multidisciplinary analysis and optimization conference

  • Beck JL, Au S-K (2002) Bayesian updating of structural models and reliability using Markov chain Monte Carlo simulation. J Eng Mech 128(4):380–391

    Article  Google Scholar 

  • Bellary SAI, Samad A (2017) An alternative approach to surrogate averaging for a centrifugal impeller shape optimisation. Int J Comput Aided Eng Technol 9(1):62–83

    Article  Google Scholar 

  • Bellary SAI, Adhav R, Siddique MH, Chon B-H, Kenyery F, Samad A (2016) Application of computational fluid dynamics and surrogate-coupled evolutionary computing to enhance centrifugal-pump performance. Eng Appl Comput Fluid Mech 10(1):171–181

    Google Scholar 

  • Bettebghor D, Bartoli N, Grihon S, Morlier J, Samuelides M (2011) Surrogate modeling approximation using a mixture of experts based on EM joint estimation. Struct Multidiscip Optim 43(2):243–259

    Article  Google Scholar 

  • Bettonvil B, Kleijnen JPC (1997) Searching for important factors in simulation models with many factors: sequential bifurcation. Eur J Oper Res 96(1):180–194

    Article  MATH  Google Scholar 

  • Bhat S, Viana FAC, Lind R, Haftka R (2010) Control-oriented design using H-infinity synthesis and multiple surrogates. 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, American Institute of Aeronautics and Astronautics

  • Bhattacharjee KS, Singh HK, Ray T, Branke J (2016) Multiple surrogate assisted multiobjective optimization using improved pre-selection. 2016 IEEE Congress on Evolutionary Computation (CEC)

  • Bhattacharjee KS, Isaacs A, Ray T (2017) Multi-objective optimization using an evolutionary algorithm embedded with multiple spatially distributed surrogates. MULTI-OBJECTIVE OPTIMIZATION: techniques and application in chemical engineering. World Scientific, Singapore, pp 135–155

    Google Scholar 

  • Bhattacharjee KS, Singh HK, Ray T (2018) Multiple surrogate-assisted many-objective optimization for computationally expensive engineering design. J Mech Des 140(5):051403

    Article  Google Scholar 

  • Bliznyuk N, Ruppert D, Shoemaker C, Regis R, Wild S, Mugunthan P (2008) Bayesian calibration and uncertainty analysis for computationally expensive models using optimization and radial basis function approximation. J Comput Graphical Stat 17(2):270–294

    Article  MathSciNet  Google Scholar 

  • Box GEP, Hunter JS (1957) Multi-factor experimental designs for exploring response surfaces. Ann Math Stat 28(1):195–241

    Article  MathSciNet  MATH  Google Scholar 

  • Box GEP, Behnken DW (1960) Some new three level designs for the study of quantitative variables. Technometrics 2(4):455–475

    Article  MathSciNet  Google Scholar 

  • Box GE, Hunter JS (1961) The 2 k—p fractional factorial designs. Technometrics 3(3):311–351

    MathSciNet  MATH  Google Scholar 

  • Box GE, Draper NR (1987) Empirical model-building and response surfaces. Wiley, Hoboken

    MATH  Google Scholar 

  • Box GE, Hunter WG, Hunter JS (1978) Statistics for experimenters. Wiley, New York

    MATH  Google Scholar 

  • Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  • Brereton RG, Lloyd GR (2010) Support vector machines for classification and regression. Analyst 135(2):230–267

    Article  Google Scholar 

  • Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks. J Complex Syst 2:321–355

    MathSciNet  MATH  Google Scholar 

  • Burnham KP, Anderson DR (2003) Model selection and multimodel inference: a practical information-theoretic approach. Springer, Berlin

    MATH  Google Scholar 

  • Campolongo F, Braddock R (1999) The use of graph theory in the sensitivity analysis of the model output: a second order screening method. Reliab Eng Syst Saf 64(1):1–12

    Article  Google Scholar 

  • Carbonell JR (1970) AI in CAI: an artificial-intelligence approach to computer-assisted instruction. IEEE Trans Man-Mach Syst 11(4):190–202

    Article  Google Scholar 

  • Chaudhuri A, Haftka RT (2014) Efficient global optimization with adaptive target setting. AIAA J 52(7):1573–1578

    Article  Google Scholar 

  • Chauvin Y, Rumelhart DE (2013) Backpropagation: theory, architectures, and applications. Psychology Press, London

    Book  Google Scholar 

  • Chen J, Lin DK (1998) On the identifiability of a supersaturated design. J Statis Plan Inference 72(1–2):99–107

    Article  MathSciNet  MATH  Google Scholar 

  • Chen VC, Tsui K-L, Barton RR, Allen JK (2003) Ch. 7. A review of design and modeling in computer experiments. Handb Stat 22:231–261

    Article  Google Scholar 

  • Chen VC, Tsui K-L, Barton RR, Meckesheimer M (2006) A review on design, modeling and applications of computer experiments. IIE Trans 38(4):273–291

    Article  Google Scholar 

  • Cheng B, Titterington DM (1994) Neural networks: a review from a statistical perspective. Stat Sci 9(1):2–54

    Article  MathSciNet  MATH  Google Scholar 

  • Cheng RC, Currie CS (2004) Optimization by simulation metamodelling methods. Proceedings of the 36th conference on Winter simulation, Winter Simulation Conference

  • Cho H, Bae S, Choi KK, Lamb D, Yang R-J (2014) An efficient variable screening method for effective surrogate models for reliability-based design optimization. Struct Multidiscip Optim 50(5):717–738

    Article  Google Scholar 

  • Clarke SM, Griebsch JH, Simpson TW (2004) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087

    Article  Google Scholar 

  • Corchado E, Corchado E, Corchado JM, Abraham A (2007) Innovations in hybrid intelligent systems (Advances in Soft Computing). Springer, Berlin

    Book  MATH  Google Scholar 

  • Cutler A, Cutler DR, Stevens JR (2012) Random forests. Ensemble machine learning. Springer, Berlin, pp 157–175

    Book  Google Scholar 

  • De'ath G (2007) Boosted trees for ecological modeling and prediction. Ecology 88(1):243–251

    Article  Google Scholar 

  • Deaton JD, Grandhi RV (2014) A survey of structural and multidisciplinary continuum topology optimization: post 2000. Struct Multidiscip Optim 49(1):1–38

    Article  MathSciNet  Google Scholar 

  • Deb K, Myburgh C (2016) Breaking the billion-variable barrier in real-world optimization using a customized evolutionary algorithm. Proceedings of the 2016 on Genetic and Evolutionary Computation Conference, ACM

  • Deb K, Hussein R, Roy P, Toscano G (2017) Classifying metamodeling methods for evolutionary multi-objective optimization: first results. Evolutionary multi-criterion optimization: 9th International Conference, EMO 2017, Münster, Germany, March 19–22, 2017, Proceedings. Trautmann H, Rudolph G, Klamroth K, Schütze O, Wiecek M, Jin Y, Grimme C Springer, Cham 160–175

  • Demuth HB, Beale MH, De Jess O, Hagan MT (2014) Neural network design. Pws Pub, Boston

    Google Scholar 

  • Deng H, Shao W, Ma Y (2012a) A Bayesian approach to kriging metamodeling for computer experiments. J Adv Mech Des Syst Manuf 6(4):404–419

    Article  Google Scholar 

  • Deng H, Shao W, Ma Y, Wei Z (2012b) Bayesian metamodeling for computer experiments using the Gaussian Kriging models. Qual Reliab Eng Int 28(4):455–466

    Article  Google Scholar 

  • Dey A (1985) Orthogonal fractional factorial designs. Wiley, NY

    MATH  Google Scholar 

  • Dey S, Mukhopadhyay T, Adhikari S (2017) Metamodel based high-fidelity stochastic analysis of composite laminates: a concise review with critical comparative assessment. Compos Struct 171:227–250

    Article  Google Scholar 

  • Dobler CP, Anderson-Cook CM (2005) Forecasting, time series, and regression: an applied approach. Am Stat 59(3):278

    Article  Google Scholar 

  • Draper NR, Guttman I (1988) An index of rotatability. Technometrics 30(1):105–111

    Article  MathSciNet  MATH  Google Scholar 

  • Duda RO, Hart PE, Stork DG (2012) Pattern classification. Wiley, NY

    MATH  Google Scholar 

  • Dumais S, Platt J, Heckerman D, Sahami M (1998) Inductive learning algorithms and representations for text categorization. Proceedings of the seventh international conference on Information and knowledge management, ACM

  • Easterling RG, Berger JO (2002) Statistical foundations for the validation of computer models. Presented at computer model verification and validation in the 21st century workshop. Johns Hopkins University, Baltimore

    Google Scholar 

  • Ezhilsabareesh K, Rhee SH, Samad A (2018) Shape optimization of a bidirectional impulse turbine via surrogate models. Eng Appl Comput Fluid Mech 12(1):1–12

    Google Scholar 

  • Fang J, Sun G, Qiu N, Kim NH, Li Q (2017) On design optimization for structural crashworthiness and its state of the art. Struct Multidiscip Optim 55(3):1091–1119

    Article  MathSciNet  Google Scholar 

  • Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1):50–79

    Article  Google Scholar 

  • Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189–1232

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning. Springer, New York

    MATH  Google Scholar 

  • Gano SE, Renaud JE, Martin JD, Simpson TW (2006) Update strategies for kriging models used in variable fidelity optimization. Struct Multidiscip Optim 32(4):287–298

    Article  Google Scholar 

  • Gao H, Breitkopf P, Coelho RF, Xiao M (2018) Categorical structural optimization using discrete manifold learning approach and custom-built evolutionary operators. Struct Multidiscip Optim 58:215–228

    Article  MathSciNet  Google Scholar 

  • Geisser S (1993) Predictive inference. CRC Press, Boca Raton

    Book  MATH  Google Scholar 

  • Geist A (2000) PVM–parallel virtual machine : a users' guide and tutorial for networked parallel computing. MIT Press, Cambridge

    MATH  Google Scholar 

  • Ghanem RG, Doostan A (2006) On the construction and analysis of stochastic models: characterization and propagation of the errors associated with limited data. J Comput Phys 217(1):63–81

    Article  MathSciNet  MATH  Google Scholar 

  • Giunta AA, Dudley JM, Narducci R, Grossman B, Haftka RT, Mason WH, Watson LT (1994) Noisy aerodynamic response and smooth approximations in HSCT design. 5th Symposium on Multidisciplinary Analysis and Optimization, American Institute of Aeronautics and Astronautics

  • Glaz B, Goel T, Liu L, Friedmann PP, Haftka RT (2009) Multiple-surrogate approach to helicopter rotor blade vibration reduction. AIAA J 47(1):271–282

    Article  Google Scholar 

  • Goel T, Haftka R, Queipo N, Shyy W (2006). Performance estimate and simultaneous application of multiple surrogates. 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference

  • Goel T, Haftka RT, Shyy W, Queipo NV (2007) Ensemble of surrogates. Struct Multidiscip Optim 33(3):199–216

    Article  Google Scholar 

  • Gorissen D, Dhaene T, Turck FD (2009) Evolutionary model type selection for global surrogate modeling. J Mach Learn Res 10:2039–2078

    MathSciNet  MATH  Google Scholar 

  • Gorissen D, Couckuyt I, Demeester P, Dhaene T, Crombecq K (2010) A surrogate modeling and adaptive sampling toolbox for computer based design. J Mach Learn Res 11:2051–2055

    Google Scholar 

  • Grama A (2013) Introduction to parallel computing. Addison-Wesley, Harlow

    Google Scholar 

  • Gunn SR (1998) Support vector machines for classification and regression. ISIS Tech Rep 14:85–86

    Google Scholar 

  • Gunst RF (1996) Response surface methodology: process and product optimization using designed experiments. Taylor & Francis Group, Routledge

    Google Scholar 

  • Gunst RF, Mason RL (2009) Fractional factorial design. Wiley Interdiscip Rev 1(2):234–244

    Article  Google Scholar 

  • Güntert P, Billeter M, Ohlenschläger O, Brown LR, Wüthrich K (1998) Conformational analysis of protein and nucleic acid fragments with the new grid search algorithm FOUND. J Biomol NMR 12(4):543–548

    Article  Google Scholar 

  • Gustafsson A, Herrmann A, Huber F (2013) Conjoint measurement: methods and applications. Springer, Berlin

    MATH  Google Scholar 

  • Gutmann H-M (2001) A radial basis function method for global optimization. J Global Optim 19(3):201–227

    Article  MathSciNet  MATH  Google Scholar 

  • Habib A, Kumar Singh H, Ray T (2017) A multiple surrogate assisted evolutionary algorithm for optimization involving iterative solvers. Eng Optim 50:1–20

    Google Scholar 

  • Haftka RT, Villanueva D, Chaudhuri A (2016) Parallel surrogate-assisted global optimization with expensive functions—a survey. Struct Multidiscip Optim 54(1):3–13

    Article  MathSciNet  Google Scholar 

  • Hammond C, Turner CJ (2010) Data Modeling using NURBS curves and modified genetic algorithms. Proceedings of the ASME 2010 International Mechanical Engineering Congress and Exposition. vol 3. Design and Manufacturing, Parts A and B, Vancouver, British Columbia, Canada, November 12–18, pp 1089–1098

  • Han J, Pei J, Kamber M (2011) Data mining: concepts and techniques. Elsevier, Amsterdam

    MATH  Google Scholar 

  • Hansen PC, Pereyra V, Scherer G (2012) Least squares data fitting with applications. JHU Press, Baltimore

    MATH  Google Scholar 

  • Hasselman T, Yap K, Lin C, Cafeo J (2005) A case study in model improvement for vehicle crashworthiness simulation. 23rd International Modal Analysis Conference

  • He X, Wang L, Hong HG (2013) Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. Ann Statist 41(1):342–369

    Article  MathSciNet  MATH  Google Scholar 

  • Hedayat AS, Sloane NJA, Stufken J (2012) Orthogonal arrays: theory and applications. Springer, Berlin

    MATH  Google Scholar 

  • Helton JC (1999) Uncertainty and sensitivity analysis in performance assessment for the Waste Isolation Pilot Plant. Comput Phys Commun 117(1):156–180

    Article  Google Scholar 

  • Higdon D, Kennedy M, Cavendish JC, Cafeo JA, Ryne RD (2004) Combining field data and computer simulations for calibration and prediction. SIAM J Sci Comput 26(2):448–466

    Article  MathSciNet  MATH  Google Scholar 

  • Ho TK (1995) Random decision forests. Document analysis and recognition, 1995, proceedings of the third international conference on, IEEE

  • Ho TK (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832–844

    Article  Google Scholar 

  • Hong C, Liao KP, Cai T (2019) Semi-supervised validation of multiple surrogate outcomes with application to electronic medical records phenotyping. Biometrics 75(1):78–89

    Article  MathSciNet  MATH  Google Scholar 

  • Hou Z, Dai Z, Lao W, Wang Y, Lu W (2019) Application of mixed-integer nonlinear optimization programming based on ensemble surrogate model for dense nonaqueous phase liquid source identification in groundwater. Environ Eng Sci. https://doi.org/10.1089/ees.2018.0366

    Article  Google Scholar 

  • Hsu C-W, Chang C-C, Lin C-J (2003) A practical guide to support vector classification. Department of Computer Science and Information Engineering, National Taiwan University, Taipei

    Google Scholar 

  • IT J (2002) Principal component analysis. Springer, New York

    Google Scholar 

  • Jaeckel P, Rebonato R (1999) The most general methodology for creating a valid correlation matrix for risk management and option pricing purposes. J Risk 2(2):17–28

    Google Scholar 

  • Jagadeesh KA, Wenger AM, Berger MJ, Guturu H, Stenson PD, Cooper DN, Bernstein JA, Bejerano G (2016) M-CAP eliminates a majority of variants of uncertain significance in clinical exomes at high sensitivity. Nat Genet 48(12):1581

    Article  Google Scholar 

  • Jiang Z, Chen S, Apley DW, Chen W (2016) Reduction of epistemic model uncertainty in simulation-based multidisciplinary design. J Mech Des 138(8):081403–081413

    Article  Google Scholar 

  • Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput 9(1):3–12

    Article  Google Scholar 

  • Jin R, Chen W, Simpson T (2000) Comparative studies of metamodeling techniques under multiple modeling criteria. 8th Symposium on Multidisciplinary Analysis and Optimization, American Institute of Aeronautics and Astronautics

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23(1):1–13

    Article  Google Scholar 

  • Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Global Optim 21(4):345–383

    Article  MathSciNet  MATH  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13(4):455–492

    Article  MathSciNet  MATH  Google Scholar 

  • Karson MJ, Manson AR, Hader RJ (1969) Minimum bias estimation and experimental design for response surfaces. Technometrics 11(3):461–475

    Article  MathSciNet  MATH  Google Scholar 

  • Karwan MH, Rardin RL (1980) Searchability of the composite and multiple surrogate dual functions. Operations Res 28(5):1251–1257

    Article  MathSciNet  MATH  Google Scholar 

  • Kathleen HVB, Cox DR (1962) Some systematic supersaturated designs. Technometrics 4(4):489–495

    Article  MathSciNet  MATH  Google Scholar 

  • Kennedy MC, O'Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13

    Article  MathSciNet  MATH  Google Scholar 

  • Kennedy MC, O'Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B (Statistical Methodology) 63(3):425–464

    Article  MathSciNet  MATH  Google Scholar 

  • Khuri A (1988) A measure of rotatability for response-surface designs. Technometrics 30(1):95–104

    Article  MathSciNet  MATH  Google Scholar 

  • Khuri AI, Mukhopadhyay S (2010) Response surface methodology. Wiley Interdiscip Rev 2(2):128–149

    Article  Google Scholar 

  • Kianifar MR, Campean F (2019) Performance evaluation of metamodelling methods for engineering problems: towards a practitioner guide. Struct Multidiscip Optim 61:1–28

    Google Scholar 

  • Kim S, Kim H (2016) A new metric of absolute percentage error for intermittent demand forecasts. Int J Forecast 32(3):669–679

    Article  Google Scholar 

  • Kleinberg E (1990) Stochastic discrimination. Ann Math Artif Intell 1(1):207–239

    Article  MATH  Google Scholar 

  • Kleinberg E (1996) An overtraining-resistant stochastic modeling method for pattern recognition. Ann Stat 24(6):2319–2349

    Article  MathSciNet  MATH  Google Scholar 

  • Kleinberg EM (2000) On the algorithmic implementation of stochastic discrimination. IEEE Trans Pattern Anal Mach Intell 22(5):473–490

    Article  Google Scholar 

  • Kleijnen JP (2009) Kriging metamodeling in simulation: A review. Eur J Oper Res 192(3):707–716

    Article  MathSciNet  MATH  Google Scholar 

  • Koch PN, Simpson TW, Allen JK, Mistree F (1999) Statistical approximations for multidisciplinary design optimization: the problem of size. J Aircr 36(1):275–286

    Article  Google Scholar 

  • Krauss C, Do XA, Huck N (2017) Deep neural networks, gradient-boosted trees, random forests: statistical arbitrage on the S&P 500. Eur J Oper Res 259(2):689–702

    Article  MATH  Google Scholar 

  • La Fuente D, Andres R (2016) Simulation metamodeling with gaussian process: a numerical study. Industrial Engineering, North Carolina State University, Raleigh

    Google Scholar 

  • Laurent L, Le Riche R, Soulier B, Boucard P-A (2017) An overview of gradient-enhanced metamodels with applications. Arch Comput Methods Eng 26:61–106

    Article  MathSciNet  Google Scholar 

  • Lemercier B, Lacoste M, Loum M, Walter C (2012) Extrapolation at regional scale of local soil knowledge using boosted classification trees: a two-step approach. Geoderma 171:75–84

    Article  Google Scholar 

  • Lewis K, Mistree F (1998) The other side of multidisciplinary design optimization: accommodating a multi-objective, uncertain and non-deterministic world. Eng Optim 31(2):161–189

    Article  Google Scholar 

  • Li K, Deb K, Altinoz T, Yao X (2017) Empirical investigations of reference point based methods when facing a massively large number of objectives: First results. International Conference on Evolutionary Multi-Criterion Optimization, Springer

  • Liang C, Mahadevan S (2016) Stochastic multidisciplinary analysis with high-dimensional coupling. AIAA J 54(2):1209–1219

    Article  Google Scholar 

  • Lim D, Ong Y-S, Jin Y, Sendhoff B (2007) A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation. Proceedings of the 9th annual conference on Genetic and evolutionary computation. London, England, ACM: 1288–1295

  • Lin DK (1993) A new class of supersaturated designs. Technometrics 35(1):28–31

    Article  Google Scholar 

  • Lin CD, Anderson-Cook CM, Hamada MS, Moore LM, Sitter RR (2015) Using genetic algorithms to design experiments: a review. Qual Reliab Eng Int 31(2):155–167

    Article  Google Scholar 

  • Liu K, Tovar A, Nutwell E, Detwiler D (2015) Thin-walled compliant mechanism component design assisted by machine learning and multiple surrogates, SAE World Congress and Exposition

  • Lucas JM (1994) Using response surface methodology to achieve a robust process. J Qual Technol 26(4):248–260

    Article  Google Scholar 

  • Luo P, Cai Z, Geng Z (2019) Criteria for multiple surrogates. Stat Sinica 29(3):1343–1366

    MathSciNet  MATH  Google Scholar 

  • Lv Z, Zhao J, Wang W, Liu Q (2019) A multiple surrogates based PSO algorithm. Artif Intell Rev 52:2169–2190

    Article  Google Scholar 

  • MacCalman A, Lesinski G, Goerger S (2016) Integrating external simulations within the model-based systems engineering approach using statistical metamodels. Proced Comput Sci 95:436–441

    Article  Google Scholar 

  • Mack Y, Goel T, Shyy W, Haftka R, Queipo N (2005) Multiple surrogates for the shape optimization of bluff body-facilitated mixing. 43rd AIAA Aerospace Sciences Meeting and Exhibit, American Institute of Aeronautics and Astronautics

  • Madala HR, Ivakhnenko AG (1994) Inductive learning algorithms for complex systems modeling. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Maier HR, Dandy GC (2000) Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications. Environ Model Softw 15(1):101–124

    Article  Google Scholar 

  • Malavé V, Turner CJ (2011) Metamodel-assisted ice detection for wind turbine blades. International Mechanical Engineering Congress and Exposition. IMECE2011–62480: 565–572.

  • McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245

    MathSciNet  MATH  Google Scholar 

  • Messer M, Panchal JH, Krishnamurthy V, Klein B, Yoder PD, Allen JK, Mistree F (2010) Model selection under limited information using a value-of-information-based indicator. J Mech Des 132(12):121008

    Article  Google Scholar 

  • Michalski RS (1983) A theory and methodology of inductive learning. Mach Learn, I edn. Elsevier, Amsterdam, p 83

    Google Scholar 

  • Montgomery DC (2017) Design and analysis of experiments. Wiley, NY

    Google Scholar 

  • Mugunthan P, Shoemaker CA (2006) Assessing the impacts of parameter uncertainty for computationally expensive groundwater models. Water Resour Res 42(10):1–15

    Article  Google Scholar 

  • Myers RH, Montgomery DC, Anderson-Cook CM (2016) Response surface methodology: process and product optimization using designed experiments. Wiley, NY

    MATH  Google Scholar 

  • Nguyen N-K (1996) An algorithmic approach to constructing supersaturated designs. Technometrics 38(1):69–73

    Article  MATH  Google Scholar 

  • Owen AB (1992) Orthogonal arrays for computer experiments, integration and visualization. Stat Sinica 2(2):439–452

    MathSciNet  MATH  Google Scholar 

  • Paiva RM, Carvalho ARD, Crawford C, Suleman A (2010) Comparison of surrogate models in a multidisciplinary optimization framework for wing design. AIAA J 48(5):995–1006

    Article  Google Scholar 

  • Panchal JH, Paredis CJJ, Allen JK, Mistree F (2008) A value-of-information based approach to simulation model refinement. Eng Optim 40(3):223–251

    Article  Google Scholar 

  • Panchal JH, Paredis CJ, Allen JK, Mistree F (2009) Managing design-process complexity: a value-of-information based approach for scale and decision decoupling. J Comput Inf Sci Eng 9(2):021005

    Article  Google Scholar 

  • Peherstorfer B, Willcox K, Gunzburger M (2018) Survey of multifidelity methods in uncertainty propagation, inference, and optimization. Siam Rev 60(3):550–591

    Article  MathSciNet  MATH  Google Scholar 

  • Phoa FKH, Pan Y-H, Xu H (2009) Analysis of supersaturated designs via the Dantzig selector. J Stat Plan Inference 139(7):2362–2372

    Article  MathSciNet  MATH  Google Scholar 

  • Picheny V, Ginsbourger D, Roustant O, Haftka RT, Kim N-H (2010) Adaptive designs of experiments for accurate approximation of a target region. J Mech Des 132(7):071008–071009

    Article  Google Scholar 

  • Pickett B, Turner CJ (2011) A review and evaluation of existing adaptive sampling criteria and methods for the creation of nurbs-based metamodels. International Design Engineering Technical Conferences, DETC2011-47288

  • Qian PZ (2009) Nested Latin hypercube designs. Biometrika 96(4):957–970

    Article  MathSciNet  MATH  Google Scholar 

  • Qian Z, Seepersad CC, Joseph VR, Allen JK, Wu CJ (2006) Building surrogate models based on detailed and approximate simulations. J Mech Des 128(4):668–677

    Article  Google Scholar 

  • Qian PZG, Wu H, Wu CJ (2008) Gaussian process models for computer experiments with qualitative and quantitative factors. Technometrics 50(3):383–396

    Article  MathSciNet  Google Scholar 

  • Qian PZ, Ai M, Wu CJ (2009a) Construction of nested space-filling designs. Ann Stat 37(6A):3616–3643

    Article  MathSciNet  MATH  Google Scholar 

  • Qian PZ, Tang B, Wu CJ (2009b) Nested space-filling designs for computer experiments with two levels of accuracy. Stat Sinica 19(1):287–300

    MathSciNet  MATH  Google Scholar 

  • Rabitz H (1989) Systems analysis at the molecular scale. Science 246(4927):221–226

    Article  Google Scholar 

  • Ratto M, Castelletti A, Pagano A (2012) Emulation techniques for the reduction and sensitivity analysis of complex environmental models. Elsevier, Amsterdam

    Book  Google Scholar 

  • Reese CS, Wilson AG, Hamada M, Martz HF, Ryan KJ (2004) Integrated analysis of computer and physical experiments. Technometrics 46(2):153–164

    Article  MathSciNet  Google Scholar 

  • Regis RG, Shoemaker CA (2007) A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J Comput 19(4):497–509

    Article  MathSciNet  MATH  Google Scholar 

  • Ripley BD (2007) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–435

    Article  MathSciNet  MATH  Google Scholar 

  • Sadaghiani M, Alizadeh R, Bahrami M (2014) Scenario-based planning for energy foresight case study: Iran’s transportation industry. The 10th international energy conference (IEC 2014) Tehran, Iran, Ministry of Energy

  • Samad A, Kim K-Y, Goel T, Haftka RT, Shyy W (2006) Shape optimization of turbomachinery blade using multiple surrogate models. Fluids Engineering Division Summer Meeting: FEDSM2006–98368

  • Samad A, Lee K-D, Kim K-Y, Haftka R (2007) Application of multiple-surrogate model to optimization of a dimpled channel. 7th World Congress on Structural and Multidisciplinary Optimization. pp. 2276–2282

  • Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117

    Article  Google Scholar 

  • Seni G, Elder JF (2010) Ensemble methods in data mining: improving accuracy through combining predictions. Synth Lect Data Min Knowl Discov 2(1):1–126

    Article  Google Scholar 

  • Shan S, Wang GG (2010) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241

    Article  MathSciNet  MATH  Google Scholar 

  • Shankar Bhattacharjee K, Kumar Singh H, Ray T (2016) Multi-objective optimization with multiple spatially distributed surrogates. J Mech Des 138(9):091401–091410

    Article  Google Scholar 

  • Shi R, Liu L, Long T, Liu J (2016) An efficient ensemble of radial basis functions method based on quadratic programming. Eng Optim 48(7):1202–1225

    Article  MathSciNet  Google Scholar 

  • Sim DYY, Teh CS, Ismail AI (2018) Improved boosted decision tree algorithms by adaptive a priori and post-pruning for predicting obstructive sleep apnea. Adv Sci Lett 24(3):1680–1684

    Article  Google Scholar 

  • Simpson TW, Poplinski JD, Koch PN, Allen JK (1997) On the use of statistics in design and the implications for deterministic computer experiments. ASME Design Engineering Technical Conferences, Sacramento, California

  • Simpson T, Mistree F, Korte, Mauery T (1998) Comparison of response surface and kriging models for multidisciplinary design optimization. 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, American Institute of Aeronautics and Astronautics

  • Simpson TW, Poplinski JD, Koch PN, Allen JK (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17(2):129–150

    Article  MATH  Google Scholar 

  • Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang R-J (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27(5):302–313

    Article  Google Scholar 

  • Simpson T, Toropov V, Balabanov V, Viana F (2008) Design and analysis of computer experiments in multidisciplinary design optimization: A review of how far we have come-or not. 12th AIAA/ISSMO multidisciplinary analysis and optimization conference

  • Sobieszczanski-Sobieski J, Haftka RT (1997) Multidisciplinary aerospace design optimization: survey of recent developments. Struct Optim 14(1):1–23

    Article  Google Scholar 

  • Sobol IM (1993) Sensitivity analysis for nonlinear mathematical models, mathematical modelling and computational experiments. Math Model Comput Exp 1(4):407–414

    MATH  Google Scholar 

  • Song X, Lv L, Li J, Sun W, Zhang J (2018) An advanced and robust ensemble surrogate model: extended adaptive hybrid functions. J Mech Des 140(4):041402

    Article  Google Scholar 

  • Stein ML (2012) Interpolation of spatial data: some theory for kriging. Springer, Berlin

    Google Scholar 

  • Steuben JC (2014) Massively parallel engineering simulations on graphics processors: parallelization, synchronization, and approximation. Colorado School of Mines, Golden

    Google Scholar 

  • Steuben J, Turner CJ (2010) Robust optimization exploration using nurbs-based metamodeling techniques. International Design Engineering Technical Conferences International Design Engineering Technical Conferences, DETC2010–28226

  • Steuben J, Turner CJ (2011a). Robust optimization and analysis of nurbs-based metamodels using graph theory. International Design Engineering Technical Conferences DETC2011–47217: 587–598

  • Steuben J, Turner CJ (2011b) Waypoint-based robot navigation using NURBs-based metamodels. International Mechanical Engeineeing Congress and Exposition. IMECE2011–62450: 1123–1133

  • Steuben JC, Turner CJ (2012) Robust optimization of mixed-integer problems using NURBS-based metamodels. J Comput Inf Sci Eng 12(4):041010–041017

    Article  Google Scholar 

  • Steuben J, Turner C (2014) Adaptive surrogate-model fitting using error monotonicity. International Design Engineering Technical Conferences, DETC2014–35037

  • Steuben JC, Turner CJ (2015) Graph analysis of non-uniform rational B-spline-based metamodels. Eng Optim 47(9):1157–1176

    Article  MathSciNet  Google Scholar 

  • Steuben JC, Turner CJ, Crawford RH (2013) Robust engineering design optimization with non-uniform rational B-splines-based metamodels. Eng Optim 45(7):767–786

    Article  Google Scholar 

  • Steuben J, Michopoulos J, Iliopoulos A, Turner C (2015a) Inverse characterization of composite materials via surrogate modeling. Compos Struct 132:694–708

    Article  Google Scholar 

  • Steuben J, Michopoulos JG, Iliopoulos A, Turner C (2015b) Towards real-time composite material characterization using surrogate models and GPGPU computing. International Design Engineering Technical Conferences, DETC2015–46476

  • Steuben JC, Birnbaum AJ, Michopoulos JG, Iliopoulos AP (2019) Enriched analytical solutions for additive manufacturing modeling and simulation. Addit Manuf 25:437–447

    Google Scholar 

  • Trucano TG, Swiler LP, Igusa T, Oberkampf WL, Pilch M (2006) Calibration, validation, and sensitivity analysis: what's what. Reliab Eng Syst Saf 91(10):1331–1357

    Article  Google Scholar 

  • Turner CJ (2005) HyPerModels: hyperdimensional performance models for engineering design. The University of Texas at Austin, Texas

    Google Scholar 

  • Turner CJ (2010) Diagnosis via NURBs metamodel. International Mechanical Engineering Congress And Exposition IMECE2010–3823: 341–346

  • Turner CJ (2011) Metamodeling in product and process design. International Design Engineering Technical Conferences, DETC2011–47833: 639–651

  • Turner CJ, Crawford RH (2005a) Adapting non-uniform rational B-spline fitting approaches to metamodeling. International Design Engineering Technical Conferences, DETC2005:85544: 165–178

  • Turner CJ, Crawford RH (2005b) Selecting an appropriate metamodel: the case for NURBs metamodels. International Design Engineering Technical Conferences, DETC2005–85043: 759–771

  • Turner CJ, Crawford RH (2009) N-Dimensional Nonuniform Rational B-Splines for Metamodeling. J Comput Inf Sci Eng 9(3):031002–031013

    Article  Google Scholar 

  • Turner CJ, Campbell MI, Crawford RH (2003) Generic sequential sampling for metamodel approximations. International Design Engineering Technical Conferences, DETC2004–48230: 555–564

  • Turner CJ, Campbell MI, Crawford RH (2004) Metamodel defined multidimensional embedded sequential sampling criteria. International Design Engineering Technical Conferences, DETC2004–57722: 703–714

  • Turner CJ, Ajetunmobi AM, Crawford RH (2006) Fault detection with NURBs-based metamodels. International Design Engineering Technical Conferences, DETC2004–99637: 439–450

  • Turner CJ, Crawford RH, Campbell MI (2007a) Global optimization of NURBs-based metamodels. Eng Optim 39(3):245–269

    Article  MathSciNet  Google Scholar 

  • Turner CJ, Crawford RH, Campbell MI (2007b) Mixed integer optimization with NURBs HyPerModels. Internationa Design Engineering Technical Conferences. DETC2007–35852: 371–379

  • Turner CJ, Crawford RH, Campbell MI (2007c) Multidimensional sequential sampling for NURBs-based metamodel development. Eng Comput 23(3):155–174

    Article  Google Scholar 

  • Turner C, Pickett B, Steuben J (2011) Design space analysis with hyperdimensional metamodels. Proceedings of the 2011 NSF-CMMI Grantees Conference, Atlanta, GA

  • Tutum CC, Deb K (2015) A multimodal approach for evolutionary multi-objective optimization (MEMO): proof-of-principle results. International Conference on Evolutionary Multi-Criterion Optimization, Springer

  • Vapnik V (2013) The nature of statistical learning theory, Springer, Berlin

  • Varadarajan S, Chen WEI, Pelka CJ (2000) Robust concept exploration of propulsion systems with enhanced model approximation capabilities. Eng Optim 32(3):309–334

    Article  Google Scholar 

  • Venter G, Haftka R, Chirehdast M, Venter G, Haftka R, Chirehdast M (1997) Response surface approximations for fatigue life prediction. 38th Structures, Structural Dynamics, and Materials Conference, American Institute of Aeronautics and Astronautics

  • Viana FAC, Haftka RT (2008a). Using multiple surrogates for metamodeling. Proceedings of the 7th ASMO-UK/ISSMO International Conference on Engineering Design Optimization

  • Viana FAC, RT Haftka (2008b) Using multiple surrogates for minimization of the rms error in meta-modeling. International Design Engineering Technical Conferences, DETC2008–49240, 851–860

  • Viana F, Haftka R, Steffen JV, Butkewitsch S, Leal MF (2008) Optimal use of multiple surrogate for reduced RMS error in meta-model. Proceedings of 2008 NSF Engineering Research and Innovation Conference, Knoxville, Tennessee

  • Viana FA, Picheny V, Haftka RT (2009) Conservative prediction via safety margin: design through cross-validation and benefits of multiple surrogates. ASME 2009 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers

  • Viana F, Haftka R, Watson L (2010a) Why not run the efficient global optimization algorithm with multiple surrogates? In 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference 18th AIAA/ASME/AHS Adaptive Structures Conference 12th, p 3090

  • Viana FA, Gogu C, Haftka RT (2010b) Making the most out of surrogate models: tricks of the trade. In ASME 2010 International Design Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers Digital Collection, pp 587–598

  • Viana FAC, Picheny V, Haftka RT (2010c) Using cross validation to design conservative surrogates. AIAA J 48(10):2286–2298

    Article  Google Scholar 

  • Viana FA, Haftka RT, Watson LT (2013) Efficient global optimization algorithm assisted by multiple surrogate techniques. J Global Optim 56(2):669–689

    Article  MATH  Google Scholar 

  • Viana FAC, Simpson TW, Balabanov V, Toropov V (2014) Special section on multidisciplinary design optimization: metamodeling in multidisciplinary design optimization: how far have we really come? AIAA J 52(4):670–690

    Article  Google Scholar 

  • Villanueva D, Haftka RT, Le Riche R, Picard G (2013) Locating multiple candidate designs with surrogate-based optimization. 10th World Congress on structural and multidisciplinary optimization, Orlando, USA, May, Citeseer

  • Vinzi V, Chin WW, Henseler J, Wang H (2010) Handbook of partial least squares. Springer, Berlin

    Book  MATH  Google Scholar 

  • Wang C, Duan Q, Gong W, Ye A, Di Z, Miao C (2014) An evaluation of adaptive surrogate modeling based optimization with two benchmark problems. Environ Model Softw 60:167–179

    Article  Google Scholar 

  • Wang H, Ye F, Li E, Li G (2016) A comparative study of expected improvement-assisted global optimization with different surrogates. Eng Optim 48(8):1432–1458

    Article  Google Scholar 

  • Witten IH, Frank E, Hall MA, Pal CJ (2016) Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, Burlington

    Google Scholar 

  • Xing Z, Qu R, Zhao Y, Fu Q, Ji Y, Lu W (2019) Identifying the release history of a groundwater contaminant source based on an ensemble surrogate model. J Hydrol 572:501–516

    Article  Google Scholar 

  • Xiong Y, Chen W, Tsui K-L, Apley DW (2009) A better understanding of model updating strategies in validating engineering models. Comput Methods Appl Mech Eng 198(15):1327–1337

    Article  MATH  Google Scholar 

  • Xiong S, Qian PZ, Wu CJ (2013) Sequential design and analysis of high-accuracy and low-accuracy computer codes. Technometrics 55(1):37–46

    Article  MathSciNet  Google Scholar 

  • Xu J, Zeger SL (2001) The evaluation of multiple surrogate endpoints. Biometrics 57(1):81–87

    Article  MathSciNet  MATH  Google Scholar 

  • Xue Z, Lietz R, Rigoni E, Parashar S, Kansara S (2013) RSM improvement methods for computationally expensive industrial CAE analysis. 10th World Congress on Structural and Multidisciplinary Optimization

  • Yang T, Lin H-C, Chen M-L (2006) Metamodeling approach in solving the machine parameters optimization problem using neural network and genetic algorithms: a case study. Robot Comput Integr Manuf 22(4):322–331

    Article  Google Scholar 

  • Ye P, Pan G, Dong Z (2018) Ensemble of surrogate based global optimization methods using hierarchical design space reduction. Struct Multidiscip Optim 58(2):537–554

    Article  MathSciNet  Google Scholar 

  • Yin H, Fang H, Wen G, Gutowski M, Xiao Y (2018) On the ensemble of metamodels with multiple regional optimized weight factors. Struct Multidiscip Optim 58(1):245–263

    Article  Google Scholar 

  • Zhao J, Liu Q, Pedrycz W, Li D (2012) Effective noise estimation-based online prediction for byproduct gas system in steel industry. IEEE Trans Industr Inf 8(4):953–963

    Article  Google Scholar 

  • Zhu Z, Du X (2016) Reliability analysis with Monte Carlo simulation and dependent Kriging predictions. J Mech Des. https://doi.org/10.1115/1.4034219

    Google Scholar 

Download references

Funding

This study was supported by Tata Consultancy Services (Grant no. 105-373200) and The University of Oklahoma (Grant nos. 122-794800, 122-763300).

Author information

Authors and Affiliations

  1. The Systems Realization Laboratory, Industrial and Systems Engineering, University of Oklahoma, Norman, Oklahoma, USA

    Reza Alizadeh & Janet K. Allen

  2. The Systems Realization Laboratory, Aerospace and Mechanical Engineering, University of Oklahoma, Norman, Oklahoma, USA

    Farrokh Mistree

Authors
  1. Reza Alizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janet K. Allen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farrokh Mistree
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janet K. Allen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alizadeh, R., Allen, J.K. & Mistree, F. Managing computational complexity using surrogate models: a critical review. Res Eng Design 31, 275–298 (2020). https://doi.org/10.1007/s00163-020-00336-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00163-020-00336-7

Keywords

Search

Navigation