Skip to main content
SpringerLink
Log in

A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design

  • REVIEW ARTICLE
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Metamodeling is becoming a rather popular means to approximate the expensive simulations in today’s complex engineering design problems since accurate metamodels can bring in a lot of benefits. The metamodel accuracy, however, heavily depends on the locations of the observed points. Adaptive sampling, as its name suggests, places more points in regions of interest by learning the information from previous data and metamodels. Consequently, compared to traditional space-filling sampling approaches, adaptive sampling has great potential to build more accurate metamodels with fewer points (simulations), thereby gaining increasing attention and interest by both practitioners and academicians in various fields. Noticing that there is a lack of reviews on adaptive sampling for global metamodeling in the literature, which is needed, this article categorizes, reviews, and analyzes the state-of-the-art single−/multi-response adaptive sampling approaches for global metamodeling in support of simulation-based engineering design. In addition, we also review and discuss some important issues that affect the success of an adaptive sampling approach as well as providing brief remarks on adaptive sampling for other purposes. Last, challenges and future research directions are provided and discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Acar E (2014) Simultaneous optimization of shape parameters and weight factors in ensemble of radial basis functions. Struct Multidiscip Optim 49(6):969–978

    Article  Google Scholar 

  • Acar E, Rais-Rohani M (2009) Ensemble of metamodels with optimized weight factors. Struct Multidiscip Optim 37(3):279–294

    Article  Google Scholar 

  • Ajdari A, Mahlooji H (2014) An adaptive exploration-exploitation algorithm for constructing metamodels in random simulation using a novel sequential experimental design. Commun Stat Simul Comput 43(5):947–968

    Article  MathSciNet  MATH  Google Scholar 

  • Alexandrov NM, Dennis JE Jr, Lewis RM, Torczon V (1998) A trust-region framework for managing the use of approximation models in optimization. Struct Optimization 15(1):16–23

    Article  Google Scholar 

  • Alvarez M, Lawrence ND (2009) Sparse convolved Gaussian processes for multi-output regression. In: Advances in Neural Information Processing Systems, pp 57–64

  • Alvarez MA, Rosasco L, Lawrence ND (2012) Kernels for vector-valued functions: a review. Found Trends Mach Learn 4(3):195–266

    Article  MATH  Google Scholar 

  • Atamturktur S, Williams B, Egeberg M, Unal C (2013) Batch sequential design of optimal experiments for improved predictive maturity in physics-based modeling. Struct Multidiscip Optim 48(3):549–569

    Article  Google Scholar 

  • Atkinson EJ, Therneau TM (2000) An introduction to recursive partitioning using the RPART routines. Mayo Foundation, Rochester

    Google Scholar 

  • Audze P, Eglais V (1977) New approach for planning out of experiments. Problems of Dynamics and Strengths 35:104–107

  • Auffray Y, Barbillon P, Marin J-M (2012) Maximin design on non hypercube domains and kernel interpolation. Stat Comput 22(3):703–712

    Article  MathSciNet  MATH  Google Scholar 

  • Aurenhammer F (1991) Voronoi diagrams—a survey of a fundamental geometric data structure. ACM Comput Surv 23(3):345–405

    Article  Google Scholar 

  • Aute VC (2009) Single and multiresponse adaptive design of experiments with application to design optimization of novel heat exchangers. University of Maryland, College Park, City of College Park

  • Aute V, Saleh K, Abdelaziz O, Azarm S, Radermacher R (2013) Cross-validation based single response adaptive design of experiments for kriging metamodeling of deterministic computer simulations. Struct Multidiscip Optim 48(3):581–605

    Article  Google Scholar 

  • Barton RR (1997) Design of experiments for fitting subsystem metamodels. In: Proceedings of the 29th conference on Winter simulation, Atlanta, Georgia, USA. IEEE, pp 303–310

  • Beck J, Guillas S (2016) Sequential design with mutual information for computer experiments (MICE): emulation of a tsunami model. SIAM/ASA J Uncertain Quantif 4(1):739–766

    Article  MathSciNet  MATH  Google Scholar 

  • Benamara T, Breitkopf P, Lepot I, Sainvitu C (2016) Adaptive infill sampling criterion for multi-fidelity optimization based on Gappy-POD. Struct Multidiscip Optim 54(4):843–855

    Article  Google Scholar 

  • Bonilla EV, Chai KMA, Williams CK (2007) Multi-task Gaussian process prediction. In: NIPs, pp 153–160

  • Borgonovo E, Plischke E (2016) Sensitivity analysis: a review of recent advances. Eur J Oper Res 248(3):869–887

    Article  MathSciNet  MATH  Google Scholar 

  • Braconnier T, Ferrier M, Jouhaud J-C, Montagnac M, Sagaut P (2011) Towards an adaptive POD/SVD surrogate model for aeronautic design. Comput Fluids 40(1):195–209

    Article  MathSciNet  MATH  Google Scholar 

  • Burbidge R, Rowland JJ, King RD (2007) Active learning for regression based on query by committee. In: International Conference on Intelligent Data Engineering and Automated Learning. Springer, pp 209–218

  • Busby D (2009) Hierarchical adaptive experimental design for Gaussian process emulators. Reliab Eng Syst Saf 94(7):1183–1193

    Article  Google Scholar 

  • Busby D, Farmer CL, Iske A (2007) Hierarchical nonlinear approximation for experimental design and statistical data fitting. SIAM J Sci Comput 29(1):49–69

    Article  MathSciNet  MATH  Google Scholar 

  • Cai X, Qiu H, Gao L, Yang P, Shao X (2016) An enhanced RBF-HDMR integrated with an adaptive sampling method for approximating high dimensional problems in engineering design. Struct Multidiscip Optim 53(6):1209–1229

    Article  Google Scholar 

  • Caruana R (1995) Learning many related tasks at the same time with backpropagation. In: Advances in Neural Information Processing Systems. pp 657–664

  • Černý V (1985) Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J Optim Theory Appl 45(1):41–51

    Article  MathSciNet  MATH  Google Scholar 

  • Chen R-B, Hsu Y-W, Hung Y, Wang W (2014) Discrete particle swarm optimization for constructing uniform design on irregular regions. Comput Stat Data Anal 72:282–297

    Article  MathSciNet  Google Scholar 

  • Chuang S, Hung Y (2010) Uniform design over general input domains with applications to target region estimation in computer experiments. Comput Stat Data Anal 54(1):219–232

    Article  MathSciNet  MATH  Google Scholar 

  • Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087

    Article  Google Scholar 

  • Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International conference on machine learning. ACM, p 160–167

  • Cressie N (1988) Spatial prediction and ordinary kriging. Math Geol 20(4):405–421

    Article  MathSciNet  MATH  Google Scholar 

  • Crombecq K (2011) Surrogate modeling of computer experiments with sequential experimental design. Ghent University, Antwerpen

    MATH  Google Scholar 

  • Crombecq K, Gorissen D, Deschrijver D, Dhaene T (2011a) A novel hybrid sequential design strategy for global surrogate modeling of computer experiments. SIAM J Sci Comput 33(4):1948–1974

    Article  MathSciNet  MATH  Google Scholar 

  • Crombecq K, Laermans E, Dhaene T (2011b) Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling. Eur J Oper Res 214(3):683–696

    Article  Google Scholar 

  • Damblin G, Couplet M, Iooss B (2013) Numerical studies of space-filling designs: optimization of Latin hypercube samples and subprojection properties. J Simul 7(4):276–289

    Article  Google Scholar 

  • De Geest J, Dhaene T, Faché N, De Zutter D (1999) Adaptive CAD-model building algorithm for general planar microwave structures. IEEE Trans Microwave Theory Tech 47(9):1801–1809

    Article  Google Scholar 

  • de Oliveira Castro P, Petit E, Beyler JC, Jalby W (2012) ASK: adaptive sampling kit for performance characterization. In: European conference on parallel processing, Rhodes Island, Greece. Springer, pp 89–101

  • Deschrijver D, Crombecq K, Nguyen HM, Dhaene T (2011) Adaptive sampling algorithm for macromodeling of parameterized-parameter responses. IEEE Trans Microwave Theory Tech 59(1):39–45

    Article  Google Scholar 

  • Devabhaktuni VK, Zhang Q-J (2000) Neural network training-driven adaptive sampling algorithm for microwave modeling. In: 2000 30th European microwave conference, Paris, France. IEEE, pp 1–4

  • dos Santos MIR, dos Santos PMR (2008) Sequential experimental designs for nonlinear regression metamodels in simulation. Simul Model Pract Theory 16(9):1365–1378

    Article  Google Scholar 

  • Douak F, Melgani F, Alajlan N, Pasolli E, Bazi Y, Benoudjit N (2012) Active learning for spectroscopic data regression. J Chemom 26(7):374–383

    Article  Google Scholar 

  • Draguljić D, Santner TJ, Dean AM (2012) Noncollapsing space-filling designs for bounded nonrectangular regions. Technometrics 54(2):169–178

    Article  MathSciNet  Google Scholar 

  • Dyn N, Levin D, Rippa S (1986) Numerical procedures for surface fitting of scattered data by radial functions. SIAM J Sci Stat Comput 7(2):639–659

    Article  MathSciNet  MATH  Google Scholar 

  • Eason J, Cremaschi S (2014) Adaptive sequential sampling for surrogate model generation with artificial neural networks. Comput Chem Eng 68:220–232

    Article  Google Scholar 

  • Efron B, Tibshirani RJ (1994) An introduction to the bootstrap. CRC press, Florida

    MATH  Google Scholar 

  • Fang K-T, Lin DK, Winker P, Zhang Y (2000) Uniform design: theory and application. Technometrics 42(3):237–248

    Article  MathSciNet  MATH  Google Scholar 

  • Farhang-Mehr A, Azarm S (2005) Bayesian meta-modelling of engineering design simulations: a sequential approach with adaptation to irregularities in the response behaviour. Int J Numer Methods Eng 62(15):2104–2126

    Article  MATH  Google Scholar 

  • Fedorov VV (1972) Theory of optimal experiments. Academic Press, New York

    Google Scholar 

  • Fernández-Godino MG, Park C, Kim N-H, Haftka RT (2016) Review of multi-fidelity models. arXiv preprint arXiv:160907196

  • Finkel RA, Bentley JL (1974) Quad trees a data structure for retrieval on composite keys. Acta Informatica 4(1):1–9

    Article  MATH  Google Scholar 

  • Forrester AI, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1):50–79

    Article  Google Scholar 

  • Forrester AI, Sóbester A, Keane AJ (2007) Multi-fidelity optimization via surrogate modelling. Proc R Soc Lond A Math Phys Eng Sci 463(2088):3251–3269

    Article  MathSciNet  MATH  Google Scholar 

  • Freund Y, Seung HS, Shamir E, Tishby N (1993) Information, prediction, and query by committee. In: Advances in Neural Information Processing Systems, pp 483–483

  • Fu Y, Zhu X, Li B (2013) A survey on instance selection for active learning. Knowl Inf Syst 35(2):249–283

    Article  Google Scholar 

  • Gao Y, Wang Y-J (2007) A memetic differential evolutionary algorithm for high dimensional functions' optimization. In: Third International Conference on Natural Computation. IEEE, pp 188–192

  • Gazut S, Martinez J-M, Dreyfus G, Oussar Y (2008) Towards the optimal design of numerical experiments. IEEE Trans Neural Netw 19(5):874–882

    Article  Google Scholar 

  • Geman S, Bienenstock E, Doursat R (1992) Neural networks and the bias/variance dilemma. Neural Comput 4(1):1–58

    Article  Google Scholar 

  • Ghoreyshi M, Badcock K, Woodgate M (2009) Accelerating the numerical generation of aerodynamic models for flight simulation. J Aircr 46(3):972–980

    Article  Google Scholar 

  • Golzari A, Sefat MH, Jamshidi S (2015) Development of an adaptive surrogate model for production optimization. J Pet Sci Eng 133:677–688

    Article  Google Scholar 

  • Gorissen D, Couckuyt I, Demeester P, Dhaene T, Crombecq K (2010) A surrogate modeling and adaptive sampling toolbox for computer based design. J Mach Learn Res 11:2051–2055

    Google Scholar 

  • Gramacy RB, Lee HK (2006) Adaptive design of supercomputer experiments. The Statistical Laboratory, University of Cambridge, UK

    Google Scholar 

  • Gramacy RB, Lee HKH (2008) Bayesian treed Gaussian process models with an application to computer modeling. J Am Stat Assoc 103(483):1119–1130

    Article  MathSciNet  MATH  Google Scholar 

  • Gramacy RB, Lee HK (2009) Adaptive design and analysis of supercomputer experiments. Technometrics 51(2):130–145

    Article  MathSciNet  Google Scholar 

  • Grosso A, Jamali A, Locatelli M (2009) Finding maximin latin hypercube designs by iterated local search heuristics. Eur J Oper Res 197(2):541–547

    Article  MATH  Google Scholar 

  • Guenther J, Lee HK, Gray GA (2015) Sequential design for achieving estimated accuracy of global sensitivities. Appl Stoch Model Bus Ind 31(6):782–800

    Article  MathSciNet  Google Scholar 

  • Gutmann H-M (2001) A radial basis function method for global optimization. J Glob Optim 19(3):201–227

    Article  MathSciNet  MATH  Google Scholar 

  • Haaland B, Qian PZ (2010) An approach to constructing nested space-filling designs for multi-fidelity computer experiments. Stat Sin 20(3):1063

    MathSciNet  MATH  Google Scholar 

  • Haftka RT, Villanueva D, Chaudhuri A (2016) Parallel surrogate-assisted global optimization with expensive functions–a survey. Struct Multidiscip Optim 54(1):3–13

    Article  MathSciNet  Google Scholar 

  • Halton JH (1960) On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer Math 2(1):84–90

    Article  MathSciNet  MATH  Google Scholar 

  • Han Z-H, Görtz S, Hain R (2010a) A variable-fidelity modeling method for aero-loads prediction. In: New results in numerical and experimental fluid mechanics VII. Springer, pp 17–25

  • Han Z-H, Zimmermann R, Goretz S (2010b) A new cokriging method for variable-fidelity surrogate modeling of aerodynamic data. In: 48th AIAA Aerospace sciences meeting including the new horizons forum and Aerospace exposition, p 1225

  • Han Z-H, Görtz S, Zimmermann R (2013) Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function. Aerosp Sci Technol 25(1):177–189

    Article  Google Scholar 

  • Helton JC, Davis F, Johnson JD (2005) A comparison of uncertainty and sensitivity analysis results obtained with random and Latin hypercube sampling. Reliab Eng Syst Saf 89(3):305–330

    Article  Google Scholar 

  • Helton JC, Johnson JD, Oberkampf W, Sallaberry CJ (2006) Sensitivity analysis in conjunction with evidence theory representations of epistemic uncertainty. Reliab Eng Syst Saf 91(10):1414–1434

    Article  Google Scholar 

  • Hendrickx W, Dhaene T (2005) Sequential design and rational metamodelling. In: Proceedings of the 37th conference on Winter simulation, Orlando, Florida. ACM, pp 290–298

  • Hernández-Lobato JM, Hoffman MW, Ghahramani Z (2014) Predictive entropy search for efficient global optimization of black-box functions. In: Advances in Neural Information Processing Systems. pp 918–926

  • Huang Z, Qiu H, Zhao M, Cai X, Gao L (2015) An adaptive SVR-HDMR model for approximating high dimensional problems. Eng Comput 32(3):643–667

    Article  Google Scholar 

  • Huang H, Lin DK, Liu M-Q, Yang J-F (2016) Computer experiments with both qualitative and quantitative variables. Technometrics 58(4):495–507

    Article  MathSciNet  Google Scholar 

  • Husslage B, Van Dam E, Den Hertog D (2005) Nested maximin Latin hypercube designs in two dimensions. CentER discussion paper no. 2005-79

  • Iooss B, Lemaître P (2015) A review on global sensitivity analysis methods. In: Uncertainty management in simulation-optimization of complex systems. Springer, pp 101–122

  • Janouchová E, Kučerová A (2013) Competitive comparison of optimal designs of experiments for sampling-based sensitivity analysis. Comput Struct 124:47–60

    Article  Google Scholar 

  • Jiang P, Shu L, Zhou Q, Zhou H, Shao X, Xu J (2015) A novel sequential exploration-exploitation sampling strategy for global metamodeling. IFAC-PapersOnLine 48(28):532–537

    Article  Google Scholar 

  • Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol Comput 1(2):61–70

    Article  Google Scholar 

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23(1):1–13

    Article  Google Scholar 

  • Jin R, Chen W, Sudjianto A (2002) On sequential sampling for global metamodeling in engineering design. In: ASME 2002 International design engineering technical Conferences and Computers and information in engineering conference, Montreal, Canada. ASME, pp 539–548

  • Jin R, Chen W, Sudjianto A (2005) An efficient algorithm for constructing optimal design of computer experiments. J Stat Plan Inference 134(1):268–287

    Article  MathSciNet  MATH  Google Scholar 

  • Jin Y, Li J, Du W, Qian F (2016) Adaptive sampling for surrogate modelling with artificial neural network and its application in an industrial cracking furnace. Can J Chem Eng 94(2):262–272

    Article  Google Scholar 

  • Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inference 26(2):131–148

    Article  MathSciNet  Google Scholar 

  • Jones DR, Perttunen CD, Stuckman BE (1993) Lipschitzian optimization without the Lipschitz constant. J Optim Theory Appl 79(1):157–181

    Article  MathSciNet  MATH  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

    Article  MathSciNet  MATH  Google Scholar 

  • Joseph VR, Hung Y (2008) Orthogonal-maximin Latin hypercube designs. Stat Sin 18(1):171–186

    MathSciNet  MATH  Google Scholar 

  • Kalagnanam JR, Diwekar UM (1997) An efficient sampling technique for off-line quality control. Technometrics 39(3):308–319

    Article  MATH  Google Scholar 

  • Kennedy MC, O'Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13

    Article  MathSciNet  MATH  Google Scholar 

  • Kennedy MC, O'Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B Stat Methodol 63(3):425–464

    Article  MathSciNet  MATH  Google Scholar 

  • Kenny QY, Li W, Sudjianto A (2000) Algorithmic construction of optimal symmetric Latin hypercube designs. J Stat Plan Inference 90(1):145–159

    Article  MathSciNet  MATH  Google Scholar 

  • Kim B, Lee Y, Choi D-H (2009) Construction of the radial basis function based on a sequential sampling approach using cross-validation. J Mech Sci Technol 23(12):3357–3365

    Article  Google Scholar 

  • Kitayama S, Arakawa M, Yamazaki K (2011) Sequential approximate optimization using radial basis function network for engineering optimization. Optim Eng 12(4):535–557

    Article  MATH  Google Scholar 

  • Kleijnen JP (2005) An overview of the design and analysis of simulation experiments for sensitivity analysis. Eur J Oper Res 164(2):287–300

    Article  MathSciNet  MATH  Google Scholar 

  • Kleijnen JP (2008) Design and analysis of simulation experiments. Springer, New York

    MATH  Google Scholar 

  • Kleijnen JP (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716

    Article  MathSciNet  MATH  Google Scholar 

  • Kleijnen JP (2015) Kriging metamodels and their designs. In: Design and analysis of simulation experiments. Springer, pp 179–239

  • Kleijnen JP, Van Beers WC (2004) Application-driven sequential designs for simulation experiments: kriging metamodelling. J Oper Res Soc 55(8):876–883

    Article  MATH  Google Scholar 

  • Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation, and active learning. In: Advances in Neural Information Processing Systems, pp 231–238

  • Kupresanin A, Johannesson G (2011) Comparison of sequential designs of computer experiments in high dimensions. Technical report LLNL-TR-491692, Lawrence Livermore National Laboratory (LLNL), Livermore

  • Lam CQ (2008) Sequential adaptive designs in computer experiments for response surface model fit. Ph.D. thesis. The Ohio State University, Columbus

  • Le Gratiet L, Cannamela C (2015) Cokriging-based sequential design strategies using fast cross-validation techniques for multi-fidelity computer codes. Technometrics 57(3):418–427

    Article  MathSciNet  Google Scholar 

  • Le Gratiet L, Garnier J (2014) Recursive co-kriging model for design of computer experiments with multiple levels of fidelity. Int J Uncertain Quantif 4(5):365–386

    Article  MathSciNet  Google Scholar 

  • Le Gratiet L, Marelli S, Sudret B (2017) Metamodel-based sensitivity analysis: polynomial chaos expansions and Gaussian processes. In: Ghanem R, Higdon D, Owhadi H (eds) Handbook of uncertainty quantification. Springer International Publishing, Cham, pp 1289–1325. doi:10.1007/978-3-319-12385-1_38

  • Li G, Azarm S, Farhang-Mehr A, Diaz A (2006) Approximation of multiresponse deterministic engineering simulations: a dependent metamodeling approach. Struct Multidiscip Optim 31(4):260–269

    Article  Google Scholar 

  • Li B, Peng L, Ramadass B (2009) Accurate and efficient processor performance prediction via regression tree based modeling. J Syst Archit 55(10):457–467

    Article  Google Scholar 

  • Li G, Aute V, Azarm S (2010) An accumulative error based adaptive design of experiments for offline metamodeling. Struct Multidiscip Optim 40(1):137–155

    Article  Google Scholar 

  • Li E, Wang H, Li G (2012) High dimensional model representation (HDMR) coupled intelligent sampling strategy for nonlinear problems. Comput Phys Commun 183(9):1947–1955

    Article  MathSciNet  Google Scholar 

  • Liefvendahl M, Stocki R (2006) A study on algorithms for optimization of Latin hypercubes. J Stat Plan Inference 136(9):3231–3247

    Article  MathSciNet  MATH  Google Scholar 

  • Lin Y (2004) An efficient robust concept exploration method and sequential exploratory experimental design. Georgia Institute of Technology, Atlanta

    Google Scholar 

  • Lin Y, Mistree F, Allen JK, Tsui K-L, Chen VC (2004) A sequential exploratory experimental design method: development of appropriate empirical models in design. In: ASME 2004 International design engineering technical Conferences and Computers and information in engineering conference, Salt Lake City, Utah, USA. ASME, pp 1021–1035

  • Lin CD, Mukerjee R, Tang B (2009) Construction of orthogonal and nearly orthogonal Latin hypercubes. Biometrika 96(1):243–247

    Article  MathSciNet  MATH  Google Scholar 

  • Liu X, Zhu Q, Lu H (2014) Modeling multiresponse surfaces for airfoil design with multiple-output-Gaussian-process regression. J Aircr 51(3):740–747

    Article  Google Scholar 

  • Liu H, Xu S, Wang X (2015a) Sequential sampling designs based on space reduction. Eng Optim 47(7):867–884

    Article  Google Scholar 

  • Liu H, Xu S, Wang X, Wu J, Song Y (2015b) A global optimization algorithm for simulation-based problems via the extended DIRECT scheme. Eng Optim 47(11):1441–1458

    Article  MathSciNet  Google Scholar 

  • Liu H, Xu S, Ma Y, Chen X, Wang X (2016a) An adaptive Bayesian sequential sampling approach for global metamodeling. J Mech Des 138(1):011404

    Article  Google Scholar 

  • Liu H, Xu S, Wang X, Meng J, Yang S (2016b) Optimal weighted pointwise ensemble of radial basis functions with different basis functions. AIAA J 3117–3133

  • Liu H, Xu S, Wang X, Yang S, Meng J (2016c) A multi-response adaptive sampling approach for global metamodeling. Proc Inst Mech Eng Pt C J Mechan Eng Sci. doi:10.1177/0954406216672250

  • Liu H, Wang X, Xu S (2017a) Generalized radial basis function-based high-dimensional model representation handling existing random data. J Mech Des 139(1):011404

    Article  Google Scholar 

  • Liu H, Xu S, Chen X, Wang X, Ma Q (2017b) Constrained optimization via a DIRECT-type constraint-handling technique and an adaptive metamodeling strategy. Struct Multidiscip Optim 55(1):155–177

    Article  MathSciNet  Google Scholar 

  • Loeppky JL, Sacks J, Welch WJ (2009) Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4):366–376

    Article  MathSciNet  Google Scholar 

  • Loeppky JL, Moore LM, Williams BJ (2010) Batch sequential designs for computer experiments. J Stat Plan Inference 140(6):1452–1464

    Article  MathSciNet  MATH  Google Scholar 

  • Loeppky JL, Moore LM, Williams BJ (2012) Projection array based designs for computer experiments. J Stat Plan Inference 142(6):1493–1505

    Article  MathSciNet  MATH  Google Scholar 

  • Lovison A, Rigoni E (2011) Adaptive sampling with a Lipschitz criterion for accurate metamodeling. Commun Appl Ind Math 1(2):110–126

    MathSciNet  MATH  Google Scholar 

  • Mackman T, Allen C (2010a) Aerodynamic data modelling using multi-criteria adaptive sampling. In: 13th AIAA/ISSMO multidisciplinary analysis optimization conference, Ft. worth. AIAA, pp AIAA 2010-9194

  • Mackman T, Allen C (2010b) Investigation of an adaptive sampling method for data interpolation using radial basis functions. Int J Numer Methods Eng 83(7):915–938

    MATH  Google Scholar 

  • Mackman T, Allen C, Ghoreyshi M, Badcock K (2013) Comparison of adaptive sampling methods for generation of surrogate aerodynamic models. AIAA J 51(4):797–808

    Article  Google Scholar 

  • Martins JR, Lambe AB (2013) Multidisciplinary design optimization: a survey of architectures. AIAA J 51(9):2049–2075

    Article  Google Scholar 

  • McKay MD, Beckman RJ, Conover WJ (1979) Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245

    MathSciNet  MATH  Google Scholar 

  • Meckesheimer M, Booker AJ, Barton RR, Simpson TW (2002) Computationally inexpensive metamodel assessment strategies. AIAA J 40(10):2053–2060

    Article  Google Scholar 

  • Mendes-Moreira J, Soares C, Jorge AM, Sousa JFD (2012) Ensemble approaches for regression: a survey. ACM Comput Surv 45(1):10

    Article  MATH  Google Scholar 

  • Morris MD, Mitchell TJ, Ylvisaker D (1993) Bayesian design and analysis of computer experiments: use of derivatives in surface prediction. Technometrics 35(3):243–255

    Article  MathSciNet  MATH  Google Scholar 

  • Osborne MA, Roberts SJ, Rogers A, Jennings NR (2012) Real-time information processing of environmental sensor network data using bayesian gaussian processes. ACM Trans Sens Netw 9(1):1

    Article  Google Scholar 

  • Owen AB (1992) Orthogonal arrays for computer experiments, integration and visualization. Stat Sin 2(2):439–452

    MathSciNet  MATH  Google Scholar 

  • Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  • Pan G, Ye P, Wang P (2014a) A novel Latin hypercube algorithm via translational propagation. Sci World J 2014:163949

    Google Scholar 

  • Pan G, Ye P, Wang P, Yang Z (2014b) A sequential optimization sampling method for metamodels with radial basis functions. Sci World J 2014:192862

    Google Scholar 

  • Park C, Haftka RT, Kim NH (2017) Remarks on multi-fidelity surrogates. Struct Multidiscip Optim 55(3):1029–1050

    Article  MathSciNet  Google Scholar 

  • Parr J, Holden CM, Forrester AI, Keane AJ (2010) Review of efficient surrogate infill sampling criteria with constraint handling. In: 2nd International conference on engineering optimization, pp 1-10

  • Patterson H (1954) The errors of lattice sampling. J R Stat Soc Ser B Methodol 16(1):140–149

    MathSciNet  MATH  Google Scholar 

  • Peherstorfer B, Willcox K, Gunzburger M (2016) Survey of multifidelity methods in uncertainty propagation, inference, and optimization. Department of Aeronautics & Astronautics, MIT, Cambridge

    MATH  Google Scholar 

  • Pholdee N, Bureerat S (2015) An efficient optimum Latin hypercube sampling technique based on sequencing optimisation using simulated annealing. Int J Syst Sci 46(10):1780–1789

    Article  MATH  Google Scholar 

  • Pickett B, Turner CJ (2011) A review and evaluation of existing adaptive sampling criteria and methods for the creation of nurbs-based metamodels. In: ASME 2011 International design engineering technical Conferences and Computers and information in engineering conference, Washington, DC, USA. ASME, p 609–618

  • Pronzato L, Müller WG (2012) Design of computer experiments: space filling and beyond. Stat Comput 22(3):681–701

    Article  MathSciNet  MATH  Google Scholar 

  • Qian PZG (2009) Nested Latin hypercube designs. Biometrika 96(4):957–970

    Article  MathSciNet  MATH  Google Scholar 

  • Qian PZ, Ai M (2010) Nested lattice sampling: a new sampling scheme derived by randomizing nested orthogonal arrays. J Am Stat Assoc 105(491):1147–1155

    Article  MathSciNet  MATH  Google Scholar 

  • Qian PZ, Wu CJ (2008) Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50(2):192–204

    Article  MathSciNet  Google Scholar 

  • Qian PZG, Wu H, Wu CJ (2008) Gaussian process models for computer experiments with qualitative and quantitative factors. Technometrics 50(3):383–396

    Article  MathSciNet  Google Scholar 

  • Qian PZ, Tang B, Wu CJ (2009) Nested space-filling designs for computer experiments with two levels of accuracy. Stat Sin 9(1):287–300

    MathSciNet  MATH  Google Scholar 

  • Qian PZ, Ai M, Hwang Y, Su H (2014) Asymmetric nested lattice samples. Technometrics 56(1):46–54

    Article  MathSciNet  Google Scholar 

  • Quan A (2014) Batch sequencing methods for computer experiments. The Ohio State University, Columbus

    Google Scholar 

  • Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28

    Article  Google Scholar 

  • Rai R, Campbell M (2008) Q2S2: a new methodology for merging quantitative and qualitative information in experimental design. J Mech Des 130(3):031103

    Article  Google Scholar 

  • Rasmussen CE (2006) Gaussian processes for machine learning. The MIT press, London

    MATH  Google Scholar 

  • RayChaudhuri T, Hamey LG (1995) Minimisation of data collection by active learning. In: IEEE International Conference on Neural Networks. IEEE, p 1338–1341

  • Razavi S, Tolson BA, Burn DH (2012a) Numerical assessment of metamodelling strategies in computationally intensive optimization. Environ Model Softw 34:67–86

    Article  Google Scholar 

  • Razavi S, Tolson BA, Burn DH (2012b) Review of surrogate modeling in water resources. Water Resour Res 48(7):1–32

    Article  Google Scholar 

  • Regis RG, Shoemaker CA (2005) Constrained global optimization of expensive black box functions using radial basis functions. J Glob Optim 31(1):153–171

    Article  MathSciNet  MATH  Google Scholar 

  • Regis RG, Shoemaker CA (2007) A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J Comput 19(4):497–509

    Article  MathSciNet  MATH  Google Scholar 

  • Reichart R, Tomanek K, Hahn U (2008) Rappoport a multi-task active learning for linguistic annotations. In: ACL, p 861–869

  • Rennen G, Husslage B, Van Dam ER, Den Hertog D (2010) Nested maximin Latin hypercube designs. Struct Multidiscip Optim 41(3):371–395

    Article  MathSciNet  MATH  Google Scholar 

  • Rimmel A, Teytaud F (2014) A survey of meta-heuristics used for computing maximin latin hypercube. In: European Conference on Evolutionary Computation in Combinatorial Optimization. Springer, p 25–36

  • Rippa S (1999) An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Adv Comput Math 11(2):193–210

    Article  MathSciNet  MATH  Google Scholar 

  • Romero DA, Amon CH, Finger S (2006) On adaptive sampling for single and multi-response bayesian surrogate models. In: ASME 2006 International design engineering technical Conferences and Computers and information in engineering conference, Philadelphia. ASME, p 393–404

  • Romero DA, Amon CH, Finger S (2012) Multiresponse metamodeling in simulation-based design applications. J Mech Des 134(9):091001

    Article  Google Scholar 

  • Rosenbaum B (2013) Efficient global surrogate models for responses of expensive simulations. Universität Trier, Trier

    Google Scholar 

  • Rosenbaum B, Schulz V (2012) Comparing sampling strategies for aerodynamic kriging surrogate models. J Appl Math Mech 92(11–12):852–868

    MathSciNet  MATH  Google Scholar 

  • Rumpfkeil M, Yamazaki W, Dimitri M (2011) A dynamic sampling method for kriging and cokriging surrogate models. In: 49th AIAA Aerospace sciences meeting including the new horizons forum and Aerospace exposition, p 883

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423

    Article  MathSciNet  MATH  Google Scholar 

  • Saka Y, Gunzburger M, Burkardt J (2007) Latinized, improved LHS, and CVT point sets in hypercubes. Int J Numer Anal Model 4(3–4):729–743

    MathSciNet  MATH  Google Scholar 

  • Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S (2008) Global sensitivity analysis: the primer. Wiley, UK

    MATH  Google Scholar 

  • Sanchez SM, Wan H (2015) Work smarter, not harder: a tutorial on designing and conducting simulation experiments. In: Proceedings of the 2015 Winter Simulation conference. IEEE Press, p 1795–1809

  • Sasena MJ (2002) Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. University of Michigan, Ann Arbor

    Google Scholar 

  • Sasena M, Parkinson M, Goovaerts P, Papalambros P, Reed M (2002) Adaptive experimental design applied to ergonomics testing procedure. In: ASME 2002 International design engineering technical Conferences and Computers and information in engineering conference, Montreal. ASME, p 529–537

  • Settles B (2010) Active learning literature survey. Computer science technical report 1648, University of Wisconsin, Madison

  • Seung HS, Opper M, Sompolinsky H (1992) Query by committee. In: Proceedings of the fifth annual workshop on computational learning theory, Pittsburgh. ACM, pp 287–294

  • Shahsavani D, Grimvall A (2009) An adaptive design and interpolation technique for extracting highly nonlinear response surfaces from deterministic models. Reliab Eng Syst Saf 94(7):1173–1182

    Article  Google Scholar 

  • Shan S, Wang GG (2010a) Metamodeling for high dimensional simulation-based design problems. J Mech Des 132(5):051009

    Article  Google Scholar 

  • Shan S, Wang GG (2010b) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241

    Article  MathSciNet  MATH  Google Scholar 

  • Shan S, Wang GG (2011) Turning black-box functions into white functions. J Mech Des 133(3):031003

    Article  Google Scholar 

  • Shewry MC, Wynn HP (1987) Maximum entropy sampling. J Appl Stat 14(2):165–170

    Article  Google Scholar 

  • Singh P, Deschrijver D, Dhaene T (2013) A balanced sequential design strategy for global surrogate modeling. In: 2013 Winter simulations conference, Georgia. IEEE, pp 2172–2179

  • Sóbester A, Leary SJ, Keane AJ (2005) On the design of optimization strategies based on global response surface approximation models. J Glob Optim 33(1):31–59

    Article  MathSciNet  MATH  Google Scholar 

  • Sobol IM (1993) Sensitivity estimates for nonlinear mathematical models. Math Model Comput Exp 1(4):407–414

    MathSciNet  MATH  Google Scholar 

  • Sobol’ I (1979) On the systematic search in a hypercube. SIAM J Numer Anal 16(5):790–793

    Article  MathSciNet  MATH  Google Scholar 

  • Spyromitros-Xioufis E, Tsoumakas G, Groves W, Vlahavas I (2016) Multi-target regression via input space expansion: treating targets as inputs. Mach Learn 104(1):55–98

    Article  MathSciNet  Google Scholar 

  • Srinivas N, Krause A, Kakade SM, Seeger M (2010) Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of the 27th International conference on machine learning. ACM, pp 1015–1022

  • Steinberg D, Colla P (2009) CART: classification and regression trees. The Top Ten Algorithms in Data Mining 9:179

  • Stinstra E, den Hertog D, Stehouwer P, Vestjens A (2003) Constrained maximin designs for computer experiments. Technometrics 45(4):340–346

    Article  MathSciNet  Google Scholar 

  • Stocki R (2005) A method to improve design reliability using optimal Latin hypercube sampling. Comput Assist Mech Eng Sci 12(4):393

    Google Scholar 

  • Sundararajan S, Keerthi SS (2001) Predictive approaches for choosing hyperparameters in Gaussian processes. Neural Comput 13(5):1103–1118

    Article  MATH  Google Scholar 

  • Tian W (2013) A review of sensitivity analysis methods in building energy analysis. Renew Sust Energ Rev 20:411–419

    Article  Google Scholar 

  • Toal DJ (2015) Some considerations regarding the use of multi-fidelity kriging in the construction of surrogate models. Struct Multidiscip Optim 51(6):1223–1245

    Article  Google Scholar 

  • Turner CJ, Crawford RH, Campbell MI (2007) Multidimensional sequential sampling for NURBs-based metamodel development. Eng Comput 23(3):155–174

    Article  Google Scholar 

  • van Dam ER, Husslage B, den Hertog D, Melissen H (2007) Maximin Latin hypercube designs in two dimensions. Oper Res 55(1):158–169

    Article  MathSciNet  MATH  Google Scholar 

  • van Dam ER, Rennen G, Husslage B (2009) Bounds for maximin Latin hypercube designs. Oper Res 57(3):595–608

    Article  MathSciNet  MATH  Google Scholar 

  • van Dam ER, Husslage B, Den Hertog D (2010) One-dimensional nested maximin designs. J Glob Optim 46(2):287–306

    Article  MathSciNet  MATH  Google Scholar 

  • van der Herten J, Couckuyt I, Deschrijver D, Dhaene T (2015) A fuzzy hybrid sequential design strategy for global surrogate modeling of high-dimensional computer experiments. SIAM J Sci Comput 37(2):A1020–A1039

    Article  MathSciNet  MATH  Google Scholar 

  • Vasile M, Minisci E, Quagliarella D, Guénot M, Lepot I, Sainvitu C, Goblet J, Filomeno Coelho R (2013) Adaptive sampling strategies for non-intrusive POD-based surrogates. Eng Comput 30(4):521–547

    Article  Google Scholar 

  • Viana FA (2013) Things you wanted to know about the Latin hypercube design and were afraid to ask. In: 10th world Congress on Structural and multidisciplinary optimization, Orlando, pp 1–9

  • Viana FA, Haftka RT, Steffen V Jr (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidiscip Optim 39(4):439–457

    Article  Google Scholar 

  • Viana FA, Picheny V, Haftka RT (2010a) Using cross validation to design conservative surrogates. AIAA J 48(10):2286–2298

    Article  Google Scholar 

  • Viana FA, Venter G, Balabanov V (2010b) An algorithm for fast optimal Latin hypercube design of experiments. Int J Numer Methods Eng 82(2):135–156

    MathSciNet  MATH  Google Scholar 

  • Viana FA, Simpson TW, Balabanov V, Toropov V (2014) Metamodeling in multidisciplinary design optimization: how far have we really come? AIAA J 52(4):670–690

    Article  Google Scholar 

  • Villemonteix J, Vazquez E, Walter E (2009) An informational approach to the global optimization of expensive-to-evaluate functions. J Glob Optim 44(4):509

    Article  MathSciNet  MATH  Google Scholar 

  • Wang GG (2003) Adaptive response surface method using inherited latin hypercube design points. J Mech Des Trans ASME 125(2):210–220

    Article  Google Scholar 

  • Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4):370–380

    Article  Google Scholar 

  • Wang L, Shan S, Wang GG (2004) Mode-pursuing sampling method for global optimization on expensive black-box functions. Eng Optim 36(4):419–438

    Article  Google Scholar 

  • Wang H, Wu Z, Rahnamayan S (2011) Enhanced opposition-based differential evolution for solving high-dimensional continuous optimization problems. Soft Comput 15(11):2127–2140

    Article  Google Scholar 

  • Wei X, Wu Y-Z, Chen L-P (2012) A new sequential optimal sampling method for radial basis functions. Appl Math Comput 218(19):9635–9646

    MathSciNet  MATH  Google Scholar 

  • Williams BJ, Loeppky JL, Moore LM, Macklem MS (2011) Batch sequential design to achieve predictive maturity with calibrated computer models. Reliab Eng Syst Saf 96(9):1208–1219

    Article  Google Scholar 

  • Xiong Y, Chen W, Apley D, Ding X (2007) A non-stationary covariance-based kriging method for metamodelling in engineering design. Int J Numer Methods Eng 71(6):733–756

    Article  MATH  Google Scholar 

  • Xiong F, Xiong Y, Chen W, Yang S (2009) Optimizing Latin hypercube design for sequential sampling of computer experiments. Eng Optim 41(8):793–810

    Article  Google Scholar 

  • Xu S, Liu H, Wang X, Jiang X (2014) A robust error-pursuing sequential sampling approach for global metamodeling based on voronoi diagram and cross validation. J Mech Des 136(7):071009

    Article  Google Scholar 

  • Yang Z, Tang K, Yao X (2007) Differential evolution for high-dimensional function optimization. In: 2007 I.E. Congress on evolutionary computation, Singapore. IEEE, pp 3523–3530

  • Yao W, Chen X, Luo W (2009) A gradient-based sequential radial basis function neural network modeling method. Neural Comput & Applic 18(5):477–484

    Article  Google Scholar 

  • Younis A, Dong Z (2010) Metamodelling and search using space exploration and unimodal region elimination for design optimization. Eng Optim 42(6):517–533

    Article  Google Scholar 

  • Zhang Y, Hoang TN, Low KH, Kankanhalli M (2016) Near-optimal active learning of multi-output Gaussian processes. In: Proceedings of the 30th AAAI conference on artificial intelligence. AAAI Press, pp 2351–2357

  • Zhao L, Choi K, Lee I, Gorsich D (2009) Sequential-sampling-based kriging method with dynamic basis selection. In: 8th world Congress on Structural and multidisciplinary optimization, Lisbon

  • Zhou Q, Shao X, Jiang P, Gao Z, Zhou H, Shu L (2016) An active learning variable-fidelity metamodelling approach based on ensemble of metamodels and objective-oriented sequential sampling. J Eng Des 27(4–6):205–231

    Article  Google Scholar 

  • Zhou Q, Jiang P, Shao X, Hu J, Cao L, Wan L (2017) A variable fidelity information fusion method based on radial basis function. Adv Eng Inform 32:26–39

    Article  Google Scholar 

Download references

Acknowledgements

This work was conducted within the Rolls-Royce@NTU Corporate Lab with support from the National Research Foundation (NRF) Singapore under the Corp Lab@University Scheme. It is also partially supported by the Data Science and Artificial Intelligence Research Center (DSAIR) and the School of Computer Science and Engineering at Nanyang Technological University.

Author information

Authors and Affiliations

  1. Rolls-Royce@NTU Corporate Lab, Singapore, Singapore

    Haitao Liu

  2. Data Science and Artificial Intelligence Research Center, Nanyang Technological University, Singapore, Singapore

    Yew-Soon Ong

  3. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Jianfei Cai

Authors
  1. Haitao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yew-Soon Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianfei Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haitao Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, H., Ong, YS. & Cai, J. A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design. Struct Multidisc Optim 57, 393–416 (2018). https://doi.org/10.1007/s00158-017-1739-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-017-1739-8

Keywords

Search

Navigation