Skip to main content
SpringerLink
Log in

Multi-information source constrained Bayesian optimization

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Design decisions for complex systems often can be made or informed by a variety of information sources. When optimizing such a system, the evaluation of a quantity of interest is typically required at many different input configurations. For systems with expensive to evaluate available information sources, the optimization task can potentially be computationally prohibitive using traditional techniques. This paper presents an information-economic approach to the constrained optimization of a system with multiple available information sources. The approach rigorously quantifies the correlation between the discrepancies of different information sources, which enables the overcoming of information source bias. All information is exploited efficiently by fusing newly acquired information with that previously evaluated. Independent decision-makings are achieved by developing a two-step look-ahead utility policy and an information gain policy for objective function and constraints respectively. The approach is demonstrated on a one-dimensional example test problem and an aerodynamic design problem, where it is shown to perform well in comparison to traditional multi-information source techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Alexandrov N, Lewis R, Gumbert C, Green L, Newman P (2000) Optimization with variable-fidelity models applied to wing design. In: 38th aerospace sciences meeting and exhibit, pp 841

  • Alexandrov NM, Dennis JE Jr, Lewis RM, Torczon V (1998) A trust-region framework for managing the use of approximation models in optimization. Struct Optim 15(1):16–23

    Article  Google Scholar 

  • Alexandrov NM, Lewis RM, Gumbert CR, Green LL, Newman PA (2001) Approximation and model management in aerodynamic optimization with variable-fidelity models. J Aircr 38(6):1093–1101

    Article  Google Scholar 

  • Allaire D, Willcox K (2012) Fusing information from multifidelity computer models of physical systems. In: 2012 15th international conference on information fusion (FUSION), pp 2458–2465. IEEE

  • Barrett R, Ning A (2016) Comparison of airfoil precomputational analysis methods for optimization of wind turbine blades. IEEE Trans Sustainable Energy 7(3):1081–1088

    Article  Google Scholar 

  • Booker AJ, Dennis JE, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Optim 17(1):1–13

    Article  Google Scholar 

  • Chaudhuri A, Jasa J, Martins J, Willcox KE (2018) Multifidelity optimization under uncertainty for a tailless aircraft. In: 2018 AIAA non-deterministic approaches conference, pp 1658

  • Draper D (1995) Assessment and propagation of model uncertainty. J R Stat Soc Ser B Methodol 57:45–97

    MathSciNet  MATH  Google Scholar 

  • Drela M (1989) Xfoil: An analysis and design system for low reynolds number airfoils. In: Low reynolds number aerodynamics, pp 1–12. Springer

  • Frazier P, Powell W, Dayanik S (2009) The knowledge-gradient policy for correlated normal beliefs. INFORMS J Comput 21(4):599–613

    Article  MathSciNet  MATH  Google Scholar 

  • Frazier PI, Powell WB, Dayanik S (2008) A knowledge-gradient policy for sequential information collection. SIAM J Control Optim 47(5):2410–2439

    Article  MathSciNet  MATH  Google Scholar 

  • Geisser S (1965) A bayes approach for combining correlated estimates. J Am Stat Assoc 60:602–607

    Article  MathSciNet  MATH  Google Scholar 

  • Ghoreishi SF (2016) Uncertainty analysis for coupled multidisciplinary systems using sequential importance resampling. Texas A&M University, Master’s thesis

    Google Scholar 

  • Ghoreishi SF, Allaire DL (2016) Compositional uncertainty analysis via importance weighted gibbs sampling for coupled multidisciplinary systems. In: 18th AIAA non-deterministic approaches conference, pp 1443

  • Ghoreishi S F, Allaire D L (2017) Adaptive uncertainty propagation for coupled multidisciplinary systems. AIAA J 55:3940–3950

    Article  Google Scholar 

  • Ghoreishi SF, Allaire D (2018) Gaussian process regression for Bayesian fusion of multi-fidelity information sources. In: 19th AIAA/ISSMO multidisciplinary analysis and optimization conference

  • Ghoreishi SF, Allaire DL (2018) A fusion-based multi-information source optimization approach using knowledge gradient policies. In: 2018 AIAA/ASCE/AHS/ASC structures, Structural dynamics, and materials conference, pp 1159

  • Ghoreishi SF, Molkeri A, Srivastava A, Arroyave R, Allaire D (2018) Multi-information source fusion and optimization to realize icme: Application to dual-phase materials. J Mech Des 140(11):111409

    Article  Google Scholar 

  • Hoeting JA, Madigan D, Raftery AE, Volinsky CT (1999) Bayesian model averaging: a tutorial. Stat Sci 14:382–401

    Article  MathSciNet  MATH  Google Scholar 

  • Huang D, Allen T T, Notz W I, Miller R A (2006) Sequential kriging optimization using multiple-fidelity evaluations. Struct Multidiscip Optim 32(5):369–382

    Article  Google Scholar 

  • Imani M., Ghoreishi S.F., Braga-Neto UM (2018) Bayesian control of large mdps with unknown dynamics in data-poor environments. In Advances in neural information processing systems

  • JJ Thibert M, Grandjacques L H, et al. (1979) Ohman Naca 0012 airfoil. AGARD Advisory Report, pp 138

  • Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21 (4):345–383

    Article  MathSciNet  MATH  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

    Article  MathSciNet  MATH  Google Scholar 

  • Julier S, Uhlmann J (2001) General decentralized data fusion with covariance intersection. In: Hall D, Llinas J (eds) Handbook of data fusion. CRC Press, Boca Raton

  • Julier S, Uhlmann J (1997) A non-divergent estimation algorithm in the presence of unknown correlations. In: Proceedings of the american control conference, pp 2369–2373

  • Kandasamy K, Dasarathy G, Schneider J, Poczos B (2017) Multi-fidelity Bayesian optimisation with continuous approximations. arXiv:1703.06240

  • Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86

    Article  MathSciNet  MATH  Google Scholar 

  • Lam R, Allaire DL, Willcox KE (2015) Multifidelity optimization using statistical surrogate modeling for non-hierarchical information sources. In: 56th AIAA/ASCE/AHS/ASC structures, Structural dynamics, and materials conference, pp 0143

  • Leamer EE (1978) Specification searches: ad hoc inference with nonexperimental data, vol 53. Wiley, New York

    MATH  Google Scholar 

  • Madigan D, Raftery AE (1994) Model selection and accounting for model uncertainty in graphical models using occam’s window. J Am Stat Assoc 89(428):1535–1546

    Article  MATH  Google Scholar 

  • March A, Willcox K (2012) Provably convergent multifidelity optimization algorithm not requiring high-fidelity derivatives. AIAA J 50(5):1079–1089

    Article  Google Scholar 

  • Morris P (1977) Combining expert judgments: a Bayesian approach. Manag Sci 23:679–693

    Article  MATH  Google Scholar 

  • Mosleh A, Apostolakis G (1986) The assessment of probability distributions from expert opinions with an application to seismic fragility curves. Risk Anal 6(4):447–461

    Article  Google Scholar 

  • Palacios F, Alonso J, Duraisamy K, Colonno M, Hicken J, Aranake A, Campos A, Copeland S, Economon T, Lonkar A et al (2013) Stanford university unstructured (su 2): an open-source integrated computational environment for multi-physics simulation and design. In: 51st AIAA aerospace sciences meeting including the new horizons forum and aerospace exposition, pp 287

  • Peherstorfer B, Willcox K, Gunzburger M (2016) Survey of multifidelity methods in uncertainty propagation, inference, and optimization. Preprint, pp 1–57

  • Poloczek M, Wang J, Frazier P (2017) Multi-information source optimization. In: Advances in Neural Information Processing Systems, pp 4291–4301

  • Powell WB, Ryzhov IO (2012) Optimal learning, vol 841. Wiley, New York

    Book  Google Scholar 

  • Rasmussen CE, Williams C (2006) Gaussian processes for machine learning. MIT Press, Cambridge

    MATH  Google Scholar 

  • Reinert JM, Apostolakis GE (2006) Including model uncertainty in risk-informed decision making. Ann Nucl Energy 33(4):354–369

    Article  Google Scholar 

  • Riley ME, Grandhi RV, Kolonay R (2011) Quantification of modeling uncertainty in aeroelastic analyses. J Aircr 48(3):866–873

    Article  Google Scholar 

  • Rumsey C (2014) 2d naca 0012 airfoil validation case. Turbulence modeling resource, NASA Langley Research Center, pp 33

  • Sasena MJ, Papalambros P, Goovaerts P (2002) Exploration of metamodeling sampling criteria for constrained global optimization. Eng Optim 34(3):263–278

    Article  Google Scholar 

  • Scott W, Frazier P, Powell W (2011) The correlated knowledge gradient for simulation optimization of continuous parameters using gaussian process regression. SIAM J Optim 21(3):996– 1026

    Article  MathSciNet  MATH  Google Scholar 

  • Thomison WD, Allaire DL (2017) A model reification approach to fusing information from multifidelity information sources. In: 19th AIAA non-deterministic approaches conference, pp 1949

  • Wang J (2017) Bayesian optimization with parallel function evaluations and multiple information sources: methodology with applications in biochemistry, aerospace engineering, and machine learning. Cornell University

  • Winkler R (1981) Combining probability distributions from dependent information sources. Manag Sci 27(4):479–488

    Article  MATH  Google Scholar 

  • Winkler RL (1981) Combining probability distributions from dependent information sources. Manag Sci 27 (4):479–488

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported by the AFOSR MURI on multi-information sources of multi-physics systems under Award Number FA9550-15-1-0038, program manager, Dr. Fariba Fahroo and by the National Science Foundation under grant no. CMMI-1663130. Opinions expressed in this paper are of the authors and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Texas A&M University, College Station, TX, 77843, USA

    Seyede Fatemeh Ghoreishi & Douglas Allaire

Authors
  1. Seyede Fatemeh Ghoreishi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Douglas Allaire
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyede Fatemeh Ghoreishi.

Additional information

Responsible Editor: Felipe A. C. Viana

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghoreishi, S.F., Allaire, D. Multi-information source constrained Bayesian optimization. Struct Multidisc Optim 59, 977–991 (2019). https://doi.org/10.1007/s00158-018-2115-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-018-2115-z

Keywords

Search

Navigation