Skip to main content
SpringerLink
Log in

Global sensitivity analysis of stochastic computer models with joint metamodels

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

The global sensitivity analysis method used to quantify the influence of uncertain input variables on the variability in numerical model responses has already been applied to deterministic computer codes; deterministic means here that the same set of input variables always gives the same output value. This paper proposes a global sensitivity analysis methodology for stochastic computer codes, for which the result of each code run is itself random. The framework of the joint modeling of the mean and dispersion of heteroscedastic data is used. To deal with the complexity of computer experiment outputs, nonparametric joint models are discussed and a new Gaussian process-based joint model is proposed. The relevance of these models is analyzed based upon two case studies. Results show that the joint modeling approach yields accurate sensitivity index estimators even when heteroscedasticity is strong.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ankenman, B., Nelson, B., Staum, J.: Stochastic kriging for simulation metamodeling. Oper. Res. 58, 371–382 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Bayarii, M.J., Berger, J., Cafeo, J., Garcia-Donato, G., Liu, F., Palomo, J., Parthasarathy, R.J., Paulo, R., Sacks, J., Walsh, D.: Computer model validation with functional output. Ann. Stat. 35, 1874–1906 (2007a)

    Article  Google Scholar 

  • Bayarii, M.J., Berger, J., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C., Tu, J.: A framework for validation of computer models. Technometrics 49, 138–154 (2007b)

    Article  MathSciNet  Google Scholar 

  • Boukouvalas, A., Cornford, D.: Learning heteroscedastic Gaussian processes for complex datasets. Technical Report, Neural Computing Research Group, Aston University, Birmingham, UK (2009)

  • Bursztyn, D., Steinberg, D.: Screening experiments for dispersion effects. In: Dean, A., Lewis, S. (eds.) Screening—Methods for Experimentation in Industry, Drug Discovery and Genetics. Springer, Berlin (2006)

    Google Scholar 

  • Chen, V., Tsui, K.-L., Barton, R., Meckesheimer, M.: A review on design, modeling and applications of computer experiments. IIE Trans. 38, 273–291 (2006)

    Article  Google Scholar 

  • Chilès, J.-P., Delfiner, P.: Geostatistics: Modeling Spatial Uncertainty. Wiley, New York (1999)

    Book  MATH  Google Scholar 

  • Dellino, G., Kleijnen, J.P.C., Meloni, C.: Robust optimization in simulation: Taguchi and response surface methodology. Int. J. Prod. Econ. 125, 52–59 (2010)

    Article  Google Scholar 

  • De Rocquigny, E., Devictor, N., Tarantola, S. (eds.): Uncertainty in Industrial Practice. Wiley, New York (2008)

    Google Scholar 

  • Fang, K.-T., Li, R., Sudjianto, A.: Design and Modeling for Computer Experiments. Chapman & Hall/CRC Press, London/Boca Raton (2006)

    MATH  Google Scholar 

  • Forrester, A.I.J., Keane, A.J., Bressloff, N.W.: Design and analysis of “Noisy” computer experiments. AIAA J. 44, 2331–2339 (2006)

    Article  Google Scholar 

  • Gijbels, I., Prosdocimi, I., Claeskens, G.: Nonparametric estimation of mean and dispersion functions in extended generalized linear models. Test 19, 580–608 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Ginsbourger, D., Roustant, O., Richet, Y.: Kriging with heterogeneous nugget effect for the approximation of noisy simulators with tunable fidelity. In: Proceedings of Joint Meeting of the Statistical Society of Canada and the Société Française de Statistique, Ottawa, Canada (2008)

    Google Scholar 

  • Gramacy, R.B., Lee, H.K.H.: Bayesian treed Gaussian process models with an application to computer modeling. J. Am. Stat. Assoc. 103, 1119–1130 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman and Hall, London (1990)

    MATH  Google Scholar 

  • Helton, J.C.: Conceptual and computational basis for the quantification of margins and uncertainty. Sandia National Laboratories, Report SAND2009-3055 (2009)

  • Helton, J.C., Johnson, J., Salaberry, C., Storlie, C.: Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab. Eng. Syst. Saf. 91, 1175–1209 (2006)

    Article  Google Scholar 

  • Homma, T., Saltelli, A.: Importance measures in global sensitivity analysis of non linear models. Reliab. Eng. Syst. Saf. 52, 1–17 (1996)

    Article  Google Scholar 

  • Iooss, B., Ribatet, M.: Global sensitivity analysis of computer models with functional inputs. Reliab. Eng. Syst. Saf. 94, 1194–1204 (2009)

    Article  Google Scholar 

  • Iooss, B., Lhuillier, C., Jeanneau, H.: Numerical simulation of transit-time ultrasonic flowmeters due to flow profile and fluid turbulence. Ultrasonics 40, 1009–1015 (2002)

    Article  Google Scholar 

  • Kelton, W.D., Sadowski, R.P., Sturrock, D.T.: Simulation with Arena, 4th edn. McGraw-Hill, Boston (2007)

    Google Scholar 

  • Kennedy, M., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. 63(3), 425–464 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Kersting, K., Plagemann, C., Pfaff, P., Burgard, W.: Most likely heteroscedastic Gaussian process regression. In: Proceedings of the 24th International Conference on Machine Learning, Corvallis, Oregon, USA (2007)

    Google Scholar 

  • Kleijnen, J.P.C.: Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57, 111–142 (1997)

    Article  MATH  Google Scholar 

  • Kleijnen, J.P.C.: Design and Analysis of Simulation Experiments. Springer, Berlin (2008)

    MATH  Google Scholar 

  • Kleijnen, J.P.C., van Beers, W.: Robustness of kriging when interpolating in random simulation with heterogeneous variances: some experiments. Eur. J. Oper. Res. 165, 826–834 (2005)

    Article  MATH  Google Scholar 

  • Lee, Y., Nelder, J.: Robust design via generalized linear models. J. Qual. Technol. 35(1), 2–12 (2003)

    Google Scholar 

  • Manceau, E., Mezghani, M., Zabalza-Mezghani, I., Roggero, F.: Combination of experimental design and joint modeling methods for quantifying the risk associated with deterministic and stochastic uncertainties—an integrated test study. In: 2001 SPE Annual Technical Conference and Exhibition, New Orleans, 30 September–3 October (2001). paper SPE 71620

    Google Scholar 

  • Marrel, A., Iooss, B., Van Dorpe, F., Volkova, E.: An efficient methodology for modeling complex computer codes with Gaussian processes. Comput. Stat. Data Anal. 52, 4731–4744 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Martin, J., Simpson, T.: Use of kriging models to approximate deterministic computer models. AIAA J. 43, 853–863 (2005)

    Article  Google Scholar 

  • McCullagh, P., Nelder, J.: Generalized Linear Models. Chapman & Hall, London (1989)

    MATH  Google Scholar 

  • Myers, R.H., Montgomery, D.C., Anderson-Cook, C.M.: Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 3rd edn. Wiley, New York (2009)

    MATH  Google Scholar 

  • Nelder, J., Pregibon, D.: An extended quasi-likelihood function. Biometrika 74, 221–232 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  • Nelder, J., Wedderburn, R.: Generalized linear models. J. R. Stat. Soc. A 135, 370–384 (1972)

    Article  Google Scholar 

  • Phadke, M.: Quality Engineering Using Robust Design. Prentice-Hall, New York (1989)

    Google Scholar 

  • Picheny, V., Ginsbourger, D., Richet, Y., Caplin, G.: Optimization of noisy computer experiments with tunable precision. Technometrics (2011, accepted, in revision). http://hal.archives-ouvertes.fr/hal-00578550_v1/

  • Pope, B.: Lagrangian pdf methods for turbulent reactive flows. Annu. Rev. Fluid Mech. 26, 23–63 (1994)

    Article  MathSciNet  Google Scholar 

  • Reich, B.J., Kalendra, E., Storlie, C.B., Bondell, H.D., Fuentes, M.: Variable selection for high dimensional Bayesian density estimation: application to human exposure simulation. J. R. Stat. Soc., Ser. C (2011). doi:10.1111/j.1467-9876.2011.01020.x

    Google Scholar 

  • Robinson, T.J., Birch, J., Alden Starnes, B.: A semi-parametric approach to dual modeling when no replication exists. J. Stat. Plan. Inference 140, 2860–2869 (2010)

    Article  MATH  Google Scholar 

  • Ruffo, P., Bazzana, L., Consonni, A., Corradi, A., Saltelli, A., Tarantola, S.: Hydrocarbon exploration risk evaluation through uncertainty and sensitivity analysis techniques. Reliab. Eng. Syst. Saf. 91, 1155–1162 (2006)

    Article  Google Scholar 

  • Sacks, J., Welch, W., Mitchell, T., Wynn, H.: Design and analysis of computer experiments. Stat. Sci. 4, 409–435 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Saltelli, A., Chan, K., Scott, E. (eds.): Sensitivity Analysis. Wiley Series in Probability and Statistics. Wiley, New York (2000)

    MATH  Google Scholar 

  • Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., Tarantola, S.: Variance based sensitivity analysis of model output. Design and estimator for total sensitivity index. Comput. Phys. Commun. 181, 259–270 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Siebers, P.O., Macal, C.M., Garnett, J., Buxton, D., Pidd, M.: Discrete-event simulation is dead, long live agent-based simulation! J. Simul. 4, 204–210 (2010)

    Article  Google Scholar 

  • Smyth, G.: Generalized linear models with varying dispersion. J. R. Stat. Soc. B 51, 47–60 (1989)

    MathSciNet  Google Scholar 

  • Sobol, I.: Sensitivity estimates for non linear mathematical models. In: Mathematical Modelling and Computational Experiments, vol. 1, pp. 407–414 (1993)

    Google Scholar 

  • Storlie, C.B., Swiler, L.P., Helton, J.C., Salaberry, C.J.: Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab. Eng. Syst. Saf. 94, 1735–1763 (2009)

    Article  Google Scholar 

  • Vining, G., Myers, R.: Combining Taguchi and response-surface philosophies—a dual response approach. J. Qual. Technol. 22, 38–45 (1990)

    Google Scholar 

  • Volkova, E., Iooss, B., Van Dorpe, F.: Global sensitivity analysis for a numerical model of radionuclide migration from the RRC “Kurchatov Institute” radwaste disposal site. Stoch. Environ. Res. Risk Assess. 22, 17–31 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Wood, S., Augustin, N.: GAMs with integrated model selection using penalized regression splines and applications to environmental modelling. Ecol. Model. 157, 157–177 (2002)

    Article  Google Scholar 

  • Yeşilyurt, S., Ghaddar, C.K., Cruz, M.E., Patera, A.T.: Bayesian-validated surrogates for noisy computer simulations; application to random media. SIAM J. Sci. Comput. 17, 973–992 (1996)

    Article  MathSciNet  Google Scholar 

  • Zabalza, I., Dejean, J., Collombier, D.: Prediction and density estimation of a horizontal well productivity index using generalized linear models. In: Proceedings of ECMOR VI, Peebles, Scotland, 8–11 September 1998

    Google Scholar 

  • Zabalza, I., Manceau, E., Roggero, F.: A new approach for quantifying the impact of geostatistical uncertainty on production forecasts: The joint modeling method. In: Proceedings of IAMG Conference, Cancun, Mexico, 6–12 September 2001

    Google Scholar 

  • Zabalza-Mezghani, I., Manceau, E., Feraille, M., Jourdan, A.: Uncertainty management: from geological scenarios to production scheme optimization. J. Pet. Sci. Eng. 44, 11–25 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IFP Energies Nouvelles, 1 & 4 avenue de Bois Préau, 92852, Rueil-Malmaison, France

    Amandine Marrel & Sébastien Da Veiga

  2. EDF, R&D, 6 Quai Watier, 78401, Chatou, France

    Bertrand Iooss

  3. Université Montpellier II, Place Bataillon, 34095, Montpellier, France

    Mathieu Ribatet

Authors
  1. Amandine Marrel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bertrand Iooss
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sébastien Da Veiga
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mathieu Ribatet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amandine Marrel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marrel, A., Iooss, B., Da Veiga, S. et al. Global sensitivity analysis of stochastic computer models with joint metamodels. Stat Comput 22, 833–847 (2012). https://doi.org/10.1007/s11222-011-9274-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-011-9274-8

Keywords

Search

Navigation