Skip to main content
Log in

Error propagation in computer models: analytic approaches, advantages, disadvantages and constraints

  • Original Paper
  • Published:
Stochastic Environmental Research and Risk Assessment Aims and scope Submit manuscript

Abstract

Uncertainty and its propagation in computer models has relevance in many disciplines, including hydrology, environmental engineering, ecology and climate change. Error propagation in a model results in uncertainty in prediction due to uncertainties in model inputs and parameters. Common methods for quantifying error propagation are reviewed, namely Differential Error Analysis and Monte Carlo Simulation, including underlying principles, together with a discussion on their differences, advantages and disadvantages. The separate case of uncertainty in the model calibration process is different to error propagation in a fixed model in that it is associated with a dynamic process of iterative parameter adjustment, and is compared in the context of non-linear regression and Bayesian approaches, such as Markov Chain Monte Carlo Simulation. Error propagation is investigated for a soil model representing the organic carbon depth profile and also a streamflow model using probabilistic simulation. Different sources of error are compared, including uncertainty in inputs, parameters and geometry. The results provided insights into error propagation and its computation in systems and models in general.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abusam A, Keesman KJ, Van Straten G (2003) Forward and backward uncertainty propagation: an oxidation ditch modelling example. Water Res 37(2):429–435

    Article  CAS  Google Scholar 

  • Bayes T (1763) An essay towards solving a problem in the doctrine of chances. Philos Trans R Soc Lond 53:1393–1442

    Google Scholar 

  • Benke KK, Benke KE (2013) Uncertainty in health risks from artificial lighting due to disruption of circadian rhythm and melatonin secretion: a review. Hum Ecol Risk Assess 19:916–929

    Article  CAS  Google Scholar 

  • Benke KK, Robinson NJ (2017) Quantification of uncertainty in mathematical models: the statistical relationship between field and laboratory pH measurements. Appl Environ Soil Sci 20:12. https://doi.org/10.1155/2017/5857139

    Article  Google Scholar 

  • Benke KK, Lowell KE, Hamilton AJ (2008) Parameter uncertainty, sensitivity analysis and prediction error in a water-balance hydrological model. Math Comput Model 47:1134–1149

    Article  Google Scholar 

  • Beven K (2008) Comment on ‘‘Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling? Vrugt J.A., ter Braak, C.J.F., Gupta, H.V., Robinson, B.A., 2008. Stoch Environ Res Risk Assess. https://doi.org/10.1007/s00477-008-0283-x

    Article  Google Scholar 

  • Beven KJ, Binley AM (1992) The future of distributed models: model calibration and predictive uncertainty. Hydrol Process 6:279–298

    Article  Google Scholar 

  • Beverly C, Christy B, Weeks A (2006) Application of the 2CSalt model to the Bet Bet, Wild Duck, Gardner and Sugarloaf Catchments in Victoria. Department of Primary Industries, Victoria

    Google Scholar 

  • Bolstad WM (2007) Introduction to Bayesian statistics, 2nd edn. Wiley, Hoboken

    Book  Google Scholar 

  • Box GEP, Cox DR (1982) An analysis of transformations revisited, rebutted. J Am. Stat Assoc 77:209–210

    Article  Google Scholar 

  • Brown JD, Heuvelink GBM (2005) 79: assessing uncertainty propagation through physically based models of soil water flow and solute transport. In: Anderson MG (ed) Encyclopaedia of hydrological sciences. Wiley, New York

    Google Scholar 

  • Buckland ST (1984) Monte Carlo confidence intervals. Biometrics 40:811–817

    Article  Google Scholar 

  • Carlin BP, Louis TA (2008) Bayesian methods for data analysis, 3rd edn. Chapman and Hall, Boca Raton

    Google Scholar 

  • Chen C-S, Chen C-S (2018) A composite spatial predictor via local criteria under a misspecified model. Stoch Environ Res Risk Assess 32:341–355

    Article  Google Scholar 

  • Doherty J (2003) MICA: model independent Markov Chain Monte Carlo analysis. Watermark Numerical Computing, Brisbane

    Google Scholar 

  • Donnelly SM, Kramer A (1999) Testing for multiple species in forest samples: an evaluation and comparison of tests for equal relative variation. Am J Phys Anthopol 108:507–529

    Article  CAS  Google Scholar 

  • Dotto CBS, Mannina G, Kleidorfer M, Vezzaro L, Henrichs M, McCarthy DT, Freni G, Rauch W, Deletic A (2012) Comparison of different uncertainty techniques in urban stormwater quantity and quality modelling. Water Res 46(8):2545–2558

    Article  CAS  Google Scholar 

  • Freeze RA (2004) The role of stochastic hydrogeological modeling in real-world engineering applications. Stoch Env Res Risk Assess 18(4):286–289

    Article  Google Scholar 

  • Freni G, Mannina G (2009) Urban runoff modelling uncertainty: comparison among Bayesian and pseudo-Bayesian methods. Environ Model Softw 24:1100–1111

    Article  Google Scholar 

  • Freni G, Mannina G (2010) Bayesian approach for uncertainty quantification in water quality modelling: the influence of prior distribution. J Hydrol 392:31–39

    Article  CAS  Google Scholar 

  • Freund JE (1998) Mathematical statistics. Prentice-Hall, New York

    Google Scholar 

  • Garg A, Vijayaraghavan V, Zhang J, Li S, Liang X (2017a) Design of robust battery capacity model for electric vehicle by incorporation of uncertainties. Int J Energy Res 41(10):1436–1451

    Article  Google Scholar 

  • Garg A, Vijayaraghavan V, Zhang J, Lam JSL (2017b) Robust model design for evaluation of power characteristics of the cleaner energy system. Renew Energy 112:302–313

    Article  CAS  Google Scholar 

  • Gelman A, Carlin JB, Stern HS, Rubin DB (2004) Bayesian data analysis. Chapman and Hall/CRC, Boca Raton

    Google Scholar 

  • Geman S, Geman D (1984) Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6:721–741

    Article  CAS  Google Scholar 

  • Goidts E, van Wesmael B, Crucifix M (2009) Magnitude and sources of uncertainties in soil organic carbon (SOC) stock assessments at various scales. Eur J Soil Sci 60:723–729

    Article  CAS  Google Scholar 

  • Gupta HV, Clark MP, Vrugt JA, Abramowitz G, Ye M (2012) Towards a comprehensive assessment of model structural adequacy. Water Resour Res 48:W08301. https://doi.org/10.1029/2011WR011044

    Article  Google Scholar 

  • Haas CN, Eisenberg JNS (2001) Risk assessment. In: Fewtrell L, Bartram J (eds) Water quality: guidelines, standards and health. Assessment of risk and risk management for water-related infectious disease. IWA Publishing, London, pp 161–183

    Google Scholar 

  • Hamilton AJ, Basset Y, Benke KK, Grimbacher PS, Miller SE, Novotný V, Samuelson GA, Stork NE, Weiblen GD, Yen JD (2010) Quantifying uncertainty in estimation of tropical arthropod species richness. Am Nat 176:90–95

    Article  Google Scholar 

  • Hamilton AJ, Novotny V, Waters EK, Basset Y, Benke KK, Grimbacher PS, Miller SE, Samuelson GA, Weiblen GD, Yen JD, Stork NE (2013) Estimating global arthropod species richness: refining probabilistic models using probability bounds analysis. Oecologia 171:357–365. https://doi.org/10.1007/s00442-012-2434-5

    Article  Google Scholar 

  • Helton JC (1993) Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliab Eng Syst Saf 42:327–367

    Article  Google Scholar 

  • Heuvelink GBM, Burrough PA (2002) Developments in statistical approaches to spatial uncertainty and its propagation. Int J Geogr Inf Sci 16:111–113

    Article  Google Scholar 

  • Heuvelink GB, Burrough PA, Stein A (1989) Propagation of errors in spatial modelling with GIS. Int J Geogr Inf Syst 3(4):303–322

    Article  Google Scholar 

  • Kavetski D, Kuczera G, Franks SW (2006a) Bayesian analysis of input uncertainty in hydrological modelling: 1. Theory. Water Resour J 42(W03407):1–9

    Google Scholar 

  • Kavetski D, Kuczera G, Franks SW (2006b) Calibration of conceptual hydrological models revisited: 1. Overcoming numerical artefacts. J Hydrol 32(1–2):173–186

    Article  Google Scholar 

  • Kavetski D, Kuczera G, Franks SW (2006c) Calibration of conceptual hydrological models revisited: 2. Improving optimisation and analysis. J Hydrol 32(1–2):187–201

    Article  Google Scholar 

  • Kline SJ, McClintock FA (1953) Describing uncertainties in single sample experiments. Mech Eng 75:3–8

    Google Scholar 

  • Kroese DP, Taimre T, Botev ZI (2011) Handbook of Monte Carlo methods. Wiley, New York

    Book  Google Scholar 

  • Kuczera G, Parent E (1998) Monte Carlo assessment of parameter uncertainty in conceptual models: the metropolis algorithm. J Hydrol 211:69–85

    Article  Google Scholar 

  • Lagos-Álvarez BM, Toribio RF, Figueroa-Zunuga J, Mateu J (2017) Geostastistical mixed beta regression: a Bayesian approach. Stoch Environ Res Risk Assess 31:571–584

    Article  Google Scholar 

  • Lark RM, Webster R (2006) Geostastistical mapping of geomorphic surfaces in the presence of trend. Earth Surf Process Landf 31:862–874

    Article  Google Scholar 

  • Littleboy M (2005) UserGuide_2CSalt, Rev 1.0, CRC for Catchment Hydrology, Australia. www.toolkit.net.au

  • Malone BP, McBratney AB, Minasny B (2011) Empirical estimates of uncertainty for mapping continuous depth functions of soil attributes. Geoderma 160:614–625

    Article  Google Scholar 

  • Mandel J (1964) The statistical analysis of experimental data. Dover Publications Inc., New York

    Google Scholar 

  • McKay MD (1995) Evaluating prediction uncertainty. Report No. LA-12915-MS, Statistics Group, Los Alamos National Laboratory, NM, USA

  • Minasny B, Vrugt JA, McBratney AB (2011) Confronting uncertainty in model-based geostatistics using Markov Chain Monte Carlo simulation. Geoderma 163:150–162

    Article  CAS  Google Scholar 

  • Nelson MA, Bishop TFA, Odeh IOA, Triantafilis J (2011) An error budget for different sources of error in digital soil mapping. Eur J Soil Sci 62:417–430

    Article  Google Scholar 

  • Oberkampf WL, Helton JC, Joslyn CA, Wojtkiewicz SF, Ferson S (2004) Challenge problems: uncertainty in system response given uncertain parameters. Reliab Eng Syst Saf 85:11–19

    Article  Google Scholar 

  • Oliver MA, Webster R (2014) A tutorial guide to geostatistics: computing and modelling variograms and kriging. Catena 113:56–69

    Article  Google Scholar 

  • Oritz JO, Felgueiras CA, Camargo ECG, Rennó CD, Oritz MJ (2017) Spatial modelling of soil lime requirements with uncertainty assessment using geostatistical sequential indicator simulation. Open J Soil Sci 7:133–148

    Article  Google Scholar 

  • Oya A, Navarro-Moreno J, Ruiz-Molina JC (2007) Spatial random field simulation by a numerical series representation. Stoch Env Res Risk Assess 21:317–326

    Article  Google Scholar 

  • Parratt LG (1971) Probability and experimental errors in science. Dover Publications Inc., New York

    Google Scholar 

  • Patil A, Deng ZQ, Malone RF (2011) Input data resolution-induced uncertainty in watershed modelling. Hydrol Process 25:2302–2312

    Article  Google Scholar 

  • Qian SS, Stow CA, Borsuk ME (2003) On Monte Carlo methods for Bayesian inference. Ecol Model 159:269–277

    Article  CAS  Google Scholar 

  • Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using Bayesian model averaging to calibrate forecast ensembles. Mon Weather Rev 133:1155–1174

    Article  Google Scholar 

  • Rayment GE, Lyons DJ (2011) Soil chemical methods: Australasia, vol 3. CSIRO publishing, Collingwood

    Google Scholar 

  • Refsgaard JC, van der Sluijs JP, Højberg AL, Vanrolleghem PA (2007) Uncertainty in the environmental modelling process—a framework and guidance. Environ Model Softw 22:1543–1556

    Article  Google Scholar 

  • Robinson NJ, Benke KK, Norng S (2015) Identification and interpretation of sources of uncertainty in soils change in a global systems-based modelling process. Soil Res 53(6):592–604

    Article  Google Scholar 

  • Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536

    Article  Google Scholar 

  • Savelyeva E, Utkin S, Kazakpv S, Demyanov V (2010) Modeling spatial uncertainty for locally uncertain data. Geoenv VII Geostat Environ Appl 16:295–306

    Google Scholar 

  • Sklar A (1959) Fonctions de répartition à n dimensions et leurs marges”. Publ Inst Stat Univ Paris 8:229–231

    Google Scholar 

  • Smith MS, Khaled MA (2012) Estimation of copula models with discrete margins via Bayesian data augmentation. J Am Stat Assoc 107:290–303

    Article  Google Scholar 

  • Stenson MP, Littleboy M, Gilfedder M (2011) Estimation of water and salt generation from unregulated upland catchments. Environ Model Softw. https://doi.org/10.1016/j.envsoft.2011.05.013

    Article  Google Scholar 

  • Taylor JR (1997) An introduction to error analysis. University Science Books, Sausalito

    Google Scholar 

  • Trucano T, Swiler L, Igusa T, Oberkampf W, Pilch W (2006) Calibration, validation, and sensitivity analysis: what’s what. Reliability engineering and system safety. In: 4th international conference on sensitivity. analysis of model output-SAMO 2004, vol 91, No. 10–11, pp 1331–1357

  • Vandenberghe V, Bauwens W, Vanrolleghem PA (2007) Evaluation of uncertainty propagation into river water quality predictions to guide future monitoring campaigns. Environ Model Softw 22:725–732

    Article  Google Scholar 

  • Vose D (2008) Risk analysis. Wiley, Chichester

    Google Scholar 

  • Vrugt JA, Gupta HV, Bouten W, Sorooshian S (2003) The shuffled complex evolutionary metropolis algorithm for optimisation and uncertainty assessment of hydrological parameters. Water Resour Res 39(8):1-1–1-9

    Google Scholar 

  • Vrugt JA, ter Braak CJF, Gupta HV, Robinson BA (2009a) Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling? Stoch Environ Res Risk Assess 23:1011–1026. https://doi.org/10.1007/s00477-008-0274-y

    Article  Google Scholar 

  • Vrugt JA, ter Braak CJF, Gupta HV, Robinson BA (2009b) Response to Keith Beven comment on “equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?”. Stoch Environ Res Risk Assess 23:1061–1062. https://doi.org/10.1007/s00477-008-0284-9

    Article  Google Scholar 

  • Wang D, Lu WZ (2006) Forecasting Ozone Levels and analyzing their dynamics by a Bayesian multilayer perceptron model for two air-monitoring sites in Hong Kong. Hum Ecol Risk Assess 12:313–327

    Article  Google Scholar 

  • Wöhling T, Vrugt JA (2008) Combining multi-objective optimization and Bayesian model averaging to calibrate forecast ensembles of soil hydraulic models. Water Resour Res 44:W12432. https://doi.org/10.1029/2008WR007154

    Article  Google Scholar 

  • Wu F, Chen C (2009) Bayesian updating of parameters for a sediment entrainment model via Markov Chain Monte Carlo. J Hydraul Eng 135:22–37

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to N. J. Robinson.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Benke, K.K., Norng, S., Robinson, N.J. et al. Error propagation in computer models: analytic approaches, advantages, disadvantages and constraints. Stoch Environ Res Risk Assess 32, 2971–2985 (2018). https://doi.org/10.1007/s00477-018-1555-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00477-018-1555-8

Keywords

Navigation