Abstract
Uncertainty and its propagation in computer models has relevance in many disciplines, including hydrology, environmental engineering, ecology and climate change. Error propagation in a model results in uncertainty in prediction due to uncertainties in model inputs and parameters. Common methods for quantifying error propagation are reviewed, namely Differential Error Analysis and Monte Carlo Simulation, including underlying principles, together with a discussion on their differences, advantages and disadvantages. The separate case of uncertainty in the model calibration process is different to error propagation in a fixed model in that it is associated with a dynamic process of iterative parameter adjustment, and is compared in the context of non-linear regression and Bayesian approaches, such as Markov Chain Monte Carlo Simulation. Error propagation is investigated for a soil model representing the organic carbon depth profile and also a streamflow model using probabilistic simulation. Different sources of error are compared, including uncertainty in inputs, parameters and geometry. The results provided insights into error propagation and its computation in systems and models in general.
Similar content being viewed by others
References
Abusam A, Keesman KJ, Van Straten G (2003) Forward and backward uncertainty propagation: an oxidation ditch modelling example. Water Res 37(2):429–435
Bayes T (1763) An essay towards solving a problem in the doctrine of chances. Philos Trans R Soc Lond 53:1393–1442
Benke KK, Benke KE (2013) Uncertainty in health risks from artificial lighting due to disruption of circadian rhythm and melatonin secretion: a review. Hum Ecol Risk Assess 19:916–929
Benke KK, Robinson NJ (2017) Quantification of uncertainty in mathematical models: the statistical relationship between field and laboratory pH measurements. Appl Environ Soil Sci 20:12. https://doi.org/10.1155/2017/5857139
Benke KK, Lowell KE, Hamilton AJ (2008) Parameter uncertainty, sensitivity analysis and prediction error in a water-balance hydrological model. Math Comput Model 47:1134–1149
Beven K (2008) Comment on ‘‘Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling? Vrugt J.A., ter Braak, C.J.F., Gupta, H.V., Robinson, B.A., 2008. Stoch Environ Res Risk Assess. https://doi.org/10.1007/s00477-008-0283-x
Beven KJ, Binley AM (1992) The future of distributed models: model calibration and predictive uncertainty. Hydrol Process 6:279–298
Beverly C, Christy B, Weeks A (2006) Application of the 2CSalt model to the Bet Bet, Wild Duck, Gardner and Sugarloaf Catchments in Victoria. Department of Primary Industries, Victoria
Bolstad WM (2007) Introduction to Bayesian statistics, 2nd edn. Wiley, Hoboken
Box GEP, Cox DR (1982) An analysis of transformations revisited, rebutted. J Am. Stat Assoc 77:209–210
Brown JD, Heuvelink GBM (2005) 79: assessing uncertainty propagation through physically based models of soil water flow and solute transport. In: Anderson MG (ed) Encyclopaedia of hydrological sciences. Wiley, New York
Buckland ST (1984) Monte Carlo confidence intervals. Biometrics 40:811–817
Carlin BP, Louis TA (2008) Bayesian methods for data analysis, 3rd edn. Chapman and Hall, Boca Raton
Chen C-S, Chen C-S (2018) A composite spatial predictor via local criteria under a misspecified model. Stoch Environ Res Risk Assess 32:341–355
Doherty J (2003) MICA: model independent Markov Chain Monte Carlo analysis. Watermark Numerical Computing, Brisbane
Donnelly SM, Kramer A (1999) Testing for multiple species in forest samples: an evaluation and comparison of tests for equal relative variation. Am J Phys Anthopol 108:507–529
Dotto CBS, Mannina G, Kleidorfer M, Vezzaro L, Henrichs M, McCarthy DT, Freni G, Rauch W, Deletic A (2012) Comparison of different uncertainty techniques in urban stormwater quantity and quality modelling. Water Res 46(8):2545–2558
Freeze RA (2004) The role of stochastic hydrogeological modeling in real-world engineering applications. Stoch Env Res Risk Assess 18(4):286–289
Freni G, Mannina G (2009) Urban runoff modelling uncertainty: comparison among Bayesian and pseudo-Bayesian methods. Environ Model Softw 24:1100–1111
Freni G, Mannina G (2010) Bayesian approach for uncertainty quantification in water quality modelling: the influence of prior distribution. J Hydrol 392:31–39
Freund JE (1998) Mathematical statistics. Prentice-Hall, New York
Garg A, Vijayaraghavan V, Zhang J, Li S, Liang X (2017a) Design of robust battery capacity model for electric vehicle by incorporation of uncertainties. Int J Energy Res 41(10):1436–1451
Garg A, Vijayaraghavan V, Zhang J, Lam JSL (2017b) Robust model design for evaluation of power characteristics of the cleaner energy system. Renew Energy 112:302–313
Gelman A, Carlin JB, Stern HS, Rubin DB (2004) Bayesian data analysis. Chapman and Hall/CRC, Boca Raton
Geman S, Geman D (1984) Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6:721–741
Goidts E, van Wesmael B, Crucifix M (2009) Magnitude and sources of uncertainties in soil organic carbon (SOC) stock assessments at various scales. Eur J Soil Sci 60:723–729
Gupta HV, Clark MP, Vrugt JA, Abramowitz G, Ye M (2012) Towards a comprehensive assessment of model structural adequacy. Water Resour Res 48:W08301. https://doi.org/10.1029/2011WR011044
Haas CN, Eisenberg JNS (2001) Risk assessment. In: Fewtrell L, Bartram J (eds) Water quality: guidelines, standards and health. Assessment of risk and risk management for water-related infectious disease. IWA Publishing, London, pp 161–183
Hamilton AJ, Basset Y, Benke KK, Grimbacher PS, Miller SE, Novotný V, Samuelson GA, Stork NE, Weiblen GD, Yen JD (2010) Quantifying uncertainty in estimation of tropical arthropod species richness. Am Nat 176:90–95
Hamilton AJ, Novotny V, Waters EK, Basset Y, Benke KK, Grimbacher PS, Miller SE, Samuelson GA, Weiblen GD, Yen JD, Stork NE (2013) Estimating global arthropod species richness: refining probabilistic models using probability bounds analysis. Oecologia 171:357–365. https://doi.org/10.1007/s00442-012-2434-5
Helton JC (1993) Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliab Eng Syst Saf 42:327–367
Heuvelink GBM, Burrough PA (2002) Developments in statistical approaches to spatial uncertainty and its propagation. Int J Geogr Inf Sci 16:111–113
Heuvelink GB, Burrough PA, Stein A (1989) Propagation of errors in spatial modelling with GIS. Int J Geogr Inf Syst 3(4):303–322
Kavetski D, Kuczera G, Franks SW (2006a) Bayesian analysis of input uncertainty in hydrological modelling: 1. Theory. Water Resour J 42(W03407):1–9
Kavetski D, Kuczera G, Franks SW (2006b) Calibration of conceptual hydrological models revisited: 1. Overcoming numerical artefacts. J Hydrol 32(1–2):173–186
Kavetski D, Kuczera G, Franks SW (2006c) Calibration of conceptual hydrological models revisited: 2. Improving optimisation and analysis. J Hydrol 32(1–2):187–201
Kline SJ, McClintock FA (1953) Describing uncertainties in single sample experiments. Mech Eng 75:3–8
Kroese DP, Taimre T, Botev ZI (2011) Handbook of Monte Carlo methods. Wiley, New York
Kuczera G, Parent E (1998) Monte Carlo assessment of parameter uncertainty in conceptual models: the metropolis algorithm. J Hydrol 211:69–85
Lagos-Álvarez BM, Toribio RF, Figueroa-Zunuga J, Mateu J (2017) Geostastistical mixed beta regression: a Bayesian approach. Stoch Environ Res Risk Assess 31:571–584
Lark RM, Webster R (2006) Geostastistical mapping of geomorphic surfaces in the presence of trend. Earth Surf Process Landf 31:862–874
Littleboy M (2005) UserGuide_2CSalt, Rev 1.0, CRC for Catchment Hydrology, Australia. www.toolkit.net.au
Malone BP, McBratney AB, Minasny B (2011) Empirical estimates of uncertainty for mapping continuous depth functions of soil attributes. Geoderma 160:614–625
Mandel J (1964) The statistical analysis of experimental data. Dover Publications Inc., New York
McKay MD (1995) Evaluating prediction uncertainty. Report No. LA-12915-MS, Statistics Group, Los Alamos National Laboratory, NM, USA
Minasny B, Vrugt JA, McBratney AB (2011) Confronting uncertainty in model-based geostatistics using Markov Chain Monte Carlo simulation. Geoderma 163:150–162
Nelson MA, Bishop TFA, Odeh IOA, Triantafilis J (2011) An error budget for different sources of error in digital soil mapping. Eur J Soil Sci 62:417–430
Oberkampf WL, Helton JC, Joslyn CA, Wojtkiewicz SF, Ferson S (2004) Challenge problems: uncertainty in system response given uncertain parameters. Reliab Eng Syst Saf 85:11–19
Oliver MA, Webster R (2014) A tutorial guide to geostatistics: computing and modelling variograms and kriging. Catena 113:56–69
Oritz JO, Felgueiras CA, Camargo ECG, Rennó CD, Oritz MJ (2017) Spatial modelling of soil lime requirements with uncertainty assessment using geostatistical sequential indicator simulation. Open J Soil Sci 7:133–148
Oya A, Navarro-Moreno J, Ruiz-Molina JC (2007) Spatial random field simulation by a numerical series representation. Stoch Env Res Risk Assess 21:317–326
Parratt LG (1971) Probability and experimental errors in science. Dover Publications Inc., New York
Patil A, Deng ZQ, Malone RF (2011) Input data resolution-induced uncertainty in watershed modelling. Hydrol Process 25:2302–2312
Qian SS, Stow CA, Borsuk ME (2003) On Monte Carlo methods for Bayesian inference. Ecol Model 159:269–277
Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using Bayesian model averaging to calibrate forecast ensembles. Mon Weather Rev 133:1155–1174
Rayment GE, Lyons DJ (2011) Soil chemical methods: Australasia, vol 3. CSIRO publishing, Collingwood
Refsgaard JC, van der Sluijs JP, Højberg AL, Vanrolleghem PA (2007) Uncertainty in the environmental modelling process—a framework and guidance. Environ Model Softw 22:1543–1556
Robinson NJ, Benke KK, Norng S (2015) Identification and interpretation of sources of uncertainty in soils change in a global systems-based modelling process. Soil Res 53(6):592–604
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536
Savelyeva E, Utkin S, Kazakpv S, Demyanov V (2010) Modeling spatial uncertainty for locally uncertain data. Geoenv VII Geostat Environ Appl 16:295–306
Sklar A (1959) Fonctions de répartition à n dimensions et leurs marges”. Publ Inst Stat Univ Paris 8:229–231
Smith MS, Khaled MA (2012) Estimation of copula models with discrete margins via Bayesian data augmentation. J Am Stat Assoc 107:290–303
Stenson MP, Littleboy M, Gilfedder M (2011) Estimation of water and salt generation from unregulated upland catchments. Environ Model Softw. https://doi.org/10.1016/j.envsoft.2011.05.013
Taylor JR (1997) An introduction to error analysis. University Science Books, Sausalito
Trucano T, Swiler L, Igusa T, Oberkampf W, Pilch W (2006) Calibration, validation, and sensitivity analysis: what’s what. Reliability engineering and system safety. In: 4th international conference on sensitivity. analysis of model output-SAMO 2004, vol 91, No. 10–11, pp 1331–1357
Vandenberghe V, Bauwens W, Vanrolleghem PA (2007) Evaluation of uncertainty propagation into river water quality predictions to guide future monitoring campaigns. Environ Model Softw 22:725–732
Vose D (2008) Risk analysis. Wiley, Chichester
Vrugt JA, Gupta HV, Bouten W, Sorooshian S (2003) The shuffled complex evolutionary metropolis algorithm for optimisation and uncertainty assessment of hydrological parameters. Water Resour Res 39(8):1-1–1-9
Vrugt JA, ter Braak CJF, Gupta HV, Robinson BA (2009a) Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling? Stoch Environ Res Risk Assess 23:1011–1026. https://doi.org/10.1007/s00477-008-0274-y
Vrugt JA, ter Braak CJF, Gupta HV, Robinson BA (2009b) Response to Keith Beven comment on “equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?”. Stoch Environ Res Risk Assess 23:1061–1062. https://doi.org/10.1007/s00477-008-0284-9
Wang D, Lu WZ (2006) Forecasting Ozone Levels and analyzing their dynamics by a Bayesian multilayer perceptron model for two air-monitoring sites in Hong Kong. Hum Ecol Risk Assess 12:313–327
Wöhling T, Vrugt JA (2008) Combining multi-objective optimization and Bayesian model averaging to calibrate forecast ensembles of soil hydraulic models. Water Resour Res 44:W12432. https://doi.org/10.1029/2008WR007154
Wu F, Chen C (2009) Bayesian updating of parameters for a sediment entrainment model via Markov Chain Monte Carlo. J Hydraul Eng 135:22–37
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Benke, K.K., Norng, S., Robinson, N.J. et al. Error propagation in computer models: analytic approaches, advantages, disadvantages and constraints. Stoch Environ Res Risk Assess 32, 2971–2985 (2018). https://doi.org/10.1007/s00477-018-1555-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00477-018-1555-8