Research papersAppropriate model selection methods for nonstationary generalized extreme value models
Introduction
Frequency analysis plays an important role in the hydraulic structure design process as well as in the management of water resources. It utilizes appropriate probability distribution models to estimate hydrologic quantiles. Frequency analysis assumes that data is both independent and stationary, i.e., data and its statistical characteristics do not vary over time. Industrialization and urbanization influence change in climatic conditions, and this has caused hydrologic and meteorological data to become nonstationary (Jain and Lall, 2000, Jain and Lall, 2001, Katz et al., 2002, Milly et al., 2008, Olsen et al., 1999). For example, statistics such as quantiles of hydrologic data, and parameters of probability models may change over time. However, there has been much controversy over the concept of nonstationarity in water resources management and planning. Milly et al. (2008) asserted that nonstationary probabilistic models should be identified and applied because anthropogenic climate change is affecting the extremes of hydrological variables (e.g., precipitation, streamflow and evapotranspiration). In contrast, several scholars have emphasized that the careless application of nonstationarity could lead to the underestimation of variability, uncertainty and risk (Koutsoyiannis, 2011, Lins and Cohn, 2011, Montanari and Koutsoyiannis, 2014, Serinaldi and Kilsby, 2015). Nonetheless, various studies on nonstationarity for hydrological modeling are still being conducted to predict future events under changing environmental conditions. There have been many studies focusing on nonstationary frequency analysis that primarily takes into account covariates, such as time, temperature, and climate indices. Examples of climate indices are Pacific Decadal Oscillation (PDO), Southern Oscillation Index (SOI), Mediterranean Oscillation Index (MOI), North Atlantic Oscillation (NAO), Sea Level Pressure (SLP), and Sea Surface Temperature (SST). These covariates are used to determine parameters of probability distribution models (Brown et al., 2008, Coles, 2001, Griffis and Stedinger, 2007, Katz et al., 2002, Sugahara et al., 2009, Tramblay et al., 2013, Vasiliades et al., 2015, Wang et al., 2004, Wi et al., 2015).
Extreme value theory is a branch of statistics that focuses on the extreme events and the tail behavior of a distribution. The theory uses the block maxima approach to derive Extreme Value (EV) distributions, including the Fréchet, Weibull, and Gumbel distributions. The GEV distribution unifies the three abovementioned EV distributions. In nonstationary frequency analysis, nonstationary GEV distributions have been proposed and widely used (Cannon, 2010, Coles, 2001, El Adlouni et al., 2007, Kharin and Zwiers, 2005, Leadbetter et al., 1983, Mailhot et al., 2010, Nadarajah, 2005, Vasiliades et al., 2015, Wang et al., 2004, Wi et al., 2015). The nonstationary GEV models proposed by Nadarajah, 2005, Vasiliades et al., 2015, and Wi et al. (2015) have been used to conduct nonstationary frequency analysis of the annual maximum rainfall series, using time as a covariate.
In conventional frequency analysis, the test, Kolmogorov–Smirnov (KS) test, Cramér von Mises (CVM) test, probability plot correlation coefficient (PPCC) test, Anderson-Darling test, and modified Anderson-Darling test have been used to examine the goodness-of-fit (GOF) for probability models (Heo et al., 2013). In addition to these GOF tests, model prediction error measured by the bootstrap or cross-validation has been used to select an appropriate probability model (Laio et al., 2009, Smyth, 2000). Burnham and Anderson (2002) and Zucchini (2000) introduced and expounded these techniques for model selection. In nonstationary frequency analysis, however, it is not simple to apply goodness-of-fit tests to nonstationary probability distribution models involving parameters that vary with time since these tests should be performed at each time step. Therefore, many studies alternatively recommend the Akaike’s information criterion (AIC), corrected Akaike’s information criterion (AICc), and Bayesian information criterion (BIC) for the selection of appropriate nonstationary models (Cannon, 2010, Strupczewski et al., 2001a, Strupczewski et al., 2001b, Sugahara et al., 2009, Villarini et al., 2009, Villarini et al., 2010). These criteria are straightforward and allow for selecting an appropriate model if the maximized likelihood is calculated. Strupczewski et al., 2001a, Strupczewski et al., 2001b used the AIC to select the most efficient model out of several nonstationary flood frequency models. Sugahara et al. (2009) applied the AICc (Hurvich and Tsai, 1995) and the rAICc (Burnham and Anderson, 2004) to select the most efficient model out of four nonstationary generalized Pareto distributions. Villarini et al., 2009, Villarini et al., 2010 employed the AIC and the BIC to find the degrees of freedom for the Generalized Additive Models of Location, Scale, and Shape parameters (GAMLSS) in a nonstationary framework. Cannon (2010) identified an appropriate nonstationary GEV model using the AICc and the BIC. Vasiliades et al. (2015) identified an appropriate nonstationary GEV model using the AICc and the BIC.
Alternatively, the Likelihood Ratio Test (LRT) has been used in several studies (Clarke, 2002, El Adlouni et al., 2007, García et al., 2007, Katz, 2013, Kharin and Zwiers, 2005, Mailhot et al., 2010, Nadarajah, 2005, Tramblay et al., 2013, Wang et al., 2013), and has been recommended for the selection of an appropriate nonstationary extreme value model (Coles, 2001). Clarke (2002) proposed the Gumbel distribution, involving time as a covariate, and used Generalized Linear Models (GLMs) to fit trend parameters. The LRT was applied to evaluate the goodness-of-fit for the GLMs. Kharin and Zwiers (2005) also evaluated nonstationary GEV models by performing the LRT. Nadarajah, 2005, El Adlouni et al., 2007, and Wang et al. (2013) proposed several nonstationary GEV models, and determined the most efficient one by using the LRT. García et al. (2007) conducted the LRT to draw a comparison between stationary and nonstationary GEV models. Mailhot et al. (2010) employed the LRT to compare the nonstationary Ensemble Members (EM) and Annual Maximum (AM) models. Katz (2013) used the AIC, the BIC, and the LRT to select appropriate nonstationary models. Tramblay et al. (2013) selected an appropriate nonstationary Peaks-Over-Threshold (POT) model with the help of the LRT.
The abovementioned studies are only a few ones that compare various model selection criteria to determine an appropriate nonstationary GEV model. Although Stone (1979) described the fundamental characteristics and comparative performance of the AIC and BIC, no specific standards have been set to determine the best criterion for such a model. Therefore, it is likely that an inappropriate model may be selected under nonstationary conditions, and this makes it necessary to determine the most appropriate criterion. Panagoulia et al. (2014) conducted a simple simulation study to evaluate the performances of the AICc and BIC for nonstationary GEV models. However, their results were limited to specific sample sizes and simulation conditions. To get more general results, this study compares the performances of the AIC, the AICc, the BIC, and the LRT, using the Monte Carlo simulation for various sample sizes as well as location, scale, and shape parameters based on stationary and nonstationary GEV distributions. To evaluate the simulation results, the AIC, AICc, BIC, and LRT were applied to the stationary and nonstationary GEV models fitted to the observed annual maximum rainfall data.
Section snippets
Model selection criteria
A number of methods can be applied to select appropriate nonstationary models. Of these, the AIC, the AICc, the BIC, and the LRT have been recommended the most. In this study, these tests were applied to various stationary and nonstationary GEV models.
General description
Monte Carlo simulations were performed to evaluate the feasibility of using the AIC, AICc, BIC, and LRT in the selection of the GEV(0,0,0), GEV(0,1,0), GEV(1,0,0), and GEV(1,1,0) models. In order to determine the magnitude of trends in the location and scale parameters for the simulation conditions, the observed annual maximum rainfall data sets were used in this study. The observed data sets were standardized on the basis of the mean value of each data set, and then the slopes of the location
Actual evaluation
In order to evaluate the results of the simulation experiments, the observed annual maximum rainfall data at four selected sites— Jinju, Gunsan, Yeongju, and Namwon—were used as examples of stationary and nonstationary cases. The data was standardized on the basis of the mean value of each data set. The location, scale, and shape parameters of these four data sets were estimated by the maximum likelihood method for the sample using 20-year moving windows. Fig. 12 shows the estimated location,
Conclusions
This simulation study was conducted to recommend appropriate model selection methods for use in nonstationary frequency analysis. Monte Carlo simulation experiments were conducted for various stationary and nonstationary GEV models, and the performances of the AIC, AICc, BIC, and LRT model selection methods were evaluated.
For stationary GEV models, the proper model selection ratios of the BIC and LRT were above 90%, whereas those of the AIC and AICc were more than 70%. This is because the AIC
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and future Planning (grant number: 2014006671).
References (46)
Significance of AIC differences for precipitation intensity distributions
Adv. Water Resour.
(2003)- et al.
Approximation of modified Anderson-Darling test statistics for extreme value distributions with unknown shape parameter
J. Hydrol.
(2013) - et al.
Statistics of extremes in hydrology
Adv. Water Resour.
(2002) - et al.
Future changes in intensity and seasonal pattern of occurrence of daily and multi-day annual maximum precipitation over Canada
J. Hydrol.
(2010) - et al.
Stationarity is undead: uncertainty dominates the distribution of extremes
Adv. Water Resour.
(2015) - et al.
Non-stationary approach to at-site flood frequency modelling I. Maximum likelihood estimation
J. Hydrol.
(2001) - et al.
Non-stationary approach to at-site flood frequency modelling III. Flood analysis of Polish rivers
J. Hydrol.
(2001) - et al.
Flood frequency analysis for nonstationary annual peak records in an urban drainage basin
Adv. Water Resour.
(2009) - et al.
Nonstationary modeling of a long record of rainfall and temperature over Rome
Adv. Water Resour.
(2010) An introduction to model selection
J. Math. Psychol.
(2000)
Information theory and an extension of the maximum likelihood principle
Global changes in extreme daily temperature since 1950
J. Geophys. Res.
Model Selection and Multimodel Inference
Multimodel inference: understanding AIC and BIC in model selection
Sociological Methods Res.
A flexible nonlinear modeling framework for nonstationary generalized extreme value analysis in hydroclimatology
Hydrol. Process
Estimating time trends in Gumbel-distributed data by means of generalized linear models
Water Resour. Res.
An Introduction to Statistical Modeling of Extreme Values
Generalized maximum likelihood estimators for the nonstationary generalized extreme value model
Water Resour. Res.
Trends in block-seasonal extreme rainfall over the Iberian peninsula in the second half of the twentieth century
J. Clim.
Regression and time series model selection in small samples
Biometrika
Model selection for extended Quasi-Likelihood Models in small samples
Biometrics
Magnitude and timing of annual maximum floods: Trends and large-scale climatic associations for the Blacksmith Fork river, Utah
Water Resour. Res.
Cited by (57)
CLformer: Constraint-based Locality enhanced Transformer for anomaly detection of ancient building structures
2023, Engineering Applications of Artificial IntelligenceCauses of 2022 summer marine heatwave in the East China Seas
2023, Advances in Climate Change ResearchQuantifying climate change impacts on low flows of small high mountain watersheds: A nonstationary approach
2023, Journal of Hydrology: Regional StudiesA multi-dimensional framework for improving flood risk assessment: Application in the Han River Basin, China
2023, Journal of Hydrology: Regional StudiesThe decomposition-based nonstationary flood frequency analysis
2022, Journal of HydrologyCitation Excerpt :In this context, the determination of a NS-FFA model, which is formed by coupling the probability distribution of floods and the nonstationary structure that governs its evolution over time, is a key to implementing the NS-FFA. The conventional approach for determining a suitable NS-FFA model is through selection by ranking a set of candidate models according to the pre-selected performance metric, e.g., information-theoretic measures such as the Akaike and Bayesian Information Criteria (AIC and BIC) (e.g., Kim et al., 2017; Li et al., 2019; Ragno et al., 2019). This approach is referred to as the performance-based approach in this paper.