Imprecise global sensitivity analysis using bayesian multimodel inference and importance sampling

Highlights

• A fully probabilistic multimodel approach for imprecise GSA is proposed.

• This approach employs Bayesian multimodel inference to quantify uncertainties in model inputs.

• The multimodel inference is combined with an importance reweighting scheme for estimation of imprecise Sobol indices.

• The methodology can be used to assess confidence in computed sensitivity indices and inform testing and data collection efforts.

Abstract

Global Sensitivity Analysis (GSA) aims to understand the relative importance of uncertain input variables to model response. Conventional GSA involves calculating sensitivity (Sobol’) indices for a model with known model parameter distributions. However, model parameters are affected by aleatory and epistemic uncertainty, with the latter often caused by lack of data. We propose a new framework to quantify uncertainty in probability model-form and model parameters resulting from small datasets and integrate these uncertainties into Sobol’ index estimates. First, the process establishes, through Bayesian multimodel inference, a set of candidate probability models and their associated probabilities. Imprecise Sobol’ indices are calculated from these probability models using an importance sampling reweighting approach. This results in probabilistic Sobol’ indices, whose distribution characterizes uncertainty in the sensitivity resulting from small dataset size. The imprecise Sobol’ indices thus provide a measure of confidence in the sensitivity estimate and, moreover, can be used to inform data collection efforts targeted to minimize the impact of uncertainties. Through an example studying the parameters of a Timoshenko beam, we show that these probabilistic Sobol’ indices converge to the true/deterministic Sobol’ indices as the dataset size increases and hence, distribution-form uncertainty reduces. The approach is then applied to assess the sensitivity of the out-of-plane properties of an E-glass fiber composite material to its constituent properties. This second example illustrates the approach for an important class of materials with wide-ranging applications when data may be lacking for some input parameters.

Introduction

Computational simulation is widely used to better understand complex mathematical and physical systems. Typically these simulations aim to model the system response, specifically targeting some quantity of interest (QoI) and its dependence on some set of input model parameters. In many cases, these input parameters are not precisely known or deterministic because of inherent randomness in the system and/or a lack of knowledge/data. In traditional analyses, uncertainty is quantified probabilistically and propagated through the computational model. Consequently, the corresponding QoI is uncertain.

Sensitivity analysis is the quantitative assessment of the influence of variations in input parameters on variations in the output QoI. Sensitivity analysis therefore enables the assessment of the relative importance of each model input to the specific QoI [1], [2]. Generally speaking, there are two classes of sensitivity analysis: Local Sensitivity Analysis (LSA) and Global Sensitivity Analysis (GSA). LSA focuses on the influence of the small variations in the input parameters around some nominal values and are typically quantified by evaluating local gradients in the system solution with respect to the input parameters. GSA, on the other hand, examines the effect of variations in the input parameters across the entire range of the parameter space. We are specifically interested in GSA in this work. Two prominent methods have been developed for GSA. The first is the one-factor-at-a-time approach in which each parameter is perturbed in turn while keeping all other parameters fixed at their nominal value. This approach has been shown to break down when the model deviates from linearity [3] and cannot account for a probability density over the parameter domain. The second is based on the estimation of variance-based sensitivity indices (so-called Sobol’ indices) [4] that prove more robust to model form and relies on each parameter having an associated probability measure.

A variety of methods have been proposed for variance-based GSA that employ, for example, Fourier analysis as in the FAST (Fourier Amplitude Sensitivity Test) algorithm [5], [6], Monte Carlo methods [7], [8], and surrogate modeling methods such as polynomial chaos [9], [10], Gaussian processes/Kriging [11], and support vector machines [12] considering both independent and correlated input parameters [13]. Moreover, the range of recent applications for GSA spans across physical systems of all types from chemistry [14] to structural acoustics [15] and structural vibration of nuclear reactor assemblies [16]. In this work, we employ Monte Carlo estimates of global sensitivity indices because they are robust, simple to use, and couple conveniently with the multimodel uncertainty quantification methods for imprecise probabilities proposed herein. Extension of the concepts proposed herein to utilize more efficient GSA algorithms thus remains an important research objective.

Conventional GSA methods are built from a classical probabilistic framework in which input uncertainty is characterized through a known probability measure on the input parameters. Hence, the first step of GSA is to identify or assume a reasonable probability distribution for the input variables. However, as commonly occurs when data characterizing the input parameters are sparse, it may be impossible to identify the appropriate distribution. This can lead to assumptions that may seem reasonable but are nonetheless subjective and may yield sensitivity indices that are inaccurate, non-conservative, or even potentially unreasonable with no quantitative error metrics. These errors arise from so-called epistemic uncertainty – or uncertainty that arises from a lack of knowledge or data and can therefore be reduced with additional data collection [17]. The inclusion of epistemic uncertainty gives rise to two distinct objectives for imprecise global sensitivity analysis:

• 1. To assess confidence in estimates of conventional global sensitivity indices.

• 2. To specifically understand the influence of epistemic uncertainty on system response, and therefore inform data collection efforts such that they reduce the impact of uncertainty in system response.

To address both objectives in this work, we propose a method that infers a probability distribution for the Sobol’ indices. Thus, toward the first object, the method provides a natural measure of confidence. Toward the second objective, the inferred distribution identifies parameters that may contribute significantly to system response and also have large uncertainty. Therefore, it is useful in identifying parameters that should be targeted for further data collection efforts. Generally speaking, several theories have been proposed to address the various forms of epistemic uncertainty. It has been argued that epistemic uncertainty requires a different mathematical treatment than aleatory uncertainty [18], which arises from irreducible randomness and is naturally treated probabilitistically. However, no scientific consensus has been achieved in terms of what that treatment should be. The larger problem is that a unified mathematical treatment of epistemic uncertainty must be general enough to accommodate the many types of epistemic uncertainty – such as those that arise from vague, qualitative, or conflicting data, total or near-total ignorance, small datasets that give limited information about a probability measure, and the “unknown unknown” among others. Different mathematical theories have been developed which account for some of these epistemic uncertainties; typically by generalizing the measure used to quantify events in the algebra of possibilities. Specific examples include possibility theory which introduce the concepts of possibility and necessity measures [19], Dempster-Shafer/Evidence theory which introduces the concepts of belief and plausibility measures [20], [21], Choquet capacities which introduce conjugage measures of lower and upper probability [22], fuzzy sets [23], [24] and fuzzy measures [25], random sets and sets of probability measures [26], [27], and interval methods [28] including interval probabilities and probability boxes (p-boxes) [18]. In the intervening years, efforts have been made to unify these theories under an over-arching theory of imprecise probabilities, most notably through the works of Walley [29], [30].

Perhaps more relevant for our purposes, many recent efforts have been made in the engineering community to translate these theories of imprecise probabilities into the practice of uncertainty quantification and stochastic analysis in computational modeling. This includes efforts to: extend Monte Carlo simulations [31], [32], [33], develop non-intrusive stochastic simulation methods for imprecise probability models [34], [35], [36], [37], construct p-boxes and Dempster-Shafer belief functions [38], understand the effects of imprecision on reliability/probability of failure [39], [40], [41], [42], [43], propagate p-boxes[44], [45], and perform Bayesian model averaging [46] to name just a few. A concise review of imprecise probability methods in engineering can be found in [47]. Among the many studies of epistemic uncertainty in engineering, relatively few have considered the problem of imprecise global sensitivity analysis. To the authors’ knowledge, such investigations date back a little more than a decade beginning with the work of Helton et al. who merged GSA with evidence theory to estimate imprecise sensitivity measures [48] and Hall who estimated the upper and lower bounds of sensitivity indices when the uncertainty in the model inputs are expressed as closed convex sets of probability measures [49]. Among the most comprehensive studies of imprecise GSA is that conducted by Oberguggenberger et al. [50] who compared classical GSA with GSA for problems with uncertainties defined by random sets, fuzzy sets, and intervals. Other recent works include those of Song et al. [51] who devised GSA for input uncertainties characterized by p-boxes and computed the imprecise sensitivity indices using extended Monte Carlo simulation [31] (EMCS). Li and Mahadevan [52] devised a scheme to estimate Sobol’ indices when both aleatory and epistemic uncertainty are present in time series data. More recently, Wei et al. [53] proposed a probabilistic framework where epistemic input uncertainties are characterized by second-order probability models to compare the relative importance of influential and non-influential input variables using EMCS. Schöbi and Sudret [54] used polynomial chaos expansions to develop interval Sobol’ indices when input uncertainties are characterized by p-boxes, and Hart and Germaud [55] provided a theoretical analysis for the robustness of the Sobol’ indices to changes in the distribution of the uncertain variables.

In this study, we investigate imprecise GSA where epistemic uncertainty specifically results fromsparse data sets, which may be compiled from disparate data sources. This focus is motivated by the difficulty of data collection under complex conditions in engineering practice. In many cases, experimental or validated simulation data for quantifying parameter uncertainties are strictly limited. As a result, it is impossible to assign an objective and accurate probability distribution for the input variables to a computational model and precisely estimate their impact on the response output. The specific contributions of this study can be summarized as follows:

• We derive a formulation for computing main effect and total sensitivity indices using importance sampling.

• We apply a Bayesian multimodel inference process in conjunction with the importance sampling-based GSA to estimate the distribution of imprecise Sobol’ indices whose uncertainty results from lack of data quantifying input uncertainty.

• We demonstrate how to obtain the optimal sampling density to efficiently employ the proposed multimodel GSA for engineering systems.

• We illustrate how to use the results of the proposed imprecise GSA to assess confidence in computed sensitivity indices and inform future testing and data collection.

It is important to emphasize that the novelty of the work presented here lies in bringing the components of importance sampling based GSA and Bayesian multimodel inference together for a robust and efficient approach to imprecise GSA. The development of the individual components of the approach is not the primary intent. Indeed previous works have, for example, derived formulations for computing sensitivity indices using importance sampling (see e.g. [53]) – although the formulation provided here has some distinct differences and advantages that are discussed below.

The paper is structured as follows. We begin in Section 2 by presenting a brief review of GSA, particularly variance-based methods and Sobol’ indices. Section 3 introduces the Bayesian multimodel inference methodology and its application for identifying imprecise probability models given limited data. An efficient importance sampling-based Monte Carlo method for imprecise global sensitivity analysis is then proposed in Section 4. The effectiveness of the proposed algorithm is illustrated by a closed-form numerical example in Section 5. Section 6 presents an application of the proposed method to predicting the sensitivity of composite material properties to the properties of its constituents. Finally, some concluding remarks are provided in Section 7.