General foundations of high‐dimensional model representations

Abstract

A family of multivariate representations is introduced to capture the input–output relationships of high‐dimensional physical systems with many input variables. A systematic mapping procedure between the inputs and outputs is prescribed to reveal the hierarchy of correlations amongst the input variables. It is argued that for most well‐defined physical systems, only relatively low‐order correlations of the input variables are expected to have an impact upon the output. The high‐dimensional model representations (HDMR) utilize this property to present an exact hierarchical representation of the physical system. At each new level of HDMR, higher‐order correlated effects of the input variables are introduced. Tests on several systems indicate that the few lowest‐order terms are often sufficient to represent the model in equivalent form to good accuracy. The input variables may be either finite‐dimensional (i.e., a vector of parameters chosen from the Euclidean space $\mathcal{R}^n$) or may be infinite‐dimensional as in the function space ${\text{C}}^n \left[ {0,1} \right]$. Each hierarchical level of HDMR is obtained by applying a suitable projection operator to the output function and each of these levels are orthogonal to each other with respect to an appropriately defined inner product. A family of HDMRs may be generated with each having distinct character by the use of different choices of projection operators. Two types of HDMRs are illustrated in the paper: ANOVA‐HDMR is the same as the analysis of variance (ANOVA) decomposition used in statistics. Another cut‐HDMR will be shown to be computationally more efficient than the ANOVA decomposition. Application of the HDMR tools can dramatically reduce the computational effort needed in representing the input–output relationships of a physical system. In addition, the hierarchy of identified correlation functions can provide valuable insight into the model structure. The notion of a model in the paper also encompasses input–output relationships developed with laboratory experiments, and the HDMR concepts are equally applicable in this domain. HDMRs can be classified as non‐regressive, non‐parametric learning networks. Selected applications of the HDMR concept are presented along with a discussion of its general utility.