ReviewNon-parametric Bayesian networks: Improving theory and reviewing applications
Introduction
Understanding and representing multivariate distributions along with their dependence structure is a highly active area of research. A large body of scientific work treating multivariate models is available. This paper advocates graphical models in general and Bayesian networks (BNs) in particular to represent high dimensional distributions with complex dependence structures.
A BN consists of a directed acyclic graph (DAG) and a set of (conditional) distributions. Each node in the graph corresponds to a random variable and the arcs represent direct qualitative dependence relationships. The absence of arcs guarantees a set of (conditional) independence facts. The direct predecessors (successors) of a node are called parents (children). A marginal distribution is specified for each node with no parents, and a conditional distribution is associated with each child node. The (conditional) distributions serve as the quantitative information about the strength of the dependencies between the variables involved. The graph with the conditional independence statements encoded by it, together with the (conditional) distributions represent the joint distribution over the random variables denoted by the nodes of the graph. The quantitative information needed can be either retrieved from data, when available, or from experts.
The relatively simple visualization of the complicated relationships among the random variables is one of the most appealing features of a BN model. The main use of BNs is to update distributions given observations. This is referred to as inference in BNs.
BNs have been successfully used to represent uncertain knowledge, in a consistent probabilistic manner, in a variety of fields [62]. Nevertheless, most of the applications use discrete BNs, i.e BNs whose nodes represent discrete random variables. These models specify marginal distributions for the nodes with no parents, and conditional probability tables for child nodes. Inference in discrete BNs can be performed using either exact algorithms (e.g. [68], [51]), or approximate algorithms (e.g. [31], [47]). Despite their popularity, discrete BNs suffer from severe limitations. Applications involving high complexity in data-sparse environments are severely limited by the excessive assessment burden which leads to rapid, informal and indefensible quantification. Furthermore, this type of representation, using only discrete variables, is inadequate for many problems of practical importance. Discrete BNs have been extensively studied, hence we omit their presentation here, but refer to Weber et al. [62] and references therein. Many domains require reasoning about the joint behaviour of a mixture of discrete and continuous variables. These domains are often called hybrid domains. Hence, BNs involving both discrete and continuous variables will be called hybrid BNs (HBNs). Working with HBNs proves considerably more challenging than working with their discrete counterparts. Several methods for HBNs are available and discussed in the literature (e.g. [38], [34]). We will only briefly mention some of them, insist on the most recent ones, and present in more detail the method called non-parametric BNs (NPBNs) that is overlooked in many HBNs review papers. The emphasis here is on the application of the NPBN methods in different fields of science and engineering in the course of approximately 10 years since NPBNs were first presented.
A study which is perhaps the most comprehensive overview study of BNs applications [62] incorporated information about most methods available for working with HBNs up to the date of its publication. Given the amount of research available on the theoretical and practical aspects of BNs, it is not surprising that even such an impressively large study overlooked a couple of methods and their applications. For instance only one application of NPBNs was mentioned, without any specification of the methodology applied. We would like to complement their presentation of NPBN applications.
We aim to provide a short introduction to the NPBNs, present a couple of theoretical refinements and an overview of their use in practice. Moreover a number of difficulties and challenges occur when working with HBNs in general, and with NPBNs in particular. We address these challenges from the practitioner׳s perspective, and in this way we provide guidance in using the NPBN framework.
The remainder of the paper is organized as follows. Section 2 provides some background on methods available for HBNs. Section 3 describes the NPBN model and gives details about quantification and inference using NPBNs. The main theorem of NPBNs is reformulated and proved for more general settings. Two NPBN structure learning algorithms are detailed. The first one represents the initially proposed algorithm used in most of the applications presented later in the paper, whereas the second one is a new proposal that shows promise. NPBNs have been or are currently being used in at least twelve professional applications. Section 4 reviews these applications. Two of the applications are described in more detail with the intention of presenting some more theoretical details on real examples. The other applications are only very briefly presented, with more details given in the Appendix. Conclusions and directions for future research are given in Section 5.
Section snippets
Hybrid Bayesian networks
One of the first ideas in dealing with HBNs is the use of the conditional Gaussian model [51], [55], [37]. The price of the discrete-normal HBNs is the restriction (of the continuous part) to the joint normal distribution. Some of the exact algorithms for inference designed for discrete BNs were extended to discrete-normal HBNs. Approximation algorithms are also available (e.g. [38]).
Nevertheless uncertainty distributions need not conform to a particular parametric form. When the joint
Non-parametric Bayesian networks
The NPBN methodology was initially developed for purely continuous BNs. NPBNs associate nodes with random variables for which no marginal distribution assumption is made, and arcs with one-parameter conditional copulae [30], parameterized by Spearman׳s rank correlations. The (conditional) copulae are assigned to the arcs of the NPBN according to a protocol that depends on a (non-unique) ordering of the parent nodes. The main result of NPBNs states that a particular choice of conditional
Applications of NPBNs
This section introduces twelve of the largest projects that use the NPBN methodology. Nine of these applications are divided according to the different subjects they cover: risk analysis, reliability of structures, properties of materials. Two applications of temporal/dynamic NPBNs: one that tackles a parameter estimation problem, and another related to traffic prediction, are coupled in Section 4.4. The last part of this section presents three ongoing research projects. In all NPBNs, the
Conclusions
We have mentioned a number of approaches to deal with HBNs, amongst which MoTBFs, eBN, and NPBNs. This paper concentrates on the NPBN methodology, hence NPBNs were presented in more detail. The presentation tried to cover three aspects: the quantification of the models, the inference, and the real-life applications. Unlike other approaches, the NPBN methodology does not build on classical BN methods, but rather proposes a new way of dealing with non-parametric continuous distributions in BNs.
References (68)
- et al.
Hybrid Bayesian network classifiersapplication to species distribution models
Environ Model Softw
(2010) - et al.
Analysis of the crash of tk 1951 using cats
Reliab Eng Syst Saf
(2010) - et al.
Mining and visualising ordinal data with non-parametric continuous BBNs
Comput Stat Data Anal
(2010) - et al.
Analysis of the Schiphol cell complex fire using a Bayesian belief net based model
Reliab Eng Syst Saf
(2012) - et al.
Mixtures of truncated basis functions
Int J Approx Reason
(2012) - et al.
Eliciting conditional and unconditional rank correlations from conditional probabilities
Reliab Eng Syst Saf
(2008) - et al.
Analysis of axle and vehicle load properties through Bayesian networks based on weigh-in-motion data
Eng Syst Saf
(2014) - Aanonsen S, Naedval G, Oliver D, Reynolds A, Valles B. The ensemble Kalman filter in reservoir engineering. SPE J,...
- Ale B, Bellamy L, Cooke R, Duyvis M, Kurowicka D, Lin P, et al. Causal model for air transport safety. Final Report...
- Aparicio A, Martínez-Austria P, Güitrón A, Ramírez A. Floods in Tabasco, Mexico: a diagnosis and proposal for courses...