Elsevier

NeuroImage

Volume 56, Issue 3, 1 June 2011, Pages 907-922
NeuroImage

A Bayesian model of shape and appearance for subcortical brain segmentation

https://doi.org/10.1016/j.neuroimage.2011.02.046Get rights and content

Abstract

Automatic segmentation of subcortical structures in human brain MR images is an important but difficult task due to poor and variable intensity contrast. Clear, well-defined intensity features are absent in many places along typical structure boundaries and so extra information is required to achieve successful segmentation. A method is proposed here that uses manually labelled image data to provide anatomical training information. It utilises the principles of the Active Shape and Appearance Models but places them within a Bayesian framework, allowing probabilistic relationships between shape and intensity to be fully exploited. The model is trained for 15 different subcortical structures using 336 manually-labelled T1-weighted MR images. Using the Bayesian approach, conditional probabilities can be calculated easily and efficiently, avoiding technical problems of ill-conditioned covariance matrices, even with weak priors, and eliminating the need for fitting extra empirical scaling parameters, as is required in standard Active Appearance Models. Furthermore, differences in boundary vertex locations provide a direct, purely local measure of geometric change in structure between groups that, unlike voxel-based morphometry, is not dependent on tissue classification methods or arbitrary smoothing. In this paper the fully-automated segmentation method is presented and assessed both quantitatively, using Leave-One-Out testing on the 336 training images, and qualitatively, using an independent clinical dataset involving Alzheimer's disease. Median Dice overlaps between 0.7 and 0.9 are obtained with this method, which is comparable or better than other automated methods. An implementation of this method, called FIRST, is currently distributed with the freely-available FSL package.

Research Highlights

► Automated segmentation of 15 subcortical structures with full Bayesian formulation. ► Vertex-analysis for detecting local geometric changes with no arbitrary smoothing. ► Good performance over a wide range of demographics and T1-weighted images. ► Avoids arbitrary scaling parameters and ill-conditioning in usual appearance models. ► Freely available as part of FSL — tool is called FIRST.

Introduction

It is important that medical image segmentation methods are accurate and robust in order to sensitively study both normal and pathological brains. Achieving this in the subcortical areas of the brain, given the typical low contrast-to-noise, is a great challenge for automated methods. When trained human specialists perform manual segmentations they draw on prior knowledge of shape, image intensities and shape-to-shape relationships. We present here a formulation of a computationally efficient shape and appearance model based on a Bayesian framework that incorporates both intra- and inter-structure variability information, while also taking account of the limited size of the training set with respect to the dimensionality of the data. The method is capable of performing segmentations of individual or multiple subcortical structures as well as analysing differences in shape between different groups, showing the location of changes in these structures, rather than just changes in the overall volume.

The Active Shape Model (ASM) is an automated segmentation method that has been widely used in the field of machine vision and medical image segmentation over the past decade (Cootes et al., 1995). Standard ASMs model the distribution of corresponding anatomical points (vertices/control points) and then parameterize the mean shape and most likely variations of this shape across a training set. Images are segmented using the model built from the training data, which specifies the range of likely shapes. In the original formulation, if the dimensionality of the shape representation exceeds the size of the training data then the only permissible shapes are linear combinations of the original training data, although some methods for generalising this have been presented in the literature (Heimann and Meinzer, 2009).

Intensity models are also useful in segmentation, and the Active Appearance Model (AAM) is an extension of the ASM framework that incorporates such intensity information (Cootes et al., 1998). As with the standard shape model, the intensity distribution is modelled as a multivariate Gaussian and is parameterized by its mean and eigenvectors (modes of variation). The AAM relates the shape and intensity models to each other with a weighting matrix estimated from the training set. Fitting shapes to new images is done by minimising the squared difference between the predicted intensities, given a shape deformation, and the observed image intensities. Again, many modifications of this basic formulation have also been proposed (Heimann and Meinzer, 2009).

In addition to the ASM and AAM methods there are many other approaches taken by fully-automated segmentation methods for subcortical structures. Some of these methods are specific to particular structures (e.g. hippocampus), others can be applied to general structures and still others can be applied to multiple structures simultaneously. The approaches can be surface-based, volumetric-based or both, and utilise methods such as: region competition (Chupin et al., 2007, Chupin et al., 2009); homotopic region deformation (Lehéricy et al., 2009); level-sets within a Bayesian framework (Cremers et al., 2006) or with local distribution models (Yan et al., 2004); 4D shape priors (Kohlberger et al., 2006); probabilistic boosting trees (Wels et al., 2008); label, or classifier, fusion (Heckemann et al., 2006); label fusion with templates (Collins and Pruessner, 2010); label fusion with graph cuts (Wolz et al., 2010); wavelets with ASM (Davatzikos et al., 2003); multivariate discriminant methods (Arzhaeva et al., 2006); medial representations or deformable M-reps (Levy et al., 2007, Styner et al., 2003); probabilistic boosting trees (Tu et al., 2008); large diffeomorphic mapping (Lee et al., 2009b); and non-linear registration combined with AAM (Babalola et al., 2007).

The most common volumetric-based approaches to segmentation are based on non-linear warping of an atlas, or atlases, to new data (Collins and Evans, 1997, Fischl et al., 2002, Pohl et al., 2006). Traditionally, a single average atlas has been used to define the structure segmentations (as in (Collins and Evans, 1997, Gouttard et al., 2007)) whereas recent methods (Gousias et al., 2008, Heckemann et al., 2006) propagate information from multiple atlases and fuse the results. Additional information such as voxel-wise intensity and shape priors can also be utilised (Fischl et al., 2002, Khan et al., 2008). When using a single atlas, only a very limited amount of information on shape variation from the training data can be retained. In place of this shape information, registration methods define the likelihood of a given shape via the space of allowable transformations and regularisation-based penalization applied to them. This potentially biases the segmented shapes to favour smooth variations about the average template. Alternatively, methods that use multiple atlases or additional voxel-wise shape priors are able to retain more variational information from the training data.

Surface-based methods, on the other hand, tend to explicitly use learned shape variation as a prior in the segmentation (Colliot et al., 2006, Pitiot et al., 2004, Tsai et al., 2004). In brain image segmentation various ways of representing shapes and relationships have been proposed, including fuzzy models (Colliot et al., 2006), level-sets (Tsai et al., 2004), and simplex meshes (Pitiot et al., 2004). In addition, an array of different approaches has been taken to couple the intensities in the image to the shape, usually in the form of energies and/or forces, which often require arbitrary weighting parameters to be set.

Our approach takes the deformable-model-based AAM and poses it in a Bayesian framework. This framework is advantageous as it naturally allows probability relationships between shapes of different structures and between shape and intensity to be utilised and investigated, while also accounting for the limited amount of training data in a natural way. It is still based on using a deformable model that restricts the topology (unlike level-sets or voxel-wise priors), which is advantageous since the brain structures we are interested in have a fixed topology, as confirmed by our training data. Another benefit of the deformable model is that point correspondence between structures is maintained. This allows vertex-wise structural changes to be detected between groups of subjects, facilitating investigations of normal and pathological variations in the brain. Moreover, this type of analysis is purely local, based directly on the geometry/location of the structure boundary and is not dependent on tissue-type classification or smoothing extents, unlike voxel-based morphometry methods.

One difficulty of working with standard shape and appearance models is the limited amount and quality of training data (Heimann and Meinzer, 2009). This means that the models cannot represent variations in shape and intensity that are not explicitly present in the training data, and that leads to restrictions in permissible shapes, and difficulties in establishing robust shape–intensity relationships. The problem is particularly acute when the number of training sets is substantially less than the dimensionality of the model (number of vertices times number of intensity samples per vertex) which is certainly the case in this application (e.g., we have 336 training sets, but models with 10,000 or more parameters). Although a number of approaches have been proposed to alleviate these problems, we find that both of these problems are dealt with automatically by formulating the model in a Bayesian framework. For example, one approach for removing shape restrictions that has been proposed previously (Cremers et al., 2002) requires the addition of a regularisation term in the shape covariance matrix, and we find that this same term arises naturally in our Bayesian formulation.

Using the AAM in a Bayesian framework also eliminates the need for arbitrary empirical weightings between intensity and shape. This is due to the use of conditional probabilities (e.g., probability of shape conditional on intensity), which underpin the method and can be calculated extremely efficiently, without any additional regularisation required. These conditional probabilities also allow the expected intensity distribution to change with the proposed shape; see Fig. 3 for an example of why this is important. Furthermore, this conditional probability formulation is very general and can be used to relate any subparts of the model (e.g., different shapes). Therefore, the method proposed in this paper cannot only be used to model and segment each structure independently, it can also be used in more flexible ways that incorporate joint shape information.

The following sections of this paper explain the details of the Bayesian Appearance Model (BAM), including our training set, provide validation experiments, and give an example application of vertex analysis for finding structural changes between disease and control cohorts.

Section snippets

Training data

The training data used in this work consists of 336 pairs of images: the original T1-weighted MR images of the brain and their corresponding manually-labelled counterparts. This dataset comprises six distinct groups of data and spans both normal and pathological brains (including cases of schizophrenia and Alzheimer's disease). The size, age, group, and resolution for each group is given in Table 1, and the T1-weighted image and manual labels of a single subject from the training set are

Probabilistic model

Our model is trained from a limited set of mesh vertices and intensity samples that correspond to the limited set of volumetric training data. We treat all training data as column vectors, either of concatenated vertex coordinates or coordinates and corresponding intensity samples. For example, for a 2D rectangle parameterized by the corner vertices V={(2,0),(2,3),(3,3),(3,0)} the training vector for shape alone would be xi = [− 2 0  2 3 3 3 3 0]T. It is essential that vertex correspondence (consistent

Bayesian appearance models

To formulate the Bayesian Appearance Model we take the general mathematical framework developed in the previous sections and apply it to intensity and shape partitions: xI and xs. The shape partition is modelled using Eq. (10), so that given the intensities from a new image, the vector bs (new shape instance) can be estimated from the posterior conditional distribution p(xs|xI,Z).

In the following sections we describe how the model is fit to new data. As discussed earlier, when fitting to the

Vertex analysis

The output from a subcortical segmentation method can be used in many ways; one application is to look for differences in these structures between different groups (e.g., disease versus healthy controls). Many such group difference studies have been carried out based on volumetric measures of the structures of interest (e.g., caudate, hippocampus, etc.). However, volumetric studies do not show where the changes are occurring in the structure, and this may be of critical importance when

Results and discussion

In this section we test the Bayesian Appearance Model both qualitatively and quantitatively on real MRI data. This includes a set of leave-one-out (LOO) cross-validation tests comparing volumetric overlap with the manual labelled images from the training set. In addition, vertex analysis is tested by comparing cohorts of patients with matched healthy controls.

All the results shown here use a C++ implementation of the Bayesian Appearance Model that is distributed as part of FSL (Woolrich et al.,

Conclusion

In this paper a Bayesian Appearance Model is proposed that incorporates both shape and intensity information from a training set. The idea is similar to that of the Active Appearance Model except that it uses a probabilistic framework to estimate the relationship between shape and intensity and makes extensive use of conditional probabilities. These probabilities are well-conditioned due to the priors, which are specified empirically, but are easy to set and only have a small influence on the

Acknowledgments

The authors wish to thank Dr Mojtaba Zarei for very helpful discussions about Alzheimer's Disease, as well as the UK EPSRC IBIM Grant and the UK BBSRC David Phillips Fellowship for funding this research. In addition, the authors extend thanks to all those involved in contributing data for this project: Christian Haselgrove, Centre for Morphometric Analysis, Harvard; Bruce Fischl, the Martinos Center for Biomedical Imaging, MGH (NIH grants P41-RR14075, R01 RR16594-01A1, and R01 NS052585-01);

References (87)

  • V. Fonov et al.

    Unbiased average age-appropriate atlases for pediatric studies

    Neuroimage

    (2011)
  • B. Franke et al.

    Genetic variation in CACNA1C, a gene associated with bipolar disorder, influences brainstem rather than gray matter volume in healthy individuals

    Biol. Psychiatry

    (2010)
  • K. Friston et al.

    Classical and Bayesian inference in neuroimaging: theory

    Neuroimage

    (2002)
  • C.R. Genovese et al.

    Thresholding of statistical maps in functional neuroimaging using the false discovery rate

    Neuroimage

    (2002)
  • E. Gerardin et al.

    Multidimensional classification of hippocampal shape features discriminates Alzheimer's disease and mild cognitive impairment from normal aging

    Neuroimage

    (2009)
  • I. Gousias et al.

    Automatic segmentation of brain MRIs of 2-year-olds into 83 regions of interest

    Neuroimage

    (2008)
  • R. Heckemann et al.

    Automatic anatomical brain MRI segmentation combining label propagation and decision fusion

    Neuroimage

    (2006)
  • T. Heimann et al.

    Statistical shape models for 3D medical image segmentation: a review

    Med. Image Anal.

    (2009)
  • E. Hermans et al.

    Effects of exogenous testosterone on the ventral striatal BOLD response during reward anticipation in healthy women

    Neuroimage

    (2010)
  • M. Jenkinson et al.

    Improved optimisation for the robust and accurate linear registration and motion correction of brain images

    Neuroimage

    (2002)
  • A. Khan et al.

    FreeSurfer-initiated fully-automated subcortical brain segmentation in MRI using large deformation diffeomorphic metric mapping

    Neuroimage

    (2008)
  • N. Lee et al.

    Improved segmentation of hippocampus using landmark based large deformation diffeomorphic mapping

    Neuroimage

    (2009)
  • R. Menke et al.

    MRI characteristics of the substantia nigra in Parkinson's disease: a combined quantitative T1 and DTI study

    Neuroimage

    (2009)
  • O. Meulenbroek et al.

    Autobiographical memory retrieval in patients with Alzheimer's disease

    Neuroimage

    (2010)
  • J. Montagnat et al.

    A review of deformable surfaces: topology, geometry and deformation

    Image Vis. Comput.

    (2001)
  • R. Morey et al.

    A comparison of automated segmentation and manual tracing for quantifying hippocampal and amygdala volumes

    Neuroimage

    (2009)
  • A. Pitiot et al.

    Expert knowledge-guided segmentation system for brain MRI

    Neuroimage

    (2004)
  • A. Qiu et al.

    Regional shape abnormalities in mild cognitive impairment and Alzheimer's disease

    Neuroimage

    (2009)
  • F. Sabattoli et al.

    Hippocampal shape differences in dementia with Lewy bodies

    Neuroimage

    (2008)
  • A. Scher et al.

    Hippocampal shape analysis in Alzheimer's disease: a population-based study

    Neuroimage

    (2007)
  • I. Seror et al.

    Putaminal volume and diffusion in early familial Creutzfeldt–Jakob Disease

    J. Neurol. Sci.

    (2010)
  • S. Sotiropoulos et al.

    Brain tractography using Q-ball imaging and graph theory: improved connectivities through fibre crossings via a model-based approach

    Neuroimage

    (2010)
  • M. Styner et al.

    Statistical shape analysis of neuroanatomical structures based on medial models

    Med. Image Anal.

    (2003)
  • A. Tsai et al.

    Mutual information in coupled multi-shape model for medical image segmentation

    Med. Image Anal.

    (2004)
  • R. Wolz et al.

    Measurement of hippocampal atrophy using 4D graph-cut segmentation: application to ADNI

    Neuroimage

    (2010)
  • M. Woolrich et al.

    Bayesian analysis of neuroimaging data in FSL

    Neuroimage

    (2009)
  • Y. Xu et al.

    Age effects on hippocampal structural changes in old men: the HAAS

    Neuroimage

    (2008)
  • M. Zarei et al.

    Combining shape and connectivity analysis: an MRI study of thalamic degeneration in Alzheimer's disease

    Neuroimage

    (2010)
  • F. Agosta et al.

    Sensorimotor network rewiring in mild cognitive impairment and Alzheimer's disease

    Hum. Brain Mapp.

    (2010)
  • Y. Arzhaeva et al.

    Image classification from generalized image distance features: application to detection of interstitial disease in chest radiographs

  • K. Babalola et al.

    Automatic segmentation of the caudate nuclei using active appearance models

  • K. Babalola et al.

    Comparison and evaluation of segmentation techniques for subcortical structures in brain MRI

  • Y. Benjamini et al.

    Controlling the false discovery rate: a practical and powerful approach to multiple testing

    R. Stat. Soc. B

    (1995)
  • Cited by (1845)

    View all citing articles on Scopus
    View full text