Technical NoteActivation likelihood estimation meta-analysis revisited
Highlights
► The permutation procedure of ALE is replaced by a faster and more accurate approach. ► Family-wise error correction and cluster-level inference are introduced into ALE. ► The current and revised implementation of ALE yields comparable results.
Introduction
Over the last decades, neuroimaging research has produced a vast amount of data localising the neural effects of cognitive and sensory processes in the brain of both healthy and diseased populations. In spite of their power to delineate the functional organisation of the human brain, however, neuroimaging also carries several limitations. The most important among these are the rather small sample sizes investigated, the consequently low reliability (Raemaekers et al., 2007) and the inherent subtraction logic which is only sensitive to differences between conditions (Price et al., 2005). Consequently, pooling data from different experiments, which investigate similar questions but employ variations of the experimental design, has become an important task. Such meta-analyses allow the identification of brain regions' locations that show a consistent response across experiments, collectively involving hundreds of subjects and numerous implementations of a particular paradigm (Laird et al., 2009a, Laird et al., 2009b). Community-wide standards of spatial normalisation and the reporting of peak activation locations in stereotaxic coordinates allow researchers to compare results across experiments when the primary data are unavailable or difficult to obtain (Poldrack et al., 2008).
Activation likelihood estimation (ALE; Laird et al., 2005, Turkeltaub et al., 2002) is probably the most common algorithm for coordinate-based meta-analyses (informative review see Wager et al., 2007b). The ALE algorithm is readily available to the neuroimaging community in form of the GingerALE desktop application (http://brainmap.org/ale). This approach treats activation foci reported in neuroimaging studies not as single points but as spatial probability distributions centred at the given coordinates. ALE maps are then obtained by computing the union of activation probabilities for each voxel. As in other algorithms for quantitative meta-analysis, the differentiation between true convergence of foci and random clustering (i.e., noise) is tested by a permutation procedure (Nichols and Hayasaka, 2003). Recently, we have proposed a revised algorithm for ALE analysis (Eickhoff et al., 2009), which models the spatial uncertainty – and thus probability distribution – of each focus using an estimation of the inter-subject and inter-laboratory variability typically observed in neuroimaging experiments, rather than using a pre-specified full-width half maximum (FWHM) for all experiments as originally proposed. In addition, it limits the meta-analysis to an anatomically constrained space specified by a grey matter mask and includes a new method of inference that calculates the above-chance clustering between experiments (i.e., random-effects analysis), rather than between foci (i.e., fixed-effects analysis).
An alternative approach to coordinate-based meta-analysis is kernel density analysis (KDA (Wager and Smith, 2003)). Both algorithms (KDE and ALE) are based on the idea of delineating those locations in the brain where the coordinates reported for a particular paradigm or comparison show an above-chance convergence. However, whereas ALE investigates where the location probabilities reflecting the spatial uncertainty associated with the foci of each experiment overlap in different voxels, KDE tests how many foci are reported close to any individual voxel. Recently, an algorithm for random-effects (RDFX) inference on KDE (termed multi-level kernel density estimation, MKDE) has been proposed (Wager et al., 2007b) which rests on a similar concept as the new random effects approach for ALE meta-analyses (Eickhoff et al., 2009). Both are based on summarising all foci reported for any given study in a single image [the “modelled activation” (MA) map in ALE and “comparison indicator maps” (CIM) in MKDE]. These are then combined across studies, and inference is subsequently sought on those voxels where MA maps (ALE) or CIMs (MKDE) overlap stronger as would be expected if there were a random spatial arrangement, i.e., no correspondence between studies.
The null-distributions for this inference on spatially continuous statistical maps computed by non-linear operations are estimated in both algorithms by using permutation procedures. More precisely, MDKE randomly redistributes the cluster centres throughout the grey matter of the brain, performs the same analysis as computed for the real data and uses the ensuing peak heights to derive FWE corrected voxel-level thresholds. This approach to statistical inference in voxel-wise meta-analysis data has the major advantage that the estimated null-distribution will reflect the spatial continuity of the statistical field of interest without requiring an exact parameterisation of the (non-linear) nature of its properties. That is, algorithms based on random relocation of foci within each experiment, generation of summary images per experiment and quantification of the convergence across these may empirically provide a good estimation on the distribution of statistical features of interest such as cluster size above a given threshold or maximum peak height (Wager et al., 2007b). Here we use this approach to derive a null-distribution of these two measures against which the results of the performed ALE analysis can then be compared for providing FWE or cluster-level corrected statistical inference.
A new approach to coordinate-based meta-analysis has very recently been proposed as signed difference map analysis (SDM; Radua et al., 2010, Radua and Mataix-Cols, 2009). SDM sums the voxel-wise activation probabilities of foci modelled as 3D Gaussian distributions like ALE, instead of counting closely activating experiments like MKDE. As opposed to ALE and MKDE, SDM emphasises foci that were derived from conservatively corrected analyses. Similar to MKDE, it avoids too high probability values through neighbouring foci in a same experiment by limiting maximum values. This feature has also very recently been introduced to ALE (Turkeltaub et al., in press) and was incorporated in the present work. Another novel feature of SDM consists in holding positive and negative values in a same map which prevents spurious overlap between those two categories of localization information rarely occurring in ALE. Analogous to MKDE and unrevised ALE implementations, significant convergence is distinguished from noise by computing a whole-brain null-distribution using a permutation procedure. Finally, SDM corrects results by FDR, unlike contemporary variants of ALE and MKDE. Taken together, ALE, MKDE and SDM all represent suitable methods for coordinate-based meta-analysis.
In the present report, we will address two remaining drawbacks of the widely used ALE algorithm. First, the null-distribution for statistical inference, reflecting a random spatial association between experiments is currently based on a permutation procedure. This approach, which has been part of all meta-analysis algorithms proposed up to now, however, has two disadvantages. First, drawing a sufficient estimate of the null-distribution may be rather time-consuming, given that a large number of permutations are required to sufficiently reflect the possible associations between experiments. If the test is underpowered, however, experimental ALE-values may exceed those observed under the null-distribution, indicating an insufficient estimation of its upper tail. Second, statistical inference on the ensuing p- or Z-maps is currently based on either uncorrected thresholds or correction for multiple comparisons using the false discovery rate (FDR) approach (Genovese et al., 2002). Whilst using uncorrected thresholds provides no protection against false positives in a situation of multiple comparisons, FDR is likewise not the optimal approach. It has rather been noted that in cases where the underlying signal is continuous (such as in neuroimaging meta-analyses), controlling the false discovery rate is not equivalent to controlling the false discovery rate of activations (Chumbley and Friston, 2009). FDR corrected inference is therefore not appropriate for inferences on the topological features (regions of activation) of a statistical map as derived from ALE meta-analysis. Finally, in order to avoid spurious clusters consisting of only a few voxels, both of these procedures are commonly combined with an (arbitrary) extent threshold, suppressing clusters that are smaller than, e.g., 50 contiguous supra-threshold voxels. However, this subjective approach neither corresponds to statistical testing nor allows inference on the significance of regional activations. To overcome these limitations and to provide a more valid framework for ALE meta-analyses, we here present an analytical approach for deriving the null-distribution reflecting a random spatial association between experiments and propose algorithms for family-wise error correction and cluster-level inference on ALE data.
Section snippets
Objective
Activation likelihood estimation (ALE) meta-analysis aims at determining above-chance convergence of activation probabilities between experiments (i.e., not between foci). To this end, ALE seeks to refute the null-hypothesis that the foci of experiments are spread uniformly throughout the brain. More specifically, ALE delineates where in the brain the convergence across all included imaging studies is higher than it would be expected if results were independently distributed (Eickhoff et al.,
Revised approach for computing the null-distribution
The analytical null-distributions for inference on both datasets were compared to those derived from the random sampling algorithm described by Eickhoff et al. (2009). For the latter approach we used between 106 and 1012 random samples. One of the most dramatic differences pertained to the computation time needed to compute the null-distribution. For the face perception dataset, 106 samples were computed in about a minute, 108 samples in about 30 min and 1010 samples about 24 h whilst 1012
Discussion
Here we outlined a revision of the activation likelihood estimation (ALE) algorithm for coordinate-based meta-analyses of neuroimaging experiments that address two potential shortcomings of the current implementation of this approach. These pertain to how the null-distribution reflecting the expected ALE values under the assumption of spatial independence is computed and to the methods for correcting the statistical inference for multiple comparisons. In summary, we demonstrated in an
Acknowledgments
We acknowledge funding by the Human Brain Project (R01-MH074457-01A1; PTF, ARL, SBE), the DFG (IRTG 1328; SBE, DB) and the Helmholtz Initiative on Systems-Biology “The Human Brain Model” (SBE).
References (49)
- et al.
Voxel-based morphometry — the methods
Neuroimage
(2000) - et al.
Processing the socially relevant parts of faces
Brain Res. Bull.
(2007) - et al.
Attention does not modulate neural responses to social stimuli in autism spectrum disorders
Neuroimage
(2006) - et al.
The influence of encoding strategy on episodic memory and cortical activity in schizophrenia
Biol. Psychiatry
(2005) - et al.
Direct comparison of prefrontal cortex regions engaged by working and long-term memory tasks
Neuroimage
(2001) - et al.
Facial expressions and complex IAPS pictures: common and differential networks
Neuroimage
(2006) - et al.
False discovery rate revisited: FDR and topological inference using Gaussian random fields
Neuroimage
(2009) - et al.
Cortical and subcortical brain effects of transcranial magnetic stimulation (TMS)-induced movement: an interleaved TMS/functional magnetic resonance imaging study
Biol. Psychiatry
(2005) - et al.
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Neuroimage
(2002) - et al.
Eccentricity bias as an organizing principle for human high-order object areas
Neuron
(2002)
Increased medial temporal lobe activation during the passive viewing of emotional and neutral facial expressions in schizophrenia
Schizophr. Res.
Neural substrates of facial emotion processing using fMRI
Brain Res. Cogn. Brain Res.
Face perception is modulated by sexual preference
Curr. Biol.
Neural correlates of rapid reversal learning in a simple model of human social interaction
Neuroimage
Guidelines for reporting an fMRI study
Neuroimage
Test-retest reliability of fMRI activation during prosaccades and antisaccades
Neuroimage
Towards a nosology for frontotemporal lobar degenerations-a meta-analysis involving 267 subjects
Neuroimage
Meta-analysis of the functional neuroanatomy of single-word reading: method and validation
Neuroimage
Effects of attention and emotion on face processing in the human brain: an event-related fMRI study
Neuron
Why are smiles contagious? An fMRI study of the interaction between perception of facial affect and facial movements
Psychiatry Res.
Differential amygdala responses to happy and fearful facial expressions depend on selective attention
Neuroimage
Controlling the false discovery rate-a practical and powerful approach to multiple testing
J. R. Stat. Soc. B. Methodol.
Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders
Nat. Neurosci.
Brain systems mediating cognitive interference by emotional distraction
J. Neurosci.
Cited by (1052)
Heterarchy or hierarchy? Insights from a new model of visual imagination
2024, Physics of Life ReviewsThe common and distinct brain basis associated with adult and adolescent risk-taking behavior: Evidence from the neuroimaging meta-analysis
2024, Neuroscience and Biobehavioral ReviewsNeural correlates of willingness to pay for items: A meta-analysis of functional magnetic resonance imaging studies
2024, Physiology and BehaviorLabel-based meta-analysis of functional brain dysconnectivity across mood and psychotic disorders
2024, Progress in Neuro-Psychopharmacology and Biological PsychiatryThe extended neural architecture of human attachment: An fMRI coordinate-based meta-analysis of affiliative studies
2024, Neuroscience and Biobehavioral Reviews