Abstract
Electro- and magneto-encephalography are functional neuroimaging modalities characterised by their ability to quantify dynamic spatiotemporal activity within the brain. However, the visualisation techniques used to illustrate these effects are currently limited to single- or multi-channel time series plots, topographic scalp maps and orthographic cross-sections of the spatiotemporal data structure. Whilst these methods each have their own strength and weaknesses, they are only able to show a subset of the data and are suboptimal at articulating one or both of the space-time components.
Here, we propose Porthole and Stormcloud, a set of data visualisation tools which can automatically generate context appropriate graphics for both print and screen with the following graphical capabilities:
-
Animated two-dimensional scalp maps with dynamic timeline annotation and optional user interaction;
-
Three-dimensional construction of discrete clusters within sparse spatiotemporal volumes, rendered with ‘cloud-like’ appearance and augmented by cross-sectional scalp maps indicating local maxima.
These publicly available tools were designed specifically for visualisation of M/EEG spatiotemporal statistical parametric maps, however, we also demonstrate alternate use cases of posterior probability maps and weight maps produced by machine learning classifiers. In principle, the methods employed here are transferrable to visualisation of any spatiotemporal image.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bach, B., Dragicevic, P., Archambault, D., Hurter, C., & Carpendale, S. (2016). A descriptive framework for temporal data visualizations based on generalized space-time cubes. Computer Graphics Forum, 36(6), 36–61. https://doi.org/10.1111/cgf.12804.
Brett, M., Penny, W. D., & Kiebel, S. J. (2004). Introduction to random field theory. In R. S. J. Frackowiak, K. J. Friston, C. D. Frith, R. J. Dolan, C. J. Price, S. Zeki, et al. (Eds.), Human brain function (2nd ed., pp. 867–879). Burlington: Academic Press.
Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167. https://doi.org/10.1023/a:1009715923555.
Christen, M., Vitacco, D. A., Huber, L., Harboe, J., Fabrikant, S. I., & Brugger, P. (2013). Colorful brains: 14 years of display practice in functional neuroimaging. NeuroImage, 73, 30–39. https://doi.org/10.1016/j.neuroimage.2013.01.068.
Friston, K. J., Ashburner, J. T., Kiebel, S. J., Nichols, T. E., & Penny, W. D. (2011). Statistical parametric mapping: The analysis of functional brain images. London: Academic Press.
Fuchs, R., & Hauser, H. (2009). Visualization of multi-Variate scientific data. Computer Graphics Forum, 28(6), 1670–1690. https://doi.org/10.1111/j.1467-8659.2009.01429.x.
Garrido, M. I., Teng, C. L. J., Taylor, J. A., Rowe, E. G., & Mattingley, J. B. (2016). Surprise responses in the human brain demonstrate statistical learning under high concurrent cognitive demand. NPJ Science of Learning, 1, 16006. https://doi.org/10.1038/npjscilearn.2016.6.
Garrido, M. I., Rowe, E. G., Halász, V., & Mattingley, J. B. (2018). Bayesian mapping reveals that attention boosts neural responses to predicted and unpredicted stimuli. Cerebral Cortex, 28(5), 1771–1782. https://doi.org/10.1093/cercor/bhx087.
Hägerstraand, T. (1970). What about people in regional science? Papers in Regional Science, 24(1), 7–24. https://doi.org/10.1111/j.1435-5597.1970.tb01464.x.
Harris, C. D., Rowe, E. G., Randeniya, R., & Garrido, M. I. (2018). Bayesian model selection maps for group studies using M/EEG data. Frontiers in Neuroscience, 12, 598. https://doi.org/10.3389/fnins.2018.00598.
Hastie, T., Friedman, J., & Tibshirani, R. (2001). The elements of statistical learning: Data mining, inference and prediction (1st ed.). New York: Springer.
Haufe, S., Meinecke, F., Görgen, K., Dähne, S., Haynes, J.-D., Blankertz, B., & Bießmann, F. (2014). On the interpretation of weight vectors of linear models in multivariate neuroimaging. NeuroImage, 87, 96–110. https://doi.org/10.1016/j.neuroimage.2013.10.067.
Koles, Z. J., & Paranjape, R. B. (1988). Topographic mapping of the EEG: An examination of accuracy and precision. Brain Topography, 1(2), 87–95. https://doi.org/10.1007/BF01129173.
Kristensson, P. O., Dahlback, N., Anundi, D., Bjornstad, M., Gillberg, H., Haraldsson, J., et al. (2009). An evaluation of space time cube representation of spatiotemporal patterns. IEEE Transactions on Visualization and Computer Graphics, 15(4), 696–702. https://doi.org/10.1109/TVCG.2008.194.
Larsen, K. M., Mørup, M., Birknow, M. R., Fischer, E., Hulme, O., Vangkilde, A., Schmock, H., Baaré, W. F. C., Didriksen, M., Olsen, L., Werge, T., Siebner, H. R., & Garrido, M. I. (2018). Altered auditory processing and effective connectivity in 22q11.2 deletion syndrome. Schizophrenia Research, 197, 328–336. https://doi.org/10.1016/j.schres.2018.01.026.
Litvak, V., Mattout, J., Kiebel, S., Phillips, C., Henson, R., Kilner, J., Barnes, G., Oostenveld, R., Daunizeau, J., Flandin, G., Penny, W., & Friston, K. (2011). EEG and MEG data analysis in SPM8. Computational Intelligence and Neuroscience, 2011, 852961. https://doi.org/10.1155/2011/852961.
Näätänen, R. (1990). The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. Behavioral and Brain Sciences, 13(2), 201–233. https://doi.org/10.1017/S0140525X00078407.
Olah, C., & Carter, S. (2017). Research debt. Distill. https://doi.org/10.23915/distill.00005.
Oostenveld, R., & Praamstra, P. (2001). The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology, 112(4), 713–719. https://doi.org/10.1016/S1388-2457(00)00527-7.
Poline, J., Kherif, F., Pallier, C., & Penny, W. D. (2007). Contrasts and classical inference. In W. D. Penny, K. J. Friston, J. T. Ashburner, S. J. Kiebel, & T. E. Nichols (Eds.), Statistical parametric mapping: The analysis of functional brain images (pp. 126–139). Amsterdam: Elsevier Science.
Rosa, M. J., Bestmann, S., Harrison, L., & Penny, W. (2010). Bayesian model selection maps for group studies. NeuroImage, 49(1–3), 217–224. https://doi.org/10.1016/j.neuroimage.2009.08.051.
Schölkopf, B., & Smola, A. J. (2000). Learning with kernels. Cambridge, MA: The MIT Press.
Schrouff, J., & Mourão-Miranda, J. (2018, June 12–14). Interpreting weight maps in terms of cognitive or clinical neuroscience: Nonsense? In 2018 international workshop on pattern recognition in neuroimaging (PRNI) (pp. 1–4). https://doi.org/10.1109/PRNI.2018.8423944.
Schrouff, J., Rosa, M. J., Rondina, J. M., Marquand, A. F., Chu, C., Ashburner, J., Phillips, C., Richiardi, J., & Mourão-Miranda, J. (2013). PRoNTo: Pattern recognition for neuroimaging toolbox. Neuroinformatics, 11(3), 319–337. https://doi.org/10.1007/s12021-013-9178-1.
Timmermann, C., Spriggs, M. J., Kaelen, M., Leech, R., Nutt, D. J., Moran, R. J., et al. (2017). LSD modulates effective connectivity and neural adaptation mechanisms in an auditory oddball paradigm. Neuropharmacology. https://doi.org/10.1016/j.neuropharm.2017.10.039.
Tufte, E. R. (2001). The visual display of quantitative information (2nd ed.). Cheshire: Graphics Press.
Worsley, K. J. (1995). Estimating the number of peaks in a random field using the Hadwiger characteristic of excursion sets, with applications to medical images. The Annals of Statistics, 23(2), 640–669.
Worsley, K. J. (1996). The geometry of random images. Chance, 9(1), 27–40.
Acknowledgements
This work was supported by the Australian Research Council Centre of Excellence for Integrative Brain Function (ARC Centre Grant CE140100007), a University of Queensland Fellowship (2016000071) and a Foundation Research Excellence Award (2016001844) to MIG. We would like to thank Tyler Hobson for discussions on computer graphics methods, Clare Harris for providing data, as well as Veronika Halász, Kit Melissa Larsen, Ilvana Dzafic, Jessica McFadyen and Chase Sherwell for providing feedback on the functionality of earlier versions of the toolbox.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare no competing financial interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A
Graphical user interface for specifying Porthole animation parameters and metadata for annotating the display
Appendix B
Graphical user interface for specifying Stormcloud preferences for volume orientation and cluster annotation
Rights and permissions
About this article
Cite this article
Taylor, J.A., Garrido, M.I. Porthole and Stormcloud: Tools for Visualisation of Spatiotemporal M/EEG Statistics. Neuroinform 18, 351–363 (2020). https://doi.org/10.1007/s12021-019-09447-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12021-019-09447-6