Abstract
Memory plays an important role in human life. Memory can be divided into two categories, i.e., long term memory and short term memory (STM). STM or working memory (WM) stores information for a short span of time and it is used for information manipulations and fast response activities. WM is generally involved in the higher cognitive functions of the brain. Different studies have been carried out by researchers to understand the WM process. Most of these studies were based on neuroimaging modalities like fMRI, EEG, MEG etc., which use standalone processes. Each neuroimaging modality has some pros and cons. For example, EEG gives high temporal resolution but poor spatial resolution. On the other hand, the fMRI results have a high spatial resolution but poor temporal resolution. For a more in depth understanding and insight of what is happening inside the human brain during the WM process or during cognitive tasks, high spatial as well as high temporal resolution is desirable. Over the past decade, researchers have been working to combine different modalities to achieve a high spatial and temporal resolution at the same time. Developments of MRI compatible EEG equipment in recent times have enabled researchers to combine EEG-fMRI successfully. The research publications in simultaneous EEG-fMRI have been increasing tremendously. This review is focused on the WM research involving simultaneous EEG-fMRI data acquisition and analysis. We have covered the simultaneous EEG-fMRI application in WM and data processing. Also, it adds to potential fusion methods which can be used for simultaneous EEG-fMRI for WM and cognitive tasks.
Similar content being viewed by others
References
Schmorrow DD, Fidopiastis CM (2011) Foundations of augmented cognition. Directing the future of adaptive systems: 6th International conference, FAC 2011, held as part of HCI/lecture notes in artificial intelligence. Springer, New York
Park Y, Luo L, Parhi KK, Netoff T (2011) Seizure prediction with spectral power of EEG using cost-sensitive support vector machines. Epilepsia 52:1761–1770
Amin HU, Malik AS, Ahmad RF, Badruddin N, Kamel N, Hussain M et al (2015) Feature extraction and classification for EEG signals using wavelet transform and machine learning techniques. Australas Phys Eng Sci Med 38:1–11
Hsu W-Y (2011) Continuous EEG signal analysis for asynchronous BCI application. Int J Neural Syst 21:335–350
Amin HU, Malik AS, Kamel N, Chooi W-T, Hussain M (2015) P300 correlates with learning & memory abilities and fluid intelligence. J Neuroeng Rehabil 12:1
Lopes da Silva F (2013) EEG and MEG: relevance to neuroscience. Neuron 80:1112–1128
Friston KJ (1994) Functional and effective connectivity in neuroimaging: a synthesis. Hum Brain Mapp 2:56–78
Chang C, Liu Z, Chen MC, Liu X, Duyn JH (2013) EEG correlates of time-varying BOLD functional connectivity. NeuroImage 72:227–236
Schoonheim MM, Geurts JJ, Landi D, Douw L, van der Meer ML, Vrenken H et al (2013) Functional connectivity changes in multiple sclerosis patients: a graph analytical study of MEG resting state data. Hum Brain Mapp 34:52–61
Dong L, Gong D, Valdes-Sosa PA, Xia Y, Luo C, Xu P et al (2014) Simultaneous EEG-fMRI: trial level spatio-temporal fusion for hierarchically reliable information discovery. NeuroImage 99:28–41
He B, Liu Z (2008) Multimodal functional neuroimaging: integrating functional MRI and EEG/MEG. Biomed Eng, IEEE Rev 1:23–40
Yang L, Liu Z, He B (2010) EEG-fMRI reciprocal functional neuroimaging. Clin Neurophysiol 121:1240–1250
Analoui M, Bronzino JD, Peterson DR (2012) Medical imaging: principles and practices, CRC Press
Ogawa S, Tank DW, Menon R, Ellermann JM, Kim S-G, Merkle H et al (1992) Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. Proc Natl Acad Sci 89:5951–5955
Karmonik C, York M, Grossman R, Kakkar E, Patel K, Haykal H et al (2010) An image analysis pipeline for the semi-automated analysis of clinical fMRI images based on freely available software. Comput Biol Med 40:279–287
Wehenkel L, Maquet P, Phillips C, Schrouff J, Kusse C (2012) Decoding semi-constrained brain activity from fMRI using support vector machines and gaussian processes. PloS One 7:e35860
Faro SH, Mohamed FB (2010) BOLD fMRI: a guide to functional imaging for neuroscientists. Springer, New York
Luo Q, Glover GH (2012) Influence of dense-array EEG cap on fMRI signal. Magn Reson Med 68:807–815
Lemieux L (2012) EEG-correlated fMRI in epilepsy. Introd Epilepsy 43:266
Binder JR (2011) Functional MRI is a valid noninvasive alternative to Wada testing. Epilepsy Behav 20:214–222
Lindauer U, Dirnagl U, Füchtemeier M, Böttiger C, Offenhauser N, Leithner C et al (2010) Pathophysiological interference with neurovascular coupling - when imaging based on hemoglobin might go blind. Front Neuroenergetics. doi:10.3389/fnene.2010.00025
Tong S, Thakor NV (2009) Quantitative EEG analysis methods and clinical applications. Artech House, Boston
Sui J, Adali T, Yu Q, Chen J, Calhoun VD (2011) A review of multivariate methods for multimodal fusion of brain imaging data. J Neurosci Methods 204:68–81
Jorge J, van der Zwaag W, Figueiredo P (2013) EEG-fMRI integration for the study of human brain function. NeuroImage 102:24–34
Herman P, Sanganahalli BG, Blumenfeld H, Rothman DL, Hyder F (2013) Quantitative basis for neuroimaging of cortical laminae with calibrated functional MRI. Proc Natl Acad Sci 110:15115–15120
Stippich C, Sartor K (2010) Clinical functional MRI: presurgical functional neuroimaging. Springer, New York
de Souza ACS, Yehia HC, Sato M-A, Callan D (2013) Brain activity underlying auditory perceptual learning during short period training: simultaneous fMRI and EEG recording. BMC Neurosci 14:8
Ostwald D, Porcaro C, Bagshaw AP (2011) Voxel-wise information theoretic EEG-fMRI feature integration. NeuroImage 55:1270–1286
Novitskiy N, Ramautar JR, Vanderperren K, De Vos M, Mennes M, Mijovic B et al (2011) The BOLD correlates of the visual P1 and N1 in single-trial analysis of simultaneous EEG-fMRI recordings during a spatial detection task. NeuroImage 54:824–835
Musso F, Brinkmeyer J, Ecker D, London MK, Thieme G, Warbrick T et al (2011) Ketamine effects on brain function–simultaneous fMRI/EEG during a visual oddball task. NeuroImage 58:508–525
Meyer P, Hoppstaedter M, Baeuchl C, Zangl M, Diener C, Flor H (2011) The N400 reflects activity in the anterior medial temporal lobe (AMTL): simultaneous EEG-fMRI reveals the neural correlates of semantic priming. Psychophysiology 48:S52
Ahmad RF, Malik AS, Kamel N, Reza F (2015) Object categories specific brain activity classification with simultaneous EEG-fMRI. In: Engineering in medicine and biology society (EMBC), 2015 37th annual international conference of the IEEE, 2015, pp 1825–1828
Jorge J, van der Zwaag W, Figueiredo P (2014) EEG-fMRI integration for the study of human brain function. NeuroImage 102 Part 1:24–34
Debener S, De Vos M (2011) The benefits of simultaneous EEG-fMRI for EEG analysis. Clin Neurophysiol 122:217–218
Moeller F, Siniatchkin M, Gotman J (2013) Simultaneous EEG and fMRI recordings (EEG-fMRI). fMRI 30:269–281
De Martino F, Valente G, de Borst AW, Esposito F, Roebroeck A, Goebel R et al (2010) Multimodal imaging: an evaluation of univariate and multivariate methods for simultaneous EEG/fMRI. Magn Reson Imaging 28:1104–1112
Debener S, Ullsperger M, Siegel M, Engel AK (2006) *Single-trial EEG-fMRI reveals the dynamics of cognitive function. Trends Cogn Sci 10:558–563
D’Esposito M (2007) From cognitive to neural models of working memory. Philos Trans R Soc B 362:761–772
Motes MA, Rypma B (2010) Working memory component processes: isolating BOLD signal changes. NeuroImage 49:1933–1941
Diamantopoulou S, Poom L, Klaver P, Talsma D (2011) Visual working memory capacity and stimulus categories: a behavioral and electrophysiological investigation. Exp Brain Res 209:501–513
Fuster JM (1988) Prefrontal cortex. Springer, Birkhäuser Boston
Baddeley A (2010) Working memory. Curr Biol 20:R136–R140
Baddeley A (1992) working memory. Science 255:556–559
Rönnberg J, Rudner M, Ingvar M (2004) Neural correlates of working memory for sign language. Brain Res Cogn Brain Res 20:165–182
Olivers CNL (2008) Interactions between visual working memory and visual attention. Front Biosci J Virtual Libr 13:1182–1191
Nystrom LE, Braver TS, Sabb FW, Delgado MR, Noll DC, Cohen JD (2000) Working memory for letters, shapes, and locations: fMRI evidence against stimulus-based regional organization in human prefrontal cortex. NeuroImage 11:424–446
D’esposito M, (ed) (2008) Neuropsyhology and behavioral neurology (Handbook of Clinical Neurology 3)
Roux F, Uhlhaas PJ (2014) Working memory and neural oscillations: alpha–gamma versus theta–gamma codes for distinct WM information? Trends Cogn Sci 18:16–25
Zanto TP, Chadick JZ, Gazzaley A (2014) Anticipatory alpha phase influences visual working memory performance. NeuroImage 85:794–802
Robinson KE, Livesay KL, Campbell LK, Scaduto M, Cannistraci CJ, Anderson AW et al (2010) Working memory in survivors of childhood acute lymphocytic leukemia: functional neuroimaging analyses. Pediatr Blood Cancer 54:585–590
Munneke J, Heslenfeld DJ, Theeuwes J (2010) Spatial working memory effects in early visual cortex. Brain Cogn 72:368–377
Fusar-Poli P, Broome MR, Matthiasson P, Woolley JB, Johns LC, Tabraham P et al (2010) Spatial working memory in individuals at high risk for psychosis: longitudinal fMRI study. Schizophr Res 123:45–52
van der Hiele K, Vein AA, Reijntjes RHAM, Westendorp RGJ, Bollen ELEM, van Buchem MA et al (2007) EEG correlates in the spectrum of cognitive decline. Clin Neurophysiol 118:1931–1939
Lee H, Simpson GV, Logothetis NK, Rainer G (2005) Phase locking of single neuron activity to theta oscillations during working memory in monkey extrastriate visual cortex. Neuron 45:147–156
Amin HU, Malik AS, Badruddin N, Chooi W-T (2012) Brain activation during cognitive tasks: an overview of EEG and fMRI studies. In: Biomedical engineering and sciences (IECBES) IEEE EMBS, pp 950–953
Heinrichs-Graham E, Wilson TW (2015) Spatiotemporal oscillatory dynamics during the encoding and maintenance phases of a visual working memory task. Cortex 69:121–130
Lenartowicz A, Delorme A, Walshaw PD, Cho AL, Bilder RM, McGough JJ et al (2014) Electroencephalography correlates of spatial working memory deficits in attention-deficit/hyperactivity disorder: vigilance, encoding, and maintenance. J Neurosci 34:1171–1182
Roberts BM, Hsieh L-T, Ranganath C (2013) Oscillatory activity during maintenance of spatial and temporal information in working memory. Neuropsychologia 51:349–357
Brookes MJ, Wood JR, Stevenson CM, Zumer JM, White TP, Liddle PF et al (2011) Changes in brain network activity during working memory tasks: a magnetoencephalography study. Neuroimage 55:1804–1815
Park H-D, Min B-K, Lee K-M (2010) EEG oscillations reflect visual short-term memory processes for the change detection in human faces. Neuroimage 53:629–637
Jaiswal N, Ray W, Slobounov S (2010) Encoding of visual–spatial information in working memory requires more cerebral efforts than retrieval: evidence from an EEG and virtual reality study. Brain Res 1347:80–89
Khader PH, Jost K, Ranganath C, Rösler F (2010) Theta and alpha oscillations during working-memory maintenance predict successful long-term memory encoding. Neurosci Lett 468:339–343
Jensen O, Tesche CD (2002) Frontal theta activity in humans increases with memory load in a working memory task. Eur J Neurosci 15:1395–1399
Rahm B, Kaiser J, Unterrainer JM, Simon J, Bledowski C (2014) fMRI characterization of visual working memory recognition. NeuroImage 90:413–422
Christophel TB, Haynes J-D (2014) Decoding complex flow-field patterns in visual working memory. NeuroImage 91:43–51
Clark VP, Maisog JM, Haxby JV (1998) fMRI study of face perception and memory using random stimulus sequences. J Neurophysiol 79:3257–3265
Manoach DS, Schlaug G, Siewert B, Darby DG, Bly BM, Benfield A et al (1997) Prefrontal cortex fMRI signal changes are correlated with working memory load. NeuroReport 8:545–549
Yancey SW, Phelps EA (2001) Functional neuroimaging and episodic memory: a perspective. J Clin Exp Neuropsychol 23:32–48
Dutta A, Shah K, Silvanto J, Soto D (2014) Neural basis of non-conscious visual working memory. NeuroImage 91:336–343
McNab F, Klingberg T (2008) Prefrontal cortex and basal ganglia control access to working memory. Nat Neurosci 11:103–107
Koshino H, Kana RK, Keller TA, Cherkassky VL, Minshew NJ, Just MA (2008) fMRI investigation of working memory for faces in autism: visual coding and underconnectivity with frontal areas. Cereb Cortex 18:289–300
Koshino H, Carpenter PA, Minshew NJ, Cherkassky VL, Keller TA, Just MA (2005) Functional connectivity in an fMRI working memory task in high-functioning autism. NeuroImage 24:810–821
Veltman DJ, Rombouts SA, Dolan RJ (2003) Maintenance versus manipulation in verbal working memory revisited: an fMRI study. Neuroimage 18:247–256
Henson R, Burgess N, Frith CD (2000) Recoding, storage, rehearsal and grouping in verbal short-term memory: an fMRI study. Neuropsychologia 38:426–440
Meltzer JA, Negishi M, Mayes LC, Constable RT (2007) Individual differences in EEG theta and alpha dynamics during working memory correlate with fMRI responses across subjects. Clin Neurophysiol 118:2419–2436
Horovitz SG, Skudlarski P, Gore JC (2002) Correlations and dissociations between BOLD signal and P300 amplitude in an auditory oddball task: a parametric approach to combining fMRI and ERP. Magn Reson Imaging 20:319–325
Michels L, Bucher K, Lüchinger R, Klaver P, Martin E, Jeanmonod D, Brandeis D (2010) Simultaneous EEG-fMRI during a working memory task: modulations in low and high frequency bands. PloS One 5:e10298–e10298
Michels L, Lüchinger R, Koenig T, Martin E, Brandeis D (2012) Developmental changes of BOLD signal correlations with global human eeg power and synchronization during working memory. PLoS One 7:e39447
Zhang Q, Zhao X, Zhu C, Yang X, Yao L (2015) Cortical activities of single-trial P300 amplitudes modulated by memory load using simultaneous EEG-fMRI. In: SPIE medical imaging. pp 94170L–94170L-6
Hoffmann S, Labrenz F, Themann M, Wascher E, Beste C (2014) Crosslinking EEG time–frequency decomposition and fMRI in error monitoring. Brain Struct Funct 219:595–605
Hoppstädter M, Baeuchl C, Diener C, Flor H, Meyer P (2015) Simultaneous EEG-fMRI reveals brain networks underlying recognition memory ERP old/new effects. NeuroImage 116:112–122
Liu Y, Bengson J, Huang H, Mangun GR, Ding M (2016) Top-down modulation of neural activity in anticipatory visual attention: control mechanisms revealed by simultaneous EEG-fMRI. Cerebral Cortex 26:517–529
Rusiniak M, Lewandowska M, Wolak T, Pluta A, Milner R, Ganc M et al (2013) A modified oddball paradigm for investigation of neural correlates of attention: a simultaneous ERP-fMRI study. Magn Reson Mater Phys, Biol Med 26:511–526
Sato H, Yahata N, Funane T, Takizawa R, Katura T, Atsumori H et al (2013) A NIRS–fMRI investigation of prefrontal cortex activity during a working memory task. NeuroImage 83:158–173
Walz JM, Goldman RI, Carapezza M, Muraskin J, Brown TR, Sajda P (2013) Simultaneous EEG-fMRI reveals temporal evolution of coupling between supramodal cortical attention networks and the brainstem. J Neurosci 33:19212–19222
Lüchinger R, Michels L, Martin E, Brandeis D (2012) Brain state regulation during normal development: intrinsic activity fluctuations in simultaneous EEG–fMRI. NeuroImage 60:1426–1439
Warbrick T, Mobascher A, Brinkmeyer J, Musso F, Richter N, Stoecker T et al (2009) Single-trial P3 amplitude and latency informed event-related fMRI models yield different BOLD response patterns to a target detection task. NeuroImage 47:1532–1544
Scheeringa R, Petersson KM, Oostenveld R, Norris DG, Hagoort P, Bastiaansen MCM (2009) Trial-by-trial coupling between EEG and BOLD identifies networks related to alpha and theta EEG power increases during working memory maintenance. NeuroImage 44:1224–1238
Mulert C, Seifert C, Leicht G, Kirsch V, Ertl M, Karch S et al (2008) Single-trial coupling of EEG and fMRI reveals the involvement of early anterior cingulate cortex activation in effortful decision making. NeuroImage 42:158–168
Wibral M, Turi G, Linden DEJ, Kaiser J, Bledowski C (2008) Decomposition of working memory-related scalp ERPs: crossvalidation of fMRI-constrained source analysis and ICA. Int J Psychophysiol 67:200–211
Bledowski C, Kadosh KC, Wibral M, Rahm B, Bittner RA, Hoechstetter K et al (2006) Mental chronometry of working memory retrieval: a combined functional magnetic resonance imaging and event-related potentials approach. J Neurosci 26:821–829
Calhoun V, Adali T, Pearlson G, Kiehl K (2005) Neuronal chronometry of target detection: fusion of hemodynamic and event-related potential data. Neuroimage 30:239–244
Pessoa L, Gutierrez E, Bandettini PA, Ungerleider LG (2002) Neural correlates of visual working memory: fMRI amplitude predicts task performance. Neuron 35:975–987
Lindquist MA (2008) The statistical analysis of fMRI data. Stat Sci 23:439–464
Perrachione TH, Ghosh SS (2013) Optimized design and analysis of sparse-sampling fMRI experiments. Front Neurosci 7:55
Chong TT-J, Cunnington R, Williams MA, Kanwisher N, Mattingley JB (2008) fMRI adaptation reveals mirror neurons in human inferior parietal cortex. Curr Biol 18:1576–1580
Spaniol J, Davidson PS, Kim AS, Han H, Moscovitch M, Grady CL (2009) Event-related fMRI studies of episodic encoding and retrieval: meta-analyses using activation likelihood estimation. Neuropsychologia 47:1765–1779
Huettel SA (2012) Event-related fMRI in cognition. NeuroImage 62:1152–1156
Klimesch W (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res Rev 29:169–195
Whitten TA, Hughes AM, Dickson CT, Caplan JB (2011) A better oscillation detection method robustly extracts EEG rhythms across brain state changes: the human alpha rhythm as a test case. NeuroImage 54:860–874
Zumer JM, Brookes MJ, Stevenson CM, Francis ST, Morris PG (2010) Relating BOLD fMRI and neural oscillations through convolution and optimal linear weighting. NeuroImage 49:1479–1489
Jensen O, Gelfand J, Kounios J, Lisman JE (2002) Oscillations in the alpha band (9–12 Hz) increase with memory load during retention in a short-term memory task. Cereb Cortex (New York, N.Y.: 1991) 12:877–882
Wager T, Smith E (2003) Neuroimaging studies of working memory. Cogn Affect Behav Neurosci 3:255–274
Ritter P, Villringer A (2006) Simultaneous EEG-fMRI. Neurosci Biobehav Rev 30:823–838
Ritter P, Moosmann M, Villringer A (2009) Rolandic alpha and beta EEG rhythms’ strengths are inversely related to fMRI-BOLD signal in primary somatosensory and motor cortex. Hum Brain Mapp 30:1168–1187
Mulert C, Louis L (eds) (2010) EEG—fMRI, physiological basis, technique and applications, Springer, New York
Rosenkranz K, Lemieux L (2010) Present and future of simultaneous EEG-fMRI. Magn Reson Mater Phys, Biol Med 23:309–316
Daunizeau J, Laufs H, Friston KJ (2010) EEG-fMRI information fusion: biophysics and data analysis. Springer, Berlin, pp 511–526
Huster RJ, Debener S, Eichele T, Herrmann CS (2012) Methods for simultaneous EEG-fMRI: an introductory review. J Neurosci 32:6053–6060
Walz JM, Goldman RI, Carapezza M, Muraskin J, Brown TR, Sajda P (2014) Simultaneous EEG-fMRI reveals a temporal cascade of task-related and default-mode activations during a simple target detection task. NeuroImage 102:229–239
Ahmad RF, Malik AS, Kamel N, Reza F, Abdul Karim AH (2015) Optimization and development of concurrent EEG-fMRI data acquisition setup for understanding neural mechanisms of brain. In: Instrumentation and measurement technology conference (I2MTC), 2015 IEEE International, pp 476–481
Bayram A, Bayraktaroglu Z, Karahan E, Erdogan B, Bilgic B, Ozker M et al (2011) Simultaneous EEG/fMRI analysis of the resonance phenomena in steady-state visual evoked responses. Clin EEG Neurosci 42:98–106
Acknowledgments
This research work has been supported by the Fundamental Research Grant Scheme (Ref: FRGS/1/2014/TK03/UTP/02/1), Ministry of High Education (MOHE), Malaysia and Graduate Assistantship scheme of Universiti Teknologi PETRONAS, Malaysia.
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Ahmad, R.F., Malik, A.S., Kamel, N. et al. Simultaneous EEG-fMRI for working memory of the human brain. Australas Phys Eng Sci Med 39, 363–378 (2016). https://doi.org/10.1007/s13246-016-0438-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13246-016-0438-x