ABSTRACT
The combination of virtual reality (VR) and brain measurements is a promising development of HCI, but the maturation of this paradigm requires more knowledge about how brain activity is influenced by parameters of VR applications. To this end we investigate the influence of two prominent VR parameters, 3d-motion and interactivity, while brain activity is measured for a mental rotation task, using functional MRI (fMRI). A mental rotation network of brain areas is identified, matching previous results. The addition of interactivity increases the activation in core areas of this network, with more profound effects in frontal and preparatory motor areas. The increases from 3d-motion are restricted to primarily visual areas. We relate these effects to emerging theories of cognition and potential applications for brain-computer interfaces (BCIs). Our results demonstrate one way to provoke increased activity in task-relevant areas, making it easier to detect and use for adaptation and development of HCI.
- Aguirre, G.K., Detre, J.A., Alsop, D.C., and D'Esposito, M. The parahippocampus subserves topographical learning in man. Cereb. Cortex 6, 6 (1996), 823--829.Google ScholarCross Ref
- Backman, A. Colosseum3D-Authoring Framework for Virtual Environments. In Proc. EUROGRAPHICS Workshop IPT & EGVE Workshop, (2005), 225--226. Google ScholarDigital Library
- Barsalou, L.W. Grounded Cognition. Annual Review of Psychology 59, 1 (2008), 617--645.Google ScholarCross Ref
- Beck, L., Wolter, M., Mungard, N., Kuhlen, T., and Sturm, W. Combining Virtual Reality and Functional Magnetic Resonance Imaging (fMRI): Problems and Solutions. In HCI and Usability for Medicine and Health Care. 2007, 335--348. Google ScholarDigital Library
- Blanke, O. and Arzy, S. The Out-of-Body Experience: Disturbed Self-Processing at the Temporo-Parietal Junction. Neuroscientist 11, 1 (2005), 16--24.Google ScholarCross Ref
- Born, R.T. and Bradley, D.C. Structure and Function of Visual Area MT. Annual Review of Neuroscience 28, 1 (2005), 157--189.Google ScholarCross Ref
- Christou, G., Law, E.L., Green, W., and Hornbaek, K. Challenges in evaluating usability and user experience of reality-based interaction. In Proc. CHI 2009 Extended Abstracts, ACM (2009), 4811--4814. Google ScholarDigital Library
- Cohen, M.S., Kosslyn, S.M., Breiter, H.C., et al. Changes in cortical activity during mental rotation A mapping study using functional MRI. Brain 119, 1 (1996), 89--100.Google ScholarCross Ref
- Constantinidis, C. and Wang, X. A neural circuit basis for spatial working memory. The Neuroscientist: A Review Journal Bringing Neurobiology, Neurology and Psychiatry 10, 6 (2004), 553--65.Google Scholar
- Curtis, C.E. and D'Esposito, M. Persistent activity in the prefrontal cortex during working memory. Trends in Cognitive Sciences 7, 9 (2003), 415--423.Google ScholarCross Ref
- Cutrell, E. and Tan, D.S. BCI for passive input in HCI. In Proc. CHI 2008 Workshop on Brain-Computer Interfaces for HCI and Games, (2008). Google ScholarDigital Library
- Decety, J. and Jeannerod, M. Mentally simulated movements in virtual reality: does Fitt's law hold in motor imagery? Behavioural Brain Research 72, 1-2 (1995), 127--134.Google ScholarCross Ref
- Dolan, R. Neuroimaging of Cognition: Past, Present, and Future. Neuron 60, 3 (2008), 496--502.Google ScholarCross Ref
- Ericsson, K.A., Prietula, M.J., and Cokely, E.T. The making of an expert. Harvard business review 85, 7/8 (2007), 114.Google Scholar
- Eriksson, J., Larsson, A., and Nyberg, L. Item-specific Training Reduces Prefrontal Cortical Involvement in Perceptual Awareness. Journal of Cognitive Neuroscience 20, 10 (2008), 1777--1787. Google ScholarDigital Library
- Farrer, C. and Frith, C.D. Experiencing Oneself vs Another Person as Being the Cause of an Action: The Neural Correlates of the Experience of Agency. NeuroImage 15, 3 (2002), 596--603.Google ScholarCross Ref
- Fitts, P.M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 6 (1954), 381--391.Google ScholarCross Ref
- Friston, K. and Stephan, K. Free-energy and the brain. Synthese 159, 3 (2007), 417--458.Google ScholarCross Ref
- Girouard, A., Hirshfield, L.M., Solovey, E., and Jacob, R.J.K. Using functional Near-Infrared Spectroscopy in HCI: Toward evaluation methods and adaptive interfaces. In Proc. CHI 2008 Workshop on Brain-Computer Interfaces for HCI and Games, (2008).Google Scholar
- Girouard, A. Adaptive brain--computer interface. In Proc. CHI 2009 Extended Abstracts, ACM Press (2009), 3097--3100. Google ScholarDigital Library
- Hawkins, J. On Intelligence. Owl Books, 2005. Google ScholarDigital Library
- Hawkins, J. and George, D. Hierarchical Temporal Memory. Numenta Inc., 2006.Google Scholar
- Hirshfield, L.M., Solovey, E.T., Girouard, A., et al. Using Brain Measurement to Evaluate Reality Based Interactions. In Proc. CHI 2009 Workshop on Challenges in the Evaluation of Usability and User Experience in Reality Based Interaction, (2009), 19.Google Scholar
- Hirshfield, L.M., Solovey, E.T., Girouard, A., et al. Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy. In Proc. CHI 2009, ACM Press (2009), 2185--2194. Google ScholarDigital Library
- Jacob, R.J.K., Girouard, A., Hirshfield, L.M., et al. Reality-based interaction: a framework for post-WIMP interfaces. In Proc. CHI 2008, ACM Press (2008), 201--210. Google ScholarDigital Library
- Jäncke, L. Virtual reality and the role of the prefrontal cortex in adults and children. Frontiers in Neuroscience 3, 1 (2009).Google ScholarCross Ref
- Kilner, J.M., Friston, K.J., and Frith, C.D. Predictive coding: an account of the mirror neuron system. Cognitive Processing 8, 3 (2007), 159--66.Google ScholarCross Ref
- Kosslyn, S.M., Ganis, G., and Thompson, W.L. Neural foundations of imagery. Nat Rev Neurosci 2, 9 (2001), 635--642.Google ScholarCross Ref
- Lécuyer, A., Lotte, F., Reilly, R.B., Leeb, R., Hirose, M., and Slater, M. Brain--Computer Interfaces, Virtual Reality, and Videogames. Computer 41, 10 (2008), 66--72. Google ScholarDigital Library
- Minnery, B.S. and Fine, M.S. Neuroscience and the future of human-computer interaction. Interactions 16, 2 (2009), 70--75. Google ScholarDigital Library
- Mourao-Miranda, J., Ecker, C., Sato, J.R., and Brammer, M. Dynamic Changes in the Mental Rotation Network Revealed by Pattern Recognition Analysis of fMRI Data. Journal of Cognitive Neuroscience 21, 5 (2009), 890--904. Google ScholarDigital Library
- Parasuraman, R. and Wilson, G.F. Putting the brain to work: neuroergonomics past, present, and future. Human Factors 50, 3 (2008), 468--474.Google ScholarCross Ref
- Postma, A. and Barsalou, L.W. Spatial working memory and imagery: From eye movements to grounded cognition. Acta Psychologica 118, 2--3 (2009).Google Scholar
- Rizzo, A.A., Schultheis, M., Kerns, K.A., and Mateer, C. Analysis of assets for virtual reality applications in neuropsychology. Neuropsychological Rehabilitation 14, 1 (2004), 207--239.Google ScholarCross Ref
- Schacter, D.L., Addis, D.R., and Buckner, R.L. Remembering the past to imagine the future: the prospective brain. Nat Rev Neurosci 8, 9 (2007), 657--661.Google ScholarCross Ref
- Shepard, R.N. and Metzler, J. Mental Rotation of Three-Dimensional Objects. Science 171, 3972 (1971), 701--703.Google Scholar
- Sjölie, D., Bodin, K., Eriksson, J., and Janlert, L. Using brain imaging to assess interaction in immersive VR. In Proc. CHI 2009 Workshop on Challenges in the Evaluation of Usability and User Experience in Reality Based Interaction, (2009), 23.Google Scholar
- Sweetser, P. and Wyeth, P. GameFlow: a model for evaluating player enjoyment in games. Comput. Entertain. 3, 3 (2005), 3--3. Google ScholarDigital Library
- Tagaris, G.A., Kim, S., Strupp, J.P., Andersen, P., U?urbil, K., and Georgopoulos, A.P. Mental Rotation Studied by Functional Magnetic Resonance Imaging at High Field (4 Tesla): Performance and Cortical Activation. Journal of Cognitive Neuroscience 9, 4 (1997), 419--432. Google ScholarDigital Library
Index Terms
- Effects of interactivity and 3D-motion on mental rotation brain activity in an immersive virtual environment
Recommendations
Virtual reality with fMRI: a breakthrough cognitive treatment tool
The impact of virtual reality (VR) has been felt in a wide range of fields over the past 10 years. VR has been shown to be an effective treatment for anxiety, phobia, pain, posttraumatic stress disorder, stress inoculation training, and drug and alcohol ...
Effects of electrical brain stimulation on brain indices and presence experience in immersive, interactive virtual reality
AbstractThe subjective presence experience in virtual reality (VR) is associated with distinct brain activation patterns. Particularly, the dorsolateral prefrontal cortex (DLPFC) seems to play a central role. We investigated the effects of electric brain ...
Virtual reality with fMRI: a breakthrough cognitive treatment tool
The impact of virtual reality (VR) has been felt in a wide range of fields over the past 10 years. VR has been shown to be an effective treatment for anxiety, phobia, pain, posttraumatic stress disorder, stress inoculation training, and drug and alcohol ...
Comments