Walking from thought

Abstract

Online analysis and classification of single electroencephalogram (EEG) trials during motor imagery were used for navigation in the virtual environment (VE). The EEG was recorded bipolarly with electrode placement over the hand and foot representation areas. The aim of the study was to demonstrate for the first time that it is possible to move through a virtual street without muscular activity when the participant only imagines feet movements. This is achieved by exploiting a brain–computer interface (BCI) which transforms thought-modulated EEG signals into an output signal that controls events within the VE. The experiments were carried out in an immersive projection environment, commonly referred to as a “Cave” (Cruz-Neira, C., Sandin, D.J., DeFanti, T.A., Surround-screen projection-based virtual reality: the design and implementation of the CAVE. Proceedings of the 20th annual conference on Computer graphics and interactive techniques, ACM Press, 1993, pp. 135–142) where participants were able to move through a virtual street by foot imagery only. Prior to the final experiments in the Cave, the participants underwent an extensive BCI training.

Introduction

A relatively recent development is called EEG-based brain–computer interface (BCI). By means of a BCI, specific features extracted from EEG signals are used, e.g. to operate devices and assist people with comprised motor functions (Wolpaw et al., 2002) or to control events in the virtual environment (VE) (Bayliss and Ballard, 2000, Leeb et al., 2004). In general, VE provides an excellent tool to test procedures which might be applied subsequently in reality, e.g. for patients with disabilities. If it is possible to show that people can learn to control their movements through space within a VE, it would justify the much bigger expense of building physical devices like, e.g. a robot arm controlled by a BCI. Another application of combining BCI and virtual reality technologies is using VE as feedback medium with the goal to enhance classification accuracy and shorten the time needed for BCI training sessions, e.g. to re-establish a communication channel in patients who are totally paralyzed (or “locked-in”) (Pfurtscheller et al., 2005, Wolpaw et al., 2002).

One of the first who combined virtual reality (VR) and BCI technologies were Bayliss and Ballard (2000) using the P300 evoked potential components in combination with a head-mounted display (HMD)-based VR system. Subjects were instructed to drive a modified go-car in a virtual tour and stop at red lights while ignoring both green and yellow lights. The red lights were made to be rare enough to make the P300 suitable (Donchin et al., 2000). In this type of BCI the subject has to focus attention to the visual cue and obtain visual feedback when the goal was achieved.

Besides focused visual attention, motor imagery is also a suitable mental strategy in BCI research. In motor imagery, the participants do not have to focus on a certain flashing object in the VE but do have to imagine a specific motor act such as hand, foot or tongue movement. Motor imagery results in a somatotopic activation pattern, similar to the same physical movement being executed (Porro et al., 1996). In particular, hand and feet motor imagery affects sensorimotor EEG rhythms in a way that allows a BCI to generate a control signal of high reliability.

The goal of this paper is (i) to demonstrate that it is possible to move through a virtual street without any muscular activity, when the participant only imagines the movement of both feet; (ii) to investigate whether feedback in a BCI experiment in form of a moving scene in VE disturbs or motivates the participants.

Three able-bodied subjects firstly underwent a basic BCI training where a computer monitor was used for feedback (FB). Thereafter, advanced training was performed within a VE delivered on a head-mounted display (HMD), and finally a “walking” task was carried out in the ReaCTor, a Cave-like system (Cruz-Neira et al., 1993) at University College London (UCL).