A motor imagery-based online interactive brain-controlled switch: Paradigm development and preliminary test
Introduction
Noninvasive EEG-based brain–computer interfaces (BCIs) provide an augmented communication channel for individuals who do not have the motor function capabilities necessary to interact with the external world by controlling a device, such as robotic arm, wheel chair, or computer (Wolpaw et al., 2002), and also for able bodied individuals to interact with media applications such as Google earth (Scherer et al., 2007), virtual environment (Scherer et al., 2008) as a new human–machine interface, or even prove suitable as a biometric measure for person identification (Pfurtscheller and Neuper, 2006, Pfurtscheller and Solis-Escalante, 2009).
The majority of the past BCI investigations have been performed on a trial-based continuous control. An example of the aforementioned control includes the cursor control introduced by Wolpaw’s group, wherein a cursor at left side of monitor starts moving at the beginning of each trial while requiring the users to keep sustained attention and continuously regulate their brain activities to control cursor vertical position until the cursor reaches the right side of the monitor (Wolpaw et al., 1991, McFarland et al., 2003). In contrast to the trial-based BCI designs, a couple of research groups explored self-paced or asynchronous designs for continuous BCI operation to differential ‘Intentional Control’ state from ‘No Control’ state; Birch’s group employed a slow cortical potentials (SCP)-like low frequency EEG signals, formally known as low frequency asynchronous switch design (LF-ASD) (Mason and Birch, 2000, Birch et al., 2002) as well as the improved versions (Bashashati et al., 2006), Pfurtscheller’s group employed Mu rhythm-based event-related desynchronization (ERD) design (Pfurtscheller et al., 2005, Muller-Putz et al., 2006), self-paced BCI using movement-related cortical potentials (event-related potentials) from electrocorticograms (Huggins et al., 1999) or from EEG (Yom-Tov and Inbar, 2002) as well as a SSVEP-based self-paced BCI approach (Parini et al., 2009). The majority of literatures related to the self-paced or asynchronous BCI were performed offline to explore optimal computational methods (Fatourechi et al., 2008b, Faradji et al., 2009, Scherer et al., 2009), whereas online study is limited.
To develop a brain-controlled switch as functional as a real-world switch, we consider the following properties:
For a real-world switch, for example, a light switch, users will pay attention to the switch only when they want to turn the switch on/off. Birch’s group addressed that the users may perform a certain motor task only when they want the ‘Intentional control’ of the BCI system, whereas users may be day-dreaming, thinking about a problem or any mental tasks other than the motor task for the ‘Intentional Control’ during the ‘No Control’ State (not to operate the switch) (Bashashati et al., 2006). Ideally, a fully self-paced brain-controlled switch will allow users any tasks other than the task for ‘Intentional Control’ or just keep relax when they do not intend to operate the switch. However, in order to improve the BCI performance affected by the low signal-to-noise (S/N) of the brain signals, in particular, when recorded noninvasively from EEG, users might be asked to be attentive to one or more particular mental tasks other than the ‘intentional Control’ task even during ‘No Control’ State (Birch et al., 2002). For example, subjects were asked to count the number of times that a white ball bounced off the monitor’s screen during the ‘No Control’ state (Fatourechi et al., 2008b). It is more plausible to develop a self-paced brain-controlled switch that may allow users not to attend to any particular task or tasks. Moreover, it is also plausible in BCI applications that users are able to pursue other control purposes through other mental tasks if they don’t need to keep attentive to the switch.
Considering that a real-world switch mostly stays in off-activation state, the brain-controlled switch should have minimal FPR to avoid too many false operations during the ‘No Control’ state, i.e., non-activated state. For example, a FPR of 10% may result about 15 incorrect operations or erroneous operations in just 10 min, assuming that the brain-controlled switch detects the brain signal every 4 s. Birch’s group suggested that 0% FPR might be required in practical applications and achieved averaged FPR below 1% in their offline or pseudo-online optimization studies (Fatourechi et al., 2008a, Faradji et al., 2009).
The operation of a real-world switch is interactive that allows users to make multiple attempts until the switch is activated, because not all the switches are necessary to be sensitive enough that can be activated in a single attempt. It is particularly important for a brain-controlled switch that the interactive operation is required because a higher activation threshold is usually adopted in order to achieve a low FPR. The current self-paced or asynchronous BCI adopted TPR to evaluate the sensitivity of BCI in the ‘Intentional Control’ state. For a real-world switch, users need not to keep operating the switch after the switch is activated. Therefore, a response time showing the time from the start of switch operation until the switch is activated will be more appropriate to provide the switch performance.
The BCI was originally proposed to restore motor functions for paralyzed patients, who are unable to make muscle contraction to produce physical movement. Pfurtscheller’s group explored motor imagery-based asynchronous BCI to allow paralyzed patients to ‘walking’ in a virtual environment (Leeb et al., 2007). In the early Brich’s group study, they also investigated the feasibility of a motor imagery-based self-paced BCI (Birch et al., 2002).
The event-related frequency power decrease, or event-related desynchronization (ERD) has patterns in certain frequency bands and specific sites characterizing the dynamics of EEG oscillation time-locked to an externally triggered event, which are highly reproducible and stable over time (Pollock et al., 1991, Kondacs and Szabo, 1999, Pfurtscheller et al., 2006). The ERD patterns and event-related synchronization (ERS) can also be observed when subjects perform voluntary movement or intended motor imagery tasks over motor areas contralateral to the hand moved or imagined (Pfurtscheller et al., 1996, Pfurtscheller and Lopes da Silva, 1999). These merits promote the ERD as a better choice as an EEG feature to drive BCIs.
In this study, we developed a novel paradigm for a brain-controlled switch based on human motor imagery-associated ERD following external sync signals. In this paradigm, the ERD feature was enhanced by consecutive event-related moving average time-locked to an external sync signal. Assuming that the smoothed EEG power of consecutive epochs provides a better S/N ratio that may tolerate the variances of EEG spontaneous activity, we expected that the novel design of a BCI switch may provide more reliable and robust performance with a minimal false positive rate along with a shorter time to turn the switch on. The interactive brain-controlled switch was designed as functional as a real-world switch which users would not need to be attentive to the switch (for sample, continuously regulating their brain activities or perform any particular mental tasks), when they did not want to operate the switch. Furthermore, the brain-controlled switch was interactive in the sense that users performed repeated attempts until the switch was turned on. The novel brain-controlled switch was tested online in real-time.
Section snippets
Subjects
Four right handed healthy volunteers (two females and two males) between the ages of 18 and 26 participated in this study. All subjects gave informed consent. Prior to this study, subjects did not receive any training related to the regulation of their brain rhythm or were informed of the experimental hypothesis. The protocol was approved by the Institutional Review Board.
Experimental paradigm and behavioral task
All the subjects participated in 6 sessions of experiments: 1 session with physical movement of right index finger pinch and
Time–frequency analysis
The time–frequency analysis during the ‘Intentional Control’ state and the ‘No Control’ state from Laplacian derived channel C3 is illustrated in Fig. 2. The first column plots were obtained from one session with motor imagery, where subjects were urging to move after they heard a high pitch sound (time 0) and imagined the finger movement after they heard a low pitch sound (time 1). Similarly, the third column plots were obtained from the session with physical movement where subjects were
The event-related design of brain-controlled switch
For a real-world switch that may be used to turn on a light in a room, to detect a button click in a computer mouse, or serve some vital purpose in a variety of other applications, we are most concerned with the sensitivity (how easily we can activate the switch on one hand), and the specificity (that the switch is only activated when we intend to activate it on the other hand). Depending on the different purposes, a real-world switch can be designed either with high sensitivity or with low
References (40)
- et al.
Exploration of computational methods for classification of movement intention during human voluntary movement from single trial EEG
Clin Neurophysiol
(2007) - et al.
Plausibility assessment of a 2-state self-paced mental task-based BCI using the no-control performance analysis
J Neurosci Methods
(2009) An on-line transformation of EEG scalp potentials into orthogonal source derivations
Electroencephalogr Clin Neurophysiol
(1975)- et al.
Long-term intra-individual variability of the background EEG in normals
Clin Neurophysiol
(1999) - et al.
Brain–computer interface (BCI) operation: optimizing information transfer rates
Biol Psychol
(2003) - et al.
Event-related EEG/MEG synchronization and desynchronization: basic principles
Clin Neurophysiol
(1999) - et al.
Future prospects of ERD/ERS in the context of brain–computer interface (BCI) developments
Prog Brain Res
(2006) - et al.
Could the beta rebound in the EEG be suitable to realize a “brain switch”?
Clin Neurophysiol
(2009) - et al.
Post-movement beta synchronization. A correlate of an idling motor area?
Electroencephalogr Clin Neurophysiol
(1996) - et al.
Reliability of topographic quantitative EEG amplitude in healthy late–middle-aged and elderly subjects
Electroencephalogr Clin Neurophysiol
(1991)