Elsevier

Clinical Neurophysiology

Volume 121, Issue 8, August 2010, Pages 1304-1313
Clinical Neurophysiology

A motor imagery-based online interactive brain-controlled switch: Paradigm development and preliminary test

https://doi.org/10.1016/j.clinph.2010.03.001Get rights and content

Abstract

Objective

To develop a practical motor imagery-based brain-controlled switch as functional as a real-world switch that is reliable with a minimal false positive operation rate and convenient for users without the need of attention to the switch during a ‘No Control’ state (when not to activate the switch).

Methods

Four healthy volunteers were instructed to perform an intended motor imagery task following an external sync signal in order to turn on a virtual switch provided on a computer screen. No specific mental task was required during the ‘No Control’ state. The beta band event-related frequency power (event-related desynchronization or ERD) from a single EEG Laplacian channel was monitored online in real-time. The computer continuously monitored the relative ERD power level until it exceeded a pre-set threshold and turned on the virtual switch.

Results

Subject 1 achieved lowest average false positive rate of 0.4 ± 0.9% in a five-session online study during the entire ‘No Control’ state, whereby the subject required 6.8 ± 0.6 s of active urging time or total response time of 20.5 ± 1.9 s to perform repeated attempts in order to turn on the switch in the online interactive switch operation. The average false positive rate among four subjects was 0.8 ± 0.4% with average active urging time of 12.3 ± 4.4 s or average response time of 36.9 ± 13.0 s. Offline analysis from subject 2 shows that the overall performance from 10-fold cross-validation was 96.2% with 3 consecutive epoch averaging, which was further improved to 99.0% by computationally intensive methods.

Conclusions

The novel design of the brain-controlled switch using the ERD feature associated with motor imagery achieved minimal false positive rate with a reasonable active urging time or response time to activate the switch.

Significance

The reliability and convenience of the developed brain-controlled switch may extend current brain–computer interface capacities in practical communication and control applications.

Introduction

Noninvasive EEG-based brain–computer interfaces (BCIs) provide an augmented communication channel for individuals who do not have the motor function capabilities necessary to interact with the external world by controlling a device, such as robotic arm, wheel chair, or computer (Wolpaw et al., 2002), and also for able bodied individuals to interact with media applications such as Google earth (Scherer et al., 2007), virtual environment (Scherer et al., 2008) as a new human–machine interface, or even prove suitable as a biometric measure for person identification (Pfurtscheller and Neuper, 2006, Pfurtscheller and Solis-Escalante, 2009).

The majority of the past BCI investigations have been performed on a trial-based continuous control. An example of the aforementioned control includes the cursor control introduced by Wolpaw’s group, wherein a cursor at left side of monitor starts moving at the beginning of each trial while requiring the users to keep sustained attention and continuously regulate their brain activities to control cursor vertical position until the cursor reaches the right side of the monitor (Wolpaw et al., 1991, McFarland et al., 2003). In contrast to the trial-based BCI designs, a couple of research groups explored self-paced or asynchronous designs for continuous BCI operation to differential ‘Intentional Control’ state from ‘No Control’ state; Birch’s group employed a slow cortical potentials (SCP)-like low frequency EEG signals, formally known as low frequency asynchronous switch design (LF-ASD) (Mason and Birch, 2000, Birch et al., 2002) as well as the improved versions (Bashashati et al., 2006), Pfurtscheller’s group employed Mu rhythm-based event-related desynchronization (ERD) design (Pfurtscheller et al., 2005, Muller-Putz et al., 2006), self-paced BCI using movement-related cortical potentials (event-related potentials) from electrocorticograms (Huggins et al., 1999) or from EEG (Yom-Tov and Inbar, 2002) as well as a SSVEP-based self-paced BCI approach (Parini et al., 2009). The majority of literatures related to the self-paced or asynchronous BCI were performed offline to explore optimal computational methods (Fatourechi et al., 2008b, Faradji et al., 2009, Scherer et al., 2009), whereas online study is limited.

To develop a brain-controlled switch as functional as a real-world switch, we consider the following properties:

For a real-world switch, for example, a light switch, users will pay attention to the switch only when they want to turn the switch on/off. Birch’s group addressed that the users may perform a certain motor task only when they want the ‘Intentional control’ of the BCI system, whereas users may be day-dreaming, thinking about a problem or any mental tasks other than the motor task for the ‘Intentional Control’ during the ‘No Control’ State (not to operate the switch) (Bashashati et al., 2006). Ideally, a fully self-paced brain-controlled switch will allow users any tasks other than the task for ‘Intentional Control’ or just keep relax when they do not intend to operate the switch. However, in order to improve the BCI performance affected by the low signal-to-noise (S/N) of the brain signals, in particular, when recorded noninvasively from EEG, users might be asked to be attentive to one or more particular mental tasks other than the ‘intentional Control’ task even during ‘No Control’ State (Birch et al., 2002). For example, subjects were asked to count the number of times that a white ball bounced off the monitor’s screen during the ‘No Control’ state (Fatourechi et al., 2008b). It is more plausible to develop a self-paced brain-controlled switch that may allow users not to attend to any particular task or tasks. Moreover, it is also plausible in BCI applications that users are able to pursue other control purposes through other mental tasks if they don’t need to keep attentive to the switch.

Considering that a real-world switch mostly stays in off-activation state, the brain-controlled switch should have minimal FPR to avoid too many false operations during the ‘No Control’ state, i.e., non-activated state. For example, a FPR of 10% may result about 15 incorrect operations or erroneous operations in just 10 min, assuming that the brain-controlled switch detects the brain signal every 4 s. Birch’s group suggested that 0% FPR might be required in practical applications and achieved averaged FPR below 1% in their offline or pseudo-online optimization studies (Fatourechi et al., 2008a, Faradji et al., 2009).

The operation of a real-world switch is interactive that allows users to make multiple attempts until the switch is activated, because not all the switches are necessary to be sensitive enough that can be activated in a single attempt. It is particularly important for a brain-controlled switch that the interactive operation is required because a higher activation threshold is usually adopted in order to achieve a low FPR. The current self-paced or asynchronous BCI adopted TPR to evaluate the sensitivity of BCI in the ‘Intentional Control’ state. For a real-world switch, users need not to keep operating the switch after the switch is activated. Therefore, a response time showing the time from the start of switch operation until the switch is activated will be more appropriate to provide the switch performance.

The BCI was originally proposed to restore motor functions for paralyzed patients, who are unable to make muscle contraction to produce physical movement. Pfurtscheller’s group explored motor imagery-based asynchronous BCI to allow paralyzed patients to ‘walking’ in a virtual environment (Leeb et al., 2007). In the early Brich’s group study, they also investigated the feasibility of a motor imagery-based self-paced BCI (Birch et al., 2002).

The event-related frequency power decrease, or event-related desynchronization (ERD) has patterns in certain frequency bands and specific sites characterizing the dynamics of EEG oscillation time-locked to an externally triggered event, which are highly reproducible and stable over time (Pollock et al., 1991, Kondacs and Szabo, 1999, Pfurtscheller et al., 2006). The ERD patterns and event-related synchronization (ERS) can also be observed when subjects perform voluntary movement or intended motor imagery tasks over motor areas contralateral to the hand moved or imagined (Pfurtscheller et al., 1996, Pfurtscheller and Lopes da Silva, 1999). These merits promote the ERD as a better choice as an EEG feature to drive BCIs.

In this study, we developed a novel paradigm for a brain-controlled switch based on human motor imagery-associated ERD following external sync signals. In this paradigm, the ERD feature was enhanced by consecutive event-related moving average time-locked to an external sync signal. Assuming that the smoothed EEG power of consecutive epochs provides a better S/N ratio that may tolerate the variances of EEG spontaneous activity, we expected that the novel design of a BCI switch may provide more reliable and robust performance with a minimal false positive rate along with a shorter time to turn the switch on. The interactive brain-controlled switch was designed as functional as a real-world switch which users would not need to be attentive to the switch (for sample, continuously regulating their brain activities or perform any particular mental tasks), when they did not want to operate the switch. Furthermore, the brain-controlled switch was interactive in the sense that users performed repeated attempts until the switch was turned on. The novel brain-controlled switch was tested online in real-time.

Section snippets

Subjects

Four right handed healthy volunteers (two females and two males) between the ages of 18 and 26 participated in this study. All subjects gave informed consent. Prior to this study, subjects did not receive any training related to the regulation of their brain rhythm or were informed of the experimental hypothesis. The protocol was approved by the Institutional Review Board.

Experimental paradigm and behavioral task

All the subjects participated in 6 sessions of experiments: 1 session with physical movement of right index finger pinch and

Time–frequency analysis

The time–frequency analysis during the ‘Intentional Control’ state and the ‘No Control’ state from Laplacian derived channel C3 is illustrated in Fig. 2. The first column plots were obtained from one session with motor imagery, where subjects were urging to move after they heard a high pitch sound (time 0) and imagined the finger movement after they heard a low pitch sound (time 1). Similarly, the third column plots were obtained from the session with physical movement where subjects were

The event-related design of brain-controlled switch

For a real-world switch that may be used to turn on a light in a room, to detect a button click in a computer mouse, or serve some vital purpose in a variety of other applications, we are most concerned with the sensitivity (how easily we can activate the switch on one hand), and the specificity (that the switch is only activated when we intend to activate it on the other hand). Depending on the different purposes, a real-world switch can be designed either with high sensitivity or with low

References (40)

  • R. Scherer et al.

    Flexibility and practicality graz brain–computer interface approach

    Int Rev Neurobiol

    (2009)
  • E.W. Sellers et al.

    A P300-based brain–computer interface: initial tests by ALS patients

    Clin Neurophysiol

    (2006)
  • J.R. Wolpaw et al.

    Brain–computer interfaces for communication and control

    Clin Neurophysiol

    (2002)
  • J.R. Wolpaw et al.

    An EEG-based brain–computer interface for cursor control

    Electroencephalogr Clin Neurophysiol

    (1991)
  • A. Bashashati et al.

    An improved asynchronous brain interface: making use of the temporal history of the LF-ASD feature vectors

    J Neural Eng

    (2006)
  • G.E. Birch et al.

    Initial on-line evaluations of the LF-ASD brain–computer interface with able-bodied and spinal-cord subjects using imagined voluntary motor potentials

    IEEE Trans Neural Syst Rehabil Eng

    (2002)
  • G.E. Birch et al.

    Single-trial processing of event-related potentials using outlier information

    IEEE Trans Biomed Eng

    (1993)
  • B. Blankertz et al.

    Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis

    IEEE Trans Neural Syst Rehabil Eng

    (2003)
  • M. Fatourechi et al.

    Performance of a self-paced brain–computer interface on data contaminated with eye-movement artifacts and on data recorded in a subsequent session

    Comput Intell Neurosci

    (2008)
  • M. Fatourechi et al.

    A self-paced brain–computer interface system with a low false positive rate

    J Neural Eng

    (2008)
  • Cited by (0)

    View full text