Dynamic time window mechanism for time synchronous VEP-based BCIs—Performance evaluation with a dictionary-supported BCI speller employing SSVEP and c-VEP

Felix Gembler; Piotr Stawicki; Abdul Saboor; Ivan Volosyak

doi:10.1371/journal.pone.0218177

Abstract

Brain-Computer Interfaces (BCIs) based on visual evoked potentials (VEPs) allow high communication speeds and accuracies. The fastest speeds can be achieved if targets are identified in a synchronous way (i.e., after a pre-set time period the system will produce a command output). The duration a target needs to be fixated on until the system classifies an output command affects the overall system performance. Hence, extracting a data window dedicated for the classification is of critical importance for VEP-based BCIs. Secondly, unintentional fixation on a target could easily lead to its selection. For the practical usability of BCI applications it is desirable to distinguish between intentional and unintentional fixations. This can be achieved by using threshold-based target identification methods. The study explores personalized dynamic classification time windows for threshold-based time synchronous VEP BCIs. The proposed techniques were tested employing the SSVEP and the c-VEP paradigm. Spelling performance was evaluated using an 8-target dictionary-supported BCI utilizing an n-gram word prediction model. The performance of twelve healthy participants was assessed with the information transfer rate (ITR) and accuracy. All participants completed sentence spelling tasks, reaching average accuracies of 94% and 96.3% for the c-VEP and the SSVEP paradigm, respectively. Average ITRs around 57 bpm were achieved for both paradigms.

Citation: Gembler F, Stawicki P, Saboor A, Volosyak I (2019) Dynamic time window mechanism for time synchronous VEP-based BCIs—Performance evaluation with a dictionary-supported BCI speller employing SSVEP and c-VEP. PLoS ONE 14(6): e0218177. https://doi.org/10.1371/journal.pone.0218177

Editor: Zhishun Wang, Columbia University, UNITED STATES

Received: December 7, 2018; Accepted: May 28, 2019; Published: June 13, 2019

Copyright: © 2019 Gembler et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data sets cannot be shared according to legal guidelines. All participants were informed that their data will be deleted after a certain time period. This study was approved by the ethics committee of: Ethik-Kommission / Medizinische Fakultät der Universität Duisburg-Essen Robert-Koch-Str. 9, 45147 Essen. Fax: +49 (0) 201/723-1847. E-mail: ethikkommission@uk-essen.de. Website: www.uni-due.de/ethikkommission.

Funding: This research was supported by the European Fund for Regional Development (EFRD - or EFRE in German) under Grants GE-1-1-047, and IT-1-2-001. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Brain-Computer Interfaces (BCIs) detect, analyze, and decode brain activities to provide communication with the external environment, without involving any muscle activities [1]. The brain activities are usually recorded non-invasively by an electroencephalogram (EEG). BCIs may be used as a communication tool for severely impaired people [2, 3].

In comparison to other BCI paradigms, BCIs based on visual evoked potentials (VEPs) yield the fastest spelling performance [4–7]. VEPs are brain responses to a visual stimulus and they are usually categorized according to the type of the modulation stimulus. The most commonly used VEPs in BCI research are the frequency modulated steady-state visual evoked potentials (SSVEPs or f-VEPs) [5, 8, 9] and the code-modulated VEPs (c-VEPs) [7, 10, 11]. Recently, an information transfer rate (ITR) of 325.33 bpm has been achieved with a 40-target BCI using the SSVEP paradigm [5]. In the SSVEP paradigm, each target to be selected flashes at a specific frequency. When such a stimulus is fixated, continuous brain responses are elicited at the occipital and parietal cortical areas of the brain; the fundamental frequency of the stimulus as well as its harmonics can be detected.

In contrast, for c-VEP BCIs, all targets are modulated with different time lags of the same code sequence [12]. Usually pseudorandom m-sequences which have good autocorrelation properties are used [12].

During the last decades, the spelling interfaces are among the most widely utilized applications in BCI research [13]. To demonstrate usability and feasibility of such applications, various performance metrics such as the ITR can be applied [14]. Evaluation of BCI performance of such applications typically requires participants to type predefined words or phrases.

While SSVEP BCIs can be realized without training sessions, highest spelling speeds are achieved if pre-recorded user EEG data are used in a specific way to classify the attended target [15]. While this approach yields high ITRs, usually static classification time windows, which are dependent on the recording session, are used. For c-VEP-based BCIs, static time windows are the standard because of the required synchronization between EEG data collection and stimulus representation, and the fixed length of the code sequence; sliding EEG data windows are technically much harder to realize.

If the synchronous approach is utilized in VEP-BCIs (employing either SSVEP or c-VEP flashing patterns), the pre-recorded template with the maximum correlation to the recorded data determines the output command. Typically, applications introduce a flickering pause after a command is classified (flickering and, if applicable, data collection pause), which allows the user to shift the gaze to the next target. In the literature, this stimulation pause is often referred to as gaze shifting period [16], gaze shifting time or cue duration [17], break between trials [18], or rest period [19]. After this pre-set gaze shifting period, the stimuli continue flickering for exactly one stimulation cycle again, i.e. the user cannot influence the duration of the flickering. The BCI outputs are determined at equidistant time points. While this approach leads to high target selection speeds if short classification windows are used, it might be impractical in longterm daily use as it can easily lead to unintentional target activation.

Besides high responsiveness, a natural user-BCI interaction requires the system to distinguish between intentional and unintentional gaze fixation. Especially, during long term use, the user might need longer time to locate a desired target on the screen; for example, in spelling applications, the time the user needs to shift his/her gaze to a desired letter depends on many factors, such as familiarity and complexity of the letter arrangement. Complex user interface extensions like dictionary-based word suggestion mechanisms might update the selection options after every command [20]. Hence, a more flexible classification approach is needed. Therefore, we utilize flexible threshold-based classification time windows in this study.

For the SSVEP paradigm, such flexible classification times have been realized using classification thresholds [18, 21]. For instance, in the SSVEP-based Bremen BCI speller, the target activation corresponding to a specific frequency is only performed, if a pre-defined threshold is surpassed; otherwise, the classified command is rejected [21].

Classification thresholds make the system more robust, but can slow down the output information transfer rate, as it takes extra time until a threshold is surpassed. In spelling applications, this performance drop can be compensated using word prediction methods, which have been widely used in BCI research [20, 22]. Prediction methods based on n-gram models have only recently been realized in BCIs [23]. The n-gram model is used for the prediction of the next items in a sequence. Item probabilities are extracted from a text database. While it is typically used with individual characters as items in BCI, here, the n-gram model was utilized at the word level, i.e., the words with highest probability were suggested based on the previously entered text.

Materials and methods

Participants

Twelve healthy participants were recruited for this experiment, eight female and four male (average age 23.75 years, SD 2.35, range 21 to 30 years). All subjects had normal or corrected to normal vision. The research was approved by the ethical committee of the medical faculty of University Duisburg-Essen. Prior to the experiment, the participants were informed about the purpose, risks, and design of the study. The subjects who agreed to participate in the study signed an informed consent in accordance with the Helsinki declaration. Information needed for the analysis of the experiments was stored anonymously during the experiments. The participants had the opportunity to opt-out of the study at any time. All subjects received a financial reward for their participation.

Hardware

The used computer (MSI GT 73VR with nVidia GTX1070 graphic card) operated on Microsoft Windows 10 Education running on an Intel processor (Intel Core i7, 2.70 GHz). A liquid crystal display screen (Asus ROG Swift PG258Q, 1920 × 1080 pixel, 240 Hz maximal refresh rate) was used.

An EEG amplifier (g.USBamp, Guger Technologies, Graz, Austria) was used, utilizing all its 16 signal channels. The 16 signal electrodes were placed according to the international 10/5 system of electrode placement (see, e.g., [24] for more details): P_Z, P₃, P₄, P₅, P₆, PO₃, PO₄, PO₇, PO₈, POO₁, POO₂, O₁, O₂, O_Z, O₉, and O₁₀. The reference electrode was placed at C_Z and the ground electrode at AF_Z. Standard abrasive electrolytic electrode gel was applied between the electrodes and the scalp to bring impedances below 5 kΩ during the preparation phase. An analogue band pass filter (between 2 and 60 Hz) and a notch filter (around 50 Hz) were applied in the amplifier.

Stimulus presentation

In this experiment, the number of stimulus classes, K, was set to 8. Therefore, a distinct flashing pattern was assigned to each target. Two stimulus types, SSVEP- and c-VEP stimuli, were tested consecutively.

c-VEP stimulus presentation.

The target stimuli consisted of eight boxes (230 × 230 pixel) arranged as 2 × 4 stimulus matrix (see section Software for more details).

The c-VEP paradigm is often realized by employing the so-called m-sequences, non-periodic binary codes with good autocorrelation properties [12]. According to Wei et al. [10], a modulation sequence with a length of 63 bit and a lag of 4 bits between adjacent stimuli yields good performance.

Hence, for the flashing pattern 63 bit m-sequences c_i, i = 1, …, K were assigned to the stimulus matrix employing a circular shift of 4 bits (c₁ had no shift, c₂ was shifted by 4 bits to the left, c₃ was shifted by 8 bits, etc.). The codes were assigned row-wise to the matrix (i.e. starting from the upper left target with c₁, further targets were assigned in row major order). The stimuli corresponding to the codes alternated between the states ‘black’ (the background color, represented by ‘0’) and ‘white’ (represented by ‘1’). Here, c₁ was defined as

The duration of a stimulus cycle in seconds can be calculated by dividing the code length by the monitor refresh rate r in Hz; in this experiment, 63/60 = 1.05s.

SSVEP stimulus presentation.

For the SSVEP flashing pattern, a specific frequency f and phase Φ were assigned to each target [25]. The flickering was realized by sinusoidally modulating their transparencies in accordance with the frequency/phase combination, as described e.g. in [26, 27].

For this, the stimulus sequence for the i-th target is calculated as follows: (1) yielding values in the range from 0 to 1. Alpha compositing, i.e., the combination of an image with the background, was utilized to modulate the transparency of the stimulus according to this sequence. The alpha channel of the RGBA color space indicates how opaque a pixel is. An alpha value of zero, α = 0, corresponds to full transparency and α = 255 corresponds to no transparency. A black background was used for the stimuli. The color value of a stimulus was set to RGBA = (255, 255, 255, α), where alpha was set to c_i(t) * 255. As a black background was used, this resulted in the target color ‘black’ if c_i(t) = 0 and ‘white’ if c_i(t) = 1.

Similar to [26], frequencies f_i = f₀ + Δf and phases Φ_i = Φ₀ + ΔΦ, i = 1, …, K, with f₀ = 8 Hz, Δf = 1 Hz, Φ₀ = 0 and ΔΦ = 0.35π, where assigned column-wise to the stimulus matrix.

This frequency range was chosen, as it avoids mutual influences between fundamental and harmonic frequencies. Further, due to the 1-Hz difference between stimuli, the stimulus cycle is of length r, in other words, the repetition period is 1 s.

Experimental protocol

Participants sat on a chair facing the LCD screen (at a distance of approximately 60 cm). After they were prepared for the EEG recording, they went through two sessions (c-VEP and SSVEP). Each session consisted of a training phase (for template recording and automated parameter setup), an on-line copy spelling phase, and a brief questionnaire. The experiment took approximately one hour for each participant. The order of the starting paradigm was altered for every other participant. Hence half of the participants started the experiment with the SSVEP paradigm, the other half with the c-VEP paradigm.

Training phase.

In the training phase, each of the eight stimuli was fixated several times. For each trial the code pattern repeated for three cycles, i.e., the stimuli flickered for 3 * 1 = 3s for the SSVEP paradigm and for 3 * 1.05 = 3.15s for the c-VEP paradigm. A green frame around the box indicated which box the user had to fixate. Initially, subjects pressed the space bar to start the stimulation.

The recording was grouped in six training blocks, n_b = 6. In each block every stimulus was attended once, resulting in 6 * 8 = 48 trials in total. After each trial, the next box was highlighted, the flickering paused for one second, and the participant shifted his/her gaze to the next target.

In order to avoid visual fatigue, subjects were allowed to take breaks after each block of eight trials (the recording automatically paused). To start another recording block, the subjects needed to click the space bar.

Copy spelling phase.

Prior to the copy spelling task, a brief familiarization run was performed were participants spelled the word KLEVE, and a word of free choice (e.g. the own first name). During this familiarization run, in some cases, the automatically determined classification thresholds were lowered manually to increase the responsiveness of the application. In the copy spelling phase, participants were asked to spell the words BCI and BRAIN as well as a longer English sentence. For each participant and paradigm, different sentences were used. Occurring errors were corrected using the UNDO function of the interface.

Questionnaires.

Prior to the training phase, participants filled in a brief questionnaire, answering questions regarding gender and age. Additionally, after each session, participants gave their subjective impressions of the BCI answering questions regarding fatigue and annoyance. The questions as well as the collected answers of this questionnaires are provided in the results section.

Software

CCA-based spatial filters.

Canonical-correlation analysis (CCA) is a statistical method which investigates the relationship between two sets of variables [28]. Given two multidimensional variables and , CCA finds weight vectors and that maximize the correlation ρ between the linear combinations x = X^Tw_X and y = Y^Tw_Y. The weight vectors w_X and w_Y are determined by solving (2)

The value ρ is the first and maximal canonical correlation and x and y represent the first pair of canonical variables. Successively, additional pairs x, y can be constructed by maximizing (2) with the constraint that they are orthogonal to the already found pairs. The number of canonical variable pairs is constrained by the dimensionality of the variables X and Y. In BCI research, CCA is used to find a linear transformation that maximizes the correlation between the recorded signal and the averaged template signals. Typically, only the first canonical correlation and corresponding weight is used for classification and construction of filters [4, 5, 29]. Nonetheless, some recent studies yielded better performance when the additional canonical variable pairs with their associated correlations and weights where also utilized [30].

Here, to improve signal-to noise ratio, CCA was used to create class-specific spatial filters (see, e.g., [5, 7]).

In this regard, filters were constructed from the training data as follows: The trials recorded during the training phase were stored in a m × n_t matrix, where m denotes the number of electrode channels (here m = 16) and n_t denotes the number of samples. All n_b trials corresponding to a specific class are averaged by calculating the arithmetic mean, yielding the template ,

For the filter, two additional matrices need to be constructed. All trials T_i are concatenated horizontally yielding a matrix ,

A second matrix with the same dimensions is constructed by replicating X,

By plugging and into Eq (2), the weight vector is obtained.

This procedure is applied for all classes, yielding a set of training templates X_i and weights w_i, i = 1, …, K.

Time window mechanism.

The output command corresponding to a classification was only performed if certain thresholds were surpassed. In this regard, sliding classification time windows of dynamic length were utilized [8], i.e., in the case where no classification could be made, a new classification was performed, after receiving the new EEG data.

The multichannel EEG signals that were about to be classified were stored in a matrix , where n_y represents the length of the classification time window in samples. In praxis, n_y needs to be selected carefully. Too small time windows can lead to errors [31]. On the other hand, if n_y is too large, data unrelated to the desired target (e.g. due to gaze movements at the beginning of the time window) remains to be considered for classification, which can slow down performance. Therefore, n_y needs to be restricted, (3)

The selection of the minimum time window, the lower bound in (3), is critical. Here, it was determined individually based on the training data as elaborated later. Recall that the stimulus cycle was 1 s and 1.05 s for the SSVEP and c-VEP approach respectively. The number of samples collected in this period, n_c, was therefore n_c = 600 for SSVEP and and n_c = 630 for c-VEP. The upper bound, n_y_max, was selected as a multiple of n_c.

In the on-line BCI, the time window extended incrementally. The amplifier transfers EEG data in blocks , where n_a denotes the number of samples per block. For the implementation of the sliding window mechanism, n_a was selected as divider of the cycle length, i.e., n_a|n_c.

A further restriction to n_a was given by the amplifier manufacturer. For the gUSBamp, the buffer needed to contain at least 20-30 ms of data. Here, n_a was set to 30 samples (50 ms recordings with the sampling rate of 600 Hz).

The amplifier blocks where accumulated in a buffer ,

If the number of samples of the buffer, , was to small, i.e., , no classification was performed.

If , the classification time window gradually increased (with step width n_a). The classification was performed using the data matrix , i.e., all data from the buffer were considered for classification. If the classifier did not meet a certain threshold criterion, as described later, further EEG data was collected.

Lastly, if , only the last n_y_max samples were used for classification. Therefore, we set Y as the sub-matrix of formed from rows 1, …, m and columns , where k is the smallest integer such that n_y follows (3). Since n_a|n_c, data collection and stimulus presentation remain synchronized.

Classification.

The test matrix is compared against individual reference templates , i = 1, …, K each of which was set as the sub-matrix of the corresponding training template X_i, formed from rows 1, …, m and columns 1, …, n_y, (4)

As the spatial filters for each target are similar to each other, we adopted the ensemble-based target identification which was proposed originally for the SSVEP paradigm in [5]. Ensemble correlations, λ_k, were determined by stacking all target-specific spatial filtered data and template vectors as follows: (5)

Additionally, the difference between target- and non-target correlations can be enhanced further by applying a filter bank method, which decomposes VEP-data in sub-band components as described in [17]. The lower and upper cut-off frequencies for the m-th sub-band were selected as m * 8 and 60 Hz. To this end, an 8th order Butterworth filter was utilized. Forward and reverse filtering were applied to cancel the phase response [32].

The ensemble approach, Eq (5), was then applied to each sub-band component individually, yielding a set of correlations , k = 1, …, K, where M denotes the number of considered sub-bands. Then, the output command candidate was determined using weighted linear combinations of the correlations, (6)

Mirroring the decrease in amplitude in the higher bands, the weights a_m in (6) where set as (7) yielding decreasing weights for the higher bands. The optimal choice of these weights needs to be investigated further (see, e.g. [17]).

The number of sub-bands, M, was set to 1 and 5, for c-VEP and SSVEP, respectively. In the on-line copy spelling phase, the action associated with a classified label C was only performed if a certain threshold criterion was met, which is described in the following. The decision certainty, Δ_C, was calculated as the distance between the highest and second highest correlation. The output was performed if Δ_C exceeded a certain threshold value, β; otherwise the, classifier output was rejected. In other words, the output command was only performed if n_y ≥ n_y_min and Δ_C ≥ β.

After a produced output command, the data buffers B and Y were cleared and a 1 second gaze shifting period followed. In this gaze shifting period, the amplifier data blocks, A_i, were ignored and the stimuli did not flicker, allowing the user to shift his/her gaze to the next target. Please note that the BCI did not require a full cycle of the stimulation pattern for classification. If a command was classified before the stimulus pattern completed a full cycle (1.05 s for c-VEP and and 1 s SSVEP), the flickering stopped.

Automatic parameter calibration.

In order to realize individually optimized system parameters, the values for the classification threshold β and for the minimum time window n_y_min were determined automatically for each participant on the basis of the training data via a leave-one-out cross-validation.

In this regard, we considered the ITR in bpm (see [1], an on-line calculation tool can be found at https://bci-lab.hochschule-rhein-waal.de/en/itr.html), (8) where the target identification accuracy, p, was calculated based on the number of correctly classified commands divided by the total number of commands, and t represented the average time needed to make a selection (in s).

Utilizing leave-one-out cross-validation on the training data, an average ITR was calculated for classification windows of n_y = 30, 60, …, 3n_c samples. The value of n_y that maximized the ITR (8) was selected as minimum time window n_y_min. The classification threshold β was selected as the minimal decision certainty, Δ_C, at that time window. An example of the parameter setup procedure is depicted in Fig 1.

Download:

Fig 1. Example of the proposed automatic selection of the classification parameters.

Displayed are the results from the leave-one-out off-line analysis of the SSVEP training data for one participant. In the training session, each of the 8 targets was attended 6 times for 3 seconds. (A) Displayed are the correlation values of the target stimulus (green) and the maximum correlation of the non-target stimuli (red). For each stimulus class the correlations calculated in the first second are displayed. (B) ITR averaged over the 6 training blocks. Here, the time window yielding the highest ITR was 0.45 seconds. (C) Correlogram of the training data. Depicted are the correlations for each target using the determined time window, averaged over the trials. This minimal difference between target and non-target stimuli was used as the threshold value. Here, the distance was minimal for the 10 and 14 Hz pair, yielding a difference of 0.2.

https://doi.org/10.1371/journal.pone.0218177.g001

As stated in the experimental protocol, the suggested thresholds were sometimes lowered manually in the on-line spelling tasks. An explanation for the sometimes lower certainty in on-line tasks is that in contrast to the cue-guided training, it was not guaranteed that participants were fixating the target when the flickering started.

Dictionary-driven spelling application.

The spelling application utilized n-gram prediction, which is used in computational linguistics. An n-gram describes a sequence of n items from a text database. An item (here, a word) x_i has the probability P(x_i|x_{i−(n − 1)}, …, x_i−1). The text database was extracted from the Leipzig Corpora Collection, a ready to use corpora [33]. It contains a word frequency list as well as a word bi-grams list (co-occurrences as next neighbors) containing observed frequency counts, which were generated from approximately 1 million sentences publicly accessible.

Here, an n-gram of size 2 (also called bi-gram) was utilized, i.e., next word candidates were weighted according to the probability on the word level.

Structured query language (SQL), a query language for relational databases, was used to retrieve word suggestions from the Leipzig text database. Based on the already typed string, three word suggestions where extracted using SQL statements. First, all co-occurrence pairs, including the previously typed word and the words beginning with the already typed part of the current word, were ordered according to their frequency. If this yielded less than three candidates, the suggestions were complemented with the word frequency list (independent of the precedent word), i.e., the most frequent words matching the already typed string were added. The speller presented eight selection options, arranged in a 2 × 4 matrix format (Fig 2). One of two layers was shown. In Layer I, the first row contained 28 characters (26 letters, underscore and full stop character) divided into four groups of seven characters. The second row contained the three dictionary suggestions as well as a correction option (undo the previous command). By selecting one of the group boxes from the first row, Layer II was presented, which allowed for the selection of individual characters.

Download:

Fig 2. Writing a sentence with the dictionary-driven speller.

A participant is writing JUST_DO_IT in seven steps. Selection of individual letters required two steps: Initially the group containing the character was selected (Layer I), and then the desired character was selected (Layer II).

https://doi.org/10.1371/journal.pone.0218177.g002

The copy-spelling sentence and the user output were presented in the center of the screen. If the classifier produced an output command, audio and visual feedback were provided: The size of the selected box increased for a short time and a sound file, voicing the selected command, was played.

The functioning of the dictionary-driven spelling application is illustrated in Fig 2.

Results

Off-line performance

Optimal time windows and ITRs were calculated using off-line leave-one-out cross-validation. As expected, the highest ITRs were achieved with different time windows for each user. The time window yielding maximal ITR, which was used as minimal time window in the on-line experiment, as well as the corresponding maximum ITR are listed in Table 1. Fig 3 shows off-line ITRs and accuracies across all participants for a time window length up to 1.05 s with an interval of 50 ms. On average, participants reached a theoretical maximal ITR of 100.90 bpm and 95.21 bpm with the optimal time window for the c-VEP and SSVEP paradigm. However, the difference between the paradigms was not statistically significant (t = 1.488, p = 0.165).

Download:

Fig 3. Comparison of SSVEP and c-VEP methods.

ITRs and accuracies of all participants are presented. The asterisks mark statistical significance determined with paired sample t-tests (*p < 0.05 and **p < 0.01).

https://doi.org/10.1371/journal.pone.0218177.g003

Download:

Table 1. Information transfer rate (ITR) in bits/min for each participant for the c-VEP and SSVEP system.

https://doi.org/10.1371/journal.pone.0218177.t001

On-line spelling performance

The on-line performance was evaluated utilizing the output command accuracy, the ITR, as well as the output characters per minute (OCM) which measures typing speed by dividing the total number of output characters by the time needed to type them [14]. Measuring the OCM therefore takes into account that all errors are corrected. Tables 1, 2 and 3 summarize spelling results across all participants for each paradigm and spelling task.

Download:

Table 2. c-VEP spelling task accuracies and output commands per minute (OCM).

https://doi.org/10.1371/journal.pone.0218177.t002

Download:

Table 3. SSVEP spelling task accuracies and output commands per minute (OCM).

https://doi.org/10.1371/journal.pone.0218177.t003

For the single word spelling tasks (BCI and BRAIN), the average ITR was 92.65 and 75.06 bpm for the c-VEP and SSVEP system; the difference between the paradigms was significant (t = 2.503, p = 0.029). As expected, these values were lower than the off-line values because of the threshold criterion described in the previous section.

For the sentence spelling tasks, no statistically significant difference between the paradigms was found; ITRs of 57.11 and 57.26 bpm were achieved for the c-VEP and SSVEP paradigm.

Due to the n-gram prediction model, the average OCM was higher in the sentence spelling tasks in comparison to the word spelling tasks for the two tested paradigms. For the single word spelling tasks, 15.43 and 12.63 OCM were reached; for the sentence spelling tasks, 18.89 and 18.31 OCM were achieved with the c-VEP and SSVEP paradigm.

Questionnaire results

Further, the results from the questionnaires are depicted in Fig 4. For an overview of the pre-and post-questionnaire answers, see also Table 4. The subjective impressions regarding fatigue level and annoyance were measured using a five-point Likert scale [34], where “1” indicated the strongest degree of disagreement and “5” the strongest degree of agreement.

Download:

Table 4. Questionnaire answers.

https://doi.org/10.1371/journal.pone.0218177.t004

Download:

Fig 4. Results from the user questionnaires.

Responses were given on a 1-5 Likert scale, 1 indicating strong disagreement and 5 indicating strong agreement.

https://doi.org/10.1371/journal.pone.0218177.g004

The majority of the users did not find the flickering annoying or fatiguing. Overall, answers regarding the user-friendliness were slightly more positive for the SSVEP paradigm. Five out of twelve participants stated that they found the flickering of the c-VEP system annoying. In contrast, only one participant found the SSVEP to be annoying. In respect to the subjective level of fatigue, the SSVEP paradigm yielded better results. Four participants found the c-VEP flickering fatiguing, but only two participants stated that the SSVEP flickering caused fatigue.

A spearman rank correlation test was performed to investigate if the tiredness affected the spelling performance. Neither for the SSVEP experiment nor for the c-VEP experiment a significant relationship was found between the subjective level of tiredness and the mean ITR of the sentence spelling task (r_s = −0.43, n.s for SSVEP and r_s = −0.44, n.s. for c-VEP).

Discussion

The main purpose of the presented study was to investigate methods of dynamic gaze classification time windows for time synchronous VEP BCIs. The proposed methods lead to a more natural user-BCI interaction. To demonstrate the robustness of the approach, a dictionary-driven spelling application was tested with the SSVEP and the c-VEP paradigm. In this sense, the study also provides a direct comparison between c-VEP and SSVEP stimulation, both in terms of performance and user-friendliness.

The comparison of the two stimulation approaches indicates that c-VEP slightly outperforms SSVEP in terms of ITR, while SSVEP is preferred by most users in terms of user-friendliness, see Figs 3 and 4.

For the SSVEP paradigm, the flickering was realized by sinusoidally modulating the transparencies. This allowed a slightly more subtle visual stimulation in comparison to the c-VEP flickering which switched from full illumination to no illumination in correspondence to the code patterns. Indeed, a slight difference regarding the subjective level of annoyance and fatigue caused by the flickering is evident from the questionnaires (Fig 4). Regarding the user-friendliness, most participants seemed to favor the SSVEP paradigm. It should be noted that a more subtle stimulation could be achieved by utilizing higher carrier frequencies for the c-VEP paradigm as well [35]. On the other hand, faster rate flickering could lead to a lower performance for some users [36]. A similar effect has been observed for SSVEP stimuli as well [37]. For SSVEP BCIs, a more subtle stimulation could also be achieved by utilizing motion-based stimulation [38]. Further, Chien et al. [39] achieved promising results with little flickering sensations by employing a composition of red/green/blue 32 Hz/40 Hz flashing lights.

As can be seen in Fig 3, the variability between subjects (inter-subject variability) seems to be slightly higher for the SSVEP paradigm than for the c-VEP paradigm. A reason for this could be that the tested frequencies interfere with the natural brain activity. As stated by Bin et al. [12], narrow-band signals in the natural EEG (e.g. alpha and beta rhythms) are likely to interfere with low frequency SSVEP stimuli.

Here, a training session was utilized in both methods for template recording and parameter optimization. It should be noted that in general, c-VEP BCIs require a training stage to obtain templates. SSVEP BCIs on the other hand can be realized without training if desired, by utilizing sine- and cosine templates. Typically, for the recording of the c-VEP templates, only one code sequence is utilized for the training session. By shifting the recorded data templates, the required amount of classes can be generated. In this study, however, templates were recorded and averaged separately for each c-VEP class. This was done for the following reasons. It facilitates an ensemble-based classification approach. Further, as demonstrated by Nagel et. al, monitor latencies differ depending on the position of the target [40]; using all targets to create one single template would require a correction of these latencies.

One of the key parameters for BCI performance is the time window used for the classification of the signals. User variability justifies user-dependent selection of a minimum value for the minimum classification time interval [31]. For the c-VEP paradigm, typically, this interval is determined by the cycle length; e.g., for 63 bit m-sequence stimuli with 60 Hz monitor refresh rate, a time window of 1.05 seconds can be utilized [10, 41]. SSVEP BCIs have been used with time windows as low as 0.3 s [5]. In some studies larger classification windows were incorporated to improve robustness of the system. For example, to outbalance the lower signal-to-noise-ratio with dry electrodes, Spüler [41] utilized larger classification windows in a c-VEP BCI by averaging over multiple trials. Similarly, in previous studies, we incorporated large classification windows to deal with age-related inter-subject variability in users [8, 42].

For the system presented in this paper, a dynamic threshold-based time window approach was utilized. Two conditions needed to be met before an output was produced:

The distance class yielding maximum correlation needed to surpass the other correlations by a certain threshold amount,
the time window for classification needed to be sufficiently long.

Here, the minimal time window was set user-specifically based on the ITR using the recorded training data. A similar approach was utilized in our previous study, where the time window was determined in relation to off-line accuracy [31].

Interestingly, although, good correlation properties of the m-sequence require a full stimulation cycle (here 1.05 s), high accuracies were achieved with an incomplete code sequence (see Fig 3). On average, the optimal classification time window in terms of off-line ITR was 0.57 s (ranging from 0.35 s to 1 s) for the c-VEP paradigm and 0.62 s (ranging from 0.25 s to 1.35 s) for the SSVEP paradigm. Hence, despite the comparably low number of targets, high ITRs were achieved. This can also be attributed to the utilization of ensemble methods, which can significantly increase system speed [5]. Additionally, for the SSVEP paradigm a filter bank approach as proposed by Chen et al. [17] was utilized to enhance target discrimination. Similar methods could also enhance the classification accuracy for the c-VEP approach. Therefore, our future work will focus on investigating suitable cut-off frequencies for the c-VEP paradigm.

Regarding the copy spelling phase, the number of output characters per minute was on average significantly higher for the sentence spelling tasks, in comparison to the word spelling tasks where no dictionary suggestions were used (see Tables 2 and 3). This demonstrates the robustness of the proposed time window approach as well as the effectiveness of the implemented word suggestion methods. It should be noted, that participants did not always use the dictionary whenever they had the chance to do so (suggestions were simply overseen). For this reason, in some cases, the single word OCM was higher in comparison to the sentence task. Improvements could be made in regards to the arrangement of the GUI targets to make the suggestions more prominent. For some participants, a longer gaze shifting phase when suggestions are presented could also be helpful. It should further be noted, that participants used the system for the first time. More experience with the GUI could improve OCM as well.

Overall, there was surprisingly little performance difference between the two stimulation modalities. The c-VEP stimulation patterns yielded slightly higher off-line ITRs and significantly higher ITRs in word copy spelling tasks. In on-line sentence spelling, the speed difference becomes negligible. This is due to the fact that in sentence spelling, usually larger search phases are required to find the next letter or word to select.

Conclusion

A dynamic classification time window approach for time synchronous VEP BCIs was proposed. The optimal time window was determined individually deduced from a training session which was also used for the generation of templates and spatial filters. An 8-target spelling application utilizing n-gram-based word suggestions was used to evaluate the usability of the developed methods. Twelve participants tested the system in on-line spelling tasks with the SSVEP and the c-VEP paradigm. The presented study demonstrates the robustness of the proposed approach. All participants completed sentence spelling and word spelling tasks with accuracies well above 90% for the two paradigms. Ensemble-based classification strategies were employed in both cases. The proposed methods were equally effective for c-VEP and SSVEP based systems in terms of ITR; mean ITRs of approximately 57 bpm were achieved in both cases. Nevertheless, in word spelling tasks, the c-VEP system (mean word spelling ITR 92.7 bpm) outperformed the SSVEP system (ITR 75.1 bpm). In terms of user-friendliness however, the SSVEP paradigm was preferred by most participants. The results suggest that the stimulation pattern (SSVEP vs. c-VEP) could be selected based on the user preference. In terms of speed, the optimal paradigm could be determined individually for each user in a short training session. However, the perceived level of user-friendliness should also be taken into account, as it might be more relevant for end users than pure system speed.

References

1. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–Computer Interfaces for Communication and Control. Clinical Neurophysiology. 2002;113(6):767–791. pmid:12048038
- View Article
- PubMed/NCBI
- Google Scholar
2. Kübler A, Furdea A, Halder S, Hammer EM, Nijboer F, Kotchoubey B. A Brain-Computer Interface Controlled Auditory Event-Related Potential (P300) Spelling System for Locked-In Patients. Annals of the New York Academy of Sciences. 2009;1157(1):90–100. pmid:19351359
- View Article
- PubMed/NCBI
- Google Scholar
3. Guy V, Soriani MH, Bruno M, Papadopoulo T, Desnuelle C, Clerc M. Brain Computer Interface with the P300 Speller: Usability for Disabled People with Amyotrophic Lateral Sclerosis. Annals of Physical and Rehabilitation Medicine. 2018;61(1):5–11. pmid:29024794
- View Article
- PubMed/NCBI
- Google Scholar
4. Wei Q, Gong H, Lu Z. Grouping Modulation with Different Codes for Improving Performance in cVEP-Based Brain–Computer Interfaces. Electronics Letters. 2017;53(4):214–216.
- View Article
- Google Scholar
5. Nakanishi M, Wang Y, Chen X, Wang YT, Gao X, Jung TP. Enhancing Detection of SSVEPs for a High-Speed Brain Speller Using Task-Related Component Analysis. IEEE Transactions on Biomedical Engineering. 2018;65(1):104–112. pmid:28436836
- View Article
- PubMed/NCBI
- Google Scholar
6. Lin K, Gao S, Gao X. Boosting the information transfer rate of an SSVEP-BCI system using maximal-phase-locking value and minimal-distance spatial filter banks. Tsinghua Science and Technology. 2019;24(3):262–270.
- View Article
- Google Scholar
7. Spüler M, Rosenstiel W, Bogdan M. Online Adaptation of a C-VEP Brain-Computer Interface(BCI) Based on Error-Related Potentials and Unsupervised Learning. PLoS ONE. 2012;7(12):e51077. pmid:23236433
- View Article
- PubMed/NCBI
- Google Scholar
8. Volosyak I, Gembler F, Stawicki P. Age-Related Differences in SSVEP-Based BCI Performance. Neurocomputing. 2017;250:57–64.
- View Article
- Google Scholar
9. Rejer I, Cieszyński L. Independent Component Analysis for a Low-Channel SSVEP-BCI. Pattern Analysis and Applications. 2019;22(1):47–62.
- View Article
- Google Scholar
10. Wei Q, Feng S, Lu Z. Stimulus Specificity of Brain-Computer Interfaces Based on Code Modulation Visual Evoked Potentials. PloS one. 2016;11(5):e0156416. pmid:27243454
- View Article
- PubMed/NCBI
- Google Scholar
11. Shirzhiyan Z, Keihani A, Farahi M, Shamsi E, GolMohammadi M, Mahnam A, et al. Introducing Chaotic Codes for the Modulation of Code Modulated Visual Evoked Potentials (c-VEP) in Normal Adults for Visual Fatigue Reduction. PLOS ONE. 2019;14(3):e0213197.
- View Article
- Google Scholar
12. Bin G, Gao X, Wang Y, Hong B, Gao S. VEP-Based Brain-Computer Interfaces: Time, Frequency, and Code Modulations [Research Frontier]. IEEE Comput Intelli Mag. 2009;4(4):22–26.
- View Article
- Google Scholar
13. Rezeika A, Benda M, Stawicki P, Gembler F, Saboor A, Volosyak I. Brain–Computer Interface Spellers: A Review. Brain Sciences. 2018;8(4). pmid:29601538
- View Article
- PubMed/NCBI
- Google Scholar
14. Speier W, Arnold C, Pouratian N. Evaluating True BCI Communication Rate through Mutual Information and Language Models. PLoS ONE. 2013;8(10):e78432. pmid:24167623
- View Article
- PubMed/NCBI
- Google Scholar
15. Zerafa R, Camilleri T, Falzon O, Camilleri KP. To Train or Not to Train? A Survey on Training of Feature Extraction Methods for SSVEP-Based BCIs. Journal of Neural Engineering. 2018;15(5):051001. pmid:29869996
- View Article
- PubMed/NCBI
- Google Scholar
16. Stawicki P, Gembler F, Rezeika A, Volosyak I. A Novel Hybrid Mental Spelling Application Based on Eye Tracking and SSVEP-Based BCI. Brain Sciences. 2017;7(4):35.
- View Article
- Google Scholar
17. Chen X, Wang Y, Gao S, Jung TP, Gao X. Filter Bank Canonical Correlation Analysis for Implementing a High-Speed SSVEP-Based Brain–Computer Interface. J Neural Eng. 2015;12(046008):046008. pmid:26035476
- View Article
- PubMed/NCBI
- Google Scholar
18. da Cruz JN, Wan F, Wong CM, Cao T. Adaptive Time-Window Length Based on Online Performance Measurement in SSVEP-Based BCIs. Neurocomputing. 2015;149:93–99.
- View Article
- Google Scholar
19. Kelly S, Lalor E, Reilly R, Foxe J. Visual Spatial Attention Tracking Using High-Density SSVEP Data for Independent Brain-Computer Communication. IEEE Trans Neural Syst Rehabil Eng. 2005;13(2):172–178. pmid:16003896
- View Article
- PubMed/NCBI
- Google Scholar
20. Stawicki P, Gembler F, Volosyak I. A User-Friendly Dictionary-Supported SSVEP-Based BCI Application. In: Gamberini L, Spagnolli A, Jacucci G, Blankertz B, Freeman J, editors. Symbiotic Interaction: 5th International Workshop, Symbiotic 2016, Padua, Italy, September 29–30, 2016, Revised Selected Papers. Cham: Springer International Publishing; 2017. p. 168–180.
21. Volosyak I, Cecotti H, Valbuena D, Graser A. Evaluation of the Bremen SSVEP Based BCI in Real World Conditions. In: Rehabilitation Robotics, 2009. ICORR 2009. IEEE International Conference On. IEEE; 2009. p. 322–331.
22. Akce A, Norton JJ, Bretl T. An SSVEP-Based Brain-Computer Interface for Text Spelling With Adaptive Queries That Maximize Information Gain Rates. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2015;23(5):857–866. pmid:25474810
- View Article
- PubMed/NCBI
- Google Scholar
23. Speier W, Arnold C, Pouratian N. Integrating Language Models into Classifiers for BCI Communication: A Review. Journal of Neural Engineering. 2016;13(3):031002. pmid:27153565
- View Article
- PubMed/NCBI
- Google Scholar
24. Jurcak V, Tsuzuki D, Dan I. 10/20, 10/10, and 10/5 Systems Revisited: Their Validity as Relative Head-Surface-Based Positioning Systems. NeuroImage. 2007;34(4):1600–1611. pmid:17207640
- View Article
- PubMed/NCBI
- Google Scholar
25. Manyakov NV, Chumerin N, Robben A, Combaz A, van Vliet M, Van Hulle MM. Sampled Sinusoidal Stimulation Profile and Multichannel Fuzzy Logic Classification for Monitor-Based Phase-Coded SSVEP Brain–Computer Interfacing. Journal of Neural Engineering. 2013;10(3):036011. pmid:23594762
- View Article
- PubMed/NCBI
- Google Scholar
26. Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S. High-Speed Spelling with a Noninvasive Brain–Computer Interface. Proceedings of the National Academy of Sciences. 2015;112(44):E6058–E6067.
- View Article
- Google Scholar
27. Nakanishi M, Wang Y, Wang YT, Mitsukura Y, Jung TP. Generating Visual Flickers for Eliciting Robust Steady-State Visual Evoked Potentials at Flexible Frequencies Using Monitor Refresh Rate. PLoS ONE. 2014;9(6):e99235. pmid:24918435
- View Article
- PubMed/NCBI
- Google Scholar
28. Hotelling H. RELATIONS BETWEEN TWO SETS OF VARIATES. Biometrika. 1936;28(3-4):321–377.
- View Article
- Google Scholar
29. Lin Z, Zhang C, Wu W, Gao X. Frequency Recognition Based on Canonical Correlation Analysis for SSVEP-Based BCIs. IEEE Transactions on Biomedical Engineering. 2007;54(6):1172–1176. pmid:17549911
- View Article
- PubMed/NCBI
- Google Scholar
30. Mondini V, Mangia AL, Talevi L, Cappello A. Sinc-Windowing and Multiple Correlation Coefficients Improve SSVEP Recognition Based on Canonical Correlation Analysis. Computational Intelligence and Neuroscience. 2018;2018:1–11.
- View Article
- Google Scholar
31. Gembler F, Stawicki P, Volosyak I. Autonomous Parameter Adjustment for SSVEP-Based BCIs with a Novel BCI Wizard. Frontiers in Neuroscience. 2015;9. pmid:26733788
- View Article
- PubMed/NCBI
- Google Scholar
32. Gustafsson F. Determining the Initial States in Forward-Backward Filtering. IEEE Transactions on Signal Processing. 1996;44(4):988–992.
- View Article
- Google Scholar
33. Eckart T, Quasthoff U. Statistical Corpus and Language Comparison on Comparable Corpora. In: Sharoff S, Rapp R, Zweigenbaum P, Fung P, editors. Building and Using Comparable Corpora. Berlin, Heidelberg: Springer Berlin Heidelberg; 2013. p. 151–165.
34. Likert R. A Technique for the Measurement of Attitudes. Archives of Psychology. 1932;22 140:55–55.
- View Article
- Google Scholar
35. Wittevrongel B, Van Wolputte E, Van Hulle MM. Code-Modulated Visual Evoked Potentials Using Fast Stimulus Presentation and Spatiotemporal Beamformer Decoding. Scientific Reports. 2017;7(1). pmid:29118386
- View Article
- PubMed/NCBI
- Google Scholar
36. Gembler F, Stawicki P, Rezeika A, Saboor A, Benda M, Volosyak I. Effects of Monitor Refresh Rates on C-VEP BCIs. In: Ham J, Spagnolli A, Blankertz B, Gamberini L, Jacucci G, editors. Symbiotic Interaction. Cham: Springer International Publishing; 2018. p. 53–62.
37. Volosyak I, Cecotti H, Gräser A. Impact of Frequency Selection on LCD Screens for SSVEP Based Brain-Computer Interfaces. In: Cabestany J, Sandoval F, Prieto A, Corchado JM, editors. Bio-Inspired Systems: Computational and Ambient Intelligence. vol. 5517. Berlin, Heidelberg: Springer Berlin Heidelberg; 2009. p. 706–713.
38. Zhang X, Xu G, Xie J, Zhang X. Brain Response to Luminance-Based and Motion-Based Stimulation Using Inter-Modulation Frequencies. PLOS ONE. 2017;12(11):e0188073. pmid:29141030
- View Article
- PubMed/NCBI
- Google Scholar
39. Chien YY, Lin FC, Zao JK, Chou CC, Huang YP, Kuo HY, et al. Polychromatic SSVEP Stimuli with Subtle Flickering Adapted to Brain-Display Interactions. Journal of Neural Engineering. 2017;14(1):016018. pmid:28000607
- View Article
- PubMed/NCBI
- Google Scholar
40. Nagel S, Dreher W, Rosenstiel W, Spüler M. The Effect of Monitor Raster Latency on VEPs, ERPs and Brain–Computer Interface Performance. Journal of Neuroscience Methods. 2018;295:45–50. pmid:29197616
- View Article
- PubMed/NCBI
- Google Scholar
41. Spüler M. A High-Speed Brain-Computer Interface (BCI) Using Dry EEG Electrodes. PLOS ONE. 2017;12(2):e0172400. pmid:28225794
- View Article
- PubMed/NCBI
- Google Scholar
42. Gembler F, Stawicki P, Volosyak I. A Comparison of SSVEP-Based BCI-Performance Between Different Age Groups. In: Rojas I, Joya G, Catala A, editors. Advances in Computational Intelligence: 13th International Work-Conference on Artificial Neural Networks, IWANN 2015, Palma de Mallorca, Spain, June 10-12, 2015. Proceedings, Part I. Cham: Springer International Publishing; 2015. p. 71–77.

[ref1] 1. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–Computer Interfaces for Communication and Control. Clinical Neurophysiology. 2002;113(6):767–791. pmid:12048038
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Kübler A, Furdea A, Halder S, Hammer EM, Nijboer F, Kotchoubey B. A Brain-Computer Interface Controlled Auditory Event-Related Potential (P300) Spelling System for Locked-In Patients. Annals of the New York Academy of Sciences. 2009;1157(1):90–100. pmid:19351359
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Guy V, Soriani MH, Bruno M, Papadopoulo T, Desnuelle C, Clerc M. Brain Computer Interface with the P300 Speller: Usability for Disabled People with Amyotrophic Lateral Sclerosis. Annals of Physical and Rehabilitation Medicine. 2018;61(1):5–11. pmid:29024794
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Wei Q, Gong H, Lu Z. Grouping Modulation with Different Codes for Improving Performance in cVEP-Based Brain–Computer Interfaces. Electronics Letters. 2017;53(4):214–216.
View Article
Google Scholar

[14] View Article

[15] Google Scholar

[ref5] 5. Nakanishi M, Wang Y, Chen X, Wang YT, Gao X, Jung TP. Enhancing Detection of SSVEPs for a High-Speed Brain Speller Using Task-Related Component Analysis. IEEE Transactions on Biomedical Engineering. 2018;65(1):104–112. pmid:28436836
View Article
PubMed/NCBI
Google Scholar

[17] View Article

[18] PubMed/NCBI

[19] Google Scholar

[ref6] 6. Lin K, Gao S, Gao X. Boosting the information transfer rate of an SSVEP-BCI system using maximal-phase-locking value and minimal-distance spatial filter banks. Tsinghua Science and Technology. 2019;24(3):262–270.
View Article
Google Scholar

[21] View Article

[22] Google Scholar

[ref7] 7. Spüler M, Rosenstiel W, Bogdan M. Online Adaptation of a C-VEP Brain-Computer Interface(BCI) Based on Error-Related Potentials and Unsupervised Learning. PLoS ONE. 2012;7(12):e51077. pmid:23236433
View Article
PubMed/NCBI
Google Scholar

[24] View Article

[25] PubMed/NCBI

[26] Google Scholar

[ref8] 8. Volosyak I, Gembler F, Stawicki P. Age-Related Differences in SSVEP-Based BCI Performance. Neurocomputing. 2017;250:57–64.
View Article
Google Scholar

[28] View Article

[29] Google Scholar

[ref9] 9. Rejer I, Cieszyński L. Independent Component Analysis for a Low-Channel SSVEP-BCI. Pattern Analysis and Applications. 2019;22(1):47–62.
View Article
Google Scholar

[31] View Article

[32] Google Scholar

[ref10] 10. Wei Q, Feng S, Lu Z. Stimulus Specificity of Brain-Computer Interfaces Based on Code Modulation Visual Evoked Potentials. PloS one. 2016;11(5):e0156416. pmid:27243454
View Article
PubMed/NCBI
Google Scholar

[34] View Article

[35] PubMed/NCBI

[36] Google Scholar

[ref11] 11. Shirzhiyan Z, Keihani A, Farahi M, Shamsi E, GolMohammadi M, Mahnam A, et al. Introducing Chaotic Codes for the Modulation of Code Modulated Visual Evoked Potentials (c-VEP) in Normal Adults for Visual Fatigue Reduction. PLOS ONE. 2019;14(3):e0213197.
View Article
Google Scholar

[38] View Article

[39] Google Scholar

[ref12] 12. Bin G, Gao X, Wang Y, Hong B, Gao S. VEP-Based Brain-Computer Interfaces: Time, Frequency, and Code Modulations [Research Frontier]. IEEE Comput Intelli Mag. 2009;4(4):22–26.
View Article
Google Scholar

[41] View Article

[42] Google Scholar

[ref13] 13. Rezeika A, Benda M, Stawicki P, Gembler F, Saboor A, Volosyak I. Brain–Computer Interface Spellers: A Review. Brain Sciences. 2018;8(4). pmid:29601538
View Article
PubMed/NCBI
Google Scholar

[44] View Article

[45] PubMed/NCBI

[46] Google Scholar

[ref14] 14. Speier W, Arnold C, Pouratian N. Evaluating True BCI Communication Rate through Mutual Information and Language Models. PLoS ONE. 2013;8(10):e78432. pmid:24167623
View Article
PubMed/NCBI
Google Scholar

[48] View Article

[49] PubMed/NCBI

[50] Google Scholar

[ref15] 15. Zerafa R, Camilleri T, Falzon O, Camilleri KP. To Train or Not to Train? A Survey on Training of Feature Extraction Methods for SSVEP-Based BCIs. Journal of Neural Engineering. 2018;15(5):051001. pmid:29869996
View Article
PubMed/NCBI
Google Scholar

[52] View Article

[53] PubMed/NCBI

[54] Google Scholar

[ref16] 16. Stawicki P, Gembler F, Rezeika A, Volosyak I. A Novel Hybrid Mental Spelling Application Based on Eye Tracking and SSVEP-Based BCI. Brain Sciences. 2017;7(4):35.
View Article
Google Scholar

[56] View Article

[57] Google Scholar

[ref17] 17. Chen X, Wang Y, Gao S, Jung TP, Gao X. Filter Bank Canonical Correlation Analysis for Implementing a High-Speed SSVEP-Based Brain–Computer Interface. J Neural Eng. 2015;12(046008):046008. pmid:26035476
View Article
PubMed/NCBI
Google Scholar

[59] View Article

[60] PubMed/NCBI

[61] Google Scholar

[ref18] 18. da Cruz JN, Wan F, Wong CM, Cao T. Adaptive Time-Window Length Based on Online Performance Measurement in SSVEP-Based BCIs. Neurocomputing. 2015;149:93–99.
View Article
Google Scholar

[63] View Article

[64] Google Scholar

[ref19] 19. Kelly S, Lalor E, Reilly R, Foxe J. Visual Spatial Attention Tracking Using High-Density SSVEP Data for Independent Brain-Computer Communication. IEEE Trans Neural Syst Rehabil Eng. 2005;13(2):172–178. pmid:16003896
View Article
PubMed/NCBI
Google Scholar

[66] View Article

[67] PubMed/NCBI

[68] Google Scholar

[ref20] 20. Stawicki P, Gembler F, Volosyak I. A User-Friendly Dictionary-Supported SSVEP-Based BCI Application. In: Gamberini L, Spagnolli A, Jacucci G, Blankertz B, Freeman J, editors. Symbiotic Interaction: 5th International Workshop, Symbiotic 2016, Padua, Italy, September 29–30, 2016, Revised Selected Papers. Cham: Springer International Publishing; 2017. p. 168–180.

[ref21] 21. Volosyak I, Cecotti H, Valbuena D, Graser A. Evaluation of the Bremen SSVEP Based BCI in Real World Conditions. In: Rehabilitation Robotics, 2009. ICORR 2009. IEEE International Conference On. IEEE; 2009. p. 322–331.

[ref22] 22. Akce A, Norton JJ, Bretl T. An SSVEP-Based Brain-Computer Interface for Text Spelling With Adaptive Queries That Maximize Information Gain Rates. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2015;23(5):857–866. pmid:25474810
View Article
PubMed/NCBI
Google Scholar

[72] View Article

[73] PubMed/NCBI

[74] Google Scholar

[ref23] 23. Speier W, Arnold C, Pouratian N. Integrating Language Models into Classifiers for BCI Communication: A Review. Journal of Neural Engineering. 2016;13(3):031002. pmid:27153565
View Article
PubMed/NCBI
Google Scholar

[76] View Article

[77] PubMed/NCBI

[78] Google Scholar

[ref24] 24. Jurcak V, Tsuzuki D, Dan I. 10/20, 10/10, and 10/5 Systems Revisited: Their Validity as Relative Head-Surface-Based Positioning Systems. NeuroImage. 2007;34(4):1600–1611. pmid:17207640
View Article
PubMed/NCBI
Google Scholar

[80] View Article

[81] PubMed/NCBI

[82] Google Scholar

[ref25] 25. Manyakov NV, Chumerin N, Robben A, Combaz A, van Vliet M, Van Hulle MM. Sampled Sinusoidal Stimulation Profile and Multichannel Fuzzy Logic Classification for Monitor-Based Phase-Coded SSVEP Brain–Computer Interfacing. Journal of Neural Engineering. 2013;10(3):036011. pmid:23594762
View Article
PubMed/NCBI
Google Scholar

[84] View Article

[85] PubMed/NCBI

[86] Google Scholar

[ref26] 26. Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S. High-Speed Spelling with a Noninvasive Brain–Computer Interface. Proceedings of the National Academy of Sciences. 2015;112(44):E6058–E6067.
View Article
Google Scholar

[88] View Article

[89] Google Scholar

[ref27] 27. Nakanishi M, Wang Y, Wang YT, Mitsukura Y, Jung TP. Generating Visual Flickers for Eliciting Robust Steady-State Visual Evoked Potentials at Flexible Frequencies Using Monitor Refresh Rate. PLoS ONE. 2014;9(6):e99235. pmid:24918435
View Article
PubMed/NCBI
Google Scholar

[91] View Article

[92] PubMed/NCBI

[93] Google Scholar

[ref28] 28. Hotelling H. RELATIONS BETWEEN TWO SETS OF VARIATES. Biometrika. 1936;28(3-4):321–377.
View Article
Google Scholar

[95] View Article

[96] Google Scholar

[ref29] 29. Lin Z, Zhang C, Wu W, Gao X. Frequency Recognition Based on Canonical Correlation Analysis for SSVEP-Based BCIs. IEEE Transactions on Biomedical Engineering. 2007;54(6):1172–1176. pmid:17549911
View Article
PubMed/NCBI
Google Scholar

[98] View Article

[99] PubMed/NCBI

[100] Google Scholar

[ref30] 30. Mondini V, Mangia AL, Talevi L, Cappello A. Sinc-Windowing and Multiple Correlation Coefficients Improve SSVEP Recognition Based on Canonical Correlation Analysis. Computational Intelligence and Neuroscience. 2018;2018:1–11.
View Article
Google Scholar

[102] View Article

[103] Google Scholar

[ref31] 31. Gembler F, Stawicki P, Volosyak I. Autonomous Parameter Adjustment for SSVEP-Based BCIs with a Novel BCI Wizard. Frontiers in Neuroscience. 2015;9. pmid:26733788
View Article
PubMed/NCBI
Google Scholar

[105] View Article

[106] PubMed/NCBI

[107] Google Scholar

[ref32] 32. Gustafsson F. Determining the Initial States in Forward-Backward Filtering. IEEE Transactions on Signal Processing. 1996;44(4):988–992.
View Article
Google Scholar

[109] View Article

[110] Google Scholar

[ref33] 33. Eckart T, Quasthoff U. Statistical Corpus and Language Comparison on Comparable Corpora. In: Sharoff S, Rapp R, Zweigenbaum P, Fung P, editors. Building and Using Comparable Corpora. Berlin, Heidelberg: Springer Berlin Heidelberg; 2013. p. 151–165.

[ref34] 34. Likert R. A Technique for the Measurement of Attitudes. Archives of Psychology. 1932;22 140:55–55.
View Article
Google Scholar

[113] View Article

[114] Google Scholar

[ref35] 35. Wittevrongel B, Van Wolputte E, Van Hulle MM. Code-Modulated Visual Evoked Potentials Using Fast Stimulus Presentation and Spatiotemporal Beamformer Decoding. Scientific Reports. 2017;7(1). pmid:29118386
View Article
PubMed/NCBI
Google Scholar

[116] View Article

[117] PubMed/NCBI

[118] Google Scholar

[ref36] 36. Gembler F, Stawicki P, Rezeika A, Saboor A, Benda M, Volosyak I. Effects of Monitor Refresh Rates on C-VEP BCIs. In: Ham J, Spagnolli A, Blankertz B, Gamberini L, Jacucci G, editors. Symbiotic Interaction. Cham: Springer International Publishing; 2018. p. 53–62.

[ref37] 37. Volosyak I, Cecotti H, Gräser A. Impact of Frequency Selection on LCD Screens for SSVEP Based Brain-Computer Interfaces. In: Cabestany J, Sandoval F, Prieto A, Corchado JM, editors. Bio-Inspired Systems: Computational and Ambient Intelligence. vol. 5517. Berlin, Heidelberg: Springer Berlin Heidelberg; 2009. p. 706–713.

[ref38] 38. Zhang X, Xu G, Xie J, Zhang X. Brain Response to Luminance-Based and Motion-Based Stimulation Using Inter-Modulation Frequencies. PLOS ONE. 2017;12(11):e0188073. pmid:29141030
View Article
PubMed/NCBI
Google Scholar

[122] View Article

[123] PubMed/NCBI

[124] Google Scholar

[ref39] 39. Chien YY, Lin FC, Zao JK, Chou CC, Huang YP, Kuo HY, et al. Polychromatic SSVEP Stimuli with Subtle Flickering Adapted to Brain-Display Interactions. Journal of Neural Engineering. 2017;14(1):016018. pmid:28000607
View Article
PubMed/NCBI
Google Scholar

[126] View Article

[127] PubMed/NCBI

[128] Google Scholar

[ref40] 40. Nagel S, Dreher W, Rosenstiel W, Spüler M. The Effect of Monitor Raster Latency on VEPs, ERPs and Brain–Computer Interface Performance. Journal of Neuroscience Methods. 2018;295:45–50. pmid:29197616
View Article
PubMed/NCBI
Google Scholar

[130] View Article

[131] PubMed/NCBI

[132] Google Scholar

[ref41] 41. Spüler M. A High-Speed Brain-Computer Interface (BCI) Using Dry EEG Electrodes. PLOS ONE. 2017;12(2):e0172400. pmid:28225794
View Article
PubMed/NCBI
Google Scholar

[134] View Article

[135] PubMed/NCBI

[136] Google Scholar

[ref42] 42. Gembler F, Stawicki P, Volosyak I. A Comparison of SSVEP-Based BCI-Performance Between Different Age Groups. In: Rojas I, Joya G, Catala A, editors. Advances in Computational Intelligence: 13th International Work-Conference on Artificial Neural Networks, IWANN 2015, Palma de Mallorca, Spain, June 10-12, 2015. Proceedings, Part I. Cham: Springer International Publishing; 2015. p. 71–77.

Figures

Abstract

Introduction

Materials and methods

Participants

Hardware

Stimulus presentation

c-VEP stimulus presentation.

SSVEP stimulus presentation.

Experimental protocol

Training phase.

Copy spelling phase.

Questionnaires.

Software

CCA-based spatial filters.

Time window mechanism.

Classification.

Automatic parameter calibration.

Dictionary-driven spelling application.

Results

Off-line performance

On-line spelling performance

Questionnaire results

Discussion

Conclusion

References