Skip to main content
SpringerLink
Log in

A CNN-based modular classification scheme for motor imagery using a novel EEG sampling protocol suitable for IoT healthcare systems

  • S.I. : Neural Computing for IOT based Intelligent Healthcare Systems
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The implementation of brain-computer interfaces (BCI) for real-time has become a paramount technology. Implementation of real-time BCI systems requires of methodologies that achieve high performance on classification over general brain signals of different subjects. Therefore, this work presents two simple and efficient methodologies to classify two and four motor imageries. The methodology to classify two motor imageries (MC-TM) includes an analysis of feature extraction methods based on spatial patterns and time–frequency transforms; and a convolutional neural network that preserves the information of the magnitude in the frequency bands of the sensorimotor rhythms (CNN-PIM). Besides, the methodology to classify four motor imageries (MC-FM) includes a modular classification scheme that instances 6 CNN-PIM; a new algorithm that uses the output of the softmax of each CNN-PIM to enhance the performance of the MC-FM methodology; an electroencephalogram sampling protocol that includes a specific procedure for 4 MIs classes; and a new dataset with the brain signals of 15 subjects. The MC-TM methodology achieved an accuracy of 94.44 ± 02.18% evaluated in BCI Competition IV dataset 2a (BCI-IV-2a), and accuracy of 97.67 ± 02.06% when evaluated in the EEGdataset. Meanwhile, the MC-FM achieved accuracies of 91.37 ± 3.29% and 86.48 ± 4.74% when evaluated in BCI-IV-2a and in the proposed dataset, respectively. These results situate our methodologies in a competitive position in comparison with the state-of-the-art methods. Moreover, the approximate processing time that the MC-FM methodology takes to classify EEG signals is 270 ms. Thus, it is suitable to be implemented in a real-time BCI system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Sholla S, Mir RN, Chishti MA (2021) A neuro fuzzy system for incorporating ethics in the internet of things. J Ambient Intell Humaniz Comput 12:1487–1501. https://doi.org/10.1007/s12652-020-02217-2

    Article  Google Scholar 

  2. El Saddik A, Hossain MS, Kantarci B (eds) (2020) Connected health in smart cities, 1st edn. Springer, Cham. https://doi.org/10.1007/978-3-030-27844-1_1

    Book  Google Scholar 

  3. Paszkiel S (2020) Analysis and classification of EEG signals for brain-computer interfaces, 1st edn. Springer, Cham. https://doi.org/10.1007/978-3-030-30581-9

    Book  Google Scholar 

  4. Idowu OP, Samuel OW, Li X, Asogbon MG, Fang P, Li G (2020) Efficient classification of motor imagery using particle swarm optimization-based neural network for IoT applications. IEEE Int Work Metrol Ind. https://doi.org/10.1109/MetroInd4.0IoT48571.2020.9138229

    Article  Google Scholar 

  5. Pfurtscheller G, Lopes da Silva FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110:1842–1857. https://doi.org/10.1016/S1388-2457(99)00141-8

    Article  Google Scholar 

  6. Nguyen T, Hettiarachchi I, Khatami A, Gordon-Brown L, Lim CP, Nahavandi S (2018) Classification of multi-class BCI data by common spatial pattern and fuzzy system. IEEE Access 6:27873–27884. https://doi.org/10.1109/ACCESS.2018.2841051

    Article  Google Scholar 

  7. Jeong JH, Shim KH, Kim DJ, Lee SW (2020) Brain-controlled robotic arm system based on multi-directional CNN-BiLSTM network using EEG signals. IEEE Trans Neural Syst Rehabil Eng 28:1226–1238. https://doi.org/10.1109/TNSRE.2020.2981659

    Article  Google Scholar 

  8. Yan N, Wang C, Tao Y, Li J, Zhang K, Chen T, Yuan Z, Yan X, Wang G (2020) Quadcopter control system using a hybrid BCI based on off-line optimization and enhanced human-machine interaction. IEEE Access 8:1160–1172. https://doi.org/10.1109/ACCESS.2019.2961246

    Article  Google Scholar 

  9. Mora-Sánchez A, Pulini AA, Gaume A, Dreyfus G, Vialatte FB (2020) A brain–computer interface for the continuous, real-time monitoring of working memory load in real-world environments. Cogn Neurodyn 14:301–321. https://doi.org/10.1007/s11571-020-09573-x

    Article  Google Scholar 

  10. BCI Competitions (2008). http://www.bbci.de/competition/. Accessed 28 May 2020

  11. Cho H, Ahn M, Ahn S, Kwon M, Jun SC (2017) EEG datasets for motor imagery brain-computer interface. Gigascience 6:1–8. https://doi.org/10.1093/gigascience/gix034

    Article  Google Scholar 

  12. Kaya M, Binli MK, Ozbay E, Yanar H, Mishchenko Y (2018) A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces. Sci Data 5:1–16. https://doi.org/10.1038/sdata.2018.211

    Article  Google Scholar 

  13. Shi T, Ren L, Cui W (2020) Feature extraction of brain-computer interface electroencephalogram based on motor imagery. IEEE Sens J 20:11787–11794. https://doi.org/10.1109/JSEN.2019.2939343

    Article  Google Scholar 

  14. Nicolas-Alonso LF, Gomez-Gil J (2012) Brain computer interfaces, a review. Sensors 12:1211–1279. https://doi.org/10.3390/s120201211

    Article  Google Scholar 

  15. Duan L, Hongxin Z, Khan MS, Fang M (2017) Recognition of motor imagery tasks for BCI using CSP and chaotic PSO twin SVM. J China Univ Posts Telecommun 24:83–90. https://doi.org/10.1016/S1005-8885(17)60215-2

    Article  Google Scholar 

  16. Aljalal M, Djemal R, Ibrahim S (2019) Robot navigation using a brain computer interface based on motor imagery. J Med Biol Eng 39:508–522. https://doi.org/10.1007/s40846-018-0431-9

    Article  Google Scholar 

  17. Lee HK, Choi YS (2018) A convolution neural networks scheme for classification of motor imagery EEG based on Wavelet time-frequecy image. In: 2018 International Conference on Information Networking, pp 2–5

  18. Sethi S, Upadhyay R, Singh HS (2018) Stockwell-common spatial pattern technique for motor imagery-based brain computer interface design. Comput Electr Eng 71:492–504. https://doi.org/10.1016/j.compeleceng.2018.07.056

    Article  Google Scholar 

  19. Lu N, Li T, Ren X, Miao H (2017) A deep learning scheme for motor imagery classification based on restricted Boltzmann machines. IEEE Trans Neural Syst Rehabil Eng 25:566–576. https://doi.org/10.1109/TNSRE.2016.2601240

    Article  Google Scholar 

  20. Zhang J, Yan C, Gong X (2017) Deep convolutional neural network for decoding motor imagery based brain computer interface. IEEE Int Conf Signal Process Commun Comput 2017:1–5. https://doi.org/10.1109/ICSPCC.2017.8242581

    Article  Google Scholar 

  21. Xu B, Zhang L, Song A, Wu C, Li W, Zhang D, Xu G, Li H, Zeng H (2019) Wavelet transform time-frequency image and convolutional network-based motor imagery EEG classification. IEEE Access 7:6084–6093. https://doi.org/10.1109/ACCESS.2018.2889093

    Article  Google Scholar 

  22. Uktveris T, Jusas V (2018) Convolutional neural networks for four-class motor imagery data classification. In: Ivanović M, Bădică C, Dix J, Jovanović Z, Malgeri M (eds) Intelligent distributed computing XI. Springer, Cham, pp 185–197. https://doi.org/10.1007/978-3-319-66379-1_17

    Chapter  Google Scholar 

  23. Olivas-Padilla BE, Chacon-Murguia MI (2018) Classification of multiple motor imagery using deep convolutional neural networks and spatial filters. Appl Soft Comput 75:461–472. https://doi.org/10.1016/j.asoc.2018.11.031

    Article  Google Scholar 

  24. Perez-Zapata AF, Cardona-Escobar AF, Jaramillo-Garzon JA, Dıaz GM (2018) Deep convolutional neural networks and power spectral density features for motor imagery classification of EEG signals. In: Schmorrow D, Fidopiastis C (eds) Augmented Cognition: Intelligent Technologies. Springer, Cham, pp 158–169. https://doi.org/10.1007/978-3-319-91470-1

    Chapter  Google Scholar 

  25. Hernandez LG, Antelis JM (2018) A comparison of deep neural network algorithms for recognition of EEG motor imagery signals. Pattern Recognit 10880:126–134. https://doi.org/10.1201/9781315119601

    Article  Google Scholar 

  26. Sakhavi S, Guan C, Yan S (2018) Learning temporal information for brain-computer interface using convolutional neural networks. IEEE Trans Neural Netw Learn Syst 29:5619–5629. https://doi.org/10.1016/j.cplett.2007.08.023

    Article  MathSciNet  Google Scholar 

  27. Braga RB, Lopes CD, Becker T (2019) Round cosine transform based feature extraction of motor imagery EEG signals. World Congr Med Phys Biomed Eng 68:511–515. https://doi.org/10.1007/978-981-10-9038-7

    Article  Google Scholar 

  28. Caruana RA (1993) Multitask learning: a knowledge-based source of inductive bias. Proc Tenth Int Conf Mach Learn. https://doi.org/10.1016/b978-1-55860-307-3.50012-5

    Article  Google Scholar 

  29. Li D, Yang Y, Song YZ, Hospedales TM (2018) Learning to generalize: meta-learning for domain generalization. In: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pp 3490–3497

  30. Xu Z, Cao L, Chen X (2019) Learning to learn: hierarchical meta-critic networks. IEEE Access 7:57069–57077. https://doi.org/10.1109/ACCESS.2019.2914469

    Article  Google Scholar 

  31. Brazdil P, Giraud-Carrier C (2018) Metalearning and algorithm selection: progress, state of the art and introduction to the 2018 special issue. Mach Learn 107:1–14. https://doi.org/10.1007/s10994-017-5692-y

    Article  MathSciNet  MATH  Google Scholar 

  32. Sun Q, Liu Y, Chua TS, Schiele B (2019) Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 403-412

  33. Duan L, Li J, Ji H, Pang Z, Zheng X, Lu R, Li M, Zhuang J (2020) Zero-shot learning for EEG classification in motor imagery-based BCI system. IEEE Trans Neural Syst Rehabil Eng 28:2411–2419. https://doi.org/10.1109/TNSRE.2020.3027004

    Article  Google Scholar 

  34. Vanschoren J (2019) Meta-learning. In: Hutter VJF, Kotthoff L (eds) Automated machine learning: methods, systems challenges. The Springer series on challenges in machine learning. Springer, Cham, pp 35–61

    Chapter  Google Scholar 

  35. Sagi O, Rokach L (2018) Ensemble learning: a survey. Wiley Interdiscip Rev Data Min Knowl Discov 8:1–18. https://doi.org/10.1002/widm.1249

    Article  Google Scholar 

  36. Chacon-Murguia MI, Rivas-Posada E (2020) Feature extraction evaluation for two motor imagery recognition based on common spatial patterns, time-frequency transformations and SVM. Proc IEEE World Congr Comput Intell. https://doi.org/10.1109/IJCNN48605.2020.9206638

    Article  Google Scholar 

  37. Tangermann M, Müller KR, Aertsen A, Birbaumer N, Braun C, Brunner C, Leeb R, Mehring C, Miller KJ, Müller-Putz GR, Nolte G, Pfurtscheller G, Preissl H, Schalk G, Schlögl A, Vidaurre C, Waldert S, Blankertz B (2012) Review of the BCI competition IV. Front Neurosci 6:1–31. https://doi.org/10.3389/fnins.2012.00055

    Article  Google Scholar 

  38. Cohen MX (2014) Analyzing neural time series data: theory and practice, 1st edn. MIT Press, Cambridge, p 600

    Book  Google Scholar 

  39. Taylor L, Nitschke G (2018) Improving deep learning with generic data augmentation. In: 2018 IEEE symposium series on computational intelligence (SSCI), pp 1542–1547. https://doi.org/10.1109/SSCI.2018.8628742

  40. I. The MathWorks, Imresize, Resize Image (2019) https://www.mathworks.com/help/matlab/ref/imresize.html#d117e714068. Accessed 12 Nov 2019

  41. I. The MathWorks, CWT, Contin. 1-D Wavelet Transform (2016) https://www.mathworks.com/help/wavelet/ref/cwt.html. Accessed 12 Nov 2019

  42. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. Comput Vis ECCV. https://doi.org/10.1016/j.ancr.2017.02.001

    Article  Google Scholar 

  43. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: International conference on learning representations (ICLR), pp 1–15

  44. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press. https://doi.org/10.1038/s41566-018-0231-3

    Article  MATH  Google Scholar 

  45. Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. https://doi.org/10.1109/CVPR.2015.7298640

    Article  Google Scholar 

  46. Keerthi Krishnan K, Soman KP (2021) CNN based classification of motor imaginary using variational mode decomposed EEG-spectrum image. Biomed Eng Lett 11:235–247. https://doi.org/10.1007/s13534-021-00190-z

    Article  Google Scholar 

  47. Jafarifarmand A, Badamchizadeh MA, Khanmohammadi S, Nazari MA, Tazehkand BM (2018) A new self-regulated neuro-fuzzy framework for classification of EEG signals in motor imagery BCI. IEEE Trans Fuzzy Syst 26:1485–1497. https://doi.org/10.1109/TFUZZ.2017.2728521

    Article  Google Scholar 

  48. García-Murillo DG, Alvarez-Meza A, Castellanos-Dominguez G (2021) Single-trial kernel-based functional connectivity for enhanced feature extraction in motor-related tasks. Sensors 21:1–17. https://doi.org/10.3390/s21082750

    Article  Google Scholar 

  49. Zhang Y, Nam CS, Zhou G, Jin J, Wang X, Cichocki A (2019) Temporally constrained sparse group spatial patterns for motor imagery BCI. IEEE Trans Cybern 49:3322–3332. https://doi.org/10.1109/TCYB.2018.2841847

    Article  Google Scholar 

  50. Jiang A, Shang J, Liu X, Tang Y, Kwan HK, Zhu Y (2020) Efficient CSP algorithm with spatio-temporal filtering for motor imagery classification. IEEE Trans Neural Syst Rehabil Eng 28:1006–1016. https://doi.org/10.1109/TNSRE.2020.2979464

    Article  Google Scholar 

  51. Teplan M (2002) Fundamentals of EEG measurement. Meas Sci Rev 2:1–11

    Google Scholar 

  52. Brunner C, Leeb R (2008) BCI Competition 2008–Graz data set A. Knowl Creat Diffus Util 2008:1–6. https://doi.org/10.1109/TBME.2004.827081

    Article  Google Scholar 

  53. Deng X, Zhang B, Yu N, Liu K, Sun K (2021) Advanced TSGL-EEGNet for motor imagery EEG-based brain-computer interfaces. IEEE Access 9:25118–25130. https://doi.org/10.1109/access.2021.3056088

    Article  Google Scholar 

  54. Yang L, Song Y, Ma K, Xie L (2021) Motor imagery EEG decoding method based on a discriminative feature learning strategy. IEEE Trans Neural Syst Rehabil Eng 29:368–379. https://doi.org/10.1109/TNSRE.2021.3051958

    Article  Google Scholar 

  55. Rashid M, Bari BS, Hasan MJ, Razman MAM, Musa RM, Nasir AFA, Majeed APPA (2021) The classification of motor imagery response: an accuracy enhancement through the ensemble of random subspace k-NN. PeerJ Comput Sci 7:1–31. https://doi.org/10.7717/peerj-cs.374

    Article  Google Scholar 

  56. She Q, Zou J, Luo Z, Nguyen T, Li R, Zhang Y (2020) Multi-class motor imagery EEG classification using collaborative Balanced Graph-based regularized semi-supervised extreme learning. Med Biol Eng Comput 58:2119–2130

    Article  Google Scholar 

  57. Ang KK, Chin ZY, Wang C, Guan C, Zhang H (2012) Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Front Neurosci 6:1–9. https://doi.org/10.3389/fnins.2012.00039

    Article  Google Scholar 

  58. Raza H, Cecotti H, Prasad G (2016) A combination of transductive and inductive learning for handling non-stationarities in motor imagery classification. Int Jt Conf Neural Netw. https://doi.org/10.1109/IJCNN.2016.7727277

    Article  Google Scholar 

  59. He L, Hu D, Wan M, Wen Y, Von Deneen KM, Zhou M (2016) Common Bayesian network for classification of EEG-based multiclass motor imagery BCI. IEEE Trans Syst Man Cybern. Syst 46:843–854. https://doi.org/10.1109/TSMC.2015.2450680

    Article  Google Scholar 

  60. Wang H, Xu T, Tang C, Yue H, Chen C, Xu L, Pei Z, Dong J, Bezerianos A, Li J (2020) Diverse feature blend based on filter-bank common spatial pattern and brain functional connectivity for multiple motor imagery detection. IEEE Access 8:155590–155601. https://doi.org/10.1109/ACCESS.2020.3018962

    Article  Google Scholar 

  61. Selim S, Tantawi M, Shedeed H, Badr A (2019) Comparing multi-class approaches for motor imagery using renyi entropy. Proc Int Conf Adv Intell Syst Inf 2018:127–136. https://doi.org/10.1007/978-3-319-99010-1

    Article  Google Scholar 

  62. Zhang S, Zhu Z, Zhang B, Feng B, Yu T, Li Z (2021) Fused group lasso: a New EEG classification model with spatial smooth constraint for motor imagery-based brain-computer interface. IEEE Sens J 21:1764–1778. https://doi.org/10.1109/JSEN.2020.3016402

    Article  Google Scholar 

Download references

Acknowledgements

The research leading to these results received funding from Tecnologico Nacional de Mexico/ I.T. Chihuahua under grant No 10071.21-P.

Author information

Authors and Affiliations

  1. Tecnologico Nacional de Mexico/Instituto Tecnologico de Chihuahua, Chihuahua, Chih., Mexico

    Mario I. Chacon-Murguia & Eduardo Rivas-Posada

Authors
  1. Mario I. Chacon-Murguia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo Rivas-Posada
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mario I. Chacon-Murguia.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article, and they certify that the manuscript complies with the Ethical Rules applicable for this journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chacon-Murguia, M.I., Rivas-Posada, E. A CNN-based modular classification scheme for motor imagery using a novel EEG sampling protocol suitable for IoT healthcare systems. Neural Comput & Applic 35, 22865–22886 (2023). https://doi.org/10.1007/s00521-021-06716-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06716-x

Keywords

Search

Navigation