Skip to main content

Advertisement

SpringerLink
Log in

Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities

  • Research Article
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

Abstract

Currently, electric wheelchairs are commonly used to improve mobility in disabled people. In severe cases, the user is unable to control the wheelchair by themselves because his/her motor functions are disabled. To restore mobility function, a brain-controlled wheelchair (BCW) would be a promising system that would allow the patient to control the wheelchair by their thoughts. P300 is a reliable brain electrical signal, a component of visual event-related potentials (ERPs), that could be used for interpreting user commands. This research aimed to propose a prototype BCW to allowed severe motor disabled patients to practically control a wheelchair for use in their home environment. The users were able to select from 9 possible destination commands in the automatic mode and from 4 directional commands (forward, backward, turn left and right) in the shared-control mode. These commands were selected via the designed P300 processing system. The wheelchair was steered to the desired location by the implemented navigation system. Safety of the user was ensured during wheelchair navigation due to the included obstacle detection and avoidance features. A combination of P300 and EOG was used as a hybrid BCW system. The user could fully operate the system such as enabling P300 detection system, mode shifting and stop/cancelation command by performing a different consecutive blinks to generate eye blinking patterns. The results revealed that the prototype BCW could be operated in either of the proposed modes. With the new design of the LED-based P300 stimulator, the average accuracies of the P300 detection algorithm in the shared-control and automatic modes were 95.31 and 83.42% with 3.09 and 3.79 bits/min, respectively. The P300 classification error was acceptable, as the user could cancel an incorrect command by blinking 2 times. Moreover, the proposed navigation system had a flexible design that could be interfaced with other assistive technologies. This research developed 3 alternative input modules: an eye tracker module and chin and hand controller modules. The user could select the most suitable assistive technology based on his/her level of disability. Other existing assistive technologies could also be connected to the proposed system in the future using the same protocol.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  • Anton P, Lee S, Aharon W, Roni K, Lior H, Yaara Y, Nachum S, Noam S (2010) Sniffing enables communication and environmental control for the severely disabled. Proc Natl Acad Sci 107:14413–14418

    Article  Google Scholar 

  • Arai K, Ronny M (2011) Eyes based eletric wheel chair control system. Int J Adv Comput Sci Appl 12:98–105

    Google Scholar 

  • Brice R, Etienne B, Cuntai G, Haihong Z, Chee L T, Qiang Z, Marcelo A, Christian L (2006) A brain controlled wheelchair based on P300 and path guidance. In: Biomechatron (Biorob), pp 1001–1006

  • Brice R, Etienne B, Cuntai G, Haihong Z (2010) A brain controlled wheelchair to navigate in familiar environments. Neural Syst Rehabil Eng 18:590–598

    Article  Google Scholar 

  • Christian M, Thorsten L, Tim L, Thomas R, Axel G, Bernd KB (2009) Navigating a smart wheelchair with a brain-computer interface interpreting steady-state visual evoked potentials. In: Intelligent robots and systems, pp 1118–1125

  • Dilok P, Yodchanan W (2011) Illuminant effect on LCD and LED stimulators for P300-based brain-controlled wheelchair. In: Biomedical engineering international conference (BMEiCON), pp 254257

  • Dilok P, Yodchanan W (2012) Semi-automatic P300-based brain-controlled wheelchair. In: Complex medical engineering (CME), pp 455–460

  • Dilok P, Sarawin K, Pongsakom W, Boonyanuch W, Yodchanan W (2014) Automated navigation system for eye-based wheelchair controls. In: Biomedical engineering international conference (BMEiCON)

  • Duan J, Li Z, Yang C, Xu P (2014) Shared control of a brainactuated intelligent wheelchair. In: 11th world congress on intelligent control and automation (WCICA), pp 341–346

  • Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70:510–523

    Article  CAS  PubMed  Google Scholar 

  • Gautam G (2014) Eye movement based electronic wheel chair for physically challenged persons. Int J Sci Technol Res 3:206–212

    Google Scholar 

  • Gray L, Galetta SL, Siegal T, Schatz N (1997) The central visual field in homonymous hemianopia. Arch Neurol 54:312–317

    Article  CAS  PubMed  Google Scholar 

  • Gwang ME, Kyeong K, Chul SK, James L, Soon CC, Bongsoo L, Hiroki H, Norio F, Ryoko F, Takashi W (2007) Gyro-mouse for the disabled: ‘Click’ and ‘position’ control of the mouse cursor. Int J Control Autom Syst 5:147–154

    Google Scholar 

  • Horacio AB, Shir BZ, Shlomo B, Marta K (2011) Parafoveal perception during sentence reading?: an ERP paradigm using rapid serial visual presentation (RSVP) with flankers. Psychophysiology 48:523–531

    Article  Google Scholar 

  • Hugh H (2009) Exoskeletons and orthoses: classification, design challenges and future directions. Neuroeng Rehabil 6:2387–2396

    Google Scholar 

  • Hugh DW, Tim B (2006a) Simultaneous localization and mapping: part I”. Robot Autom Mag 13:99–110

    Google Scholar 

  • Hugh DW, Tim B (2006b) Hugh DW, Tim B (2006) Simultaneous localization and mapping (SLAM): part II”. IEEE Robot Autom Mag 13:108–117

    Article  Google Scholar 

  • Inaki I, Javier MA, Andrea K, Javier M (2009) Non-invasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation. IEEE Trans Robot 25:614–627

    Article  Google Scholar 

  • Jeonghee K, Hangue P, Joy B et al (2013) The tongue enables computer and wheelchair control for people with spinal cord injury. Sci Transl Med 5:213

    Google Scholar 

  • Johan P, Jose del RM, Gerolf V, Eileen L, Ferran G, Pierre WF, Hendrik VB, Mamix N (2007) Adaptive shared control of a brain-actuated simulated wheelchair. In: Rehabilitation robotics

  • John C, Lynne S, Jayme K (2008) Neuromuscular electrical stimulation for motor restoration in hemiplegia. Topics Stroke Rehabil 15:412–426

    Article  Google Scholar 

  • Jonathan RW, Niels B, Dennis JM, Gert P, Theresa MV (2002) Brain-computerinterfaces for communication and control. Clin Neurophysiol 113:767–791

    Article  Google Scholar 

  • Karl L, Kaitlin C, Alexander D, Kaleb S (2013) Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface. J Neural Eng 10:4

    Google Scholar 

  • Kazuo T, Kazuyuki M, Hua OW (2005) Electroencephalogram-based control of an electric wheelchair. Robotics 21:762–766

    Google Scholar 

  • Krzysztof D, Piotr W (2015) Brain-computer interface for mobile devices. J Med Inf Technol 24

  • Kyuwan C, Andrzej C (2008) Control of a wheelchair by motor imagery in real time. In: Intelligent data engineering and automated learning, pp 330–337

  • Miami B (2009) Feature extraction and classification of EEG signals for rapid P300 mind spelling. In: International conference on machine learning and applications

  • Michael J, Kevin G, John A, Ruth F (2008) A sip-and-puff wireless remote control for the Apple iPod. Assist Technol 20:107–110

    Article  Google Scholar 

  • Morgan Q, Brian G, Ken C, Josh F, Tully F, Jeremy L, Eric B, Rob W, Andrew N (2009) ROS: an open-source Robot Operating System. In: ICRA workshop on open source software

  • Ng DW-K, Soh Y-W, Goh S-Y (2014) Development of an autonomous BCI wheelchair. In: IEEE symposium computational intelligence brain computer interfaces, pp 1–4

  • Rossella B, Simone C, Bernardo DS, Giulio F, Matteo M, Davide M (2008) Brain control of a smart wheelchair. Intell Auton Syst 10:221–228

    Google Scholar 

  • Stefan K, Johannes M, Thorsten G, Karen P, Uwe K, Oskar S (2013) Hector open source modules for autonomous mapping and navigation with rescue robots. In: RoboCup 2013: Robot World Cup XVII, pp 624–631

  • Susumu H, James AL, Jonathan M, Xiao L, Jeff AB (2006) The vocal joystick: evaluation of voice based cursor control techniques. In: ASSETS, pp 197–204

  • Torsten F, Rainer N (2007) Nordmann alternative wheelchair control. In: Proc RAT’07, pp 67–74

  • Tyler S, Colin B, Michel JAG, Arthur P (2008) Tooth-click control of a hands-free computer interface”. IEEE Trans Biomed 55:2050–2056

    Article  Google Scholar 

  • Ulrich H, Jean-M V, Touradj E, Karin D (2008) An efficient P300-based brain–computer interface for disabled subjects. J Neurosci Methods 167:115–125

    Article  Google Scholar 

  • Varona-Moya S, Velasco-Álvarez F, Sancha-Ros S, Fernández-Rodríguez Á, Blanca MJ, Ron-Angevin R (2015) Wheelchair navigation with an audio-cued, two-class motor imagery-based brain—computer interface system. In: 7th international IEEE/EMBS conference on neural engineering (NER), pp 22–24

  • Yunyong P, Yodchanan W (2010) Hybrid EEG-EOG brain-computer interface system for practical machine control. In: EMBC

  • Yunyong P, Yodchanan W (2012) Minimal-assisted SSVEP-based brain-computer interface device. In: APSIPA ASC, pp 1–4

  • Zhang R et al (2015) Control of a wheelchair in an indoor environment based on a brain-computer interface and automated navigation. IEEE Trans Neural Syst Rehabil Eng 24:128–139

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This projected was supported by the National Research Council of Thailand (NRCT). The development of the EEG acquisition system was supported by the National Broadcasting and Telecommunication Commission (NBCT). Finally, many thanks go to all of the disable subjects from Putthamonthon Independent Living Center (PILC) and the healthy subjects who volunteered to participate in the experiments described in this paper.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Faculty of Engineering, Mahidol Unversity, 25/25, Putthamonthol 4 Road, Salaya, Putthamonthol, Nakhon Pathom, 73170, Thailand

    Dilok Puanhvuan, Sarawin Khemmachotikun, Pongsakorn Wechakarn, Boonyanuch Wijarn & Yodchanan Wongsawat

Authors
  1. Dilok Puanhvuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarawin Khemmachotikun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pongsakorn Wechakarn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Boonyanuch Wijarn
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yodchanan Wongsawat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yodchanan Wongsawat.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Puanhvuan, D., Khemmachotikun, S., Wechakarn, P. et al. Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities. Cogn Neurodyn 11, 117–134 (2017). https://doi.org/10.1007/s11571-017-9424-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11571-017-9424-6

Keywords

Search

Navigation