Xiaoyi Zhang
Harish Kulkarni
Meredith Ringel Morris
ABSTRACT
Current eye-tracking input systems for people with ALS or other motor impairments are expensive, not robust under sunlight, and require frequent re-calibration and substantial, relatively immobile setups. Eye-gaze transfer (e-tran) boards, a low-tech alternative, are challenging to master and offer slow communication rates. To mitigate the drawbacks of these two status quo approaches, we created GazeSpeak, an eye gesture communication system that runs on a smartphone, and is designed to be low-cost, robust, portable, and easy-to-learn, with a higher communication bandwidth than an e-tran board. GazeSpeak can interpret eye gestures in real time, decode these gestures into predicted utterances, and facilitate communication, with different user interfaces for speakers and interpreters. Our evaluations demonstrate that GazeSpeak is robust, has good user satisfaction, and provides a speed improvement with respect to an e-tran board; we also identify avenues for further improvement to low-cost, low-effort gaze-based communication technologies.
Supplemental Material
- Javier S. Agustin, Henrik Skovsgaard, John P. Hansen, and Dan W. Hansen. 2009. Low-cost Gaze Interaction: Ready to Deliver the Promises. In Conference on Human Factors in Computing (CHI), 4453--4458. http://doi.org/10.1145/1520340.1520682 Google Scholar
Digital Library
- Monica Anderson. 2015. Technology Device Ownership: 2015. Retrieved September 15, 2016 from http://www.pewinternet.org/2015/10/29/technologydevice-ownership-2015/Google Scholar
- Gary Becker. 1996. Vocal Eyes Becker Communication System. Retrieved September 15, 2016 from http://jasonbecker.com/eye_communication.htmlGoogle Scholar
- Pradipta Biswas and Pat Langdon. 2011. A new input system for disabled users involving eye gaze tracker and scanning interface. Journal of Assistive Technologies 5, 2: 58--66. http://doi.org/10.1108/17549451111149269 Google ScholarCross Ref
- Gary Bradski. 2000. The OpenCV Library. Doctor Dobbs Journal 25: 120--126.Google Scholar
- Mark Davies. 2008. The Corpus of Contemporary American English: 520 million words, 1990-present. Retrieved from http://corpus.byu.edu/coca/Google Scholar
- Kristen Grauman, Margrit Betke, Jonathan Lombardi, James Gips, and Gary R. Bradski. 2003. Communication via eye blinks and eyebrow raises: Video-based human-computer interfaces. Universal Access in the Information Society 2, 4: 359--373. http://doi.org/10.1007/s10209-003-0062-x Google ScholarDigital Library
- Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology 52, C: 139--183. http://doi.org/10.1016/S0166--4115(08)62386--9Google Scholar
- Shaun K. Kane, Barbara Linam-Church, Kyle Althoff, and Denise McCall. 2012. What We Talk About: Designing a Context-Aware Communication Tool for People with Aphasia. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility - ASSETS '12, 49. http://doi.org/10.1145/2384916.2384926Google ScholarDigital Library
- Vahid Kazemi and Josephine Sullivan. 2014. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1867--1874. http://doi.org/10.1109/CVPR.2014.241 Google ScholarDigital Library
- Davis E. King. 2009. Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research 10: 1755--1758.Google ScholarDigital Library
- Kyle Krafka, Aditya Khosla, Petr Kellnhofer, et al. 2016. Eye tracking for Everyone. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Google ScholarCross Ref
- Per Ola Kristensson and Keith Vertanen. 2012. The potential of dwell-free eye-typing for fast assistive gaze communication. In Proceedings of the Symposium on Eye Tracking Research and Applications, 241--244. http://doi.org/10.1145/2168556.2168605 Google ScholarDigital Library
- Rene De La Briandais. 1959. File searching using variable length keys. In Papers presented at the western joint computer conference, 295--298. Google ScholarDigital Library
- Chris Lankford. 2000. Effective eye-gaze input into windows. In Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, 23--27. http://doi.org/10.1145/355017.355021 Google ScholarDigital Library
- Erich Leo Lehmann and George Casella. 2006. Theory of point estimation. Springer Science & Business Media.Google Scholar
- Low Tech Solutions. 2016. E-tran Board. Retrieved September 15, 2016 from http://store.lowtechsolutions.org/e-tran-board/Google Scholar
- LusoVu. 2016. EyeSpeak. Retrieved September 15, 2016 from http://www.myeyespeak.com/eyespeak/Google Scholar
- Scott MacKenzie and William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems, 754--755. http://doi.org/10.1145/765968.765971 Google ScholarCross Ref
- Päivi Majaranta, Ulla-Kaija Ahola, and Oleg Špakov. 2009. Fast gaze typing with an adjustable dwell time. In Proceedings of the 27th international conference on Human factors in computing systems CHI 09, 357. http://doi.org/10.1145/1518701.1518758 Google ScholarDigital Library
- Microsoft Research. 2016. Hands-Free Keyboard. Retrieved September 15, 2016 from https://www.microsoft.com/enus/research/project/hands-free-keyboard/Google Scholar
- Steven Nowlan, Ali Ebrahimi, David Richard Whaley, Pierre Demartines, Sreeram Balakrishnan, and Sheridan Rawlins. 2001. Data entry apparatus having a limited number of character keys and method. Retrieved from https://www.google.com/patents/US6204848Google Scholar
- Nuance. 2016. T9 Text Input. Retrieved September 15, 2016 from http://www.nuance.com/for-business/byproduct/t9/index.htmGoogle Scholar
- Alexandra Papoutsaki, Patsorn Sangkloy, James Laskey, Nediyana Daskalova, Jeff Huang, and James Hays. 2016. WebGazer: Scalable Webcam Eye Tracking Using User Interactions. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, 3839--3845.Google Scholar
- Sarah Marie Swift. 2012. Low-Tech, Eye-MovementAccessible AAC and Typical Adults.Google Scholar
- The ALS Association. 2016. Facts You Should Know. Retrieved September 15, 2016 from http://www.alsa.org/about-als/facts-you-shouldknow.htmlGoogle Scholar
- TheEyeTribe. 2016. The Eye Tribe. Retrieved September 15, 2016 from https://theeyetribe.com/Google Scholar
- Tobii. 2016. Tobii Dynavox Webshop. Retrieved September 15, 2016 from https://www.tobiiatiwebshop.com/Google Scholar
- Tobii. 2016. Communicator 5 - Tobii Dynavox. Retrieved September 15, 2016 from http://www.tobiidynavox.com/communicator5/Google Scholar
- Tobii. 2016. Sono Lexis - Tobii Dynavox. Retrieved September 15, 2016 from http://www.tobiidynavox.com/sono-lexis/Google Scholar
- Vytautas Vaitukaitis and Andreas Bulling. 2012. Eye gesture recognition on portable devices. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing - UbiComp '12, 711. http://doi.org/10.1145/2370216.2370370 Google ScholarDigital Library
- David Vitter. 2015. S.768 Steve Gleason Act of 2015. Retrieved September 15, 2016 from https://www.congress.gov/bill/114th-congress/senatebill/768Google Scholar
- Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4: 600--612. http://doi.org/10.1109/TIP.2003.819861 Google ScholarDigital Library
- David J. Ward, Alan F. Blackwell, and David J. C. MacKay. 2002. Dasher: A Gesture-Driven Data Entry Interface for Mobile Computing. Human-Computer Interaction 17, 2/3: 199--228. Google ScholarCross Ref
- Craig Watman, David Austin, Nick Barnes, Gary Overett, and Simon Thompson. 2004. Fast sum of absolute differences visual landmark detector. In IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004, 4827--4832 Vol.5. http://doi.org/10.1109/ROBOT.2004.1302482 Google ScholarCross Ref
- Jacob O. Wobbrock, Shaun K. Kane, Krzysztof Z. Gajos, Susumu Harada, and Jon Froehlich. 2011. Ability-Based Design: Concept, Principles and Examples. ACM Transactions on Accessible Computing 3, 3: 1--27. http://doi.org/10.1145/1952383.1952384 Google ScholarDigital Library
- Jacob O. Wobbrock, James Rubinstein, Michael W. Sawyer, and Andrew T. Duchowski. 2008. Longitudinal evaluation of discrete consecutive gaze gestures for text entry. In Proceedings of the 2008 symposium on Eye tracking research & applications, 11. http://doi.org/10.1145/1344471.1344475 Google ScholarDigital Library
Index Terms
- Smartphone-Based Gaze Gesture Communication for People with Motor Disabilities
Recommendations
Optimizing gaze typing for people with severe motor disabilities: the iWriter arabic interface
ASSETS '12: Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibilityCommunication in the Arabic language with gaze using dwell time has been made possible by the development of eye typing interfaces. This paper describes the design process for developing iWriter, an Arabic gaze communication system. Design ...
Kinect as an access device for people with cerebral palsy
We look at the potential of the Kinect sensor as an access device for people with involuntary movements.We develop a speed-based algorithm to transform Kinect data into discrete events.We assess the behavior of when is used by people with cerebral palsy ...
People with Motor Disabilities Using Gaze to Control Telerobots
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing SystemsTelerobots may give people with motor disabilities access to education, events and places. Eye-gaze interaction with these robots is an option when hands are not functional. Gaze control of telerobots has not yet been evaluated by people from this target ...
Comments