Nathan Dunkelberger
Ali Israr
ABSTRACT
In our daily lives, we rely heavily on our visual and auditory channels to receive information from others. In the case of impairment, or when large amounts of information are already transmitted visually or aurally, alternative methods of communication are needed. A haptic language offers the potential to provide information to a user when visual and auditory channels are unavailable. Previously created haptic languages include deconstructing acoustic signals into features and displaying them through a haptic device, and haptic adaptations of Braille or Morse code; however, these approaches are unintuitive, slow at presenting language, or require a large surface area. We propose using a multi-sensory haptic device called MISSIVE, which can be worn on the upper arm and is capable of producing brief cues, sufficient in quantity to encode the full English phoneme set. We evaluated our approach by teaching subjects a subset of 23 phonemes, and demonstrated an 86% accuracy in a 50 word identification task after 100 minutes of training.
- Edoardo Battaglia, Janelle P Clark, Matteo Bianchi, Manuel G Catalano, Antonio Bicchi, and Marcia K O'Malley. 2017. The Rice Haptic Rocker: skin stretch haptic feedback with the Pisa/IIT SoftHand. In World Haptics Conf. IEEE, 7--12.Google Scholar
Cross Ref
- P. L. Brooks and B. J. Frost. 1983. Evaluation of a tactile vocoder for word recognition. The Journal of the Acoustical Society of America 74, 1 (1983), 34--39.Google ScholarCross Ref
- Zvi Drezner. 2003. A new genetic algorithm for the quadratic assignment problem. INFORMS Journal on Computing 15, 3 (2003), 320 -- 330. Google ScholarDigital Library
- Nathan Dunkelberger, Joshua Bradley, Jennifer L. Sullivan, Ali Israr, Frances Lau, Keith Klumb, Freddy Abnousi, and Marcia K. O'Malley. 2018. Improving Perception Accuracy with Multi-sensory Haptic Cue Delivery. In Haptics: Science, Technology, and Applications. Springer International Publishing, Cham, 289--301.Google Scholar
- Mario Enriquez, Karon MacLean, and Christian Chita. 2006. Haptic Phonemes: Basic Building Blocks of Haptic Communication. In Proceedings of the 8th Int. Conf. on Multimodal Interfaces (ICMI '06). ACM, New York, NY, USA, 302--309. Google ScholarDigital Library
- Ali Israr, Peter H Meckl, Charlotte M Reed, and Hong Z Tan. 2009. Controller design and consonantal contrast coding using a multi-finger tactual display. The Journal of the Acoustical Society of America 125, 6 (2009), 3925--3935.Google ScholarCross Ref
- Ali Israr, Peter H Meckl, and Hong Z Tan. 2004. A two DOF controller for a multi-finger tactual display using a loop-shaping technique. In Proceedings of the ASME Int. Mechanical Engineering Congress and Exposition (IMECE04). 1083--1089.Google ScholarCross Ref
- Chandrika Jayant, Christine Acuario, William Johnson, Janet Hollier, and Richard Ladner. 2010. V-braille: Haptic Braille Perception Using a Touch-screen and Vibration on Mobile Phones. In Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '10). 295--296. Google ScholarDigital Library
- Yang Jiao, Frederico M. Severgnini, Juan Sebastian Martinez, Jaehong Jung, Hong Z. Tan, Charlotte M. Reed, E. Courtenay Wilson, Frances Lau, Ali Israr, Robert Turcott, Keith Klumb, and Freddy Abnousi. 2018. A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin. In Haptics: Science, Technology, and Applications. Springer International Publishing, Cham, 623--635.Google Scholar
- Edward T. Auer Jr., Lynne E. Bernstein, and David C. Coulter. 1998. Temporal and spatio-temporal vibrotactile displays for voice fundamental frequency: An initial evaluation of a new vibrotactile speech perception aid with normal-hearing and hearing-impaired individuals. The Journal of the Acoustical Society of America 104, 4 (1998), 2477--2489.Google ScholarCross Ref
- Vincent Lévesque, Jérôme Pasquero, Vincent Hayward, and Maryse Legault. 2005. Display of virtual braille dots by lateral skin deformation: feasibility study. ACM Trans. on Applied Perception (TAP) 2, 2 (2005), 132--149. Google ScholarDigital Library
- Eliane M. Loiola, Nair M. Maia de Abreu, Paulo O. Boaventura-Netto, Peter Hahn, and Tania Querido. 2007. A survey for the quadratic assignment problem. European Journal of Operational Research 176, 2 (2007), 657--690.Google ScholarCross Ref
- Granit Luzhnica, Eduardo Veas, and Viktoria Pammer. 2016. Skin Reading: Encoding Text in a 6-channel Haptic Display. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, New York, NY, USA, 148--155. Google ScholarDigital Library
- Kimberly Myles and Mary S. Binseel. 2007. The Tactile Modality: A Review of Tactile Sensitivity and Human Tactile Interfaces, U.S. Army Research Laboratory, ARL-TR-4115.Google Scholar
- Scott D. Novich and David M. Eagleman. 2015. Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput. Experimental Brain Research 233, 10 (2015), 2777--2788.Google ScholarCross Ref
- E. Piateski and L. Jones. 2005. Vibrotactile pattern recognition on the arm and torso. In First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference. 90--95. Google ScholarDigital Library
- Charlotte M Reed, Nathaniel I Durlach, and Louis D Braida. 1982. Research on tactile communication of speech: a review. ASHA monographs 20 (1982), 1.Google Scholar
- Hong Tan, Robert Gray, J Jay Young, and Ryan Taylor. 2003. A haptic back display for attentional and directional cueing. Haptics-e, The electronic journal of haptics research (2003).Google Scholar
- Hong Z. Tan, Nathaniel I. Durlach, William M. Rabinowitz, Charlotte M. Reed, and Jonathan R. Santos. 1997. Reception of Morse code through motional, vibrotactile, and auditory stimulation. Perception & Psychophysics 59, 7 (1997), 1004--1017.Google ScholarCross Ref
- Hong Z. Tan, Charlotte M. Reed, and Nathaniel I. Durlach. 2010. Optimum Information Transfer Rates for Communication through Haptic and Other Sensory Modalities. IEEE Trans. on Haptics 3, 2 (2010), 98--108. Google ScholarDigital Library
- Emma Treadway, Brent Gillespie, Darren Bolger, Amy Blank, Marcia K. O'Malley, and Alicia Davis. 2015. The role of auxiliary and referred haptic feedback in myoelectric control. In World Haptics Conf. IEEE, Northwestern University, 13--18.Google ScholarCross Ref
- Robert Turcott, Jennifer Chen, Pablo Castillo, Brian Knott, Wahyudinata Setiawan, Forrest Briggs, Keith Klumb, Freddy Abnousi, Prasad Chakka, Frances Lau, and Ali Israr. 2018. Efficient Evaluation of Coding Strategies for Transcutaneous Language Communication. In Haptics: Science, Technology, and Applications. Springer International Publishing, Cham, 600--611.Google Scholar
- Siyan Zhao, Ali Israr, Frances Lau, and Freddy Abnousi. 2018. Coding Tactile Symbols for Phonemic Communication. In Conference on Human Factors in Computing Systems. Google ScholarDigital Library
- Conveying language through haptics: a multi-sensory approach
Recommendations
Miniature Haptics: Experiencing Haptic Feedback through Hand-based and Embodied Avatars
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsWe present Miniature Haptics, a new approach to providing realistic haptic experiences by applying miniaturized haptic feedback to hand-based, embodied avatars. By shrinking haptics to a much smaller scale, Miniature Haptics enables the exploration of ...
ARm Haptics: 3D-Printed Wearable Haptics for Mobile Augmented Reality
MHCIAugmented Reality (AR) technology enables users to superpose virtual content onto their environments. However, interacting with virtual content while mobile often requires users to perform interactions in mid-air, resulting in a lack of haptic feedback. ...
Adding Force Feedback to Mixed Reality Experiences and Games using Electrical Muscle Stimulation
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsWe present a mobile system that enhances mixed reality experiences and games with force feedback by means of electrical muscle stimulation (EMS). The benefit of our approach is that it adds physical forces while keeping the users' hands free to interact ...
Comments