François Bérard
Amélie Rochet-Capellan
ABSTRACT
Human-computer interaction should be natural. However, the notion of natural is questioned due to a lack of theoretical background and methods to objectively measure the naturalness of a HCI. A frequently cited aspect of natural HCIs is their ability to benefit from knowledge and skills that users develop in their interaction with the real (non-digital) world. Among these skills, sensory-motor abilities are essential to operate many HCIs. This suggests that the transfer of these abilities between physical and digital interactions could be used as an experimental tool to assess the sensory-motor similarity between interactions, and could be considered as an objective measurement of the sensory-motor grounding of naturalness. In this framework, we introduce a new experimental paradigm inspired by motor learning research to assess sensory-motor similarity, as revealed by the transfer of learning. We tested this paradigm in an empirical study to question the naturalness of three HCIs: direct-touch, mouse pointing and absolute indirect-touch. The study revealed how skill learning transfers from these three digital interactions towards an equivalent physical interaction. We observed strong transfer of skill between direct-touch and physical interaction, but no transfer from the other two interactions. This work provides a first objective assessment of the sensory-motor basis of direct-touch naturalness, and a new empirical path to question HCI similarity and naturalness.
Supplemental Material
- Accot, J., and Zhai, S. More than dotting the i's -- foundations for crossing-based interfaces. In ACM Conference on Human Factors in Computing Systems (CHI), ACM (2002), 73--80. Google Scholar
Digital Library
- Adams, J. A. Historical review and appraisal of research on the learning, retention, and transfer of human motor skills. Psychological Bulletin 101, 1 (1987), 41.Google ScholarCross Ref
- Beaudouin-Lafon, M. Designing interaction, not interfaces. In Proceedings of the working conference on Advanced visual interfaces, ACM (2004), 15--22. Google ScholarDigital Library
- Belda-Lois, J.-M., de Rosario, H., Pons, R., Poveda, R., Morón, A., Porcar, R., Gómez, A., et al. Can human movement analysis contribute to usability understanding? Human movement science 29, 4 (2010), 529--541.Google Scholar
- Card, S., Moran, T. P., and Newell, A. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, 1983. Google ScholarDigital Library
- Clower, D. M., and Boussaoud, D. Selective use of perceptual recalibration versus visuomotor skill acquisition. Journal of Neurophysiology 84, 5 (2000), 2703--2708.Google ScholarCross Ref
- Fallman, D. The new good: exploring the potential of philosophy of technology to contribute to human-computer interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2011), 1051--1060. Google ScholarDigital Library
- Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 6 (1954), 381--391.Google ScholarCross Ref
- Gandolfo, F., Mussa-Ivaldi, F., and Bizzi, E. Motor learning by field approximation. Proceedings of the National Academy of Sciences 93, 9 (1996), 3843--3846.Google ScholarCross Ref
- Ghilardi, M. F., Moisello, C., Silvestri, G., Ghez, C., and Krakauer, J. W. Learning of a sequential motor skill comprises explicit and implicit components that consolidate differently. J Neurophysiol 101, 5 (May 2009), 2218--2229.Google ScholarCross Ref
- Gillies, M., and Kleinsmith, A. Non-representational interaction design. In Contemporary Sensorimotor Theory. Springer, 2014, 201--208.Google Scholar
- Holden, M. K. Virtual environments for motor rehabilitation: review. Cyberpsychology & behavior 8, 3 (2005), 187--211.Google ScholarCross Ref
- Houde, J. F., and Jordan, M. I. Sensorimotor adaptation in speech production. Science 279, 5354 (1998), 1213--1216.Google ScholarCross Ref
- Jacob, R. J., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., and Zigelbaum, J. Reality-based interaction: a framework for post-wimp interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (2008), 201--210. Google ScholarDigital Library
- Jetter, H.-C., Leifert, S., Gerken, J., Schubert, S., and Reiterer, H. Does (multi-)touch aid users' spatial memory and navigation in 'panning' and in 'zooming & panning' uis? In Proceedings of AVI, ACM (2012), 83--90. Google ScholarDigital Library
- Jetter, H.-C., Reiterer, H., and Geyer, F. Blended interaction: understanding natural human-computer interaction in post-wimp interactive spaces. Personal and Ubiquitous Computing 18, 5 (2014), 1139--1158. Google ScholarDigital Library
- Kirsh, D. Embodied cognition and the magical future of interaction design. ACM Transactions on Computer-Human Interaction (TOCHI) 20, 1 (2013), 3. Google ScholarDigital Library
- Krakauer, J. W., Ghez, C., and Ghilardi, M. F. Adaptation to visuomotor transformations: consolidation, interference, and forgetting. The Journal of Neuroscience 25, 2 (2005), 473--478.Google ScholarCross Ref
- Krakauer, J. W., and Mazzoni, P. Human sensorimotor learning: adaptation, skill, and beyond. Current opinion in neurobiology 21, 4 (2011), 636--644.Google Scholar
- Krakauer, J. W., Mazzoni, P., Ghazizadeh, A., Ravindran, R., and Shadmehr, R. Generalization of motor learning depends on the history of prior action. PLoS biology 4, 10 (2006), e316.Google Scholar
- Mattar, A. A., and Ostry, D. J. Modifiability of generalization in dynamics learning. Journal of neurophysiology 98, 6 (2007), 3321--3329.Google Scholar
- Norman, D. A. The way i see it: Natural user interfaces are not natural. interactions 17, 3 (2010), 6--10. Google ScholarDigital Library
- O'hara, K., Harper, R., Mentis, H., Sellen, A., and Taylor, A. On the naturalness of touchless: putting the "interaction" back into nui. ACM Transactions on Computer-Human Interaction (TOCHI) 20, 1 (2013), 5. Google ScholarDigital Library
- Poddar, I., Sethi, Y., Ozyildiz, E., and Sharma, R. Toward natural gesture/speech hci: A case study of weather narration. In Proc. Workshop on Perceptual User Interfaces (PUI), Citeseer (1998).Google Scholar
- Shadmehr, R. Generalization as a behavioral window to the neural mechanisms of learning internal models. Human movement science 23, 5 (2004), 543--568.Google Scholar
- Soukoreff, R. W., and MacKenzie, I. S. Towards a standard for pointing device evaluation, perspectives on 27 years of fitts' law research in hci. International Journal of Human-Computer Studies 61, 6 (2004), 751--789. Google ScholarDigital Library
- Tan, D. S., Pausch, R., Stefanucci, J. K., and Proffitt, D. R. Kinesthetic cues aid spatial memory. In CHI '02 Extended Abstracts on Human Factors in Computing Systems, ACM (2002), 806--807. Google ScholarDigital Library
- Tremblay, S., Houle, G., and Ostry, D. J. Specificity of speech motor learning. The Journal of Neuroscience 28, 10 (2008), 2426--2434.Google ScholarCross Ref
- Van Dam, A. Post-wimp user interfaces. Communications of the ACM 40, 2 (1997), 63--67. Google ScholarDigital Library
- Wei, K., Yan, X., Kong, G., Yin, C., Zhang, F., Wang, Q., and Kording, K. P. Computer use changes generalization of movement learning. Current Biology 24, 1 (2014), 82--85.Google ScholarCross Ref
- Wigdor, D., and Wixon, D. Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier, 2011. Google ScholarDigital Library
- Wobbrock, J. O., Aung, H. H., Rothrock, B., and Myers, B. A. Maximizing the guessability of symbolic input. In ACM Conference on Human Factors in Computing Systems (CHI) Extended Abstracts, ACM (2005), 1869--1872. Google ScholarDigital Library
- Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In ACM Conference on Human Factors in Computing Systems (CHI), ACM (2009), 1083--1092. Google ScholarDigital Library
- Wolpert, D. M., Ghahramani, Z., and Flanagan, J. R. Perspectives and problems in motor learning. Trends in cognitive sciences 5, 11 (2001), 487--494.Google Scholar
- Zhu, F. F., Poolton, J. M., Maxwell, J. P., Fan, J. K. M., Leung, G. K. K., and Masters, R. S. W. Refining the continuous tracking paradigm to investigate implicit motor learning. Exp Psychol 61, 3 (2014), 196--204.Google ScholarCross Ref
Index Terms
- The Transfer of Learning as HCI Similarity: Towards an Objective Assessment of the Sensory-Motor Basis of Naturalness
Recommendations
Touchless circular menus: toward an intuitive UI for touchless interactions with large displays
AVI '14: Proceedings of the 2014 International Working Conference on Advanced Visual InterfacesResearchers are exploring touchless interactions in diverse usage contexts. These include interacting with public displays, where mouse and keyboards are inconvenient, activating kitchen devices without touching them with dirty hands, or supporting ...
Investigating the interaction of children through NUI in e-learning applications
IHC '17: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing SystemsAlthough the use of natural user interfaces (NUI) has been growing, there is still a discussion of what is "natural" to a specific audience. This article presents an investigation on how children use and understand hands' natural movements (gestures) ...
Involving users in the gestural language definition process for the NInA framework
IHC '13: Proceedings of the 12th Brazilian Symposium on Human Factors in Computing SystemsAlong the past years, many interaction devices had been introduced to support the interaction between humans and computers, as the digital pen, touch sensitive devices and, more recently, gesture recognition devices. The last one allows a human-computer ...
Comments