Gökçe Elif Baykal
Ali Leylekoğlu
Sude Arslan
ABSTRACT
In this paper, we propose a taxonomy for the classification of children’s gestural input elicited from spatial puzzle play in VR hand tracking. The taxonomy builds on the existing manipulative gesture taxonomy in human-computer interaction, and offers two main analytical categories; Goal-directed actions and Hand kinematics as complementary dimensions for analysing gestural input. Based on our study with eight children (aged between 7-14), we report the qualitative results for describing the categories for analysis and quantitative results for their frequency in occurring in children’s interaction with the objects during the spatial task. This taxonomy is an initial step towards capturing the complexity of manipulative gestures in relation to mental rotation actions, and helps designers and developers to understand and study children’s gestures as an input for object interaction as well as an indicator for spatial thinking strategies in VR hand tracking systems.
Supplemental Material
- Dimitra Anastasiou, Valérie Maquil, Eric Ras, and Mehmetcan Fal. 2016. Design implications for a user study on a tangible tabletop. In Proceedings of the The 15th International Conference on Interaction Design and Children. 499–505.Google Scholar
Digital Library
- Jakki O Bailey and Jeremy N Bailenson. 2017. Immersive virtual reality and the developing child. In Cognitive development in digital contexts. Elsevier, 181–200.Google Scholar
- Gökçe Elif Baykal, I Veryeri Alaca, Asim Evren Yantaç, and Tilbe Göksun. 2018. A review on complementary natures of tangible user interfaces (TUIs) and early spatial learning. International journal of child-computer interaction 16 (2018), 104–113.Google ScholarCross Ref
- Gökçe Elif Baykal, Ali Leylekoğlu, Can Bora Sezer, and Işıl Oygür. 2022. Studying Children’s Manipulative Gestures in Spatial Puzzle Play with VR Hand Tracking: Analysis of Goal-directed Actions. In Proceedings of the 2022 ACM Symposium on Spatial User Interaction. 1–2.Google ScholarDigital Library
- Gökçe Elif Baykal, Maarten Van Mechelen, Tilbe Göksun, and Asim Evren Yantaç. 2019. Embedded figures in stories (EFiS): A method to observe preschoolers’ interactions with spatial manipulatives. International Journal of Child-Computer Interaction 21 (2019), 121–129.Google ScholarDigital Library
- Shimmila Bhowmick, Keyur Sorathia, and Pratul Chandra Kalita. 2021. Understanding Gesture Performance for Object Selection in VR: Classification and Taxonomy of Gestures in HCI. In India HCI 2021. 90–93.Google Scholar
- Eunjung Choi, Heejin Kim, and Min K Chung. 2014. A taxonomy and notation method for three-dimensional hand gestures. International Journal of Industrial Ergonomics 44, 1 (2014), 171–188.Google ScholarCross Ref
- Andrew Clark, Anban W Pillay, and Deshendran Moodley. 2020. A system for pose analysis and selection in virtual reality environments. In Conference of the South African Institute of Computer Scientists and Information Technologists 2020. 210–216.Google ScholarDigital Library
- Stacy B Ehrlich, Susan C Levine, and Susan Goldin-Meadow. 2006. The importance of gesture in children’s spatial reasoning.Developmental psychology 42, 6 (2006), 1259.Google Scholar
- Susan Goldin-Meadow. 2015. From action to abstraction: Gesture as a mechanism of change. Developmental review 38 (2015), 167–184.Google Scholar
- Kara M Gregory, An Sook Kim, and Alice Whiren. 2003. The effect of verbal scaffolding on the complexity of preschool children’s block constructions. (2003).Google Scholar
- Celeste Groenewald, Craig Anslow, Junayed Islam, Chris Rooney, Peter J Passmore, and BL Wong. 2016. Understanding 3D mid-air hand gestures with interactive surfaces and displays: a systematic literature review. (2016).Google Scholar
- Kathy Hirsh-Pasek, Jennifer M Zosh, Helen Shwe Hadani, Roberta M Golinkoff, Kevin Clark, Chip Donohue, and Ellen Wartella. 2022. A whole new world: Education meets the metaverse. Policy (2022).Google Scholar
- Maria Karam 2005. A taxonomy of gestures in human computer interactions. (2005).Google Scholar
- The Language Archive Max Planck Institute for Psycholinguistics. 2022. ELAN (Version 6.4). (2022). https://archive.mpi.nl/tla/elanGoogle Scholar
- Joseph Maxwell. 1992. Understanding and Validity in Qualitative Research. Harvard Educational Review 62 (01 1992), 279–300.Google Scholar
- Philipp Mayring. 2000. Qualitative Content Analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research [On-line Journal], http://qualitative-research.net/fqs/fqs-e/2-00inhalt-e.htm 1 (06 2000).Google Scholar
- Joan McComas, Jayne Pivik, and Marc Laflamme. 1998. Children’s transfer of spatial learning from virtual reality to real environments. CyberPsychology & Behavior 1, 2 (1998), 121–128.Google ScholarCross Ref
- Jess McIntosh, Charlie McNeill, Mike Fraser, Frederic Kerber, Markus Löchtefeld, and Antonio Krüger. 2016. EMPress: Practical hand gesture classification with wrist-mounted EMG and pressure sensing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2332–2342.Google ScholarDigital Library
- D McNeill. 1992. Hand and Mind: What Gestures Reveal About Thought. Chicago: Univ.Google Scholar
- Nora S Newcombe and Thomas F Shipley. 2015. Thinking about spatial thinking: New typology, new assessments. In Studying visual and spatial reasoning for design creativity. Springer, 179–192.Google Scholar
- Vladimir I Pavlovic, Rajeev Sharma, and Thomas S. Huang. 1997. Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on pattern analysis and machine intelligence 19, 7 (1997), 677–695.Google ScholarDigital Library
- Francis KH Quek. 1994. Toward a vision-based hand gesture interface. In Virtual reality software and technology. World Scientific, 17–31.Google Scholar
- Amber Solomon, Mark Guzdial, Betsy DiSalvo, and Ben Rydal Shapiro. 2018. Applying a gesture taxonomy to introductory computing concepts. In Proceedings of the 2018 ACM Conference on International Computing Education Research. 250–257.Google ScholarDigital Library
- David H Uttal, David I Miller, and Nora S Newcombe. 2013. Exploring and enhancing spatial thinking: Links to achievement in science, technology, engineering, and mathematics?Current Directions in Psychological Science 22, 5 (2013), 367–373.Google ScholarCross Ref
- Dylan Yamada-Rice, Faisal Mushtaq, Adam Woodgate, D Bosmans, A Douthwaite, I Douthwaite, W Harris, R Holt, D Kleeman, J Marsh, 2017. Children and virtual reality: Emerging possibilities and challenges. (2017).Google Scholar
- Ying Yin. 2012. A hierarchical approach to continuous gesture analysis for natural multi-modal interaction. In Proceedings of the 14th ACM international conference on Multimodal interaction. 357–360.Google ScholarDigital Library
Index Terms
- Studying Children’s Object Interaction in Virtual Reality: A Manipulative Gesture Taxonomy for VR Hand Tracking
Recommendations
Studying Children’s Manipulative Gestures in Spatial Puzzle Play with VR Hand Tracking: Analysis of Goal-directed Actions
SUI '22: Proceedings of the 2022 ACM Symposium on Spatial User InteractionThis paper presents insights about children’s manipulative gestures in a spatial puzzle play (i.e. tangram) in both real and virtual environments. We present our initial work with 11 children (aged between 7 and 14) and preliminary results based on a ...
The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments
MuC '22: Proceedings of Mensch und Computer 2022Natural user interfaces are on the rise. Manufacturers for Augmented, Virtual, and Mixed Reality head mounted displays are increasingly integrating new sensors into their consumer grade products, allowing gesture recognition without additional hardware. ...
Real-time 3D hand-computer interaction: optimization and complexity reduction
NordiCHI '08: Proceedings of the 5th Nordic conference on Human-computer interaction: building bridgesThis paper presents a low-cost method for enabling 3D hand-computer interaction. The method, accompanied by a system, uses the frame capturing functionality of a single consumer-grade webcam. Our recent work has been focused on examining and realizing a ...
Comments