Skip to main content
SpringerLink
Log in

From psychomotor to ‘motorpsycho’: learning through gestures with body sensory technologies

  • Development Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

As information and communication technology continues to evolve, body sensory technologies, like the Microsoft Kinect, provide learning designers new approaches to facilitating learning in an innovative way. With the advent of body sensory technology like the Kinect, it is important to use motor activities for learning in good and effective ways. In this article, we aim to examine both empirical illustrations and theoretical underpinnings for the gesture-based or motor-based learning enabled by the body sensory technology. We review and distill salient concepts and ideas from the existing theoretical and empirical literature related to body-movement- and gesture-based learning, and propose a motorpsycho learning approach. In our discussion, the word/affix motor is synonym to gestures and body movements, and psycho is synonym to cognitive activities. We explore the important role that motors play in psychological activities, especially in cognitive learning. We argue that motors can facilitate psychological activities in learning by enhancing information processing, encoding, representing, and communicating. We also call for more empirical studies on technology-enhanced and gesture-based learning to design, practice, and examine the motorpsycho learning approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Alibali, M. W., & DiRusso, A. A. (1999). The function of gesture in learning to count: More than keeping track. Cognitive Development, 14(1), 37–56.

    Article  Google Scholar 

  • Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of Learning Science, 21(2), 247–286.

    Article  Google Scholar 

  • Allen, L. Q. (1995). The effects of emblematic gestures on the development and access of mental representations of French expressions. The Modern Language Journal, 79(4), 521–529.

    Article  Google Scholar 

  • Amorim, M.-A., Isableu, B., & Jarraya, M. (2006). Embodied spatial transformations: “body analogy” for the mental rotation of objects. Journal of Experimental Psychology: General, 135(3), 327–347.

    Article  Google Scholar 

  • Anderson, M. L. (2003). Embodied cognition: A field guide. Artificial Intelligence, 149(1), 91–130.

    Article  Google Scholar 

  • Arzarello, F., Paola, D., Robutti, O., & Sabena, C. (2009). Gestures as semiotic resources in the mathematics classroom. Educational Studies in Mathematics, 70(2), 97–109.

    Article  Google Scholar 

  • Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation (Vol. Two, pp. 89–195). New York: Academic Press.

    Google Scholar 

  • Aymerich-Franch, L. (2010). Presence and emotions in playing a group game in a virtual environment: The influence of body participation. Cyberpsychology, behavior, and networking, 13(6), 649–654.

    Article  Google Scholar 

  • Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. Eight, pp. 47–89). New York: Academic Press.

    Google Scholar 

  • Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.

    Article  Google Scholar 

  • Barsalou, L. W. (2010). Grounded cognition: past, present, and future. Topics in Cognitive Science, 2(4), 716–724.

    Article  Google Scholar 

  • Barsalou, L. W., Niedenthal, P. M., Barbey, A., & Ruppert, J. (2003). Social embodiment. In B. Ross (Ed.), The psychology of learning and motivation (Vol. 43, pp. 43–92). San Diego: Academic.

    Google Scholar 

  • Bautista, A., Roth, W. M., & Thom, J. S. (2011). Knowing, insightful learning, and the integrity of kinetic movement. Interchange, 42(4), 363–388.

    Article  Google Scholar 

  • Bavelas, J. B., Chovil, N., Coates, L., & Roe, L. (1995). Gestures specialized for dialogue. Personality and Social Psychology Bulletin, 21, 394–405.

    Article  Google Scholar 

  • Birchfield, D., & Johnson-Glenberg, M. (2010). A next gen interface for embodied learning: SMALLab and the geological layer cake. International Journal of Gaming and Computer-Mediated Simulations, 2(1), 49–58.

    Article  Google Scholar 

  • Bloom, B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co., Inc.

    Google Scholar 

  • Bloom, B. S. (1994). Reflections on the development and use of the taxonomy. In L. W. Anderson & L. A. Sosniak (Eds.), Bloom’s Taxonomy: A Forty-Year Retrospective (pp. 1–8). Chicago: National Society for the Study of Education.

    Google Scholar 

  • Borghi, A. M., & Cimatti, F. (2010). Embodied cognition and beyond: Acting and sensing the body. Neuropsychologia, 48(3), 763–773.

    Article  Google Scholar 

  • Broaders, S., Cook, S. W., Mitchell, Z., & Goldin-Meadow, S. (2007). Making children gesture brings out implicit knowledge and leads to learning. Journal of Experimental Psychology: General, 136(4), 539–550.

    Article  Google Scholar 

  • Chambers, R. (2011). Using Innovation as well as the Kinect in Education. http://raychambers.wordpress.com/.

  • Chang, Y. J., Chen, S. F., & Huang, J. D. (2011). A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities, 32, 2566–2570.

    Article  Google Scholar 

  • Chang, C. Y., Chien, Y. T., Chiang, C. Y., Lin, M. C., & Lai, H. C. (2013). Embodying gesture-based multimedia to improve learning. British Journal of Educational Technology, 44(1), 5–9.

    Article  Google Scholar 

  • Chui, K. (2011). Conceptual metaphors in gesture. Cognitive Linguistics, 22(3), 437–458.

    Article  Google Scholar 

  • Church, R. B., & Goldin-Meadow, S. (1986). Using the relationship between gesture and speech to capture transitions in learning. Cognition, 2(3), 43–71.

    Article  Google Scholar 

  • Cook, S. W., & Goldin-Meadow, S. (2006). The role of gesture in learning: Do children use their hands to change their minds? Journal of Cognition and Development, 7(2), 211–232.

    Article  Google Scholar 

  • Cook, S. W., Yip, T. K., & Goldin-Meadow, S. (2010). Gesturing makes memories that last. Journal of Memory and Language, 63(4), 465–475.

    Article  Google Scholar 

  • Crowder, E. M., & Newman, D. (1993). Telling what they know: The role of gestures and language in children’s science explanations. Pragmatics & Cognition, 1, 341–376.

    Article  Google Scholar 

  • Davis, J. I., & Markman, A. B. (2012). Embodied cognition as a practical paradigm: Introduction to the topic, he future of embodied cognition. Topics in Cognitive Science, 4(4), 685–691.

    Article  Google Scholar 

  • Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323(5910), 66–69.

    Article  Google Scholar 

  • DePaulo, B. M. (1992). Nonverbal behavior and self-presentation. Psychological Review, 111, 203–243.

    Google Scholar 

  • Donald, M. (1991). Origins of the modern mind: Three stages in the evolution of culture and cognition. Cambridge, MA: Harvard University Press.

  • Driscoll, M. P. (2005). Psychology of Learning for Instruction (3rd ed.). New York: Allyn & Bacon.

    Google Scholar 

  • Edwards, L. D. (2009). Gestures and conceptual integration in mathematical talk. Educational Studies in Mathematics, 70(2), 127–141.

    Article  Google Scholar 

  • Ezequiel, M., & Robert, M. K. (2004). The role of gestures in spatial working memory and speech. The American Journal of Psychology, 117(3), 411–424.

    Article  Google Scholar 

  • Feyereisen, P. (1987). Gestures and speech, interactions and separations: A reply to McNeill (1985). Psychological Review, 94, 493–498.

    Article  Google Scholar 

  • Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565.

    Article  Google Scholar 

  • Goldin-Meadow, S., & Alibali, M. W. (2013). Gesture’s role in speaking, learning, and creating language. Annual Review of Psychology, 64, 257–283.

    Article  Google Scholar 

  • Goldin-Meadow, S., & Wagner, S. M. (2005). How our hands help us learn. TRENDS in Cognitive Sciences, 9(5), 234–241.

    Article  Google Scholar 

  • Goldin-Meadow, S., Alibali, M. W., & Church, R. B. (1993). Transitions in concept acquisition: Using the hand to read the mind. Psychological Review, 100(2), 279–297.

    Article  Google Scholar 

  • Goldin-Meadow, S., Cook, S. W., & Mitchell, Z. A. (2009). Gesturing gives children new ideas about math. Psychological Science, 20, 267–272.

    Article  Google Scholar 

  • Gullberg, M. (2006). Some reasons for studying gesture and second language acquisition (Hommage à Adam Kendon). IRAL-International Review of Applied Linguistics in Language Teaching, 44(2), 103–124.

    Article  Google Scholar 

  • Hachman, M. (2010). Microsoft’s ‘avatar’ project builds on ‘Kinect’. PC Magazine Online. http://www.pcmag.com/article2/0,2817,2367163,00.asp.

  • Harel, I. E., & Papert, S. E. (1991). Constructionism. New york: Ablex Publishing.

    Google Scholar 

  • Heeter, C. (1992). Being there: The subjective experience of presence. Presence, 1, 262–271.

    Google Scholar 

  • Held, R., Gupta, A., Curless, B., & Agrawala, M. (2012). 3D puppetry: A kinect-based interface for 3D animation. In R. Miller, H. Benko & C. Latulipe (eds.),UIST(pp. 423–434). New york: ACM.

  • Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137(2), 297–315.

    Article  Google Scholar 

  • Hsiao, K. F., & Rashvand, H. F. (2011). Integrating body language movements in augmented reality learning environment. Human-centric Computing and Information Sciences, 1(1), 1–10.

    Article  Google Scholar 

  • Johnson-Glenberg, M. (2012). GEARS. Retrieved from http://egl.lsi.asu.edu/gear.html.

  • Joo, Y. J., Joung, S., & Kim, E. K. (2013). Structural relationships among e-learners’ sense of presence, usage, flow, satisfaction, and persistence. Educational Technology & Society, 16(2), 310–324.

    Google Scholar 

  • Jumpido (Version 1) [Software]. (2013). Sofia, Bulgaria: Nimero Ltd. http://www.jumpido.com/en/education/kinect/school/download.

  • Kelly, S. D., Özyürek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21, 260–267.

    Article  Google Scholar 

  • Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207–227). The Hague: Mouton Publishers.

    Google Scholar 

  • Kissko, J. (2011). Welcome! What is KinectEDucation? http://www.kinecteducation.com/blog/kinect-in-education/.

  • Kita, S., & Davies, T. S. (2009). Competing conceptual representations trigger co-speech representational gestures. Language & Cognitive Processes, 24, 761–775.

    Article  Google Scholar 

  • Kita, S., Furman, R., Brown, A., Ishizuka, T., Allen, S., & Özyürek, A. (2007). Relations between syntactic encoding and co-speech gestures: Implications for a model of speech and gesture production. Journal of Language and Cognitive Processes, 22(8), 1212–1236.

    Article  Google Scholar 

  • Lee, W., Huang, C., Wu, C., Huang, S., & Chen, G. (2012). The effects of using embodied interactions to improve learning performance. 2012 IEEE 12th International Conference on Advanced Learning Technologies (ICALT), July 4–6 2012, (pp.557–559).

  • Leeds, J. (2007). Attention and motor skill learning. International Journal of Sports Science and Coaching, 2(3), 329–334.

    Article  Google Scholar 

  • Leite, L., & Orvalho, V. (2011). Anim-actor: Understanding interaction with digital puppetry using low-cost motion capture. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (pp. 65). ACM.

  • Luft, A. R., & Buitrago, M. M. (2005). Stages of motor skill learning. Molecular Neurobiology, 32(3), 205–216.

    Article  Google Scholar 

  • Macedonia, M., & Knosche, T. R. (2011). Body in mind: How gestures empower foreign language learning. Mind, Brain, and Education, 5(4), 196–211.

    Article  Google Scholar 

  • Mayer, R. E., & DaPra, C. S. (2012). An embodiment effect in computer-based learning with animated pedagogical agents. Journal of Experimental Psychology: Applied, 18(3), 239–252.

    Google Scholar 

  • McNeill, D. (1992). Hand and Mind: What Gestures Reveal about Thought. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Mizelle, J. C., & Wheaton, L. A. (2010). The neuroscience of storing and molding tool action concepts: How “plastic” is grounded cognition? Frontiers in psychology, 1, 1–9.

    Article  Google Scholar 

  • Niedenthal, P. M., Barsalou, L. W., Winkielman, P., Krauth-Gruber, S., & Ric, F. (2005). Embodiment in attitudes, social perception, and emotion. Personality and Social Psychology Review, 9(3), 184–211.

    Article  Google Scholar 

  • Ozcelik, E., & Sengul, G. (2012). Gesture-based interaction for learning: Time to make the dream a reality. British Journal of Educational Technology, 43(3), 86–89.

    Article  Google Scholar 

  • Ping, R., & Goldin-Meadow, S. (2010). Gesturing saves cognitive resources when talking about nonpresent objects. Cognitive Science, 34, 602–619.

    Article  Google Scholar 

  • Qiang, W., & Ke, F. (2013, June). Using virtual world Lego to develop fraction understanding. Poster presentation at the Games+Learning+Society Conference 2013, the Games+Learning+Society Group, Madison, Wisconsin.

  • Recuay, E. F. M. (2011). A gaming platform for enhance physical and cognitive activity training in older adults (master’s thesis). Wake Forest University, Winston-Salem, NC.

  • Reynolds, F. J., & Reeve, R. A. (2002). Gesture in collaborative mathematics problem-solving. Journal of Mathematical Behavior, 20, 447–460.

    Article  Google Scholar 

  • Richey, R. C. (1986). The Theoretical and Conceptual Bases of Instructional Design. New York: Kogan Page Ltd.

    Google Scholar 

  • Riseborough, M. G. (1982). Meaning in movement: An investigation into the interrelationship of physiographic gestures and speech in seven-year-olds. British Journal of Psychology, 73, 497–503.

    Article  Google Scholar 

  • Riva, G. (2009). Is presence a technology issue? Some insights from cognitive sciences. Virtual Reality, 13(3), 159–169.

    Article  Google Scholar 

  • Rizzolatti, G., & Luppino, G. (2001). The cortical motor system. Neuron, 31, 889–901.

    Article  Google Scholar 

  • Robbins, P., & Aydele, M. (Eds.). (2009). The Cambridge handbook of situated cognition. New York: Cambridge University Press.

    Google Scholar 

  • Roth, W. M. (1996). Thinking with hands, eyes, and signs: Multimodal science talk in a grade 6/7 unit on simple machines. Interactive Learning Environments, 4, 170–187.

    Article  Google Scholar 

  • Roth, W. M. (2001). Gestures: Their role in teaching and learning. Review of Educational Research, 71(3), 365–392.

    Article  Google Scholar 

  • Rumme, P., Saito, H., Ito, H., Oi, M., & Lepe, A. (2008). Gestures as effective teaching tools: Are students getting the point? – A study in pointing gesture in the English as a second language classroom. International Journal of Psychology, 43(3), 604–609.

    Google Scholar 

  • Sauter, M., Uttal, D., Alman, A. S., Goldin-Meadow, S., & Levine, S. C. (2012). Learning what children know about space from looking at their hands: The added value of gesture in spatial communication. Journal of Experimental Child Psychology, 111(4), 587–606.

    Article  Google Scholar 

  • Schloerb, D. W. (1995). A quantitative measure of telepresence. Presence, 4, 64–80.

    Google Scholar 

  • Schuemie, M. J., Van Der Straaten, P., Krijn, M., & Van Der Mast, C. A. (2001). Research on presence in virtual reality: A survey. CyberPsychology & Behavior, 4(2), 183–201.

    Article  Google Scholar 

  • Shapiro, L. A. (2011). Embodied cognition. London: Routledge.

    Google Scholar 

  • Shoval, E. (2011). Using mindful movement in cooperative learning while learning about angles. Instructional Science, 39(4), 453–466.

    Article  Google Scholar 

  • Simpson, E. J. (1972). The Classification of Educational Objectives in the Psychomotor Domain. Washington, DC: Gryphon House.

    Google Scholar 

  • Slater, M., Steed, A., McCarthy, J., & Maringelli, F. (1998). The influence of body movement on subjective presence in virtual environments. Human factors: The Journal of the Human Factors and Ergonomics Society, 40(3), 469–477.

    Article  Google Scholar 

  • So, W. C., Chen-Hui, C. S., & Wei-Shan, J. L. (2012). Mnemonic effect of iconic gesture and beat gesture in adults and children: Is meaning in gesture important for memory recall? Language and Cognitive Processes, 27(5), 665–681.

    Article  Google Scholar 

  • Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (pp. 409–426). Cambridge: Cambridge University Press.

    Google Scholar 

  • Suma, E., Lange, B., Rizzo, A., Krum, D., & Bolas, M. (2011). FAAST: The flexible action and articulated skeleton toolkit. Proceedings of IEEE Virtual Reality, 2(pp. 47–248).

  • Suma, E., Krum, D., Lange, B., Koenig, S., Rizzo, A., & Bolas, M. (2013). Adapting user interfaces for gestural interaction with the flexible action and articulated skeleton toolkit. Computers & Graphics, 37(3), 193–201.

    Article  Google Scholar 

  • Valenzeno, L., Alibali, M. W., & Klatzky, R. (2003). Teachers’ gestures facilitate students’ learning: A lesson in symmetry. Contemporary Educational Psychology, 28, 187–204.

    Article  Google Scholar 

  • VanderVen, M. (2012). Try Kinect in education—free trial for K-12. Microsoft Education Tech Upstate New York, Microsoft. http://blogs.technet.com/b/microsoft_education_tech_upstate_new_york/archive/2012/09/24/try-kinect-in-education-free-trial-for-k-12.aspx.

  • Vinayak, V., Murugappan, S., Liu, H., & Ramani, K. (2013). Shape-it-up: Hand gesture based creative expression of 3D shapes using intelligent generalized cylinders. Computer-Aided Design, 45(2), 277–287.

    Article  Google Scholar 

  • Vogel, B., Pettersson, O., Kurti, A., & Huck, A. S. (2012). Utilizing gesture based interaction for supporting collaborative explorations of visualizations in TEL. Proceedings of the 2012 Seventh IEEE International Conference on Wireless, Mobile and Ubiquitous Technology in Education.

  • Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4), 625–636.

    Article  Google Scholar 

  • Wilson, R. A., & Foglia, L. (2011). Embodied cognition. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy, Fall 2011 Edition. http://plato.stanford.edu/archives/fall2011/entries/embodied-cognition.

  • Xu, X., Xue, X., Dai, Z., Pan, Y., & Ke, F. (2013, June ). The Island of Pi—Facilitating math learning through a virtual-reality-based game intervention. Poster presentation at the Games+Learning+Society Conference 2013, the Games+Learning+Society Group, Madison, Wisconsin.

  • Yoon, C., Thomas, M. O. J., & Dreyfus, T. (2011). Grounded blends and mathematical gesture spaces: Developing mathematical understandings via gestures. Educational Studies in Mathematics, 78, 371–393.

    Article  Google Scholar 

  • Yu, C., Smith, L. B., Shen, H., Pereira, A. F., & Smith, T. (2009). Active information selection: Visual attention through the hands. IEEE Transactions on Autonomous Mental Development, 1(2), 141–151.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Educational Psychology and Learning Systems, Florida State University, Tallahassee, FL, USA

    Xinhao Xu & Fengfeng Ke

  2. School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China

    Xinhao Xu

Authors
  1. Xinhao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fengfeng Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinhao Xu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, X., Ke, F. From psychomotor to ‘motorpsycho’: learning through gestures with body sensory technologies. Education Tech Research Dev 62, 711–741 (2014). https://doi.org/10.1007/s11423-014-9351-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-014-9351-8

Keywords

Search

Navigation