David Kirsh
Abstract
The theory of embodied cognition can provide HCI practitioners and theorists with new ideas about interaction and new principles for better designs. I support this claim with four ideas about cognition: (1) interacting with tools changes the way we think and perceive -- tools, when manipulated, are soon absorbed into the body schema, and this absorption leads to fundamental changes in the way we perceive and conceive of our environments; (2) we think with our bodies not just with our brains; (3) we know more by doing than by seeing -- there are times when physically performing an activity is better than watching someone else perform the activity, even though our motor resonance system fires strongly during other person observation; (4) there are times when we literally think with things. These four ideas have major implications for interaction design, especially the design of tangible, physical, context aware, and telepresence systems.
- Aglioti, S. M., Cesari, P., Romani, M., and Urgesi, C. 2008. Action anticipation and motor resonance in elite basketball players. Nature Neurosci. 11, 9, 1109--1116.Google Scholar
Cross Ref
- Agnew, Z. K., Bhakoo, K. K., and Puri, B. K. 2007. The human mirror system: A theory of mind reading. Brain Res. Rev. 54, 2, 286--293.Google ScholarCross Ref
- Barsalou, L. 1983. Ad hoc categories. Memory Cogn. 11, 211--227.Google ScholarCross Ref
- Barsalou, L. W. 1999. Perceptual symbol systems. Behav. Brain Sci. 22, 577--660.Google ScholarCross Ref
- Barsalou, L. W. 2008. Grounded cognition. Annu. Rev. Psychol. 59, 617--645.Google ScholarCross Ref
- Bisiach, E., Perani, D.,Vallar, G., and Beerti, A. 1986. Unilateral neglect: Personal and extrapersonal. Neuropsychologia 24, 759--767.Google ScholarCross Ref
- Blakeslee, S. 2004. When the brain says, “don't get too close”. The New York Times, July 13.Google Scholar
- Brain, W. R. 1941. Visual disorientation with special reference to lesions of the right hemisphere. Brain 64, 224--272.Google ScholarCross Ref
- Buccino, G., Binkofski, F., and Riggio, L. 2004. The mirror neuron system and action recognition. Brain Lang. 89, 370--376.Google ScholarCross Ref
- Buxton, B. 2007. Sketching User Experience. Sketching User Experiences: Getting The Design Right And The Right Design (Interactive Technologies). Morgan Kaufmann. Google ScholarDigital Library
- Carlson, R. A., Avraamides, M. N., Cary, M., and Strasberg, S. 2007. What do the hands externalize in simple arithmetic? J. Exp. Psychol. Learn. Mem. Cogn. 33, 4, 747--756.Google ScholarCross Ref
- Clark, A. 1997. Being There Putting Brain, Body, and World Together Again. MIT Press, Cambridge, MA. Google ScholarDigital Library
- Clark, A. 2008. Supersizing the Mind: Embodiment, Action, and Cognitive Extension. Oxford University Press.Google Scholar
- Colby, C. L. 1998. Action oriented spatial reference frames in cortex. Neuron 20, 15--24.Google ScholarCross Ref
- Coslett, H. B. 1998. Evidence for a disturbance of the body schema in neglect. Brain Cogn. 37, 529--544.Google ScholarCross Ref
- Cross, E., Kraemer, D. J. M., Hamilton, A. F. De C., Kelley, W. M., and Grafton, S. T. 2009. Sensitivity of the action observation network to physical and observational learning. Cereb. Cortex 19, 315--326.Google ScholarCross Ref
- Eck, D., Gasser, M., and Port, R. 2000. Dynamics and embodiment in beat induction. In Rhythm Perception and Production, P. Desain and L. Windsor, Eds., Swets and Zeitlinger, Exton, PA.Google Scholar
- Endsley, M. 1995. Toward a theory of situation awareness in dynamic systems. Hum. Factors J. Hum. Factors Ergon. Soc. 37, 1, 32--64.Google ScholarCross Ref
- Fodor, J. A. 1975. The Language of Thought. Harvard University Press, Cambridge, MA.Google Scholar
- Frank, M. and Barner, D. 2012. Representing exact number visually using mental abacus. J. Exp. Psychol. General. 141, 1, 134--149.Google ScholarCross Ref
- Gibson, J. J. 1966. The Senses Considered as Perceptual Systems. Houghton Mifflin, Boston.Google Scholar
- Gibson, J. J. and Crooks, L. E. 1938. A theoretical field-analysis of automobile-driving. Amer. J. Psychol. 51, 453--471.Google ScholarCross Ref
- Goldin-Meadow, S. 2005. Hearing Gestures: How Our Hands Help Us to Think. Harvard University Press.Google Scholar
- Goldin-Meadow, S. and Beilock, S. L. 2010. Action's influence on thought: The case of gesture. Persepct. Psychol. Sci. 5, 664--674.Google ScholarCross Ref
- Goodwin, C. 1994. Professional vision. Amer. Anthropol. 96, 3, 606--633.Google ScholarCross Ref
- Graziano, M. S. A. and Gross, C. G. 1995. The representation of extrapersonal space: A possible role for bimodal, visuo--tactile neurons. In The Cognitive Neurosciences, M. S. Gazzaniga, Ed., MIT Press, 1021--1034.Google Scholar
- Harman, G. 1986. Change in View: Principles of Reasoning. MIT Press.Google Scholar
- Hatano, G., Miyake, Y., and Binks, M. G. 1977. Performance of expert abacus operators. Cogn. 5, 47--55.Google ScholarCross Ref
- Hinz, B. 2008. Practice exaggeration for large intervals and leaps. http://www.creativekeyboard.com/oct08/hinz.html.Google Scholar
- Iriki, A. 2009. Using tools: The moment when mind, language, and humanity emerged. Frontlines Riken Res. 4, 5.Google Scholar
- Iriki, A., Tanaka, M., and Iwamura, Y. 1996. Coding of modified body schema during tool use by macaque postcentral neurones. Neurorep. 7, 2325--2330.Google ScholarCross Ref
- Jeannerod, M. 1994. The representing brain: Neural correlates of motor intension and imagery. Behav. Brain Sci. 17, 187--245.Google ScholarCross Ref
- Jeannerod, M. 2001. Neural simulation of action: A unifying mechanism for motor cognition. NeuroImage 14, S103--S109.Google ScholarCross Ref
- Johnson-Laird, P. 1989. Mental models. In The Foundations of Cognitive Science, M. Posner, Ed., MIT Press, Cambridge, MA, Chapter 8. Google ScholarDigital Library
- Jones, K. 2003. What is an affordance? Ecol. Psychol. 15, 2, 107--114.Google ScholarCross Ref
- Kaschak, M. P., Zwaan, R. A., Aveyard, M., and Yaxley, R. H. 2006. Perception of auditory motion affects language processing. Cogn. Sci. 30, 733--744.Google ScholarCross Ref
- Kirsh, D. 1992. When is information explicitly represented? In The Vancouver Studies in Cognitive Science. Oxford University Press, 340--365.Google Scholar
- Kirsh, D. 1995. Complementary strategies: Why we use our hands when we think. In Proceedings of the 17th Annual Conference of the Cognitive Science Society, J. D. Moore and J. F. Lehman, Eds., 212--217.Google Scholar
- Kirsh, D. 2005. Multi-Tasking and cost structure: Implications for design. In Proceedings of the 27th Annual Conference of the Cognitive Science Society. Lawrence Erlbaum, Mahwah, NJ.Google Scholar
- Kirsh, D. 2009a. Problem solving and situated cognition. In The Cambridge Handbook of Situated Cognition, P. Robbins and M. Aydede, Eds., Cambridge University Press.Google Scholar
- Kirsh, D. 2009b. Projection, problem space and anchoring. In Proceedings of the 31st Annual Conference of the Cognitive Science Society, N. A. Taatgen and H. van Rijn, Eds., Cognitive Science Society, 2310--2315.Google Scholar
- Kirsh, D. 2009c. Interaction, external representations and sense making. In Proceedings of the 31st Annual Conference of the Cognitive Science Society, N. A. Taatgen and H. van Rijn, Eds., Cognitive Science Society, 1103--1108.Google Scholar
- Kirsh, D. 2009d. Knowledge, implicit versus explicit. In Oxford Companion to Consciousness. Oxford University Press, Cambridge, UK.Google Scholar
- Kirsh, D. 2010. Explaining artifact evolution. In The Cognitive Life of Things: Recasting the Boundaries of the Mind, L. Malafouris, and C. Renfrew, Eds., McDonald Institute for Archaeological Research.Google Scholar
- Kirsh, D. 2012a. How marking in dance constitutes thinking with the body. In The External Mind: Perspectives on Mediation, Distribution and Situation in Cognition and Semiosis, R. Fusaroli, T. Granelli, and C. Paolucci, Eds., 112--113.Google Scholar
- Kirsh, D. 2012b. Running it through the body. In Proceedings of the 34th Annual Cognitive Science Society. Lawrence Erlbaum.Google Scholar
- Kirsh, D., Caballero R., and Cuykendall, S. 2012. When doing the wrong thing is right. In Proceedings of the 34th Annual Cognitive Science Society. Lawrence Erlbaum.Google Scholar
- Kirsh, D. And Maglio, P. 1995. On distinguishing epistemic from pragmatic actions. Cogn. Sci. 18, 513--549.Google ScholarCross Ref
- Kirsh, D. Muntanyola, D., Jao, J., Lew, A., and Sugihara, M. 2009. Choreographic methods for creating novel, high quality dance. In Proceedings of the 5th Internation al Workshop on Design and Semantics and Form (DESFORM). Kluwer.Google Scholar
- Kossyln, H. and Moulton, S. T. 2009. Mental imagery and implicit memory. In Handbook of Imagination and Mental Imagery, K. D. Markman, W. M. P. Klein, and J. A. Suhr, Eds., Psychology Press, New York.Google Scholar
- Knoblich, G. and Flach, R. 2001. Predicting the effects of actions: Interactions of perception and action. Psychol. Sci. 12, 467--472.Google ScholarCross Ref
- Làdavas, E. 2002. Functional and dynamic properties of visual peripersonal space. Trends Cogn. Sci. 6, 1.Google ScholarCross Ref
- Ladavas, E., Di Pellegrino, G., Farne, A., and Zeloni, G. 1998. Neuropsychological evidence of an integrated visuotactile representation of peripersonal space in humans. J. Cogn. Neurosci. 10, 581--589. Google ScholarDigital Library
- Makin, T. R., Holmes N. P., and Zohary, E. 2007. Is that near my hand? Multisensory representation of peripersonal space in human intraparietal sulcus. J. Neurosci. 27, 731--740.Google ScholarCross Ref
- Malafouris, L. 2008. At the potter's wheel: An argument for material agency. In Material Agency, C. Knappett, and L. Malafouris, Eds., Springer, 19--36.Google Scholar
- Maravita, A. and Iriki, A. 2004. Tools for the body (schema). Trends Cogn. Sci. 8, 2, 79--86.Google ScholarCross Ref
- Maravita, A., Spence, C., Kennett, S., and Driver, J. 2002. Tool-Use changes multimodal spatial interactions between vision and touch in normal humans. Cogn. 83, B25--B34.Google ScholarCross Ref
- Mcluhan, M. 1964. Understanding Media: The Extensions of Man 1st Ed. McGraw Hill, New York.Google Scholar
- Myin, E. and O'Regan, J. K. 2008. Situated perception and sensation in vision and other modalities: Form an active to a sensorimotor account. In Cambridge Handbook of Situated Cognition, P. Robbins and A. Aydede, Eds., Cambridge University Press, 185--200.Google Scholar
- Noë, A. 2005. Action in Perception. MIT Press.Google Scholar
- O'Connor, E. 2006. Glassblowing tools: Extending the body towards practical knowledge and informing a social world. Qual. Sociol. 29, 2, 177--193.Google ScholarCross Ref
- O'Regan, J. K. and Noe, A. 2001. A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24, 939--1031.Google ScholarCross Ref
- Potter, L. 1980. The Art of Cello Playing: A Complete Textbook Method for Private or Class Instruction. Alfred Music Publishing.Google Scholar
- Previc, F. H. 1998. The neuropsychology of 3-D space. Psychol. Bull. 124, 123--163.Google ScholarCross Ref
- Prinz, W. 1997. Perception and action planning. Euro. J. Cogn. Psychol. 9, 129--154.Google ScholarCross Ref
- Proffitt, D. 2006. Embodied perception and the economy of action. Perspect. Psychol. Sci. 1, 110.Google ScholarCross Ref
- Rensink, R. A. 2002. Change detection. Annu. Rev. Psychol. 53, 245--277.Google ScholarCross Ref
- Rizzolatti, G. and Craighero, L. 2004. The mirror-neuron system. Annu. Rev. Neurosci. 27, 169--192.Google ScholarCross Ref
- Rizzolatti, G. and Sinigaglia, C. 2007. Mirrors in the Brain: How Our Minds Share Actions and Emotions. Oxford University Press.Google Scholar
- Scaife, M., and Rogers, Y. 1996. External cognition: How do graphical representations work? Int. J. Hum.-Comput. Stud. 45, 185--213. Google ScholarDigital Library
- Sebanz, N. and Shiffrar, M. 2007. Bodily bonds: Effects of social context on ideomotor movements. In Sensorimotor Foundations of Higher Cognition, P. Haggard, Y. Rosetti, and M. Kawato, Eds., Oxford University Press.Google Scholar
- Shin, Y. K., Proctor, R. W., and Capaldi, E. J. 2010. A review of contemporary ideomotor theory. Psychol. Bull. 136, 6, 943--974.Google ScholarCross Ref
- Simons, D. J. and Chabris, C. F. 1999. Gorillas in our midst: Sustained inattentional blindness for dynamic events. Percept. 28, 9, 1059--1074.Google ScholarCross Ref
- Vaishnavi, S., Calhoun, J., and Chatterjee, A. 1999. Crossmodal and sensorimotor integration in tactile awareness. Neurol. 53, 1596--1598.Google ScholarCross Ref
- Varela, F., Thompson, E., and Rosch, E. 1991. The Embodied Mind: Cognitive Science and Human Experience. MIT Press.Google Scholar
- Viviani, P. 2002. Motor competence in the perception of dynamic events: A tutorial. In Common Mechanisms in Perception and Action, W. Prinz and B. Hommel, Eds., Oxford University Press, 406--442.Google Scholar
- Wexler, M., Kosslyn, S., and Berthoz, A. 1998. Motor processes in mental rotation. Cogn. 68, 77--94.Google ScholarCross Ref
- Williams, A. and Gribble, P. L. 2012. Observed effector-independent motor learning by observing. J. Neurophysiol. 107, 1564--1570.Google ScholarCross Ref
- Wilson, M. and Knoblich, G. 2005. The case for motor involvement in perceiving conspecifics. Psychol. Bull. 131, 460--473.Google ScholarCross Ref
- Winter, B. and Bergen, B. 2012. Language comprehenders represent object distance both visually and auditorily. Lang. Cogn. 4, 1, 1--16.Google ScholarCross Ref
Index Terms
- Embodied cognition and the magical future of interaction design
Recommendations
Distributed cognition: toward a new foundation for human-computer interaction research
Special issue on human-computer interaction in the new millennium, Part 2We are quickly passing through the historical moment when people work in front of a single computer, dominated by a small CRT and focused on tasks involving only local information. Networked computers are becoming ubiquitous and are playing increasingly ...
Cultivating kinaesthetic awareness through interaction: Perspectives from somatic practices and embodied cognition
MOCO '17: Proceedings of the 4th International Conference on Movement ComputingDesigning for kinaesthetic awareness, the perception of our body's position and movement, presents a unique set of challenges and opportunities. While these implications are relatively new in the HCI community, they resonate with experiential knowledge ...
Workshop on embodied interaction: theory and practice in HCI
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing SystemsFor over ten years researchers in human-computer interaction (HCI) have explored an embodied perspective that seeks to describe and explain the fundamental role played by the physical body in how we experience, interact with and understand computation ...
Reviews
Andrea Ballatore
In this paper, cognitive scientist David Kirsh provides an accessible and engaging overview of state-of-the-art research on human cognition in relation to tools. Starting from McLuhan's famous statement, "We shape our tools and thereafter our tools shape us," Kirsh outlines the key tenets of the embodied cognition paradigm, which aims to overcome the Cartesian body/mind dualism that still permeates our culture, and shows clear examples from his studies on expert dancers. Embodied cognition claims that thought is not confined to the brain, but extends and relies on our body parts and external objects in complex ways, enabling us to "think with things." Thought is not an abstract computation; it is firmly grounded in our physical constitution. When we interact with a tool, we rapidly absorb it into our cognitive apparatus, and we enter a new "enactive landscape" with novel affordances that we could not imagine without the tool. Tools impact our motor system, our synesthetic perception, and our conceptualization of reality, redrawing the boundaries of our world. The author formulates questions relevant to interaction design and human-computer interaction: How far can we rewire ourselves into tools__?__ What are the limits of this neuroadaptation__?__ Even tentative answers, of course, will require a lot of work. Although the paper alludes to a "magical future of interaction design," the author fails to explain how these insights can feed back into actual interfaces for information systems, and offers vague and cautious predictions. I recommend this paper to anybody with an interest in philosophy, cognitive science, and human-computer interaction. It offers much food for multidisciplinary thought. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments