Elsevier

Computers & Education

Volume 53, Issue 3, November 2009, Pages 667-676
Computers & Education

Haptic feedback and students’ learning about levers: Unraveling the effect of simulated touch

https://doi.org/10.1016/j.compedu.2009.04.004Get rights and content

Abstract

While there has been extensive experimental research on haptics, less has been conducted on cross-modal interactions between visual and haptic perception and even less still on cross-modal applications in instructional settings. This study looks at a simulation on the principles of levers using both visual and haptic feedback: one group received visual and haptic feedback while the other just visual feedback. Using the triangulation of learning scores, eye tracking data, and video analysis of interaction with the levers, the efficacy of haptic feedback to improve learning was explored. The results indicate that while the total fixation time on the levers and numeric readout was greater for the visual and haptic group, very similar patterns of visual attention were seen between groups. Perhaps surprisingly, the visual only group scored higher on an embedded assessment. Explanations for these results are synthesized from theories of cross-modal perception and cognitive architecture.

Introduction

Despite ever increasing interest in the creation and use of computer-based instructional programs (e.g. interactive simulations, virtual labs, digital learning environments) for the teaching of school science concepts (Hennessy et al., 2007), the extent to which these technologies impact students’ understandings is still unclear. While numerous studies (e.g. Bransford et al., 2000, Doerr, 1997, Linn, 2003, Winn, 2002, Zacharia, 2003) point to potentially positive impacts, other work (e.g. Bayraktar, 2002, de Jong and van Joolingen, 1998, Hsu and Thomas, 2002, Steinberg, 2000) suggests a rather tenuous link between the use of these technologies and learning gains.

Proponents of computer-based simulations note that these virtual environments allow students to make comparisons between elements of a system and witness the outcomes much as you would with their physical counterparts (Linn, 2004). In addition, students may also be able to explore relationships in simulations not feasible in the physical realm because of limits of time, space, cost, or safety. It is likely that the efficacy of this experience will be determined in part by how effectively key information about the phenomena can be communicated to the student, mediated by the computer-based system. Improvements in computer-based graphics have meant that very rich, high-resolution color graphics can be communicated visually to the student through most computer systems. Similarly, improvement in audio technology now means that most auditory-based information can also be communicated at high fidelity. While these two modalities cover much of the sensory information instructional designers might want to communicate to learners, it does not cover the full sensory range of what could be communicated nor does it address the general limitations in how learners communicate back to the simulation environment.

Along these lines, evolving technologies now make it possible to extend students’ interactions with these computer-based learning environments beyond the audio and visual realm to include haptic (i.e., simulated touch) feedback (Burdea, 1996, Kátai et al., 2008, Revesz, 1950, Robles-De-La-Torre, 2006). Haptic feedback devices provide a whole new modality of experience that can be tied directly to user input devices, more tightly binding the user experience directly to the simulated environment. Haptically augmented multimodal interfaces can be programmed to provide realistic force feedback (e.g. simulating object compliance, weight, and inertia) and/or tactile feedback (e.g. simulating surface contact geometry, smoothness, slippage, and temperature) by employing physical receptors in the hand and arm that gather sensory information as users “feel” and manipulate two and three-dimensional virtual objects and events (Jacobson et al., 2002, Jones et al., 2006, Minogue and Jones, 2006).

With haptic devices, such as the one seen in Fig. 1, not only can the simulation communicate haptic information about the phenomenal response to the learner, but also provide a more robust feedback mechanism as the learner interacts with the system. This point-probe device tracks the x, y, and z coordinates, as well as the pitch, roll, and yaw of the virtual point-probe that the user moves about a 3D workspace. Actuators (motors within the device) communicate preprogrammed forces back to the user’s fingertips and arm as it detects collisions with the virtual objects on the screen, simulating the sense of touch. While potentially a breakthrough technology for instructional simulation environments, there is a paucity of research to guide instructional designers. This paper will explore the efficacy of a haptically augmented simulation environment for use in middle school science education, capitalizing on eye tracking data to help unravel the influence of simulated touch on student cognition.

Section snippets

Impetus for the study

An appropriate area of science education to employ haptic interfaces may be simulations that require the learner to both apply and respond to force feedback from a system. In upper elementary and middle school science, the study of levers is a common topic where students typically interact with constructed lever systems via the direct application of force with their hands and similarly receive feedback through the same pathway. It would, therefore, seem logical that the inclusion of haptic

The instructional program

For this study, a computer-based virtual lab was developed and tested with middle school (ages 11–14) students attending a week-long science summer camp offered at a middle school by the university. The participants were from three different school systems that served rural, urban, and suburban students. The interactive program introduced participants to first, second, and third class levers and allowed students to create a lever by manipulating the fulcrum location, beam length, and amount of

Results

Written test data, eye tracking, and lever push data were triangulated to provide a better understanding of how haptic feedback influenced learning outcomes and what role changes in visual attentional distribution across the key information sources in the virtual lab may have had in these learning outcomes.

Discussion

The results are both interesting and somewhat surprising but should be viewed cautiously until further studies can replicate these results. Since learning about the principles of levers requires students to understand the relationship between the spatial location of the load, fulcrum, and force along with the relationship between the amount of load and the amount of force needed to move it, visual attention to and integration between these elements will be important. The eye tracking data

Acknowledgements

I would like to sincerely thank Deyao “Daniel” Ren for his expert programming on the haptic lever application. In addition, I would also like to thank all of the counselors working with the students at the Zoom into Science summer camp where this study was conducted.

References (43)

  • G.C. Burdea

    Force and touch feedback for virtual reality

    (1996)
  • T. de Jong et al.

    Scientific discovery learning with computer simulations of conceptual domains

    Review of Educational Research

    (1998)
  • H.M. Doerr

    Experiment, simulation and analysis: An integrated instructional approach to the concept of force

    International Journal of Science Education

    (1997)
  • A.T. Duchowski

    Eye tracking methodology: Theory and practice

    (2003)
  • B. Florence et al.

    The visuo-haptic and haptic exploration of letters increases the kindergarten-children’s understanding of the alphabetic principle

    Cognitive Development

    (2004)
  • J.M. Gottman et al.

    Sequential analysis: A guide for behavioral researchers

    (1990)
  • M.A. Heller

    The psychology of touch

  • J.M. Henderson

    Visual attention and eye movement control during reading and picture viewing

  • Y. Hsu et al.

    The impacts of a web-aided instructional simulation on science learning

    International Journal of Science Education

    (2002)
  • Jacobson, R. D., Kitchen, R., & Golledge, R. (2002). Multimodal virtual reality for presenting geographic information....
  • M.G. Jones et al.

    Learning at the nanoscale: The impact of students’ use of remote microscopy on concepts of viruses, scale, and microscopy

    Journal of Research in Science Teaching

    (2003)
  • Cited by (45)

    • Effect of haptic feedback on a virtual lab about friction

      2019, Virtual Reality and Intelligent Hardware
    • Embodied learning using a tangible user interface: The effects of haptic perception and selective pointing on a spatial learning task

      2016, Computers and Education
      Citation Excerpt :

      Although a rapidly growing body of research within cognitive psychology and neuroscience supports these claims, attempts to utilize these results in more applied fields of psychology have led to mixed results. A number of studies found positive effects of perceptually enhanced learning materials featuring haptic feedback (Minogue & Jones, 2006; Schönborn, Bivall, & Tibell, 2011; Wiebe, Minogue, Gail Jones, Cowley, & Krebs, 2009), and interactivity (for a review, see Plass, Homer, & Hayward, 2009). Consequently, researchers have asked for updates to the “less is more” stance of CLT to be compatible with findings from EC (Paas & Sweller, 2011).

    • Examining whether touch sensory feedback is necessary for science learning through experimentation: A literature review of two different lines of research across K-16

      2015, Educational Research Review
      Citation Excerpt :

      Much remains to be learned about the relative efficacy of PM and VM when they are used in different subject domains, with different instructional goals, approaches, outcome measures, and types of students (Klahr et al., 2007). Research focusing on the integration of haptics – devices providing simulated tactile and/or kinesthetic feedback – in virtual learning environments (e.g., simulations) has also revealed instances when touch sensory feedback positively impacts students' science learning (e.g., Bivall et al., 2011; Brooks et al., 1990; Dede et al., 2000; Hallman et al., 2009; Han & Black, 2011; Jones et al., 2003, 2006; Minogue & Jones, 2009; Reiner, 1999; Schönborn et al., 2011) and instances when touch sensory feedback does not make a difference to students' science learning (e.g., Bivall et al., 2007; Minogue et al., 2006; Wiebe et al., 2009). Even though there are a greater number of studies showing a positive effect of touch sensory feedback on students' science learning, the serious lack of empirical studies overall on this issue (Minogue & Jones, 2006; Schönborn et al., 2011; Wiebe et al., 2009) prohibits arriving at solid conclusions regarding the effect of touch sensory feedback through haptic devices within the context of learning through science experimentation.

    View all citing articles on Scopus
    View full text