Fostering multimedia learning of science: Exploring the role of an animated agent’s image

Abstract

Research suggests that students learn better when studying a picture coupled with narration rather than on-screen text in a computer-based multimedia learning environment. Moreover, combining narration with the visual presence of an animated pedagogical agent may also encourage students to process information deeper than narration or on-screen text alone. The current study was designed to evaluate three effects among students learning about the human cardiovascular system: the modality effect (narration vs. on-screen text), the embodied agent effect (narration + agent vs. on-screen text), and the image effect (narration + agent vs. narration). The results of this study document large and significant embodied agent and image effects on the posttest (particularly retention items) but surprisingly no modality effect was found. Overall, the results suggest that incorporating an animated pedagogical agent – programmed to coordinate narration with gaze and pointing – into a science-focused multimedia learning environment can foster learning.

Introduction

As new technologies emerge that enable instructional designers to create powerful multimedia learning environments for desktop computers, the interest in incorporating animated pedagogical agents into instruction has increased. According to Dehn and van Mulken (2000), animated pedagogical agents are characters on the computer screen with embodied life-like behaviors such as speech, emotions, locomotion, gestures, and movements of the head, the eye, or other parts of the body. Because these behaviors mimic the type of non-verbal communication that usually occur in human interaction, animated agents capable of these behaviors are considered powerful additions to multimedia learning environments since they enable these environments to combine verbal and non-verbal forms of communication (Atkinson, 2002, Atkinson et al., 2005).

Currently, there are a number of animated agents used in a variety of computer-based multimedia learning environments. For instance, Herman the Bug is a talkative agent that provides students real time advice in an environment designed to focus on botanical anatomy (Lester and Stone, 1997, Moreno et al., 2001). Peedy the Parrot is an animated agent programmed to fly across the screen and use gesture or gaze to help the learners associate verbal information (textual or aural) with visual information in an environment involving multi-step proportion word problems (Atkinson, 2002, Atkinson et al., 2005). In a learning environment focusing on how an electric motor works, Dr. Phyz uses his voice to answer questions popped on the screen while moving to point on the explanatory visuals (Mayer, Dow, & Mayer, 2003). On the other hand, Amy Baylor explores how multiple animated agents can by deployed in learning environments in ways that enable learners to develop multiple perspectives by watching several animated agents interact in association with a single learning task (Baylor, 1999, Baylor, 2000, Baylor, 2001, Baylor, 2002a, Baylor, 2002b, Baylor and Ryu, 2003).

One theoretical perspective that helps account for the positive impact of animated agents used in computer-based learning environments is social agency theory (Atkinson et al., 2005, Mayer, Sobko, et al., 2003, Moreno et al., 2001). Social agency theory contends that bringing verbal (spoken words) and non-verbal social cues (e.g., gestures, gaze, locomotion) into a multimedia environment can simulate the human-to-human connection, therefore facilitating the students’ engagement in the learning process (Atkinson et al., 2005, Mayer, Sobko, et al., 2003, Moreno et al., 2001). Once such a simulated human-to-human connection is established, the social communication between the student and computer is thought to be natural and automatic, following the rules of human communication. Furthermore, these automatic responses to computers can be evoked by minimal social cues (Reeves & Nass, 1996). A gesture, gaze, or nod from an animated pedagogical agent can create an environment in which the computer invokes the social cues typically found in human-to-human communication (Atkinson, 2002), thus encouraging the learner to behave as if he or she is communicating with another human. However, the learner may not be consciously aware of the humanlike qualities of the computer in this social communication.

Although social agency theory is one theoretical perspective that justifies why animated agents should rely on spoken rather than written text in multimedia learning environments, multimedia learning theory provides additional support for this practice (Mayer, 2001). According to this theory, our cognitive architecture is capable of dual-channel information processing, that is, that we process visual and aural information through two channels that work independently – the auditory/verbal channel for processing words and the visual/pictorial channel for processing pictures. While words can be delivered either aurally (i.e., narration) or visually (i.e., on-screen text), images can only be delivered in the visual mode. Some researchers contend that when both words and images are involved in the instruction, verbal information delivered in an auditory rather than a visual mode decreases the cognitive load and increases the size of working memory because our cognitive architecture can receive information from both auditory and visual channels simultaneously as opposed to sequentially (Jeung, Chandler, & Sweller, 1997). One outgrowth of this theory is the modality principle (Mayer, 2001), which suggests that if a multimedia environment contains words and images, words should be spoken (e.g., narration) rather than written (e.g., on-screen text). Consistent with the modality principle, a number of studies have reported that aurally delivered instruction in combination with pictures is superior to instructions delivered by text (Atkinson, 2002, Mayer and Moreno, 1998, Moreno and Mayer, 1999, Mousavi et al., 1995).

The use of an agent’s non-verbal humanlike features has been investigated. A number of studies have explored the effect of an agent’s presence in computer-based multimedia learning environments, including embodied agent effect and image effect. The embodied agent effect compares the agent-delivered to the text-delivered instructions to assess the impact on learner performance and attitudes brought by a fully embodied agent that delivers aural instructions and adopts non-verbal cues such as gesture and gaze to direct a learner’s attention. Moreno et al. (2000) found that students had more positive attitudes and better achievement when the lesson was taught by an agent rather than by on-screen text (referred to as personal-agent effect in their study). Atkinson (2002) also found that an embodied agent capable of narration was more effective at fostering learning than a comparable text-based learning environment. The results of his research revealed that students learned from an animated agent reported problem solving less difficulty and produced more conceptually accurate solutions than their peers in the text-only conditions.

The image effect compares two aurally delivered instructions with or without the animated pedagogical agent on the screen and measures the impact of the presence of the agent on learner performance and attitudes. The on-screen agent may use gestures and gaze to direct a learner’s attention. Lester et al. (1997) found that even when the agent was not very expressive, the “very presence” of the agent could strongly and positively affect student perception of their learning experience. Students reported that learning was more enjoyable with the pedagogical agent on the screen. Lester and his colleges referred to this as “persona effect”. Atkinson (2002) also found an image effect when the agent functioned as a visual indicator to bring the learner’s attention to the relevant materials. Therefore the agent can help the learner to dedicate his limited cognitive resources to the important information and transfer knowledge to problem solving.

Nonetheless, the research findings on whether an animated pedagogical agent can facilitate learning are mixed. For example, although Lester et al. (1997) found favorable results of the presence of an animated pedagogical agent even if it is not fully expressive, Moreno et al. (2001) found that the visual presence of the agent did not affect students’ learning although it did not hurt students’ performance. Along the same line, van Mulken, André, and Müller (1998) conducted an empirical study which indicated neither a positive nor a negative effect on comprehension and recall performance from the agent’s presence. One argument against using an animated pedagogical agent is that it might produce a split-attention effect. Split-attention effect occurs when the presence of an agent cannot be intellectually integrated with the instruction but serves as additional cognitive load. A possible reason is that the students might focus their visual attention on the agent instead of the visuals that need to be integrated with the spoken text (Craig, Gholson, & Driscoll, 2002). However, in Craig et al.’s study (2002) that compared learning performance under agent-only, agent-plus-gesture, and no-agent conditions, no split-attention effect was found in any of the dependent measures. Detractors of agents also suggest that the agent’s physical image on the screen might act as a source of “seductive detail” (Mayer, Dow, et al., 2003). Based on their research involving an animated pedagogical agent named Dr. Phyz who flies around to gesture in addition to on-screen arrows, Mayer et al. concluded that the image of the agent might not be as important as the agent’s voice in learning improvement.