Elsevier

Educational Research Review

Volume 15, June 2015, Pages 59-74
Educational Research Review

Review
The promised land of blended learning: Quizzes as a moderator

https://doi.org/10.1016/j.edurev.2015.05.001Get rights and content

Highlights

  • Quizzes positively affect the effectiveness and attractiveness of blended learning.

  • Blended learning is slightly more effective than more traditional learning.

  • Blended learning is equally attractive as more traditional learning.

  • Blended learning seems to be more demanding than more traditional learning.

  • Effects on effectiveness, attractiveness and perceived demands differ a lot.

Abstract

Blended learning, defined as a combination of face-to-face and online learning, is expected to lead to improved education. Besides, practical reasons, like increased access to education and resource management, are mentioned for its implementation. To examine whether the expectation of improved education is met, meta-analyses were conducted. They revealed that, on average, blended learning is somewhat more effective than more traditional learning. Additionally, students evaluated it as equally attractive, but seemed to perceive it as more demanding. In sum, blended learning is equal, or maybe even better, than more traditional learning. However, the effects on effectiveness, attractiveness and perceived demands differed much between studies. Moderator analyses found that quizzes positively affect the effectiveness and attractiveness of blended learning. Concluding, blended learning has potential to improve education, when thoughtfully designed, for example by the inclusion of frequent quizzes.

Introduction

Blended learning is a popular concept. A common aspect in many definitions of blended learning is that it combines online and face-to-face learning (e.g., Graham, 2013). Hence, it is assumed to combine the advantages of both (e.g., Delialioglu, Yildirim, 2008, Feist et al, 2013, Vernadakis et al, 2012). Blended learning is widely used in higher education (Allen et al, 2007, Bonk et al, 2006); and it is also used in K-12 education (e.g., Picciano, Seaman, Shea, & Swan, 2012) and in the corporate world (e.g., Bonk et al, 2006, Kim et al, 2008).

Blended learning is the focus of many research studies (Drysdale, Graham, Spring, & Halverson, 2013). Part of the studies on blended learning are comparisons of blended and face-to-face learning (Drysdale et al, 2013, Halverson et al, 2014). These studies are particularly interesting, as they enable us to draw conclusions about the effects of blended learning on effectiveness, student satisfaction and evaluations of required investments, compared to more traditional education. With a meta-analysis the results from these comparisons can be combined into one statistical synthesis, to draw stronger overall conclusions. This article presents meta-analyses examining the effects of blended learning on effectiveness, student satisfaction and evaluations of required investments. However, before describing our meta-analyses, first the main reasons for implementing blended learning are described in the next section. Subsequently findings of previous meta-analyses are discussed as well as the main methodological differences between those previous meta-analyses and our meta-analyses.

Improving education is one of the main reasons to implement blended learning (e.g., Graham, 2006, Graham, 2009). Internet and computers, mostly used to access the internet in studies on blended learning, offer opportunities to include more, authentic, varied and different instructional materials and innovative learning activities. It is, for example, easier to incorporate frequent quizzes or self-tests in a blended learning environment than in a more traditional learning environment. Moreover, the computer can automatically score the answers of the students on those quizzes or self-tests and provide the students with feedback (e.g., Cole, Robertson, 2006, Jia et al, 2012, Riffell, Sibley, 2005).

Additionally, the introduction of blended learning in a course might lead to rethinking of the instructional design and the investment of additional time and effort in the design (Kaleta, Skibba, & Joosten, 2007). The capabilities and affordances of computers and internet as well as the additional time and effort put into the course design might trigger a shift toward a more active and learner-centered approach (Graham, Robison, 2007, Kaleta et al, 2007; see also e.g., Adileh, 2012, Taradi et al, 2005, Vernadakis et al, 2011, Yang, 2012).

Furthermore, in blended learning learners have, to some extent, control over their learning. They can, for example, work on their own pace, choose to revisit materials and/or choose when to study (e.g., Feist et al, 2013, Salyers, 2007, Yapici, Akbayin, 2012, Yang, 2012). This control might give them the possibility to adapt the learning materials to their individual needs or preferences (e.g., Cole, Robertson, 2006, Kavadella et al, 2012). However, also other capabilities of blended learning might give students the possibility to adapt the learning materials to their individual needs or preferences. Examples of such capabilities are the choice between a variety of learning materials explaining concepts (Chandra & Watters, 2012) or the choice to use e-mail and discussion fora (e.g., Cole & Robertson, 2006). Blended learning is also assumed to better meet the needs of a diverse population of students (e.g., Adileh, 2012, Cole, Robertson, 2006, Edwards et al, 2012, Picciano, 2009, Yapici, Akbayin, 2012). It is, for example, stated to better meet the needs of students with different learning styles (Adileh, 2012, Rudestam, Schoenholtz-Read, 2010, Yapici, Akbayin, 2012). Blended learning has this capability, because it combines different forms of instruction (Picciano, 2009). Additionally, by the incorporation of technology, blended learning might also be better able to meet the needs of the current generation of students, who are surrounded by computers in their daily life (Costello et al, 2004, Lancaster et al, 2012). Additionally, blended learning can include activities which guide, support or force students to space their learning (e.g., Chandra, Watters, 2012, Riffell, Sibley, 2005). This spacing can positively affect the students' learning outcomes (Budé et al, 2011, Dempster, 1988, Smith, Rothkopf, 1984). An example of an activity which might support or force students in spacing their learning activities are frequent quizzes (Mawhinney et al, 1971, Riffell, Sibley, 2005). Frequent after school chat sessions in small groups moderated by the teacher might have a similar effect. They can also force or encourage students to space their learning activities (Chandra & Watters, 2012).

Besides improving education, two other main reasons for the implementation of blended learning are specified by Graham, 2006, Graham, 2009: Increased access to education and resource management. Blended learning might, for example, be more accessible for students who live far from the educational institute (e.g., Forte, Root, 2011, Newhouse et al, 2013, Tamim, 2012). It might also be more accessible for students who have to combine the study with work or care for a family (e.g., Cole, Robertson, 2006, Newhouse et al, 2013, Tamim, 2012). Additionally, when face-to-face contact time or seat time is reduced in blended learning, fewer classrooms might be needed. This might lead to a reduction in costs (e.g., Grasl et al, 2012, Perkins et al, 2012). However, these more practical reasons are according to some authors not reasonable or at least disputable (Jaggars, Bailey, 2010, Means et al, 2013).

Blended learning is expected to improve education, because the advantages of online and face-to-face learning are combined (e.g., Delialioglu, Yildirim, 2008, Taradi et al, 2005, Vernadakis et al, 2012). Possible advantages of the online part could be the opportunities to include more, authentic, varied and different instructional materials and innovative learning activities. In blended learning there is also still some face-to-face contact and interaction between students and between the students and teacher. Blended learning might, however, also combine the disadvantages of both (Graham, 2006, Graham, 2009). Students might, for example, have more difficulties in blended learning courses with managing their time, with self-regulating their learning (e.g., Aycock et al, 2002, Kaleta et al, 2007, Senn, 2008) or with dealing with the complexity of authentic tasks (Van Merriënboer, Kirschner, & Kester, 2003). Furthermore, giving students access to more learning materials and activities might have the consequence that they spend more time on the course. This might lead to improvements in learning outcomes, but it might make the course also more demanding (Kaleta et al., 2007). This could make learning less efficient rather than more efficient (Graham & Robison, 2007).

Consequently, it is not sure that blended learning improves education. Hence, a study examining the effects of blended learning on effectiveness, student satisfaction and evaluations of required investments is needed. Our study investigates with meta-analyses whether blended learning is more, equally or less effective and attractive as more traditional learning. Besides, it will explore whether blended learning is evaluated to require more, equal or less investments than more traditional learning. Learning environments do not directly influence students' learning behavior and outcomes. Their effects are mediated by the students' perceptions (Könings et al, 2005, Könings et al, 2008). Hence, it is important to study the effects of a learning environment on students' satisfaction and evaluations of required investments, in addition to its effects on effectiveness.

Previous meta-analyses examining the effect of blended learning or blended and computer technology-enhanced learning on learning outcomes and student satisfaction are presented in five articles: Bernard, Borokhovski, Schmid, Tamim, and Abrami (2014); Means et al. (2013); Schmid et al. (2014); Sitzmann, Kraiger, Stewart, and Wisher (2006); and Tamim, Bernard, Borokhovski, Abrami, and Schmid (2011). As can be seen in Table 1, these previous meta-analyses mostly found positive effects in the small to medium range for learning outcomes. The only exception is the medium positive effect size for procedural knowledge found by Sitzmann et al. (2006). However, this effect size is based on a small sample (i.e., six studies). Only a few of these previous meta-analyses examined also students' evaluation, satisfaction, or reactions, and their results were less consistent (see Table 1). Schmid et al. (2014) mostly found positive effects in the small to medium range. The exception is one trivial positive effect size. In contrast to these results of Schmid et al, 2014, Sitzmann et al, 2006 found a trivial negative effect.

Our meta-analyses, however, aim to give a better overview of the effects on effectiveness and student satisfaction of current implementation of blended learning than previous meta-analyses. They differ from previous meta-analyses in four ways (see Table 1). First, previous meta-analyses also included older studies. The possibilities of the internet and web-based tools have increased a lot, since the nineties. Many more videos, animations and other types of multimedia are available online yet than in the nineties, for example. Additionally, the availability and the capabilities of the computers and infrastructure in schools have improved much since then. This might have affected the design of blended learning environments, and as a consequence, also the effects of blended learning on effectiveness and student satisfaction. Therefore, we have excluded publications from before 2005. Second, previous meta-analyses often did not include all levels of education and training, which might have limited their findings. Hence, in our meta-analyses all levels of education and training are included. Third, in previous meta-analyses the affective measures also included self-evaluations of learning progress (Schmid et al., 2014) and perceptions of difficulty (Sitzmann et al., 2006). By consequence, measures combined rather distinct concepts that might not clearly be related to each other. Our satisfaction measure focuses more on student satisfaction with the course and/or teacher rather than on a combination of evaluations. Fourth, previous meta-analyses have used different search terms and different definitions of blended learning. This has influenced the studies found and included, and, by consequence, might have affected the effects of blended learning on learning outcomes and satisfaction found. By following the considerations of the authors of articles, we aim to provide a purer measure of the effects of blended learning on learning outcomes and satisfaction as it is defined by the authors of articles.

In our meta-analyses, the considerations of the authors of articles were followed with some restrictions. In order to be included, blended learning had to be a combination of online and face-to-face education in which the e-learning part was more than only an addition to the more traditional education in the control condition. So, authors of articles were not followed when blended learning was not a combination of online and face-to-face learning. When the online part was only an addition in time, resources or activities to the more traditional education, the comparison between education with this addition and without it would not be a sensible comparison. Therefore, studies were excluded when the e-learning part did not replace a part of the learning time, resources or activities from the more traditional learning environment.

Blended learning might require the investment of more time, effort or work, as suggested by Sitzmann et al. (2006) when explaining the slightly negative effect size for learners' reactions in their study (see Table 1). This investment of more time, effort or work could also have led to higher effectiveness of blended learning, and, hence, to the positive effect sizes for learning outcomes found by them (see Table 1). However, they state that more research is needed to examine this suggestion. Therefore this issue will be explored in our meta-analysis on evaluations of required investments.

The objective of this study is to investigate the effectiveness, student satisfaction and student evaluations of required investments of time, work or effort with blended learning compared with more traditional learning. In addition to the average effects, the variability of the effects of blended learning on these different outcome measures is examined. When significant variability is found, it is interesting to consider conditions under which blended learning is more or less successful. For this purpose the following variables were examined in moderator analyses for learning outcomes and satisfaction: Allocation (i.e., way of assigning students to either blended learning or control condition), educational level, subject matter, duration, communication and quizzes.

Allocation is studied because it might affect the results when students are in the condition which they prefer. Post-secondary education differs in a number of ways from primary and secondary education. To explore whether this affects the results, this categorization was included in the moderator analyses. Additionally, it was explored whether blended learning was more effective or attractive for particular types of subject matter. It might also be that students need to get used to the new type of education. Hence, it might be that effects on effectiveness and satisfaction can only be found in courses with a longer duration. On the other hand, it might be that the effects of a new type of education on effectiveness and satisfaction can only be found as long as the education is perceived as new and special by the students. This latter would imply that effects of blended learning on effectiveness and satisfaction would only be found with short courses or trainings. The effects of blended learning on effectiveness and satisfaction probably also depend on how blended learning is implemented. Hence, two features of the implementation are included as moderators. It is examined whether the type of communication in the distance part of the blended learning environment affects the results for effectiveness and satisfaction. A distinction is made between no information on communication, asynchronous communication only, and both asynchronous and synchronous communication. Asynchronous communication provides learners with more flexibility with regard to the moment at which they study. It might also lead to more reflective learner behavior. But, on the other hand, asynchronous communication lacks the spontaneity and more social interactions of synchronous communication (Rudestam & Schoenholtz-Read, 2010). Quizzes, tests or self-assessments are learning activities which can easily be implemented online, especially because the computer can score the answers of the students and provide students with feedback (e.g., Cole, Robertson, 2006, Jia et al, 2012, Riffell, Sibley, 2005). Quizzes have been found to lead to better learning outcomes (Bangert-Drowns et al, 1991, Dirkx et al, 2014, Pashler et al, 2007, Roediger, Karpicke, 2006) and more favorable student opinions (Bangert-Drowns et al., 1991). Hence, the effects of including quizzes on the effectiveness and satisfaction of blended learning are examined.

Section snippets

Literature search

Records with one or more terms from each of three groups of terms (see Table 2) in the title or abstract were searched in PubMed on August 12, 2013 and in ERIC and PsycInfo on August 14, 2013. Additionally, a limitation to peer reviewed sources was set in PsycInfo and ERIC and a limitation to the English language in PsycInfo and PubMed. Subsequently, records of sources published before 2005 were excluded. In total 2463 non-duplicate records of sources published in or after 2005 were found.

The

Results

Sample characteristics, the average weighted Hedges' g and associated 95% confidence intervals as well as the results for the Q-test for heterogeneity and I2 are presented in Table 5.

Discussion

Significantly positive average effect sizes in the small to medium range were found for both objective and subjective effectiveness and a trivial non-significant effect size was found for satisfaction. So, the effectiveness of blended learning was on average better than for more traditional learning. Additionally, on average students are equally satisfied with both types of learning. A large negative effect size was found for investment evaluations. This implies that on average, students

Acknowledgements

This research was funded by Kennisnet, the knowledge center for information and communication technologies and education in The Netherlands.

References (0)

Cited by (128)

View all citing articles on Scopus
1

Currently working at ResearchNed, Nijmegen, The Netherlands.

View full text