Award-winning faculty online teaching practices: Course design, assessment and evaluation, and facilitation

https://doi.org/10.1016/j.iheduc.2019.04.001Get rights and content

Highlights

  • Award-winning online faculty use a systematic design process, backwards design, considered learner needs, and designed learner interaction during the design process.

  • Award-winning online faculty use variety of assessments, using traditional and authentic assessments and used rubrics to assess students.

  • Some of the facilitation strategies award-winning online faculty provide include timely response and feedback, availability and presence, and periodic communication.

Abstract

The purpose of this study was to identify the course design, assessment and evaluation, and facilitation practices from the perspectives of award-winning online faculty. Aligned with this purpose, we developed a conceptual framework focused on online course design, assessment and evaluation, and facilitation; and review relevant literature in light of this framework. We interviewed eight award-winning online faculty members from across the United States. These faculty received online teaching awards from one of the following professional associations: Online Learning Consortium (OLC), Association for Educational Communications and Technology (AECT), or United States Distance Learning Association. Based on the interviews, it was found that online instructors use a systematic design process, backwards design, considered learner needs, and designed learner interaction during the design process. Faculty recommended using a variety of assessments, using traditional and authentic assessments and used rubrics to assess students, course templates and quality assurance process and surveys, learning analytics, and peer reviews for assessment and evaluation. Timely response and feedback, availability and presence, and periodic communication were some facilitation strategies the award-winning instructors used. We discuss these findings and provide suggestions for future research and practice. These findings can add to what is known about effective online teaching best practices, standards, and competencies.

Introduction

As online learning continues to mature and evolve in higher education, both faculty and supporting staff (e.g., instructional designers, developers and technologists) need guidance on how to best design and deliver effective online courses. Recent reports show that online enrollments in institutions of higher education in the United States have continued to grow with approximately 31.6% of all higher education enrollments in 2016, which is a 4.5% increase from 2012 (Seaman, Allen, & Seaman, 2018). Although online environments continue to grow, faculty can be resistant to the adoption of online courses because of a number of perceived barriers, including perceived barriers to student success in online classes, uncertainty about their image as online instructors, technical support needs, and their desire for reasonable workload and manageable class enrollments in online classes (Wingo, Ivankova, & Moss, 2017). Notably, experienced online faculty have noted several supports institutions of higher education can provide to assist faculty with online course design and delivery, ranging from release time to the engagement of support staff (e.g., instructional designers, developers and technologists) to assist in the full life-cycle of an online course (Wingo et al., 2017). In the spirit of gaining insights from experienced online faculty, this paper extends the idea to those faculty members in our community recognized for exemplary practice and excellence on the design and delivery of effective online courses, or more specifically, faculty who have won awards from established professional associations for online teaching.

Award-winning online faculty are an untapped source of useful insights and practices on how to best design and deliver effective online courses. Along these lines, the notion of collecting useful information from “award-winning” faculty is not a new idea, and has been firmly grounded in higher education teaching practices in several published research studies as early as the 1990s (Dunkin & Precians, 1992; Kember & McNaught, 2007; Morris & Usher, 2011), even in the domain of online learning in more recent history (Bailey, 2008). These award-winning faculty perspectives can be translated into actionable and valuable best practices, standards, and competencies for faculty designing and delivering online courses in institutions of higher education. Best practices in online courses are commonly shared in peer-reviewed academic journals, and are read and practiced by both faculty and instructional designers to create effective online courses (Fish & Wickersham, 2009; Keengwe & Kidd, 2010). Award-winning faculty can offer guidance on online course standards (e.g., Quality Matters), which are commonly used to assess quality and guide the production of online courses in higher education (Baldwin, Ching, & Hsu, 2018). Effective online teaching practices from award winning faculty can be used by administrators, instructional designers, and faculty to plan relevant professional development experiences for faculty teaching online (Baran & Correia, 2014; Baran & Correia, 2017) and be used as a yardstick to assess individual faculty “readiness” to teaching in online spaces (Gay, 2016).

The broad dimensions of effective online courses are a topic of interest in both research and practice, and have led to new bodies of scholarship in distance education (Boling, Hough, Krinsky, Saleem, & Stevens, 2012; Margaryan, Bianco, & Littlejohn, 2015), and standards to guide the efforts of faculty in designing and delivering these courses, such as the well-known Quality Matter Rubrics (QualityMatters, 2018). In developing our conception of effective online courses, we reviewed extant literature and chose to organize this body of work into three broad areas: online course design, online course assessment and evaluation, and online course facilitation. Our literature search strategy involved using the words “online learning” with terms like “standards,” “best practices,” “roles and competencies,” and “outcomes” in a wide array of academic databases. This literature was carefully traced to the three domains mentioned.

This conceptual framework is illustrated in Fig. 1 and includes before, during, and after the online course delivery, or more precisely, the full life-cycle of an online course. Notably, online courses are generally not taught and forgotten. The courses are carefully designed before, facilitated with intention during, systematically evaluated after, and revised accordingly to support learning objectives. Our overall position is that effective online courses are a function of design, assessment and evaluation, and facilitation. This is not to say that there are no other elements of effective online courses (e.g., learner support), but rather, we focus on these three areas to organize the literature and operationalize our research study. We put forth the claim that effective online learning must account for the design, assessment and evaluation, and facilitation before, during, and after the student learning experience as illustrated as the intersecting domains in the diagram shown in Fig. 1.

As studies on the effectiveness of online learning have long been documented in the empirical research literature (Means et al., 2013), researchers and practitioners have spent a considerable amount of time trying to formulate the best practices and standards in online course design. However, these course design features are not always clearly connected to evidence-based student learning outcomes (Jaggars & Xu, 2016). Meanwhile, much emphasis in online learning has been placed on the development of standards to both assess quality and guide development efforts in course design, such as the well-known and widely adopted Quality Matters rubrics (Legon, 2015). While many scholars of online learning continue to ask questions about the efficacy of best practices and standards on student learning outcomes, as noted by Legon (2015), there are other impacts that the adoption of the Quality Matters rubrics offer institutions of higher education, such as a focus on continuous process improvement, fostering a dialog within an institution or academic unit on what constitutes quality in online learning, and encouraging consistency of online course design to name a few. That is, the adoption of best practice and standards for online courses helps to create a culture of intentionality with carefully constructed learning outcomes connected to engaging learning materials, systematic procedures and processes used throughout an online course's life-cycle, and an overall focus on quality leading to ongoing evaluation and revision of online courses. Online course design is effectively a context-specific form of instructional design oriented to online learning spaces. Therefore, online course design includes both the features of the online course, and the processes and procedures used to create that online course.

Guidance from the online learning literature has presented countless best practices and standards for online courses on a multitude of dimensions (Crews, Wilkinson, & Neill, 2015). Some of these best practices and standards are inspired by theories and models of online learning (e.g., Community of Inquiry), while others are based on other existing learning theories applied to online settings (e.g., Cognitive Load Theory). As noted by Tallent-Runnels et al. (2006), online learning research still does not have a unified theory of learning, and thus, much of the research in online learning is often fragmented with few online course design principles supported by a multitude of studies. Further, the confusion and use of varying terms to describe similar constructs, such as e-learning, distance learning, or distributed learning make it difficult to pinpoint the empirical research to support the best practices and standards (Moore, Dickson-Deane, & Galyen, 2011). Over the last several decades, instructional design models focusing on online learning experiences have been developed and tested (e.g., Czerkawski & Lyman, 2016; Dunlap, Verma, & Johnson, 2016; Puzziferro & Shelton, 2008), while existing instructional design models have also been adapted and applied to online learning (e.g., Kidney & Puckett, 2003; Shelton & Saltsman, 2011). While no single approach or model seems superior for all online learning situations, it is clear that the use of instructional design processes to guide the design and development of online courses is an effective practice. That is, elements of instructional design practice can be found in the literature supporting the effectiveness of online learning ranging from the formulation of well-written learning outcomes to “chunking” and sequencing an online course to assessing student learning and evaluating course outcomes.

Online course assessment and evaluation serves multiple purposes for students, teachers, and administrators of educational systems and programs. Assessment is an integral part of any form of teaching and learning, and can be found in most instructional design models as a salient feature connecting to the evaluation of the instruction (Branch, 2009). Assessment information from online courses can assist the faculty in making decisions about students' attaining the learning outcomes, diagnosing problems with student learning in specific areas, providing targeted feedback or additional scaffolding to students (Peterson, 2016), and making summative judgements pertaining to grades or retention. However, this assessment information is also useful for the evaluation of the online course in achieving the student learning outcomes, and can inform decisions about revising instructional material or assessments not performing properly, making larger curricular changes to the academic program, selecting appropriate learning experiences and technologies to support those experiences, or even help educational administrators document the outcomes of courses and programs for accreditation purposes (Guerrero-Roldán & Noguera, 2018). Online learning literature is relatively consistent on the importance of online courses clearly communicating the student learning outcomes and how those student learning outcomes will be assessed (Moore & Kearsley, 2011). Best practices and instructional design models also support the use of the assessment information to evaluate the online course to inform revisions (Morrison, Ross, Kalman, & Kemp, 2012).

Online assessment methods supported by empirical literature vary widely based on nature of the student learning outcomes to be achieved, and notably, the belief system of the faculty members designing the online course (Conrad & Openo, 2018). The assessment approaches in online courses can include a combination of techniques all within the same online course experience, such as traditional assessment methods like quizzes and examinations (e.g., multiple choice questions) or those labeled as authentic assessment methods, like ePortfolios, online journals, or group work. Diversity of assessment methods in online courses has been linked to overall learner satisfaction (Sun, Tsai, Finger, Chen, & Yeh, 2008). Much of the contemporary dialog on online assessment methods emphasize authenticity and engagement of the learning task and assessment method by creating “real-world” problems with ill-defined tasks, and that require integration and collaboration among learners (Conrad & Openo, 2018; Herrington, Reeves, & Oliver, 2006). This approach to online assessment requires careful attention to student and group progress with frequent check-points and opportunities for both peer- and instructor- feedback cycles. However, the more “established” approaches such as online discussion forums associated with clearly defined rubrics are still pervasive in practice and are frequently discussed in best practices for online learning literature (Wang & Chen, 2017). Ultimately, online faculty and researchers are rightfully concerned about issues of quality and validity with online assessment methods (Kirkwood & Price, 2015). As online learning technologies continue to evolve, new approaches to assessment are being tested, such as the growth in popularity of learning analytics (Nyland, Davies, Chapman, & Allen, 2017) or stealth assessment (Bhagat & Spector, 2017). Online instructors use various types of evaluation for their online courses including course certifications, peer evaluation and student evaluation. Baldwin et al. (2018) identified several national and statewide evaluation instruments for online courses including quality maters, blackboard exemplary rubric, Open SUNY Course Quality Review Rubric. Online instructors also rely on student evaluation and peer evaluation to enhance the quality of the online courses.

Online course facilitation broadly refers to how, what, when, and why an online faculty member makes decisions and takes actions to help students meet the learning outcomes. Online course facilitation refers to the day-to-day operations of a “live” course. Online course facilitation best practices, standards, and competencies have been developed and tested in varying online courses across studies. As online learning has the potential to isolate learners, the facilitation strategies used by faculty should target student behaviors that mitigate this threat (Gillett-Swan, 2017). Research on effective online course facilitation strategies has shown that some strategies are more effective than others in helping with instructor presence, instructor connectedness, engagement, and learning across students (Banna, Lin, Stewart, & Fialkowski, 2015). Specifically, facilitation strategies like timely instructor responses to email and discussion forums, timely instructor grading and feedback of assignments, and instructor personal response to reflections appear more influential on key outcomes, whereas other strategies like synchronous learning sessions or an interactive syllabus were less influential (Martin, Wang, & Sadaf, 2018). In their study, Martin et al. examined student perception of 12 facilitation strategies in online courses. Research suggests that instructor-led facilitation strategies leads to a stronger sense of community among the students (Epp, Phirangee, & Hewitt, 2017). These findings are consistent with recent research findings illustrating that students valued instructor-to-student interactions most when compared to student-to-student and student-to-content strategies (Martin & Bolliger, 2018). Salmon (2011) uses the term moderating for facilitation. In her book on E-Moderating discusses a five-stage model including access and motivation, online socialization, information exchange, knowledge construction, and development to prepare instructors for online moderation. Berge, 1995, Berge, 2008 proposed the Instructor's Roles Model which shifted focus of an instructor from a subject matter expert to a course facilitator, and categorized facilitation into four categories: Pedagogical, Social, Managerial, and Technical. Researchers have examined specific aspects of facilitation. Hosler and Arend (2012) found that discourse facilitation is key to elicit critical thinking or cognitive presence and noted that course organization and timely specific feedback improved students' participation. Shea, Li, and Pickett (2006) added that instructors' questioning and feedback have positive impact on students' perception of learning and connectedness. It is relatively clear that the choices an online faculty member makes to facilitate learning in the course can have dramatic effects on desired outcomes.

Researchers have focused on examining course design and facilitation practices from faculty who have won awards for their online courses. Bailey and Card (2009) in their study found that award-winning experienced instructors who won the South Dakota e-learning award identified theories of andragogy, constructivism and transformative learning as effective theories for online teaching. They also identified eight pedagogical practices including fostering relationships, engagement, timeliness, communication, organization, technology, flexibility and high expectations. Baran, Correia, and Thompson (2013) in their interviews with six exemplary online teachers found tasks that the online teachers prioritized when transitioning to online teaching. Structuring and organizing the course content, extensive planning and designing, tailoring the course to meet student needs, enhancing student-instructor relationship, providing timely feedback to students, constant communication, establishing teacher presence and formatively evaluating the course were some of the tasks identified as common themes.

The purpose of this study was to identify online course design, assessment and evaluation, and facilitation practices from the perspectives of award-winning online instructors based on interviews. Three research questions were answered:

  • 1.

    How do award-winning online faculty design an online course?

  • 2.

    How do award-winning online faculty assess student outcomes?

  • 3.

    How do award-wining online faculty evaluate their online course?

  • 4.

    How do award-winning online faculty facilitate an online course?

Section snippets

Method

The research project was executed in three distinct phases: 1) reviewed existing research literature on online course design, assessment and evaluation, and facilitation to formulate a conceptual framework for the study; 2) developed as interview guide based on the conceptual framework to interview award-winning online faculty; and 3) interviewed the award-winning faculty teaching online at various institutions of higher education within the United States. In this section, we outline the

Results

The results are presented in sections below according to the research questions: a) online course design, where award-winning online instructors describe their approaches to course design for student learning and engagement, b) online course assessment where they describe their use of assessments and rubrics c) online course evaluation where they describe quality assurance and student or peer evaluations and d) online course facilitation where they highlight feedback, presence, and

Discussion

Online teaching, if done effectively, has the potential to significantly enhance the quality of learning environments and learning outcomes (Perry & Edwards, 2005). Digging deeper into the approaches and strategies used by award-winninng online educators in online course design, assessment and evaluation, and facilitation is necessary to identify what makes online educators not only effective, but exemplary. Within most disciplines, there are expert faculty and novice faculty at any point.

Limitations

Qualitative research methods are influenced by a variety of techniques to ensure and provide evidence of the rigor of the research study. Specifically, this study did not employ some of these techniques, like triangulation (e.g., only data source was the interview), prolonged engagement with the interviewee (e.g., we only conducted a single interview), or member-checking. However, we did manage to use purposeful sampling, audit trails, peer debriefing, and experienced researchers to enhance the

Implications and future research

This study has implications for all faculty who teach online, for instructional designers who assist faculty in designing online courses, and for administrators who provide the necessary support for online faculty and designers. Further, the findings from this research also point to some promising avenues for future online learning research.

Declaration of interest

None

Acknowledgements

This project was funded by a Scholarship of Teaching and Learning Grant from University of North Carolina Charlotte.

References (74)

  • J. Wang et al.

    Instructor presence in instructional video: Effects on visual attention, recall, and perceived learning

    Computers in Human Behavior

    (2017)
  • A. Young et al.

    Assessing the quality of online courses from the students' perspective

    The Internet and Higher Education

    (2006)
  • AECT

    Association for educational communications and technology

  • T. Anderson

    Towards a theory of online learning. Theory and practice of online learning

    (2004)
  • C.J. Bailey

    Best practices for online teaching: Perceptions of South Dakota award-winning online university faculty

    (2008)
  • S. Baldwin et al.

    Online course design in higher education: A review of national and statewide evaluation instruments

    TechTrends

    (2018)
  • J. Banna et al.

    Interaction matters: Strategies to promote engaged learning in an online introductory nutrition course

    Journal of online learning and teaching/MERLOT

    (2015)
  • E. Baran et al.

    A professional development framework for online teaching

    TechTrends

    (2014)
  • E. Baran et al.

    What motivates exemplary online teachers? A multiple-case study

  • E. Baran et al.

    Tracing successful online teaching in higher education: Voices of exemplary online teachers

    Teachers College Record

    (2013)
  • Z. Berge

    Changing instructor's roles in virtual worlds

    The Quarterly Review of Distance Education

    (2008)
  • Z.L. Berge

    Facilitating computer conferencing: Recommendations from the field

    Educational Technology

    (1995)
  • K.K. Bhagat et al.

    Formative assessment in complex problem-solving domains: The emerging role of assessment technologies

    Journal of Educational Technology & Society

    (2017)
  • R.M. Branch

    Instructional design: The ADDIE approach

    (2009)
  • D. Conrad et al.

    Assessment strategies for online learning: Engagement and authenticity

    (2018)
  • T.B. Crews et al.

    Online quality course design vs. quality teaching: Aligning quality matters standards to principles for good teaching

    The Journal of Research in Business Education

    (2015)
  • T.B. Crews et al.

    Principles for good practice in undergraduate education: Effective online course design to assist students' success

    Journal of Online Learning and Teaching

    (2015)
  • B.C. Czerkawski et al.

    An instructional design framework for fostering student engagement in online learning environments

    TechTrends

    (2016)
  • M.J. Dunkin et al.

    Award-winning university teachers' concepts of teaching

    Higher Education

    (1992)
  • J.C. Dunlap et al.

    Presence+ Experience: A framework for the purposeful design of presence in online courses

    TechTrends

    (2016)
  • C.D. Epp et al.

    Student actions and community in online courses: The roles played by course length and facilitation method

    Online Learning Journal

    (2017)
  • L. Fayer

    A multi-case study of student perceptions of online course design elements and success

    International Journal for the Scholarship of Teaching & Learning

    (2014)
  • W.W. Fish et al.

    Best practices for online instructors: Reminders

    The Quarterly Review of Distance Education

    (2009)
  • G.H. Gay

    An assessment of online instructor e-learning readiness before, during, and after course delivery

    Journal of Computing in Higher Education

    (2016)
  • J. Gaytan et al.

    Effective online instructional and assessment strategies

    The American Journal of Distance Education

    (2007)
  • J. Gillett-Swan

    The challenges of online learning: Supporting and engaging the isolated learner

    Journal of Learning Design

    (2017)
  • J. Herrington et al.

    Authentic tasks online: A synergy among learner, task, and technology

    Distance Education

    (2006)
  • Cited by (0)

    View full text