Namrata Srivastava
Eduardo Velloso
Jason M. Lodge
ABSTRACT
With the increased reach and impact of video lectures, it is crucial to understand how they are experienced. Whereas previous studies typically present questionnaires at the end of the lecture, they fail to capture students' experience in enough granularity. In this paper we propose recording the lecture difficulty in real-time with a physical slider, enabling continuous and fine-grained analysis of the learning experience. We evaluated our approach in a study with 100 participants viewing two variants of two short lectures. We demonstrate that our approach helps us paint a more complete picture of the learning experience. Our analysis has design implications for instructors, providing them with a method that helps them compare their expectations with students' beliefs about the lectures and to better understand the specific effects of different instructional design decisions.
Supplemental Material
Available for Download
zip
paper586.zip (315.9 KB)
The supplementary files include the following study-design documents 1. Demographics and Prior Knowledge Questionnaire 2. Pre-test and Post-test questionnaires 3. Lecture Feedback forms All the files are saved together in a single PDF file, reflecting the original study design used in the paper.
- Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web. ACM, 687--698. Google Scholar
Digital Library
- Amaël Arguel, Jason Lodge, Mariya Pachman, and Paula de Barba. 2016. Confidence drives exploration strategies in interactive simulations. ASCILITE.Google Scholar
- S Adams Becker, Michele Cummins, Adam Davis, Alex Freeman, C Glesinger Hall, and Vanish Ananthanarayanan. 2017. NMC horizon report: 2017 higher education edition. Technical Report. The New Media Consortium.Google Scholar
- Alberto Betella and Paul FMJ Verschure. 2016. The affective slider: A digital self-assessment scale for the measurement of human emotions. PloS one 11, 2 (2016), e0148037.Google ScholarCross Ref
- Lori Breslow, David E Pritchard, Jennifer DeBoer, Glenda S Stump, Andrew D Ho, and Daniel T Seaton. 2013. Studying learning in the worldwide classroom research into edX's first MOOC. Research & Practice in Assessment 8 (2013), 13--25.Google Scholar
- Roland Brünken, Jan L Plass, and Detlev Leutner. 2004. Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects. Instructional Science 32, 1--2 (2004), 115--132.Google ScholarCross Ref
- Kelly Caine. 2016. Local Standards for Sample Size at CHI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 981--992. Google ScholarDigital Library
- Hee Jun Choi and Scott D Johnson. 2005. The effect of context-based video instruction on learning and motivation in online courses. The American Journal of Distance Education 19, 4 (2005), 215--227.Google ScholarCross Ref
- Konstantinos Chorianopoulos. 2013. Collective intelligence within web video. Human-centric Computing and Information Sciences 3, 1 (15 Jun 2013), 10.Google Scholar
- Andrew Cross, Mydhili Bayyapunedi, Edward Cutrell, Anant Agarwal, and William Thies. 2013. TypeRighting: combining the benefits of handwriting and typeface in online educational videos. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 793--796. Google ScholarDigital Library
- Michael N. Giannakos, Konstantinos Chorianopoulos, and Nikos Chrisochoides. 2014. Collecting and making sense of video learning analytics. In 2014 IEEE Frontiers in Education Conference (FIE) Proceedings. 1--7.Google ScholarCross Ref
- Michail N Giannakos, Konstantinos Chorianopoulos, and Nikos Chrisochoides. 2014. Collecting and making sense of video learning analytics. In Frontiers in Education Conference (FIE), 2014 IEEE. IEEE, 1--7.Google ScholarCross Ref
- Philip J Guo, Juho Kim, and Rob Rubin. 2014. How video production affects student engagement: an empirical study of MOOC videos. In Proceedings of the first ACM conference on Learning@ scale conference. ACM, 41--50. Google ScholarDigital Library
- Päivi Karppinen. 2005. Meaningful learning with digital and online videos: Theoretical perspectives. AACE Journal 13, 3 (2005), 233--250.Google Scholar
- Juho Kim, Philip J Guo, Daniel T Seaton, Piotr Mitros, Krzysztof Z Gajos, and Robert C Miller. 2014. Understanding in-video dropouts and interaction peaks inonline lecture videos. In Proceedings of the first ACM conference on Learning@ scale conference. ACM, 31--40. Google ScholarDigital Library
- René F Kizilcec, Jeremy N Bailenson, and Charles J Gomez. 2015. The instructor's face in video instruction: Evidence from two large-scale field studies. Journal of Educational Psychology 107, 3 (2015), 724.Google ScholarCross Ref
- René F Kizilcec, Kathryn Papadopoulos, and Lalida Sritanyaratana. 2014. Showing face in video instruction: effects on information retention, visual attention, and affect. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 2095--2102. Google ScholarDigital Library
- KH Vincent Lau, Pue Farooque, Gary Leydon, Michael L Schwartz, R Mark Sadler, and Jeremy J Moeller. 2018. Using learning analytics to evaluate a video-based lecture series. Medical teacher 40, 1 (2018), 91--98.Google Scholar
- Gaël Laurans, Pieter MA Desmet, and Paul Hekkert. 2009. The emotion slider: A self-report device for the continuous measurement of emotion. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops (ACII 2009). Ieee, 1--6.Google ScholarCross Ref
- Nan Li, Lukasz Kidzinski, Patrick Jermann, and Pierre Dillenbourg. 2015. How Do In-video Interactions Reflect Perceived Video Difficulty? Proceedings Papers (2015), 112.Google Scholar
- Shih-Yin Lin, John M Aiken, Daniel T Seaton, Scott S Douglas, Edwin F Greco, Brian D Thoms, and Michael F Schatz. 2016. Exploring University Students' Engagement with Online Video Lectures in a Blended Introductory Mechanics Course. arXiv preprint arXiv:1603.03348 (2016).Google Scholar
- Tharindu Rekha Liyanagunawardena, Andrew Alexandar Adams, and Shirley Ann Williams. 2013. MOOCs: A systematic study of the published literature 2008--2012. The International Review of Research in Open and Distributed Learning 14, 3 (2013), 202--227.Google ScholarCross Ref
- Jason Lodge, Jared Cooney Horvath, Alex Horton, Gregor Kennedy, Sven Venema, and Shane Dawson. 2017. Designing videos for learning: Separating the good from the bad and the ugly. (01 2017).Google Scholar
- Jason Lodge and Gregor Kennedy. 2015. Prior knowledge, confidence and understanding in interactive tutorials and simulations. ASCILITE.Google Scholar
- Jason M Lodge, Gregor Kennedy, Lori Lockyer, Amael Arguel, and Mariya Pachman. 2018. Understanding difficulties and resulting confusion in learning: An integrative review. In Frontiers in Education, Vol. 3. Frontiers, 49.Google ScholarCross Ref
- Richard E Mayer and Roxana Moreno. 2003. Nine ways to reduce cognitive load in multimedia learning. Educational psychologist 38, 1 (2003), 43--52.Google Scholar
- Barbara Mitra, Jenny Lewin-Jones, Heather Barrett, and Stella Williamson. 2010. The use of video to enable deep learning. Research in Post-Compulsory Education 15, 4 (2010), 405--414.Google ScholarCross Ref
- Dan R Olsen and Brandon Moon. 2011. Video summarization based on user interaction. In Proceedings of the 9th European Conference on Interactive TV and Video. ACM, 115--122. Google ScholarDigital Library
- Fred Paas, Juhani E Tuovinen, Huib Tabbers, and Pascal WM Van Gerven. 2003. Cognitive load measurement as a means to advance cognitive load theory. Educational psychologist 38, 1 (2003), 63--71.Google Scholar
- Fred GWC Paas and Jeroen JG Van Merriënboer. 1993. The efficiency of instructional conditions: An approach to combine mental effort and performance measures. Human factors 35, 4 (1993), 737--743.Google Scholar
- Evan F Risko, Tom Foulsham, Shane Dawson, and Alan Kingstone. 2013. The collaborative lecture annotation system (CLAS): A new TOOL for distributed learning. IEEE Transactions on Learning Technologies 6, 1 (2013), 4--13. Google ScholarDigital Library
- Jennifer Sabourin, Bradford Mott, and James C Lester. 2011. Modeling learner affect with theoretically grounded dynamic Bayesian networks. In International Conference on Affective Computing and Intelligent Interaction. Springer, 286--295. Google ScholarDigital Library
- Ryan Shaw and Marc Davis. 2005. Toward emergent representations for video. In Proceedings of the 13th annual ACM international conference on Multimedia. ACM, 431--434. Google ScholarDigital Library
- Hyungyu Shin, Eun-Young Ko, Joseph Jay Williams, and Juho Kim. 2018. Understanding the Effect of In-Video Prompting on Learners and Instructors. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 319. Google ScholarDigital Library
- Nicholas C Soderstrom, Carole L Yue, and Elizabeth Ligon Bjork. 2016. 11 Metamemory and Education. The Oxford Handbook of Metamemory (2016), 197.Google Scholar
- Jason M Tangen, Merryn D Constable, Eric Durrant, Chris Teeter, Brett R Beston, and Joseph A Kim. 2011. The role of interest and images in slideware presentations. Computers & Education 56, 3 (2011), 865--872. Google ScholarDigital Library
- Peter Tiernan. 2015. An inquiry into the current and future uses of digital video in University teaching. Education and Information Technologies 20, 1 (2015), 75--90. Google ScholarDigital Library
- Frans Van der Sluis, Jasper Ginn, and Tim Van der Zee. 2016. Explaining Student Behavior at Scale: The influence of video complexity on student dwelling time. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale. ACM, 51--60. Google ScholarDigital Library
Index Terms
- Continuous Evaluation of Video Lectures from Real-Time Difficulty Self-Report
Recommendations
The Effects of Flipping an English for Academic Purposes Course
This article addresses the lack of research into the effects of flipping tertiary level English for Academic Purposes courses. An experimental method was used to compare the outcomes and satisfaction of students n=29 enrolled in a flipped and a ...
Pedagogy Experiments with Recorded Video Lectures
T4E '14: Proceedings of the 2014 IEEE Sixth International Conference on Technology for EducationThis paper explores the effectiveness of various video based learning methods. In recent years the flipped classroom has emerged as an innovative teaching-learning process. In a flipped classroom, study materials are made available to students for self ...
Usability design for video lectures
EuroITV '13: Proceedings of the 11th European Conference on Interactive TV and VideoThere is a growing number and variety of educational video lectures online, but there is limited understanding of their effectiveness in terms of learning and usability. Although there is significant research literature within the individual domains of ...
Comments