Jacob Whitehill
Margo Seltzer
ABSTRACT
We investigated the feasibility of crowdsourcing full- fledged tutorial videos from ordinary people on the Web on how to solve math problems related to logarithms. This kind of approach (a form of learnersourcing [9, 11]) to efficiently collecting tutorial videos and other learning resources could be useful for realizing personalized learning-at-scale, whereby students receive specific learning resources -- drawn from a large and diverse set -- that are tailored to their individual and time-varying needs. Results of our study, in which we collected 399 videos from 66 unique "teachers" on Mechanical Turk, suggest that (1) approximately 100 videos -- over 80% of which are mathematically fully correct -- can be crowdsourced per week for $5/video; (2) the average learning gains (posttest minus pretest score) associated with watching the videos was stat. sig. higher than for a control video (0.105 versus 0.045); and (3) the average learning gains (0.1416) from watching the best tested crowdsourced videos was comparable to the learning gains (0.1506) from watching a popular Khan Academy video on logarithms.
- T. Aleahmad, V. Aleven, and R. Kraut. Creating a corpus of targeted learning resources with a web-based open authoring tool. IEEE Transactions on Learning Technologies, 2(1):3--9, 2009. Google Scholar
Digital Library
- J. R. Anderson, C. F. Boyle, and B. J. Reiser. Intelligent tutoring systems. Science, 228(4698):456--462, 1985. Google Scholar
- J. L. Booth, K. E. Lange, K. R. Koedinger, and K. J. Newton. Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples. Learning and Instruction, 25:24--34, 2013. Google ScholarCross Ref
- P. Brusilovsky and C. Peylo. Adaptive and intelligent web-based educational systems. International Journal of Artificial Intelligence in Education (IJAIED), 13:159--172, 2003.Google Scholar
- C.-M. Chen. Intelligent web-based learning system with personalized learning path guidance. Computers & Education, 51(2):787--814, 2008. Google ScholarDigital Library
- Y. Chen, T. Mandel, Y.-E. Liu, and Z. Popovic. Crowdsourcing accurate and creative word problems and hints. 2016.Google Scholar
- N. T. Heffernan, K. S. Ostrow, K. Kelly, D. Selent, E. G. Inwegen, X. Xiong, and J. J. Williams. The future of adaptive learning: Does the crowd hold the key? International Journal of Artificial Intelligence in Education, pages 1--30, 2016. Google ScholarCross Ref
- G.-J. Hwang, F.-R. Kuo, P.-Y. Yin, and K.-H. Chuang. A heuristic algorithm for planning personalized learning paths for context-aware ubiquitous learning. Computers & Education, 54(2):404--415, 2010. Google ScholarDigital Library
- J. Kim, P. J. Guo, D. T. Seaton, P. Mitros, K. Z. Gajos, and R. C. Miller. Understanding in-video dropouts and interaction peaks inonline lecture videos. In Proceedings of Learning at Scale, pages 31--40. ACM, 2014. Google ScholarDigital Library
- J. Kim, R. C. Miller, and K. Z. Gajos. Learnersourcing subgoal labeling to support learning from how-to videos. In CHI'13 Extended Abstracts on Human Factors in Computing Systems, pages 685--690. ACM, 2013. Google ScholarDigital Library
- J. Kim, P. T. Nguyen, S. Weir, P. J. Guo, R. C. Miller, and K. Z. Gajos. Crowdsourcing step-by-step information extraction to enhance existing how-to videos. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems, pages 4017--4026. ACM, 2014. Google ScholarDigital Library
- O. Polozov, E. O'Rourke, A. M. Smith, L. Zettlemoyer, S. Gulwani, and Z. Popovic. Personalized mathematical word problem generation. In IJCAI, pages 381--388, 2015.Google Scholar
- L. P. Salamanca, A. R. Carini, M. A. Lee, K. Dykstra, J. Whitehill, D. Angus, J. Wiles, J. S. Reilly, and M. S. Bartlett. Characterizing the temporal dynamics of student-teacher discourse. In International conference on Development and Learning, pages 1--2, 2012. Google ScholarCross Ref
- S. Türkay. The effects of whiteboard animations on retention and subjective experiences when learning advanced physics topics. Computers & Education, 98:102--114, 2016. Google ScholarDigital Library
- K. VanLehn, C. Lynch, K. Schulze, J. A. Shapiro, R. Shelby, L. Taylor, D. Treacy, A. Weinstein, and M. Wintersgill. The andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15(3):147--204, 2005.Google ScholarDigital Library
- J. J. Williams, J. Kim, A. Rafferty, S. Maldonado, W. Lasecki, K. Gajos, and N. Heffernan. Axis: Generating explanations at scale with learnersourcing and machine learning. In ACM Learning at Scale, 2016.Google Scholar
- B. Woolf, W. Burleson, I. Arroyo, T. Dragon, D. Cooper, and R. Picard. Affect-aware tutors: recognising and responding to student affect. International Journal of Learning Technology, 4(3--4):129--164, 2009.Google ScholarDigital Library
Index Terms
- A Crowdsourcing Approach to Collecting Tutorial Videos -- Toward Personalized Learning-at-Scale
Recommendations
Blended Learning in Personalized Assistive Learning Environments
In this paper, the special needs/requirements of disabled students and cost-benefits for applying blended learning in Personalized Educational Learning Environments PELE in Higher Education are studied. The authors describe how blended learning can form ...
Architecture of an Adaptive Personalized Learning Environment (APLE) for Content Recommendation
ICDTE '18: Proceedings of the 2nd International Conference on Digital Technology in EducationWith the development of sophisticated learning environments and learner-centric didactic approaches, personalized learning is in high demand. Personalization in learning environments occurs when such systems fit the learner profiles, which help in ...
Where's the Learning in Education Crowdsourcing?
L@S '20: Proceedings of the Seventh ACM Conference on Learning @ ScaleCrowdsourcing has shown promise in education domains. For example, researchers have leveraged the wisdom of the crowd to process grading in MOOCs and develop learning resources. An untapped domain is harnessing the crowd to systematically process ...
Comments