Xavier Ochoa
Katherine Chiluiza
Gonzalo Méndez
Gonzalo Luzardo
ABSTRACT
Multimodal Learning Analytics is a field that studies how to process learning data from dissimilar sources in order to automatically find useful information to give feedback to the learning process. This work processes video, audio and pen strokes information included in the Math Data Corpus, a set of multimodal resources provided to the participants of the Second International Workshop on Multimodal Learning Analytics. The result of this processing is a set of simple features that could discriminate between experts and non-experts in groups of students solving mathematical problems. The main finding is that several of those simple features, namely the percentage of time that the students use the calculator, the speed at which the student writes or draws and the percentage of time that the student mentions numbers or mathematical terms, are good discriminators be- tween experts and non-experts students. Precision levels of 63% are obtained for individual problems and up to 80% when full sessions (aggregation of 16 problems) are analyzed. While the results are specific for the recorded settings, the methodology used to obtain and analyze the features could be used to create discriminations models for other contexts.
- P. Blikstein, "Multimodal learning analytics," in Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 102--106, ACM, 2013. Google Scholar
Digital Library
- M. Worsley, "Multimodal learning analytics: enabling the future of learning through multimodal data analysis and interfaces," in Proceedings of the 14th ACM international conference on Multimodal interaction, pp. 353--356, ACM, 2012. Google ScholarDigital Library
- S. Scherer, N. Weibel, L.-P. Morency, and S. Oviatt, "Multimodal prediction of expertise and leadership in learning groups," in Proceedings of the 1st International Workshop on Multimodal Learning Analytics, MLA '12, p. 1:8, ACM, 2012. Google ScholarDigital Library
- A. Yilmaz, O. Javed, and M. Shah, "Object tracking: A survey," Acm Computing Surveys (CSUR), vol. 38, no. 4, p. 13, 2006. Google ScholarDigital Library
- A. J. Lipton, H. Fujiyoshi, and R. S. Patil, "Moving target classification and tracking from real-time video," in Applications of Computer Vision, 1998. WACV'98. Proceedings., Fourth IEEE Workshop on, pp. 8--14, IEEE, 1998. Google ScholarDigital Library
- T. Hammond and R. Davis, "Tahuti: A geometrical sketch recognition system for uml class diagrams," in ACM SIGGRAPH 2006 Courses, p. 25, ACM, 2006. Google ScholarDigital Library
- S. Oviatt, A. Cohen, and N. Weibel, "Multimodal learning analytics: Description of math data corpus for icmi grand challenge workshop," Second International Workshop on Multimodal Learning Analytics, December 2013. Google ScholarDigital Library
- H. Bay, T. Tuytelaars, and L. V. Gool, "SURF: speeded up robust features," in Computer Vision - ECCV 2006, no. 3951 in Lecture Notes in Computer Science, pp. 404--417, Springer Berlin Heidelberg, Jan. 2006. Google ScholarDigital Library
- G. Bradski, "The OpenCV Library," Dr. Dobb's Journal of Software Tools, 2000.Google Scholar
- M. Muja and D. G. Lowe, "Fast approximate nearest neighbors with automatic algorithm configuration," in In VISAPP International Conference on Computer Vision Theory and Applications, pp. 331--340, 2009.Google Scholar
- K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, "Real-time foreground-background segmentation using codebook model," Real-Time Imaging, vol. 11, pp. 172--185, June 2005. Google ScholarDigital Library
- G. R. Bradski and A. Kaehler, Learning OpenCV: Computer Vision in CGoogle Scholar
- with the OpenCV Library. Oreilly & Associates Incorporated, Mar. 2013.Google Scholar
- Z. Kalal, K. Mikolajczyk, and J. Matas, "Tracking-learning-detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 7, pp. 1409--1422, 2012. Google ScholarDigital Library
- Z. Kalal, J. Matas, and K. Mikolajczyk, "P-N Learning: Bootstrapping Binary Classifiers by Structural Constraints," Conference on Computer Vision and Pattern Recognition, 2010.Google Scholar
- C. Chelba, D. Bikel, M. Shugrina, P. Nguyen, and S. Kumar, "Large scale language modeling in automatic speech recognition," tech. rep., Google, 2012.Google Scholar
- J. Schalkwyk, D. Beeferman, F. Beaufays, B. Byrne, C. Chelba, M. Cohen, M. Kamvar, and B. Strope, ""your word is my command": Google search by voice: A case study," in Advances in Speech Recognition, pp. 61--90, Springer, 2010.Google Scholar
- J. B. Lovins, Development of a Stemming Algorithm. M.I.T. Information Processing Group, Electronic Systems Laboratory, 1968.Google Scholar
- B. Paulson and T. Hammond, "Paleosketch: accurate primitive sketch recognition and beautification," in Proceedings of the 13th international conference on Intelligent user interfaces, IUI '08, pp. 1--10, ACM, 2008. Google ScholarDigital Library
- L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, "Classification and regression trees. wadsworth & brooks," Monterey, CA, 1984.Google Scholar
- T. Therneau, B. Atkinson, and B. Ripley, rpart: Recursive Partitioning, 2013. R package version 4.1--1.Google Scholar
- R Core Team, R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2013.Google Scholar
- D. Sanchez-Cortes, O. Aran, M. S. Mast, and D. Gatica-Perez, "A nonverbal behavior approach to identify emergent leaders in small groups," Multimedia, IEEE Transactions on, vol. 14, no. 3, pp. 816--832, 2012.Google ScholarDigital Library
- M. J. Cole, J. Gwizdka, C. Liu, N. J. Belkin, and X. Zhang, "Inferring user knowledge level from eye movement patterns," Information Processing & Management, 2012. Google ScholarDigital Library
- M. Worsley and P. Blikstein, "Towards the development of multimodal action based assessment," in Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 94--101, ACM, 2013. Google ScholarDigital Library
- P. J. Kellman and P. Garrigan, "Perceptual learning and human expertise," Physics of life reviews, vol. 6, no. 2, pp. 53--84, 2009.Google ScholarCross Ref
- M. T. Chi, P. J. Feltovich, and R. Glaser, "Categorization and representation of physics problems by experts and novices," Cognitive science, vol. 5, no. 2, pp. 121--152, 1981.Google ScholarCross Ref
- L. Jiang, J. Elen, and G. Clarebout, "The relationships between learner variables, tool-usage behaviour and performance," Computers in Human Behavior, vol. 25, no. 2, pp. 501--509, 2009. Google ScholarDigital Library
Index Terms
- Expertise estimation based on simple multimodal features
Recommendations
Written and multimodal representations as predictors of expertise and problem-solving success in mathematics
ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interactionOne aim of multimodal learning analytics is to analyze rich natural communication modalities to identify domain expertise and learning rapidly and reliably. In this research, written and multimodal representations are analyzed from the Math Data Corpus, ...
Problem solving, domain expertise and learning: ground-truth performance results for math data corpus
ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interactionProblem solving, domain expertise, and learning are analyzed for the Math Data Corpus, which involves multimodal data on collaborating student groups as they solve math problems together across sessions. Compared with non-expert students, domain experts ...
Combining empirical and machine learning techniques to predict math expertise using pen signal features
MLA '14: Proceedings of the 2014 ACM workshop on Multimodal Learning Analytics Workshop and Grand ChallengeMultimodal learning analytics aims to automatically analyze students' natural communication patterns based on speech, writing, and other modalities during learning activities. This research used the Math Data Corpus, which contains time-synchronized ...
Comments