Abstract
Review strategies after learning new knowledge are essential for students to consolidate the key points, understand the subject content, analyze aspects of the learning topics, and summarize the knowledge content of learning while mastering new knowledge. However, educators have found that students generally have difficulties seeking help when they encounter learning problems. This could significantly affect their after-class review performances. To cope with this problem, an after-class review approach with an AI (Artificial Intelligence)-based chatbot is proposed in this study to provide students with immediate and quality feedback during the learning process. Moreover, a quasi-experiment was conducted to explore students’ learning motivation, attitude, and academic performance when using the AI-based chatbot. Participants were two classes of students from a university in Taiwan. One class with 18 students was the experimental group and the other with 20 students was the control group. The experimental group used the AI-based chatbot in the after-class review, while the control group used the conventional after-class review approach. Research results showed that the application of AI-based chatbots in the review process of public health courses could improve students’ academic performance, self-efficacy, learning attitude, and motivation. In other words, chatbots could help students become more active in the learning process. It is noted that after students asked questions, providing them with sufficient feedback during the review process could make them feel recognized and help to establish a relaxing and friendly interaction, thereby improving their academic performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abd-Alrazaq, A. A., Rababeh, A., Alajlani, M., Bewick, B. M., & Househ, M. (2020). Effectiveness and safety of using Chatbots to improve mental health: Systematic review and meta-analysis. Journal of Medical Internet Research, 22(7), e16021. https://doi.org/10.2196/16021
Barrouillet, P., Bernardin, S., & Camos, V. (2004). Time constraints and resource sharing in adults’ working memory spans. Journal of Experimental Psychology: General, 133(1), 83–100. https://doi.org/10.1037/0096-3445.133.1.83
Beaudry, J., Consigli, A., Clark, C., & Robinson, K. J. (2019). Getting ready for adult healthcare: Designing a chatbot to coach adolescents with special health needs through the transitions of care. Journal of Pediatric Nursing, 49, 85–91. https://doi.org/10.1016/j.pedn.2019.09.004
Belcadhi, L. C. (2016). Personalized feedback for self assessment in lifelong learning environments based on semantic web. Computers in Human Behavior, 55, 562–570. https://doi.org/10.1016/j.chb.2015.07.042
Bhargava, M., Varshney, R., & Anita, R. (2020). Emotionally intelligent chatbot for mental healthcare and suicide prevention. International Journal of Advanced Science and Technology, 29(6), 2597–2605.
Bii, P., Too, J., & Mukwa, C. (2018). Teacher attitude towards use of chatbots in routine teaching. Universal Journal of Educational Research, 6(7), 1586–1597. https://doi.org/10.13189/ujer.2018.060719
Binder, T., Sandmann, A., Sures, B., Friege, G., Theyssen, H., & Schmiemann, P. (2019). Assessing prior knowledge types as predictors of academic achievement in the introductory phase of biology and physics study programmes using logistic regression. International Journal of STEM Education, 6(1), 1–14. https://doi.org/10.1186/s40594-019-0189-9
Blank, J. J., & McElmurry, B. J. (1988). A paradigm for baccalaureate public health nursing education. Public Health Nursing, 5(3), 153–159.
Bruner, J. S. (1979). On knowing: Essays for the left hand. Harvard University Press.
Butler, A. C., & Roediger, H. L., III. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19(4–5), 514–527. https://doi.org/10.1080/09541440701326097
Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219–233. https://doi.org/10.1080/03075070600572132
Casillo, M., Clarizia, F., D’Aniello, G., De Santo, M., Lombardi, M., & Santaniello, D. (2020). CHAT-Bot: A cultural heritage aware teller-bot for supporting touristic experiences. Pattern Recognition Letters, 131, 234–243. https://doi.org/10.1016/j.patrec.2020.01.003
Chen, H. L., Vicki Widarso, G., & Sutrisno, H. (2020). A ChatBot for Learning Chinese: Learning Achievement and Technology Acceptance. Journal of Educational Computing Research, 58(6), 1161–1189. https://doi.org/10.1177/0735633120929622
Chen, Y., & Hoshower, L. B. (2003). Student evaluation of teaching effectiveness: An assessment of student perception and motivation. Assessment & Evaluation in Higher Education, 28(1), 71–88. https://doi.org/10.1080/02602930301683
Chen, Y. T., & Kuo, C. L. (2022). Applying the smartphone-based chatbot in clinical nursing education. Nurse Educator, 47(2), E29. https://doi.org/10.1097/NNE.0000000000001131
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Dirkx, K. J., Thoma, G.-B., Kester, L., & Kirschner, P. A. (2015). Answering questions after initial study guides attention during restudy. Instructional Science, 43(1), 59–71. https://doi.org/10.1007/s11251-014-9330-9
Dunlosky, J. (2005). Why does rereading improve metacomprehension accuracy? Evaluating the levels-of-disruption hypothesis for the rereading effect. Discourse Processes, 40(1), 37–55. https://doi.org/10.1207/s15326950dp4001_2
Epstein, M. L., Lazarus, A. D., Calvano, T. B., Matthews, K. A., Hendel, R. A., Epstein, B. B., & Brosvic, G. M. (2002). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record, 52(2), 187–201. https://doi.org/10.1007/BF03395423
Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6(4), 50–72. https://doi.org/10.1111/j.1937-8327.1993.tb00605.x
Ferguson, P. (2011). Student perceptions of quality feedback in teacher education. Assessment & Evaluation in Higher Education, 36(1), 51–62. https://doi.org/10.1080/02602930903197883
Fisher, C. W., Berliner, D. C., Filby, N. N., Marliave, R., Cahen, L. S., & Dishaw, M. M. (1981). Teaching behaviors, academic learning time, and student achievement: An overview. The Journal of Classroom Interaction, 17(1), 2–15.
Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human Behavior, 75, 461–468. https://doi.org/10.1016/j.chb.2017.05.045
Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023
Fryer, L. K., Thompson, A., Nakao, K., Howarth, M., & Gallacher, A. (2020). Supporting self-efficacy beliefs and interest as educational inputs and outcomes: Framing AI and Human partnered task experiences. Learning and Individual Differences, 80, 101850. https://doi.org/10.1016/j.lindif.2020.101850
Fryer, L., & Carpenter, R. (2006). Bots as language learning tools. Language Learning & Technology, 10(3), 8–14.
Fu, S., Gu, H., & Yang, B. (2020). The affordances of AI-enabled automatic scoring applications on learners’ continuous learning intention: An empirical study in China. British Journal of Educational Technology, 51(5), 1674–1692. https://doi.org/10.1111/bjet.12995
Gbenga, O., Okedigba, T., & Oluwatobi, H. (2020). An improved rapid response model for university admission enquiry system using chatbot. International Journal of Computer, 38(1), 123–131.
Go, E., & Sundar, S. S. (2019). Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior, 97, 304–316. https://doi.org/10.1016/j.chb.2019.01.020
Hailikari, T., Katajavuori, N., & Lindblom-Ylanne, S. (2008). The relevance of prior knowledge in learning and instructional design. American Journal of Pharmaceutical Education. https://doi.org/10.5688/aj7205113
Heflin, H., Shewmaker, J., & Nguyen, J. (2017). Impact of mobile technology on student attitudes, engagement, and learning. Computers & Education, 107, 91–99. https://doi.org/10.1016/j.compedu.2017.01.006
Higgins, R., Hartley, P., & Skelton, A. (2002). The conscientious consumer: Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53–64. https://doi.org/10.1080/03075070120099368
Hsieh, W. M., & Tsai, C. C. (2017). Taiwanese high school teachers’ conceptions of mobile learning. Computers and Education, 115, 82–95. https://doi.org/10.1016/j.compedu.2017.07.013
Hwang, G. J., Yang, T. C., Tsai, C. C., & Yang, S. J. (2009). A context-aware ubiquitous learning environment for conducting complex science experiments. Computers & Education, 53(2), 402–413. https://doi.org/10.1016/j.compedu.2009.02.016
Hwang, G. J., Yang, L. H., & Wang, S. Y. (2013). A concept map-embedded educational computer game for improving students' learning performance in natural science courses. Computers & Education, 69, 121–130. https://doi.org/10.1016/j.compedu.2013.07.008
Hwang, G. J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence, 1, 100001. https://doi.org/10.1016/j.caeai.2020.100001
Kabudi, T., Pappas, I., & Olsen, D. H. (2021). AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and Education: Artificial Intelligence, 2, 100017. https://doi.org/10.1016/j.caeai.2021.100017
Lawson, A. E. (1983). Predicting science achievement: The role of developmental level, disembedding ability, mental capacity, prior knowledge, and beliefs. Journal of Research in Science Teaching, 20(2), 117–129. https://doi.org/10.1002/tea.3660200204
Lin, M. P. C., & Chang, D. (2020). Enhancing Post-secondary Writers’ Writing Skills with a Chatbot. Journal of Educational Technology & Society, 23(1), 78–92.
Little, J. L., & McDaniel, M. A. (2015). Metamemory monitoring and control following retrieval practice for text. Memory & Cognition, 43(1), 85–98. https://doi.org/10.3758/s13421-014-0453-7
Mishra, P., Singh, U., Pandey, C. M., Mishra, P., & Pandey, G. (2019). Application of student’s t-test, analysis of variance, and covariance. Annals of Cardiac Anaesthesia, 22(4), 407–411. https://doi.org/10.4103/aca.ACA_94_19
Moriuchi, E., Landers, V. M., Colton, D., & Hair, N. (2020). Engagement with chatbots versus augmented reality interactive technology in e-commerce. Journal of Strategic Marketing. https://doi.org/10.1080/0965254X.2020.1740766
Naylor, S., & Keogh, B. (1999). Constructivism in classroom: Theory into practice. Journal of Science Teacher Education, 10(2), 93–106. https://doi.org/10.1023/A:1009419914289
Nuruzzaman, M., & Hussain, O. K. (2020). IntelliBot: A Dialogue-based chatbot for the insurance industry. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2020.105810
Ochoa, X., & Wise, A. F. (2021). Supporting the shift to digital with student-centered learning analytics. Educational Technology Research and Development, 69(1), 357–361. https://doi.org/10.1007/s11423-020-09882-2
Ouyang, F., & Jiao, P. (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2, 100020. https://doi.org/10.1016/j.caeai.2021.100020
Peklaj, C., Podlesek, A., & Pečjak, S. (2015). Gender, previous knowledge, personality traits and subject-specific motivation as predictors of students’ math grade in upper-secondary school. European Journal of Psychology of Education, 30(3), 313–330. https://doi.org/10.1007/s10212-014-0239-0
Piaget, J., & Duckworth, E. (1970). Genetic epistemology. American Behavioral Scientist, 13(3), 459–480. https://doi.org/10.1177/000276427001300320
Piau, A., Crissey, R., Brechemier, D., Balardy, L., & Nourhashemi, F. (2019). A smartphone Chatbot application to optimize monitoring of older patients with cancer. International Journal of Medical Informatics, 128, 18–23. https://doi.org/10.1016/j.ijmedinf.2019.05.013
Pintrich, P.R., Smith, D.A.F., Garcia, T., & McKeachie, W.J. (1991). A manual for the use of the motivated strategies for learning questionnaire (MSLQ). MI: National Center for Research to Improve Postsecondary Teaching and Learning. (ERIC Document Reproduction Service No. ED 338122)
Rajaobelina, L., & Ricard, L. (2021). Classifying potential users of live chat services and chatbots. Journal of Financial Services Marketing, 26(2), 81–94.
Rao, M. S., Mounika, M., & Fareed, S. (2020). Implementation of service based chatbot using deep learning. Test Engineering and Management, 83, 2013–2019.
Roblyer, M. D., McDaniel, M., Webb, M., Herman, J., & Witty, J. V. (2010). Findings on Facebook in higher education: A comparison of college faculty and student uses and perceptions of social networking sites. The Internet and Higher Education, 13(3), 134–140. https://doi.org/10.1016/j.iheduc.2010.03.002
Roque, G. D. S. L., de Souza, R. R., do Nascimento, J. W. A., de Campos Filho, A. S., de Melo Queiroz, S. R., & Santos, I. C. R. V. (2021). Content validation and usability of a chatbot of guidelines for wound dressing. International Journal of Medical Informatics, 151, 104473. https://doi.org/10.1016/j.ijmedinf.2021.104473
Schmidlen, T., Schwartz, M., DiLoreto, K., Kirchner, H. L., & Sturm, A. C. (2019). Patient assessment of chatbots for the scalable delivery of genetic counseling. Journal of Genetic Counseling, 28(6), 1166–1177. https://doi.org/10.1002/jgc4.1169
Shah, H., Warwick, K., Vallverdú, J., & Wu, D. (2016). Can machines talk? Comparison of Eliza with modern dialogue systems. Computers in Human Behavior, 58, 278–295. https://doi.org/10.1016/j.chb.2016.01.004
Shawar, B. A., & Atwell, E. (2007, April). Different measurement metrics to evaluate a chatbot system. In Proceedings of the workshop on bridging the gap: Academic and industrial research in dialog technologies (pp. 89–96). https://doi.org/10.3115/1556328.1556341
Sheehan, B., Jin, H. S., & Gottlieb, U. (2020). Customer service chatbots: Anthropomorphism and adoption. Journal of Business Research, 115, 14–24. https://doi.org/10.1016/j.jbusres.2020.04.030
Shimaya, J., Yoshikawa, Y., Ogawa, K., & Ishiguro, H. (2021). Robotic question support system to reduce hesitation for face-to-face questions in lectures. Journal of Computer Assisted Learning, 37(3), 621–631. https://doi.org/10.1111/jcal.12511
Shinogaya, K. (2012). Learning strategies: A review from the perspective of the relation between learning phases. Japanese Journal of Educational Psychology, 60(1), 92–105.
Slavuj, V., Meštrović, A., & Kovačić, B. (2017). Adaptivity in educational systems for language learning: A review. Computer Assisted Language Learning, 30(1–2), 64–90. https://doi.org/10.1080/09588221.2016.1242502
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers and Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
Sotolongo, N., & Copulsky, J. (2018). Conversational marketing: Creating compelling customer connections. Applied Marketing Analytics, 4(1), 6–21.
Srinivas, S., & Sashi Rekha, K. (2019). Health care chatbot system. Test Engineering and Management, 81(11–12), 5607–5611.
Stowell, J. R., Oldham, T., & Bennett, D. (2010). Using student response systems (“clickers”) to combat conformity and shyness. Teaching of Psychology, 37(2), 135–140. https://doi.org/10.1080/00986281003626631
Sung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252–275. https://doi.org/10.1016/j.compedu.2015.11.008
Tai, T. Y., & Chen, H. H. J. (2020). The impact of Google Assistant on adolescent EFL learners’ willingness to communicate. Interactive Learning Environments. https://doi.org/10.1080/10494820.2020.1841801
Terwel, J. (1999). Constructivism and its implications for curriculum theory and practice. Journal of Curriculum Studies, 31(2), 195–199.
Thompson, C. P., & Barnett, C. (1985). Review, recitation, and memory monitoring. Journal of Educational Psychology, 77(5), 533–538.
Timms, M. J. (2016). Letting artificial intelligence in education out of the box: Educational cobots and smart classrooms. International Journal of Artificial Intelligence in Education, 26(2), 701–712. https://doi.org/10.1007/s40593-016-0095-y
Tyler, I. V., Hau, M., Buxton, J. A., Elliott, L. J., Harvey, B. J., Hockin, J. C., & Mowat, D. L. (2009). Canadian medical students’ perceptions of public health education in the undergraduate medical curriculum. Academic Medicine, 84(9), 1307–1312. https://doi.org/10.1097/ACM.0b013e3181b189b4
Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S., & Torous, J. B. (2019). Chatbots and conversational agents in mental health: A review of the psychiatric landscape. The Canadian Journal of Psychiatry, 64(7), 456–464. https://doi.org/10.1177/0706743719828977
Valtolina, S., & Neri, L. (2021). Visual design of dialogue flows for conversational interfaces. Behaviour & Information Technology. https://doi.org/10.1080/0144929X.2021.1918249
Vattøy, K.-D., & Smith, K. (2019). Students’ perceptions of teachers’ feedback practice in teaching English as a foreign language. Teaching and Teacher Education, 85, 260–268. https://doi.org/10.1016/j.tate.2019.06.024
Virvou, M., & Alepis, E. (2005). Mobile educational features in authoring tools for personalised tutoring. Computers & Education, 44(1), 53–68. https://doi.org/10.1016/j.compedu.2003.12.020
Vozenilek, J., Huff, J. S., Reznek, M., & Gordon, J. A. (2004). See one, do one, teach one: Advanced technology in medical education. Academic Emergency Medicine, 11(11), 1149–1154. https://doi.org/10.1197/j.aem.2004.08.003
Wang, L. C., & Chen, M. P. (2010). The effects of game strategy and preference-matching on flow experience and programming performance in game-based learning. Innovations in Education and Teaching International, 47(1), 39–52. https://doi.org/10.1080/14703290903525838
Wilson, J., Ahrendt, C., Fudge, E. A., Raiche, A., Beard, G., & MacArthur, C. (2021). Elementary teachers’ perceptions of automated feedback and automated scoring: Transforming the teaching and learning of writing using automated writing evaluation. Computers & Education, 168, 104208. https://doi.org/10.1016/j.compedu.2021.104208
Zhang, J., Oh, Y. J., Lange, P., Yu, Z., & Fukuoka, Y. (2020). Artificial intelligence chatbot behavior change model for designing artificial intelligence chatbots to promote physical activity and a healthy diet. Journal of Medical Internet Research, 22(9), e22845. https://doi.org/10.2196/22845
Acknowledgements
This study is supported in part by the Ministry of Science and Technology of Taiwan under contract numbers MOST-109-2511-H-011-002-MY3 and MOST-108-2511-H-011-005-MY3.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors would like to declare that there is no conflict of interest in this study.
Consent to participant
The participants were protected by hiding their personal information during the research process. They knew that the participation was voluntary and they could withdraw from the study at any time.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lee, YF., Hwang, GJ. & Chen, PY. Impacts of an AI-based chabot on college students’ after-class review, academic performance, self-efficacy, learning attitude, and motivation. Education Tech Research Dev 70, 1843–1865 (2022). https://doi.org/10.1007/s11423-022-10142-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-022-10142-8