Psychometric study and validation of a questionnaire for the evaluation of online university lecturers
Article Sidebar
Main Article Content
Abstract
In this paper we present the psychometric study of the Questionnaire for the Evaluation of Online University Lecturers (version D), whose purpose is to evaluate the teaching of professors who work online mode, on the basis of the previous considerations of the student. The questionnaire has been constructed using a combination of qualitative and quantitative methodology. The particularly relevant feature of this scale is that it has been generated from the students' opinion about the characteristics that online lecturers must have in order to be considered a good teacher. In previous works we have presented the procedure that has been used to build the scale; in this we present the psychometric study of the specific questionnaire for online tutors. The sample was selected by random sampling by conglomerates, considering the Campus as a conglomerate. An exploratory factorial analysis (AFE) was carried out and subsequently a confirmatory factorial analysis (CFA). The results indicate that the constructed questionnaire has a high reliability and validity, which makes it a useful and relevant evaluation tool.
Keywords:
Downloads
Article Details
References
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological bulletin, 103(3), 411-423. https://doi.org/10.1037/0033-2909.103.3.411
Apodaca, P., & Grad, H. (2005). The dimensionality of student ratings of teaching: integration of uni- and multidimensional mols. Studies in Higher Education, 30(6), 723-748. https://doi.org/10.1080/03075070500340101
Barth, M.M. (2008). Deciphering student evaluations of teaching: A factor analysis approach. Journal of Education for Business, 84, 40–46. https://doi.org/10.3200/JOEB.84.1.40-46
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY, US: Guilford Press.
Brown, T., Williams, B., Jaberzadeh, S., Roller, L., Palermo, C., McKenna, L. … Sim, J. (2010). Predictors of attitudes to e-learning of Australian health care students. Journal of Applied Research in Higher Education, 2(1), 60-76. https://doi.org/10.1108/17581184201000006
Buchanan, J. (2011). Quality teaching: means for its enhancement? Australian Universities’ Review, 53(1), 66–72. Recuperado de https://eric.ed.gov/?id=EJ926451
Burdsal, C. A., & Harrison, P. D. (2008). Further evidence supporting the validity of both a multidimensional profile and an overall evaluation of teaching effectiveness. Assesment and Evaluation in Higher Education, 33, 567–576. https://doi.org/10.1080/02602930701699049
Cañadas, I. (2013). Study of the qualities of a good university lecturer of health sciences. Procedia – Social and Behavioral Sciences, 89, 342-350. https://doi.org/10.1016/j.sbspro.2013.08.858
Cañadas, I., Cuétara, I., & San Luis, C. (2014). Qualities demanded by on-line universties students about their tutors. International Conference. The Future of Education. 4 Edition. 12, 13 de junio de 2014. Florence.
Casero, A. (2010). Factores moduladores de la percepción de la calidad
docente. RELIEVE, 16(2), 1-17. Recuperado de http://www.uv.es/RELIEVE/v16n2/RELIEVEv16n2_3.htm
Carrascosa, J.R. y Molero, D. (2005). La valoración de la docencia universitaria en función del sexo y del centro de los estudiantes encuestados. Revista Española de Orientación y Psicopedagogía, 16(2), 301-318. Recuperado de http://www.redalyc.org/pdf/3382/338230773007.pdf
De Juan, J, Pérez, R.M, Gómez-Torres, M.J., Vizcaya, M.F. y Mora, J.M. (2007). Buenas prácticas en la evaluación de la docencia y del profesorado universitario. En M. A. Martínez y V. Carrasco (eds.) La multidimensionalidad de la Educación Universitaria. Redes de investigación Docente-Espacio Europeo de Enseñanza Superior, pp. 155-182.
Ferguson, J.M., & De Felice, A.F. (2010). Length of online course and student satisfaction, perceived learning, and academic performance. International Review of Research in Open and Distance Learning. 11(2), 73-84. Recuperado de https://files.eric.ed.gov/fulltext/EJ895748.pdf
Ferrando, P. J. y Anguiano-Carrasco, C. (2010). El análisis factorial como técnica de investigación en psicología. Papeles del Psicológo, 31 (1), 18-33.
Fu, F.L. (2010). Comparison of students’ satisfaction and dissatisfaction factors in different classroom types in higher education. En Tsang P., Cheung S.K.S., Lee V.S.K. y Huang R. (eds) Hybrid Learning. ICHL 2010. Lecture Notes in Computer Science, vol 6248, pp. 2-38. https://doi.org/10.1007/978-3-642-14657-2_38
Harman, H.H. (1976) Modern factor analysis. Third edition. Chicago: University of Chicago Press.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrica, 30(2), 179–185. https://doi.org/10.1007/BF02289447
Hung, M.L., & Chou, Ch. (2014). Students’ perceptions of instructors’ roles in blended and online learning environments: A comparative study. Computers & Education, 81, 315–325. https://doi.org/10.1016/j.compedu.2014.10.022
Hu, L. & Bentler, P. (1995). Evaluating model fit. In R. Hoyle (Ed.), Structural equation modelling: Concepts, issues and applications (pp.76-99). Thousand Oaks, CA: Sage Publications.
Kuo, Y.C., Walker, A.E., Belland, B.R., & Schroder, K.E.E., (2013). A predictive study of satisfaction in online education programs. The International Review of Research in Open and Distance Learnig, 14(1), 16-39. https://doi.org/10.19173/irrodl.v14i1.1338
Lattin, J. M., Carroll, J. D., Green, P. E., & Green, P. E. (2003). Analyzing multivariate data. Pacific Grove, CA: Thomson Brooks/Cole.
Lorenzo-Seva, U., & Ferrando, P. J. (2013). FACTOR 9.2: A Comprehensive program for fitting exploratory and semiconfirmatory factor analysis and IRT models. Applied Psychological Measurement, 37(6), 497–498. https://doi.org/10.1177/0146621613487794
MacDonald, K., & Gobson, C.E. (2011) Your tutor is your friend: using experiential learning to enhance second year transition. Journal of Applied Research in Higher Education, 3(2), pp.107–115. https://doi.org/10.1108/17581181111198656
Martínez Arias, R. (1995). Psicometría: Teoría de los tests psicológicos y educativos. Madrid: Síntesis.
Marsh, H.W. (2007). Students’ evaluations of university teaching: dimensionality, reliability, validity, potential biases and usefulness. En R.P. Perry y J.C. Smart (eds.) The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective, pp. 319–383.
Mortelmans, D., & Spooren, P. (2009). A revalidation of the SET37-questionnaire for student evaluations of teaching. Educational Studies, 35, pp.547–552. https://doi.org/10.1080/03055690902880299
Muthén, B., & Kaplan D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables: A note on the size of the mode. British Journal of Mathematical and Statistical Psychology, 45, 19-30. https://doi.org/10.1111/j.2044-8317.1985.tb00832.x
Naaj, M., Nachouki, M., & Ankit, A. (2012). Evaluating student satisfaction with blended learning in a gender-segregated environment. Journal of Information Technology Education: Research, 11(1), 185-200. Recuperado de https://www.learntechlib.org/p/111500/
Okoro, C.O., & Chukwudi, E.K. (2011). The ideal teacher and the motivated student in a changing environment. Journal of Educational and Social Research, 1(3), 107 112.Recuperado de http://www2.ohchr.org/english/bodies/cedaw/docs/co/NigeriaCO30.pdf
Onwuegbuzie, A.J., Daniel, L.G., & Collins, K.M.T. (2009). A metavalidation model for assessing the score validity of student teaching evaluations. Quality & Quantity, 43, 197–209. https://doi.org/10.1007/s11135-007-9112-4
Pan, D., Tan, G.S.H., Ragupathi, K., Booluck, K., Roop, R., & Ip,Y.K. (2009). Profiling teacher/teaching using descriptors derived from qualitative feedback: Formative and summative applications. Research in Higher Education, 50, 73–100. https://doi.org/10.1007/s11162-008-9109-4
Pozo-Munoz, C., Rebolloso-Pacheco, E., & Fernandez-Ramirez, B. (2000). The 'Ideal Teacher'. Implications for student evaluation of teacher effectiveness. Assessment & Evaluation in Higher Education 25(3), 253-263. https://doi.org/10.1080/02602930050135121
Rogers, J., & Smith, M. (2011). Demonstrating genuine interest in students' needs and progress: Implications for student satisfaction with courses. Journal of Applied Research in Higher Education, 3(1), 6–14. https://doi.org/10.1108/17581181111150865
Rothman, T., Romeo, L., Brennan, M., & Mitchell, D. (2011). Criteria for assessing student satisfaction with online courses. International Journal for e-Learning Securit, 1, 27-32. Recuperado de http://www.infonomics-society.org/IJeLS/Criteria for Assessing Student Satisfaction with Online Courses.pdf
San Luis, C., & Cañadas, I. (2014). Qualitative and quantitative methods to
assess the qualities of a lecturer: What qualities are demanded by on-line
and on-site students? Procedia - Social and Behavioral Sciences, 143,
-111. https://doi.org/10.1016/j.sbspro.2014.07.369
Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53-61. http://dx.doi.org/10.1007/BF02289209
Snook, S. C., & Gorsuch, R. L. (1989). Component analysis versus common factor analysis: A Monte Carlo study. Psychological Bulletin, 106(1), 148-154. https://doi.org/10.1037/0033-2909.106.1.148
Spencer, K. J. and Schmelkin, L. P. (2002). Student perspectives on teaching and its evaluation. Assessment & Evaluation in Higher Education, 27, 397–409. https://doi.org/10.1080/0260293022000009285
Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The State of the Art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870
Su, F., & Wood, M. (2012). What makes a good university lecturer? Students’ perceptions of teaching excellence Journal of Applied Research in Higher Education, 4 (2), 142-155. https://doi.org/10.1108/17581181211273110
Tanaka, J.S., & Huba, G.J. (1985). A fit index for covariance structure models under arbitrary GLS estimation. British Journal of Mathematical and Statistical Psychology, 38, 197–201. https://doi.org/10.1111/j.2044-8317.1985.tb00834.x
Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209-220. https://doi.org/10.1037/a0023353